Access File using FILE_FLAG_RANDOM_ACCESS with a limit on cache size
I want to access a file uisng using FILE_FLAG_RANDOM_ACCESS. But when a large file is accessed via FILE_FLAG_RANDOM_ACCESS, huge memory consumption could result in bad system performance. Is there a way to put a limit on cache size for a specific file handle?
c++ windows
asked Jun 19 '14 at 4:36
Sravan Goud
487
add a comment |
I want to access a file uisng using FILE_FLAG_RANDOM_ACCESS. But when a large file is accessed via FILE_FLAG_RANDOM_ACCESS, huge memory consumption could result in bad system performance. Is there a way to put a limit on cache size for a specific file handle?
c++ windows
asked Jun 19 '14 at 4:36
Sravan Goud
487
1
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53
add a comment |
I want to access a file uisng using FILE_FLAG_RANDOM_ACCESS. But when a large file is accessed via FILE_FLAG_RANDOM_ACCESS, huge memory consumption could result in bad system performance. Is there a way to put a limit on cache size for a specific file handle?
c++ windows
asked Jun 19 '14 at 4:36
Sravan Goud
487
I want to access a file uisng using FILE_FLAG_RANDOM_ACCESS. But when a large file is accessed via FILE_FLAG_RANDOM_ACCESS, huge memory consumption could result in bad system performance. Is there a way to put a limit on cache size for a specific file handle?
c++ windows
c++ windows
asked Jun 19 '14 at 4:36
Sravan Goud
487
asked Jun 19 '14 at 4:36
Sravan Goud
487
asked Jun 19 '14 at 4:36
Sravan Goud
487
asked Jun 19 '14 at 4:36
Sravan Goud
487
asked Jun 19 '14 at 4:36
Sravan Goud
487
487
1
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53
add a comment |
1
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53
1
1
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53
add a comment |
2 Answers
2
active
oldest
votes
Windows doesn't provide a way to set a maximum cache size, but this is relatively easy to implement on your own. Open the file with FILE_FLAG_NO_BUFFERING and implement your own cache with an LRU list to determine when to evict blocks. Note that starting with Windows Server 2016, FILE_FLAG_RANDOM_ACCESS is less aggressive about retaining data in the cache, so this is less of a concern. Microsoft still recommends avoiding this flag, however.
Further reading: Troubleshoot Cache and Memory Manager Performance Issues
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
add a comment |
Yes there is a way to control file buffering in Windows but you will need to ditch the C++ standard library and use low-level OS file I/O APIs. If you are still interested, start reading here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365199(v=vs.85).aspx
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f24298878%2faccess-file-using-file-flag-random-access-with-a-limit-on-cache-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Windows doesn't provide a way to set a maximum cache size, but this is relatively easy to implement on your own. Open the file with FILE_FLAG_NO_BUFFERING and implement your own cache with an LRU list to determine when to evict blocks. Note that starting with Windows Server 2016, FILE_FLAG_RANDOM_ACCESS is less aggressive about retaining data in the cache, so this is less of a concern. Microsoft still recommends avoiding this flag, however.
Further reading: Troubleshoot Cache and Memory Manager Performance Issues
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
add a comment |
Windows doesn't provide a way to set a maximum cache size, but this is relatively easy to implement on your own. Open the file with FILE_FLAG_NO_BUFFERING and implement your own cache with an LRU list to determine when to evict blocks. Note that starting with Windows Server 2016, FILE_FLAG_RANDOM_ACCESS is less aggressive about retaining data in the cache, so this is less of a concern. Microsoft still recommends avoiding this flag, however.
Further reading: Troubleshoot Cache and Memory Manager Performance Issues
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
add a comment |
Windows doesn't provide a way to set a maximum cache size, but this is relatively easy to implement on your own. Open the file with FILE_FLAG_NO_BUFFERING and implement your own cache with an LRU list to determine when to evict blocks. Note that starting with Windows Server 2016, FILE_FLAG_RANDOM_ACCESS is less aggressive about retaining data in the cache, so this is less of a concern. Microsoft still recommends avoiding this flag, however.
Further reading: Troubleshoot Cache and Memory Manager Performance Issues
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
Windows doesn't provide a way to set a maximum cache size, but this is relatively easy to implement on your own. Open the file with FILE_FLAG_NO_BUFFERING and implement your own cache with an LRU list to determine when to evict blocks. Note that starting with Windows Server 2016, FILE_FLAG_RANDOM_ACCESS is less aggressive about retaining data in the cache, so this is less of a concern. Microsoft still recommends avoiding this flag, however.
Further reading: Troubleshoot Cache and Memory Manager Performance Issues
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
answered Nov 22 at 20:42
Peter Ruderman
10.1k2352
10.1k2352
add a comment |
add a comment |
Yes there is a way to control file buffering in Windows but you will need to ditch the C++ standard library and use low-level OS file I/O APIs. If you are still interested, start reading here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365199(v=vs.85).aspx
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
add a comment |
Yes there is a way to control file buffering in Windows but you will need to ditch the C++ standard library and use low-level OS file I/O APIs. If you are still interested, start reading here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365199(v=vs.85).aspx
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
add a comment |
Yes there is a way to control file buffering in Windows but you will need to ditch the C++ standard library and use low-level OS file I/O APIs. If you are still interested, start reading here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365199(v=vs.85).aspx
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
Yes there is a way to control file buffering in Windows but you will need to ditch the C++ standard library and use low-level OS file I/O APIs. If you are still interested, start reading here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365199(v=vs.85).aspx
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
answered Jun 19 '14 at 5:17
ThomasMcLeod
4,69242260
4,69242260
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
add a comment |
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
I don't see a way to control the cache size for a specific file
– Sravan Goud
Jun 19 '14 at 19:45
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
@SravanGoud, I think you would need to write your own buffering / cache with the low level APIs.
– ThomasMcLeod
Jun 20 '14 at 3:04
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f24298878%2faccess-file-using-file-flag-random-access-with-a-limit-on-cache-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Pretty sure Windows doesn't have a (documented) way to set a limit on the amount of cache space devoted to a particular file.
– Jerry Coffin
Jun 19 '14 at 4:42
Thank you. Could you please point me to the documentation.
– Sravan Goud
Jun 19 '14 at 8:55
You can flush the file system cache for a specific file by openening it from time to time with the FILE_FLAG_NO_BUFFERING flag. See geekswithblogs.net/akraus1/archive/2014/12/14/160652.aspx
– Alois Kraus
Nov 22 at 23:53