Read filenames in chunks in Java streams from a folder
I have a directory with millions of files in it. I want to read the filenames into an ArrayList. If I read all filenames into an ArrayList, it consumes more memory. I suspect if a huge number of files are present in the directory, Java might throw heap space error.
Is there a way to read files in a directory in chunks/batches, Say 5 filenames each time.
java file arraylist heap-memory file-read
asked Nov 27 '18 at 10:57
NishanthNishanth
227
add a comment |
I have a directory with millions of files in it. I want to read the filenames into an ArrayList. If I read all filenames into an ArrayList, it consumes more memory. I suspect if a huge number of files are present in the directory, Java might throw heap space error.
Is there a way to read files in a directory in chunks/batches, Say 5 filenames each time.
java file arraylist heap-memory file-read
asked Nov 27 '18 at 10:57
NishanthNishanth
227
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13
add a comment |
I have a directory with millions of files in it. I want to read the filenames into an ArrayList. If I read all filenames into an ArrayList, it consumes more memory. I suspect if a huge number of files are present in the directory, Java might throw heap space error.
Is there a way to read files in a directory in chunks/batches, Say 5 filenames each time.
java file arraylist heap-memory file-read
asked Nov 27 '18 at 10:57
NishanthNishanth
227
I have a directory with millions of files in it. I want to read the filenames into an ArrayList. If I read all filenames into an ArrayList, it consumes more memory. I suspect if a huge number of files are present in the directory, Java might throw heap space error.
Is there a way to read files in a directory in chunks/batches, Say 5 filenames each time.
java file arraylist heap-memory file-read
java file arraylist heap-memory file-read
asked Nov 27 '18 at 10:57
NishanthNishanth
227
asked Nov 27 '18 at 10:57
NishanthNishanth
227
asked Nov 27 '18 at 10:57
NishanthNishanth
227
asked Nov 27 '18 at 10:57
NishanthNishanth
227
asked Nov 27 '18 at 10:57
NishanthNishanth
227
227
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13
add a comment |
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13
add a comment |
2 Answers
2
active
oldest
votes
You might use Path.list for this, it will return you a stream which is a lazy evaluated:
List<String> fileNames = Path.list("path_to_directory")
.map(Path::getFileName)
.collect(Collectors.toList());
The files will be processed one by one and this will consume less memory. However you still might get memory issues if the final list fileNames will get too big. So the terminal (collect) operation in the stream pipeline might cause some issues.
But for example if you process the filename directly on the stream (using forEach for example without collecting them) you can avoid loading all the names into the memory.
Path.list("path_to_directory")
.map(Path::getFileName)
.forEach(System.out::println);
// print files one by one without loading all of them at the same time.
I hope this helps.
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:Path.list("path_to_directory").parallel()will convert stream into aparallelStream.
– Anton Balaniuc
Nov 27 '18 at 12:44
add a comment |
You can use FileVisitor class to traverse and read one file at a time. In this way you will not get OOM error.
Use Files.html#walkFileTree method to visit files in directory.
General example is below.
Path path = FileSystems.getDefault().getPath("D:\path\with\lots\of\files");
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
// here you have the files to process
System.out.println(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
return FileVisitResult.TERMINATE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
Here is link to Java SE tutorial.
For more examples, refer here.
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53498125%2fread-filenames-in-chunks-in-java-streams-from-a-folder%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You might use Path.list for this, it will return you a stream which is a lazy evaluated:
List<String> fileNames = Path.list("path_to_directory")
.map(Path::getFileName)
.collect(Collectors.toList());
The files will be processed one by one and this will consume less memory. However you still might get memory issues if the final list fileNames will get too big. So the terminal (collect) operation in the stream pipeline might cause some issues.
But for example if you process the filename directly on the stream (using forEach for example without collecting them) you can avoid loading all the names into the memory.
Path.list("path_to_directory")
.map(Path::getFileName)
.forEach(System.out::println);
// print files one by one without loading all of them at the same time.
I hope this helps.
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:Path.list("path_to_directory").parallel()will convert stream into aparallelStream.
– Anton Balaniuc
Nov 27 '18 at 12:44
add a comment |
You might use Path.list for this, it will return you a stream which is a lazy evaluated:
List<String> fileNames = Path.list("path_to_directory")
.map(Path::getFileName)
.collect(Collectors.toList());
The files will be processed one by one and this will consume less memory. However you still might get memory issues if the final list fileNames will get too big. So the terminal (collect) operation in the stream pipeline might cause some issues.
But for example if you process the filename directly on the stream (using forEach for example without collecting them) you can avoid loading all the names into the memory.
Path.list("path_to_directory")
.map(Path::getFileName)
.forEach(System.out::println);
// print files one by one without loading all of them at the same time.
I hope this helps.
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:Path.list("path_to_directory").parallel()will convert stream into aparallelStream.
– Anton Balaniuc
Nov 27 '18 at 12:44
add a comment |
You might use Path.list for this, it will return you a stream which is a lazy evaluated:
List<String> fileNames = Path.list("path_to_directory")
.map(Path::getFileName)
.collect(Collectors.toList());
The files will be processed one by one and this will consume less memory. However you still might get memory issues if the final list fileNames will get too big. So the terminal (collect) operation in the stream pipeline might cause some issues.
But for example if you process the filename directly on the stream (using forEach for example without collecting them) you can avoid loading all the names into the memory.
Path.list("path_to_directory")
.map(Path::getFileName)
.forEach(System.out::println);
// print files one by one without loading all of them at the same time.
I hope this helps.
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
You might use Path.list for this, it will return you a stream which is a lazy evaluated:
List<String> fileNames = Path.list("path_to_directory")
.map(Path::getFileName)
.collect(Collectors.toList());
The files will be processed one by one and this will consume less memory. However you still might get memory issues if the final list fileNames will get too big. So the terminal (collect) operation in the stream pipeline might cause some issues.
But for example if you process the filename directly on the stream (using forEach for example without collecting them) you can avoid loading all the names into the memory.
Path.list("path_to_directory")
.map(Path::getFileName)
.forEach(System.out::println);
// print files one by one without loading all of them at the same time.
I hope this helps.
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
edited Nov 27 '18 at 11:52
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
answered Nov 27 '18 at 11:20
Anton BalaniucAnton Balaniuc
4,5871532
4,5871532
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:Path.list("path_to_directory").parallel()will convert stream into aparallelStream.
– Anton Balaniuc
Nov 27 '18 at 12:44
add a comment |
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:Path.list("path_to_directory").parallel()will convert stream into aparallelStream.
– Anton Balaniuc
Nov 27 '18 at 12:44
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
This will also load all the filenames into fileNames . I need to have only a subset of filenames from a folder in the memory. Say if there was 100 files in a folder, i need only 5 files in the memory. After I print the 5 filenames, I should get the next 5 filenames and print them. The objective is to use less memory by not loading all filenames in memory
– Nishanth
Nov 27 '18 at 11:34
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
@Nishanth if you will not use collect, but print the filenames directory you can avoid loading all of them into the memory. Please see second part of my answer. What exactly do you need to do with the filenames? Just print them?
– Anton Balaniuc
Nov 27 '18 at 11:53
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
No Mr.Anton Balaniuc, I need a chunk of filenames to be streamed from the folder. If I get filenames one by one, it consumes more time. Getting filenames in chunks will be faster. For Example: f there are 100 files in a folder, and if I print them one by one, it consumes more time. Rather printing 10 filenames at a time will consume less amount of time. Instead of using "forEach" is there any possible way to process like ".forEachChunkOfSize(10)" ?
– Nishanth
Nov 27 '18 at 12:32
1
1
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:
Path.list("path_to_directory").parallel() will convert stream into a parallelStream.– Anton Balaniuc
Nov 27 '18 at 12:44
@Nishanth, there is no such possibility with streams. By nature stream process elements one by one. But you can do this in parallel:
Path.list("path_to_directory").parallel() will convert stream into a parallelStream.– Anton Balaniuc
Nov 27 '18 at 12:44
add a comment |
You can use FileVisitor class to traverse and read one file at a time. In this way you will not get OOM error.
Use Files.html#walkFileTree method to visit files in directory.
General example is below.
Path path = FileSystems.getDefault().getPath("D:\path\with\lots\of\files");
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
// here you have the files to process
System.out.println(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
return FileVisitResult.TERMINATE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
Here is link to Java SE tutorial.
For more examples, refer here.
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
add a comment |
You can use FileVisitor class to traverse and read one file at a time. In this way you will not get OOM error.
Use Files.html#walkFileTree method to visit files in directory.
General example is below.
Path path = FileSystems.getDefault().getPath("D:\path\with\lots\of\files");
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
// here you have the files to process
System.out.println(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
return FileVisitResult.TERMINATE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
Here is link to Java SE tutorial.
For more examples, refer here.
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
add a comment |
You can use FileVisitor class to traverse and read one file at a time. In this way you will not get OOM error.
Use Files.html#walkFileTree method to visit files in directory.
General example is below.
Path path = FileSystems.getDefault().getPath("D:\path\with\lots\of\files");
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
// here you have the files to process
System.out.println(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
return FileVisitResult.TERMINATE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
Here is link to Java SE tutorial.
For more examples, refer here.
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
You can use FileVisitor class to traverse and read one file at a time. In this way you will not get OOM error.
Use Files.html#walkFileTree method to visit files in directory.
General example is below.
Path path = FileSystems.getDefault().getPath("D:\path\with\lots\of\files");
Files.walkFileTree(path, new FileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
// here you have the files to process
System.out.println(file);
return FileVisitResult.CONTINUE;
}
@Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
return FileVisitResult.TERMINATE;
}
@Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
return FileVisitResult.CONTINUE;
}
});
Here is link to Java SE tutorial.
For more examples, refer here.
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
answered Nov 27 '18 at 11:14
Akhil JainAkhil Jain
10.2k124680
10.2k124680
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53498125%2fread-filenames-in-chunks-in-java-streams-from-a-folder%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
i'm not sure as i don't have a directory with millions of files. But maybe you can use a stream? Like for instance Arrays.stream(Paths.get("path/to/file").toFile().list()); You can than use the StringStream to do the things you wanted to do with the fileNames
– Stephan Hogenboom
Nov 27 '18 at 11:07
Even If I use code like Arrays.stream(Paths.get("path/to/file").toFile().list());, This part Paths.get("path/to/file").toFile().list() will still bring all the filenames into memory. My problem is to use less memory to get all filenames.
– Nishanth
Nov 27 '18 at 11:13