How to read a large file of Strings in chunks, each time from different offset, in Java? -
i have large file consists of several lines, each line 1 word of english. complete file not fit in memory. such, want process chunk chunk. so, need implement this:
- method1 : read 1 chunk, call method 2.
- method2: processing on strings in chunk , go step 1.
i have 2 questions.
1. how can 1 implement method1? know how implement method2.
i know how use bufferedreader read large file line-by-line. e.g.,
bufferedreader br = new bufferedreader(new filereader(file)) { string line; while ((line = br.readline()) != null) { // process line. }
but read more lines instead of 1 line, , process of them in method2. then, when method1, want load several lines once again...
2. assumption is more efficient (performance wise), process chunk of lines rather process each line. assumption correct?
the bufferedreader takes care of reading lines in chunks.
so choice between:
- after each call readline(), call method process it
- after each call readline(), add line list, , every time list gets size, call method process list, clear down
so main factor "processing" does: logically make sense run process on several lines @ once, , there optimisation processing method can include if several lines passed @ once?
Comments
Post a Comment