mercredi 19 octobre 2016

read large chunk into memory and process line by line

I want to read and process a large text file efficiently, accoding to many posts on SO, it's advised to read large chunk of file into memory and do processing in memory.

I try to do it this way:

FILE* fp = fopen(path, "r");
char chunk[SIZE];
while (fread(chunk, SIZE, 1, fp)) {
  process_chunk_line_by_line(chunk);
}

The problem is, what if the last line in chunkis incomplete, I should not process that last line, should seek fp back by the length of the incomplete last line? Is there any more efficient way to do this?

Aucun commentaire:

Enregistrer un commentaire