Skip to content

Add option to read more than one line at a time #13

@mifrandir

Description

@mifrandir

Currently, the program is bottlenecked by the fact that we are only reading one line at a time.

I might lead to serious performance gains if we limited the number of I/O operations (especially writes) by reading a variable number of lines. This number could be

  1. specified by the user,
  2. inferred from a given memory target and a line length heuristic, or
  3. inferred from a memory limit and actual line lengths and therefore dynamically change per batch.

Metadata

Metadata

Assignees

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions