Support N-x configuration for To option to skip trailer rows
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 4.3k
- Forks
- 299
- Avg merge
- 16h 19m
- Merged PRs (30d)
- 1
Description
Summary
In csv-parse, the 'to' and 'to_line' options do not adequately support stripping trailing records. Recommend adding functionality that would either allow a syntax to these options, eg: 'T.1' to indicate stopping at end-1.
Motivation
We process some files that include a trailing "summary" record for the file. These are difficult to deal with.
Alternative
We could pre-process the file to strip these trailing rows, but it can be tricky if they contain fields with embedded newlines in field.
Draft
This can be easily implemented by dropping each parsed row into a holding queue; by pushing the configured "T-minus" row along the pipe rather than the current one, and simply dropping the queue at EOF.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the handling for the to and to_line options and the parser's end-of-file path, then inspect the existing parser tests. Confirm how a trailing-record offset should be represented and verify that configured trailing records are omitted without changing current stopping behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100