mafintosh / mafintosh/csv-parser
Invalid row produced when input data is differently chunked
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1.5k
- Forks
- 143
- PR merge metrics
- No merged PRs in 30d
Description
I encountered an issue where csv-parser would randomly append an invalid row to the parsed data, depending on how the data was chunked:
// Normally the last row:
{
STATIONS_ID: '535',
MESS_DATUM: '20200309',
QN_3: '-999',
FX: '-999',
FM: '-999',
QN_4: '1',
RSK: '6.4',
RSKF: '4',
SDK: '4.217',
SHK_TAG: '0',
NM: '-999',
VPM: '7.5',
PM: '-999',
TMK: '5.0',
UPM: '85.71',
TXK: '7.5',
TNK: '3.1',
TGK: '1.3',
eor: 'eor'
}
// Randomly (differently chunked) we get another row:
{ STATIONS_ID: '' }
- Operating System: macOS Catalina 10.15.3
- Node Version: v12.10.0
- NPM Version: 6.10.3
- csv-parser: Version: 2.3.2
Expected Behavior
The same result for the same data, no matter how it was chunked.
Actual Behavior
Different results for the same data.
How Do We Reproduce?
https://gist.github.com/hs-rhelm/e95623d9f09621aed96b93b269352049
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the linked reproduction gist against csv-parser 2.3.2 and compare results for differently chunked input. Trace the parser's handling of the final row and chunk boundaries. Done means identical parsed output for the same data, without an extra { STATIONS_ID: '' } row.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100