adaltas / adaltas/node-csv

`parse` with `{raw: true}` and windows line endings loses the newline

Open
#332 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
4.3k
Forks
299
Avg merge
16h 19m
Merged PRs (30d)
1

Description

Describe the bug

I'm calling parse with {raw: true} to get the original line. When lines end in just "\n", things work fine. When lines end in "\r\n", the raw output is missing the final "\n".

To Reproduce

const {parse} = require('csv-parse');

const inputs = [
    'a,b\nc,d\n',
    'a,b\r\nc,d\r\n',
    'a,b\r\nc,d\n',
];

for (const input of inputs) {
    parse(input, {raw: true}, (err, data) => {
        if (err) {
            console.error(err);
            return;
        }
        console.log(data);
    });
}

Output:

[
  { record: [ 'a', 'b' ], raw: 'a,b\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd' ], raw: 'c,d\r' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd\n' ], raw: 'c,d\n' }
]

First output: correct.

Second output: the raw fields are missing "\n" at the end. What I'd expect:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\r\n' }
]

Third output: In addition to the missing "\n" in the first raw, the 'd\n' cell value is surprising! Mixed line endings are tricky, so maybe there's no 100% correct answer here. But mixed line endings do come up in practice, and I think this might be better:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]

Additional context

My higher-level goal: I'm trying to add a column of cells to an existing CSV. I'd like to avoid modifying the format of any of the existing cells, since subtle changes in quoting can change how tools like Excel interpret a CSV. I was hoping the {raw: true} option would give me a way to do that, but I ran into this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the parse implementation and the handling for {raw: true}, then reproduce the three line-ending cases from the issue. Done means raw output preserves both characters for CRLF input and does not leave a newline in the final cell for mixed endings, with regression coverage for the reported examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.