mafintosh / mafintosh/csv-parser

Provide "isQuoted" information to mapValues function

Open
#172 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
1.5k
Forks
143
PR merge metrics
No merged PRs in 30d

Description

### Feature Proposal
My feature proposal is to include information on whether a particular cell was quoted or not in the mapValues function. This would add a new boolean parameter that contains this information for each cell.

```javascript
csv({
mapValues: ({ header, index, value, isQuoted }) => value.toLowerCase()
})
```

This information is available and checked for in the following source lines in the [parseCell method](https://github.com/mafintosh/csv-parser/blob/master/index.js#L67):

```javascript
parseCell (buffer, start, end) {
...
// remove quotes from quoted cells
if (buffer[start] === quote && buffer[end - 1] === quote) {
...
}
```
### Feature Use Case
Currently there is no way for consumers to know whether a particular cell is quoted in the csv document. This is information that may be relevant to certain consumers, eg. to infer data types when deserializing JSON Data.

**Example**
Suppose you have a CSV file which contains data serialized from JSON, ie. you are writing CSV values using JSON.stringify(value).

```csv
NAME,COMMENT,KARMA
"Daffy",null,0
"Bugs","I'd like to interject...",22
```

You could write a mapValues function like so:
```javascript
csv({
mapValues: ({ header, index, value, isQuoted }) => isQuoted ? value : JSON.parse(value),
})
```

And you would get the following outputs:
```
{ NAME: "Daffy", COMMENT: null, KARMA: 0 }
{ NAME: "Bugs", COMMENT: "I'd like to interject...", KARMA: 22 }
```

Explanation: JSON.parse() accepts literal values like `'0.345'`, `'2000'`, `'true'`, `'false'`, `'null'`, but will throw for strings that don't also have literal double-quotes around them (ie. `'"str"'` is ok, `'str'` will throw). Without providing the cell information. there's no way to get the same JSON data types out - they will all be strings.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in index.js at the parseCell method and trace how parsed cell values are passed to mapValues. Add the requested quoted-state information to each mapValues call while preserving the existing value behavior. Done means consumers can distinguish quoted from unquoted cells in the callback, including the examples described in this issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.