adaltas / adaltas/node-csv

Ignore comment lines for the from_line value

Open
#422 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
4.3k
Forks
299
Avg merge
16h 19m
Merged PRs (30d)
1

Description

Summary

I would like it if the from_line option could ignore comment lines when determining the line number.

Motivation

I receive a CSV structured like this:

// Some
// Number of
// Comments
thing1=value1
thing2=value2
header1,header2,header3
value,value,value
...

Where there's some arbitrary number of comment lines at the top, followed by exactly 2 lines of unrelated data that I need to skip before the actual csv content begins. The exact starting line can vary.

Alternative

I'm fetching the csv using got.stream and piping it into parse, then processing rows using for await. I might be able to add some middleware to filter out the unwanted rows before it goes to the parser but I'm not sure the best way to approach that.

Draft

Ideally I would like to be able to use {comment: '//', from_line: 3} to skip over all comments as well as the 2 unrelated lines. I.e. it would start parsing from 3rd non-comment line.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading how the parser handles the comment and from_line options, including streaming input from got.stream and row processing with for await. Confirm that {comment: '//', from_line: 3} begins parsing at the third non-comment line while still skipping the two unrelated lines, then cover the behavior with parser tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
data, stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.