adaltas / adaltas/node-csv

Support N-x configuration for To option to skip trailer rows

Open
#356 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
4.3k
Forks
299
Avg merge
16h 19m
Merged PRs (30d)
1

Description

Summary

In csv-parse, the 'to' and 'to_line' options do not adequately support stripping trailing records. Recommend adding functionality that would either allow a syntax to these options, eg: 'T.1' to indicate stopping at end-1.

Motivation

We process some files that include a trailing "summary" record for the file. These are difficult to deal with.

Alternative

We could pre-process the file to strip these trailing rows, but it can be tricky if they contain fields with embedded newlines in field.

Draft

This can be easily implemented by dropping each parsed row into a holding queue; by pushing the configured "T-minus" row along the pipe rather than the current one, and simply dropping the queue at EOF.

Additional context

Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the handling for the to and to_line options and the parser's end-of-file path, then inspect the existing parser tests. Confirm how a trailing-record offset should be represented and verify that configured trailing records are omitted without changing current stopping behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.