denoland / denoland/std

CsvParseStream: negative `fieldsPerRecord` doesn't work with `skipFirstRow` or `columns`

Open
#6,434 1 comment 2 reactions 0 assignees View on GitHub
bug csv
Dominant language
TypeScript
Stars
3.6k
Forks
681
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**

According to [`fieldsPerRecord` documentation](https://github.com/denoland/std/blob/360851f6957702caf2fa8c998c02fbb7389ce21c/csv/parse_stream.ts#L43-L54), I should be able to parse CSV with variable length records. I can successfully do so but not when I use `skipFirstRow` and/or `columns`.

working example

```ts
import { CsvParseStream } from "jsr:@std/csv@1.0.5/parse-stream";
import { assertEquals } from "jsr:@std/assert@1.0.11/equals";
import { assertType, IsExact } from "jsr:@std/testing@1.0.9/types";

const source = ReadableStream.from([
"name,age\n",
"Alice,34\n",
"Bob\n", // incomplete record
]);
const stream = source.pipeThrough(new CsvParseStream({ fieldsPerRecord: -1 }));
const result = await Array.fromAsync(stream);

assertEquals(result, [
["name", "age"],
["Alice", "34"],
["Bob"],
]);
assertType>(true);
```

**Steps to Reproduce**

1. Create a script with the contents of any of the following examples



not working with skipFirstRow

```ts
import { CsvParseStream } from "jsr:@std/csv@1.0.5/parse-stream";
import { assertEquals } from "jsr:@std/assert@1.0.11/equals";
import { assertType, IsExact } from "jsr:@std/testing@1.0.9/types";

const source = ReadableStream.from([
"name,age\n",
"Alice,34\n",
"Bob\n", // incomplete record
]);
const stream = source.pipeThrough(
new CsvParseStream({
fieldsPerRecord: -1,
skipFirstRow: true,
}),
);
const result = await Array.fromAsync(stream);

assertEquals(result, [
{ name: "Alice", age: "34" },
{ name: "Bob", age: undefined },
]);
assertType[]>>(true);
```





not working with columns

```ts
import { CsvParseStream } from "jsr:@std/csv@1.0.5/parse-stream";
import { assertEquals } from "jsr:@std/assert@1.0.11/equals";
import { assertType, IsExact } from "jsr:@std/testing@1.0.9/types";

const source = ReadableStream.from([
"Alice,34\n",
"Bob\n", // incomplete record
]);
const stream = source.pipeThrough(
new CsvParseStream({
fieldsPerRecord: -1,
columns: ["name", "age"],
}),
);
const result = await Array.fromAsync(stream);

assertEquals(result, [
{ name: "Alice", age: "34" },
{ name: "Bob", age: undefined },
]);
assertType<
IsExact[]>
>(true);
```





not working with skipFirstRow and columns

```ts
import { CsvParseStream } from "jsr:@std/csv@1.0.5/parse-stream";
import { assertEquals } from "jsr:@std/assert@1.0.11/equals";
import { assertType, IsExact } from "jsr:@std/testing@1.0.9/types";

const source = ReadableStream.from([
"name,age\n",
"Alice,34\n",
"Bob\n", // incomplete record
]);
const stream = source.pipeThrough(
new CsvParseStream({
fieldsPerRecord: -1,
skipFirstRow: true,
columns: ["name", "age"],
}),
);
const result = await Array.fromAsync(stream);

assertEquals(result, [
{ name: "Alice", age: "34" },
{ name: "Bob", age: undefined },
]);
assertType<
IsExact[]>
>(true);
```


1. Execute the script with `deno run`

1. See error:

```console
error: Uncaught (in promise) Error: Syntax error on line 3: The record has 1 fields, but the header has 2 fields
throw new Error(
^
at convertRowToObject (https://jsr.io/@std/csv/1.0.5/_io.ts:233:11)
at CsvParseStream.#pull (https://jsr.io/@std/csv/1.0.5/parse_stream.ts:460:28)
```

**Expected behavior**

1. The scripts should run successfully.
2. TypeScript type checking should also pass.
3. `Record` or `Partial>` should be used instead of `Record`

**Environment**

- OS: MacOS 15.3.1
- deno version: 2.2.0
- std version: jsr:@std/csv@1.0.5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.