nushell / nushell/nushell

`from csv`: Disable type inference for quoted values

Open
#16,987 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A:file-format category:enhancement status:needs-triage
Dominant language
Rust
Stars
40.5k
Forks
2.3k
Avg merge
1d 14h
Merged PRs (30d)
79

Description

Basics
  • I have done a basic search through the issue tracker to find similar or related issues.
  • I have made myself familiar with the available features of Nushell for the particular area this enhancement request touches.
Related problem

Numeric values in CSV files always get converted by type inference, even when they are quoted, which causes problems for numeric identifiers which should be strings because the leading zeros matter. The only way that I found to keep them as strings is to use --no-infer to disable type inference, but that applies to everything in the file, and therefore requires manually converting other columns back to numbers, which is not very convenient.

It would be really nice to be able to keep type inference except for quoted values, either as an option or by default.

Describe the solution you'd like

Either:

  • Add a --no-infer-quoted flag to disable type inference only for quoted values

Or:

  • Disable type inference for quoted values by default (if you're quoting numbers, you probably want them to be strings). But in that case you'd probably want a --infer-quoted flag to force type inference in quoted values if you use CSV files that quote everything by default.
Describe alternatives you've considered

No response

Additional context and details

Use case: Files with both numeric values that should be numbers, and numeric values that should be strings like barcodes and other identifiers. Especially when all the identifiers have leading zeros and are not of the same length.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the from csv command and trace its type-inference handling. Decide whether the behavior should be a new flag or the default, including the proposed override for quoted values. Done means quoted numeric identifiers can remain strings while unquoted numeric fields still infer as numbers, with the relevant behavior covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.