queryverse / queryverse/CSVFiles.jl
Problem with nastring for non-numeric columns
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 52
- Forks
- 12
- Avg merge
- 18h 21m
- Merged PRs (30d)
- 3
Description
Because the default nastring is NA there is a following problem:
- take a data structure that has e.g.
Stringcolumn with missing data in it; saveit to disk using default parameters; missings get converted toNAon diskloadit back and you have"NA"string where you earlier had missings
The same problem occurs with e.g. Char data.
While NA is a sensible default for numeric columns it is a bit confusing for non-numeric columns (and actually can lead to wrong results as it is fully possible to have NA string in data).
I think that it would be best to have an empty string for missings in non-numeric data.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in CSVFiles.jl by locating the default nastring handling for non-numeric columns and the save/load path. Add coverage for String and Char missing-value round trips with default parameters, and verify that existing numeric-column behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100