queryverse / queryverse/CSVFiles.jl
Errors in saving non-standard element types
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 52
- Forks
- 12
- Avg merge
- 18h 21m
- Merged PRs (30d)
- 3
Description
Consider the following code:
julia> df = DataFrame(x = [',','\n', ','])
3×1 DataFrame
│ Row │ x │
│ │ Char │
├─────┼──────┤
│ 1 │ ',' │
│ 2 │ '\n' │
│ 3 │ ',' │
julia> df |> save("test.csv")
julia> println(read("test.csv", String))
"x"
,
,
julia>
And the saved file is broken because non-strings are saved as not quoted.
Here is an extreme example (not to say it happens in reality, but just shows that it could be handled better). The code is a continuation of the earlier code:
julia> DataFrame(d=[df, df]) |> save("test2.csv")
julia> println(read("test2.csv", String))
"d"
3×1 DataFrame
│ Row │ x │
│ │ Char │
├─────┼──────┤
│ 1 │ ',' │
│ 2 │ '\n' │
│ 3 │ ',' │
3×1 DataFrame
│ Row │ x │
│ │ Char │
├─────┼──────┤
│ 1 │ ',' │
│ 2 │ '\n' │
│ 3 │ ',' │
and it is completely unreadable back (even as string) because it is not quoted again.
Finally let us consider a more normal scenario, which is again broken because of non-quoting:
julia> df = DataFrame(a=Date("2000-10-10"), b=Date("2000-11-11"))
1×2 DataFrame
│ Row │ a │ b │
│ │ Date │ Date │
├─────┼────────────┼────────────┤
│ 1 │ 2000-10-10 │ 2000-11-11 │
julia> df |> save("test3.csv", delim="-")
julia> println(read("test3.csv", String))
"a"-"b"
2000-10-10-2000-11-11
@davidanthoff Not sure which of the issues above can be fixed but at least I wanted you to be aware of them.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the three examples using save("test.csv"), save("test2.csv"), and save("test3.csv", delim="-"). Trace the save path for non-string values and check how delimiters, newlines, and nested displayed values are handled. Done means the resulting CSV remains readable when values contain delimiters or newlines and the reported examples no longer produce broken rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100