queryverse / queryverse/Query.jl
@mutate changes other column type
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 403
- Forks
- 48
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 6
Description
Repost from https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292 as it looks like a bug.
This is my smallest code with repro:
contents = """
"5674012","aa66aa66"
"5674012","b036aa66,b036aa67,b036aa68"
""";
batches = CSV.File(IOBuffer(contents); header = ["X1", "Splits"], delim = ',') |> DataFrame;
emptyStringArray = Array{SubString{String},1}()
batchesAny = batches |>
@mutate(Splits = length(_.Splits) > 0 ? split(_.Splits, ',') : emptyStringArray) |> DataFrame
batchesString = batches |>
@mutate(Splits = length(_.Splits) > 0 ? split(_.Splits, ',') : Array{SubString{String},1}()) |> DataFrame
The output is like this:
julia> batchesAny
2×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Any │ Any │
├─────┼─────────┼──────────────────────────────────────┤
│ 1 │ 5674012 │ ["aa66aa66"] │
│ 2 │ 5674012 │ ["b036aa66", "b036aa67", "b036aa68"] │
julia> batchesString
2×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Int64 │ Array{SubString{String},1} │
├─────┼─────────┼──────────────────────────────────────┤
│ 1 │ 5674012 │ ["aa66aa66"] │
│ 2 │ 5674012 │ ["b036aa66", "b036aa67", "b036aa68"] │
What I don’t understand:
-
:Splitscolumn type differs - Any vs. Array{SubString{String},1}. (I just wanted to save memory so I stored the value (that can be repeated) toemptyStringArray.) -
Even if I understand that I made something bad to column
:Splits, I thinkX1's type shouldn't be changed toAnyinbatchesAny.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the Julia reproduction in the issue and inspect the implementation of @mutate and its DataFrame conversion path. Compare batchesAny with batchesString and verify that changing Splits does not also change X1 from Int64 to Any; done means the regression is covered and the demonstrated output preserves X1's type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100