queryverse / queryverse/Query.jl
Error when filtering by Union{Bool, Null}
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 403
- Forks
- 48
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 6
Description
On Julia v0.6, using DataFrames, DataValues and Query masters:
julia> using Query, DataFrames
julia> df = DataFrame(a=[1,2,3], b=Union{Null,Bool}[true, true, false], c=[true, true, false])
3×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼───────┼───────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
│ 3 │ 3 │ false │ false │
julia> df|>@where(_.c)|>DataFrame # 1. filtering by non-nullable, OK
2×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼──────┼──────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
julia> df|>@where(_.b)|>DataFrame # 2. filtering by nullable, FAILS
ERROR: TypeError: non-boolean (DataValues.DataValue{Bool}) used in boolean context
Stacktrace:
[1] start(::QueryOperators.EnumerableWhere{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},QueryOperators.EnumerableIterable{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},IterableTables.DataFrameIterator{NamedTuples._NT_a_b_c{Int64,DataValues.DataValue{Bool},Bool},Tuple{Array{Int64,1},Array{Union{Bool, Nulls.Null},1},Array{Bool,1}}}},##14#16}) at /home/astukalov/.julia/v0.6/QueryOperators/src/enumerable/enumerable_where.jl:33
[2] macro expansion at /home/astukalov/.julia/v0.6/IterableTables/src/integrations/dataframes-null.jl:91 [inlined]
....
julia> df|>@where(_.b==true)|>DataFrame # 3. comparison of nullable with the value, OK
2×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼───┼──────┼──────┤
│ 1 │ 1 │ true │ true │
│ 2 │ 2 │ true │ true │
I don't know how to properly fix the second test (I assume it should have the same behaviour as the 1st), e.g. should Base.convert(::Type{Bool}, ::DataValue{Bool}) be defined or filtering lambda generation tweaked?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the nullable and non-nullable filtering examples on the Julia v0.6 setup, then inspect QueryOperators/src/enumerable/enumerable_where.jl and IterableTables/src/integrations/dataframes-null.jl at the stack-trace locations. Determine how nullable boolean predicates should behave and add coverage so filtering by Union{Bool, Null} has the intended consistent result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100