apache / apache/arrow-julia

another schema violation

Open
#203 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

similar to https://github.com/JuliaData/Arrow.jl/issues/167
```julia
julia> signals
Arrow.Table with 15 rows, 11 columns, and schema:
:recording Base.UUID
:file_path String
:file_format String
:span TimeSpans.TimeSpan
:kind String
:channels Vector{String} (alias for Array{String, 1})
:sample_unit String
:sample_resolution_in_unit Float64
:sample_offset_in_unit Float64
:sample_type String
:sample_rate Float64

julia> Tables.rowtable(signals)
ERROR: TypeError: in new, expected Vector{String}, got a value of type SubArray{String, 1, Arrow.List{String, Int32, Vector{UInt8}}, Tuple{UnitRange{Int64}}, true}
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/Tables/YzCZp/src/namedtuples.jl:46 [inlined]
[2] _iterate(rows::Tables.NamedTupleIterator{Tables.Schema{(:recording, :file_path, :file_format, :span, :kind, :channels, :sample_unit, :sample_resolution_in_unit, :sample_offset_in_unit, :sample_type, :sample_rate), Tuple{Base.UUID, String, String, TimeSpans.TimeSpan, String, Vector{String}, String, Float64, Float64, String, Float64}}, Tables.RowIterator{Tables.CopiedColumns{Arrow.Table}}}, st::Tuple{})
@ Tables ~/.julia/packages/Tables/YzCZp/src/namedtuples.jl:37
[3] iterate
@ ~/.julia/packages/Tables/YzCZp/src/namedtuples.jl:58 [inlined]
[4] iterate
@ ~/.julia/packages/Tables/YzCZp/src/namedtuples.jl:57 [inlined]
[5] copyto!(dest::Vector{NamedTuple{(:recording, :file_path, :file_format, :span, :kind, :channels, :sample_unit, :sample_resolution_in_unit, :sample_offset_in_unit, :sample_type, :sample_rate), Tuple{Base.UUID, String, String, TimeSpans.TimeSpan, String, Vector{String}, String, Float64, Float64, String, Float64}}}, src::Tables.NamedTupleIterator{Tables.Schema{(:recording, :file_path, :file_format, :span, :kind, :channels, :sample_unit, :sample_resolution_in_unit, :sample_offset_in_unit, :sample_type, :sample_rate), Tuple{Base.UUID, String, String, TimeSpans.TimeSpan, String, Vector{String}, String, Float64, Float64, String, Float64}}, Tables.RowIterator{Tables.CopiedColumns{Arrow.Table}}})
@ Base ./abstractarray.jl:843
[6] _collect(cont::UnitRange{Int64}, itr::Tables.NamedTupleIterator{Tables.Schema{(:recording, :file_path, :file_format, :span, :kind, :channels, :sample_unit, :sample_resolution_in_unit, :sample_offset_in_unit, :sample_type, :sample_rate), Tuple{Base.UUID, String, String, TimeSpans.TimeSpan, String, Vector{String}, String, Float64, Float64, String, Float64}}, Tables.RowIterator{Tables.CopiedColumns{Arrow.Table}}}, #unused#::Base.HasEltype, isz::Base.HasLength)
@ Base ./array.jl:608
[7] collect(itr::Tables.NamedTupleIterator{Tables.Schema{(:recording, :file_path, :file_format, :span, :kind, :channels, :sample_unit, :sample_resolution_in_unit, :sample_offset_in_unit, :sample_type, :sample_rate), Tuple{Base.UUID, String, String, TimeSpans.TimeSpan, String, Vector{String}, String, Float64, Float64, String, Float64}}, Tables.RowIterator{Tables.CopiedColumns{Arrow.Table}}})
@ Base ./array.jl:602
[8] rowtable(itr::Arrow.Table)
@ Tables ~/.julia/packages/Tables/YzCZp/src/namedtuples.jl:99
[9] top-level scope
@ REPL[15]:1

(jl_0M34Wq) pkg> st
Status `/tmp/jl_0M34Wq/Project.toml`
[fbe9abb3] AWS v1.41.0
[1c724243] AWSS3 v0.8.3
[e853f5be] Onda v0.13.6
[bd369af6] Tables v1.4.3
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the Julia example with Arrow.Table and Tables.rowtable using the shown schema, especially the Vector{String} channels column. Start at the Tables namedtuples.jl locations in the stack trace and compare with Arrow.jl issue 167. Done means rowtable can process this table without the reported TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.