apache / apache/arrow-julia

Custom type cannot round trip (Colors.jl)

Open
#484 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

```julia
julia> using Colors, Arrow

julia> colors = fill(colorant"red", 10)
10-element Array{RGB{N0f8},1} with eltype RGB{FixedPointNumbers.N0f8}:
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)
RGB{N0f8}(1.0,0.0,0.0)

julia> ArrowTypes.JuliaType(::Val{:Color}) = RGB{Colors.N0f8}

julia> ArrowTypes.arrowname(::Type{RGB{Colors.N0f8}}) = :Color

julia> df = (c = colors,);

julia> a = tempname();

julia> Arrow.write(a, df)
"/tmp/jl_kVGtyjAbml"

julia> Arrow.Table(a)
Arrow.Table with 10 rows, 1 columns, and schema:
:c RGB{FixedPointNumbers.N0f8}

julia> Arrow.Table(a).c
10-element Arrow.Struct{RGB{FixedPointNumbers.N0f8}, Tuple{Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}, Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}, Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}}}:
Error showing value of type Arrow.Struct{RGB{FixedPointNumbers.N0f8}, Tuple{Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}, Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}, Arrow.Struct{@NamedTuple{i::UInt8}, Tuple{Arrow.Primitive{UInt8, Vector{UInt8}}}}}}:
ERROR: MethodError: no method matching RGB{FixedPointNumbers.N0f8}(::@NamedTuple{i::UInt8}, ::@NamedTuple{i::UInt8}, ::@NamedTuple{i::UInt8})

Closest candidates are:
RGB{T}(::T, ::T, ::T) where T
@ ColorTypes ~/.julia/packages/ColorTypes/1dGw6/src/types.jl:97
RGB{T}(::Any) where T
@ ColorTypes ~/.julia/packages/ColorTypes/1dGw6/src/types.jl:465

Stacktrace:
[1] fromarrow(::Type{RGB{FixedPointNumbers.N0f8}}, ::@NamedTuple{i::UInt8}, ::@NamedTuple{i::UInt8}, ::@NamedTuple{i::UInt8})
@ ArrowTypes ~/.julia/packages/ArrowTypes/Nb4EC/src/ArrowTypes.jl:168
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the Colors.jl example using Arrow.write and Arrow.Table, then inspect the ArrowTypes.fromarrow method at the stack-trace location in ArrowTypes.jl:168. The fix is complete when the RGB{Colors.N0f8} column can be written and read back without the constructor MethodError.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.