apache / apache/arrow-julia

DictEncode a DictEncode

Open
#331 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

DictEncode signals that a column/array should be dictionary encoded when serialized to the arrow streaming/file format.

The current constructor will happily wrap a DictEncode in another DictEncode.

https://github.com/apache/arrow-julia/blob/v2.2.1/src/arraytypes/dictencoding.jl#L69

Does it make sense to add a no-op constructor `DictEncode(x::DictEncode) = x`?
Right now `Arrow.write` fails to if I run this line twice. I know it's not good practice and it's no big deal, but wanted to bring it up for discussion anyway.

```julia
df.col= Arrow.DictEncode(df.col)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at src/arraytypes/dictencoding.jl around line 69 and inspect the DictEncode constructor. Reproduce the issue by applying Arrow.DictEncode twice to df.col and then calling Arrow.write; done means repeated wrapping no longer causes Arrow.write to fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.