apache / apache/arrow-julia

Segfault when accessing chunked data with out of bounds range

Open
#262 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

Indexing an Arrow file that was written in multiple chunks with an out of bounds range sometimes produces a segfault.

Tested on Arrow 2.2.0 and Julia 1.6.3.

MWE:
```
using Arrow

data = (; x=1:1000)
Arrow.write("test.arrow", data; file=false)
Arrow.append("test.arrow", data)

Arrow.Table("test.arrow").x[3000] # errors as expected with BoundsError
Arrow.Table("test.arrow").x[1:3000] # may segfault, but may also throw an UndefRefError
```

Here is the full error message:
```
signal (11): Segmentation fault
in expression starting at REPL[17]:1
getindex at ./array.jl:801 [inlined]
getindex at /home/m/.julia/packages/Arrow/x6smw/src/arraytypes/primitive.jl:48 [inlined]
getindex at /home/m/.julia/packages/SentinelArrays/iHRpO/src/chainedvector.jl:178
unknown function (ip: 0x7f40d324edcc)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:115
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:204
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:155 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:561
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:669jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:877
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:825
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:825
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:929
eval at ./boot.jl:360 [inlined]
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:139
repl_backend_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:200
start_repl_backend at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:185
#run_repl#42 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:317
run_repl at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:305
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
#874 at ./client.jl:387
jfptr_YY.874_46600.clone_1 at /home/m/programs/julia-1.6.0/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
jl_f__call_latest at /buildworker/worker/package_linux64/build/src/builtins.c:714
#invokelatest#2 at ./essentials.jl:708 [inlined]
invokelatest at ./essentials.jl:706 [inlined]
run_main_repl at ./client.jl:372
exec_options at ./client.jl:302
_start at ./client.jl:485
jfptr__start_41020.clone_1 at /home/m/programs/julia-1.6.0/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
true_main at /buildworker/worker/package_linux64/build/src/jlapi.c:560
repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:702
main at julia-stable (unknown line)
__libc_start_main at /usr/lib/libc.so.6 (unknown line)
unknown function (ip: 0x4007d8)
Allocations: 71857055 (Pool: 71829980; Big: 27075); GC: 50
```

Let me know if you need any other information.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the MWE with a multi-chunk Arrow file and an out-of-bounds range. Inspect src/arraytypes/primitive.jl and the chained-vector getindex path shown in the stack trace. Done means the range fails safely with a Julia bounds-related error rather than a segfault or UndefRefError.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.