ACCESS_VIOLATION reading Arrow record batch
- Dominant language
- Julia
- Stars
- 312
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
I get this access violation when I read the attached Arrow file. I suspect there is an empty record batch in the Arrow file:
```
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x5fb26056 -- uncompress at .\boot.jl:0
in expression starting at E:\Source\PIPCore.jl\src\Examples\browsearrow.jl:11
uncompress at .\boot.jl:0
buildbitmap at C:\Users\kobus.herbst\.julia\packages\Arrow\PQ5Mm\src\table.jl:416
build at C:\Users\kobus.herbst\.julia\packages\Arrow\PQ5Mm\src\table.jl:566
unknown function (ip: 000000005fb28cc4)
build at C:\Users\kobus.herbst\.julia\packages\Arrow\PQ5Mm\src\table.jl:402
unknown function (ip: 000000005fb28158)
iterate at C:\Users\kobus.herbst\.julia\packages\Arrow\PQ5Mm\src\table.jl:378
unknown function (ip: 000000005fb21d3d)
copyto! at .\abstractarray.jl:847
_collect at .\array.jl:608 [inlined]
collect at .\array.jl:602 [inlined]
macro expansion at C:\Users\kobus.herbst\.julia\packages\Arrow\PQ5Mm\src\table.jl:288 [inlined]
#82 at .\threadingconstructs.jl:169
unknown function (ip: 000000005fb22363)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:839
Allocations: 27852364 (Pool: 27841941; Big: 10423); GC: 29
```
This is the code:
```
using Arrow
using DataFrames
using TableBrowse
function browsearrow(path::String, file::String)
df = Arrow.Table(joinpath(path,"$(file).arrow")) |> DataFrame
#TableBrowse.browse(df)
println("Rows $(nrow(df))")
end
browsearrow("D:\\Data\\Demography\\AHRI\\DayExtraction","IndividualResidencyDays_batched")
```
[IndividualResidencyDays_batched.zip](https://github.com/JuliaData/Arrow.jl/files/7193829/IndividualResidencyDays_batched.zip)
The Arrow is in the zip archive
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the crash with Arrow.Table(... ) |> DataFrame using the attached IndividualResidencyDays_batched.arrow file. Start by inspecting src/table.jl around buildbitmap (line 416), build (lines 402 and 566), and iterate (line 378). Done means the file can be read without an access violation and the row count is printed; add coverage for the suspected empty record batch if the cause is confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100