apache / apache/arrow-julia

Underlying file still referenced even with `copycols=true`

Open
#226 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

Hello,

I am getting errors when trying to delete a file loaded with `DataFrame(Arrow.Table(…))`.
It seems that the underlying file is still referenced even when I do it with `copycols=true`.

```
n = 1
filename = "$(randstring(12)).arrow"
df = DataFrame(x=rand(n))
Arrow.write("$filename", df)
res1 = DataFrame(Arrow.Table("$filename"); copycols=true)
rm(filename)
```

This the output:

```
PS > julia -q --project .\test.jl
ERROR: LoadError: IOError: unlink("iPJQf2OSiQaT.arrow"): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ .\libuv.jl:97 [inlined]
[2] unlink(p::String)
@ Base.Filesystem .\file.jl:934
[3] rm(path::String; force::Bool, recursive::Bool)
@ Base.Filesystem .\file.jl:272
[4] rm(path::String)
@ Base.Filesystem .\file.jl:263
[5] top-level scope
@ D:\source\COMPILETHIS\test.jl:13
in expression starting at test.jl:13
```

This happens with Julia 1.6.1, Arrow v1.5.0, Windows 10

Contributor guide

No contributing guide indexed for this repository

Research direction

Run the supplied Julia reproduction on Windows 10, starting with DataFrame(Arrow.Table("$filename"); copycols=true) and the subsequent rm(filename). Trace how the Arrow.Table input is retained during DataFrame construction; done means the file can be deleted successfully after construction with copycols=true.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.