write fails on large dataframes
- Dominant language
- Julia
- Stars
- 312
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
hi :)
as indicated in the Tables.jl docs, for large dataframes (>10k columns), Tables.Schema stores its values in
.storenames and .storedtypes instead of the type parameters. This needs to be taken into account in,
eg. makeschema in write.jl and possibly other places.
EDIT: I just added if isnothing(names) names_ = sch.names else names_ = names end
(and replace names with names_ below) in my checkout and that fixed writing for me.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with makeschema in write.jl and the Tables.jl documentation on Tables.Schema for large dataframes. Check how .storenames and .storedtypes are handled when schema values are not in the type parameters, then verify that writing dataframes with more than 10k columns succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100