Precompilation / latency?
- Dominant language
- Julia
- Stars
- 312
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
With the release 1.6, I was reading a bit about precompilation. I noticed that Arrow package has quite high latency for the first time a file / message is read. I wonder if it would be practical to generate precompile statements for the most common column types - e.g. Int/Float primitives, Strings, Dates/DateTimes + batched/dictionary encoded versions of these. Perhaps the number of methods is quite large, but it could be interesting.
Contributor guide
No contributing guide indexed for this repository
Research direction
No source files, tests, or entry points are named. Reproduce the first-read latency described for Arrow 1.6, measure it across the listed primitive and encoded column types, and clarify the desired precompilation scope before implementation; completion criteria are not defined in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100