queryverse / queryverse/Query.jl
Support implicit multi-threading?
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 403
- Forks
- 48
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 6
Description
Since Query.jl is already not iterating but using getindex, is it possible to support multi-threading? Either baked in or via some externally passed in "executor" style like FLoops.jl
As an example to Query-style thingy with multi-threading: https://root.cern/doc/master/classROOT_1_1RDataFrame.html
In Julia we can match and surpass their speed: https://github.com/Moelf/UnROOT_RDataFrame_MiniBenchmark
And using Query.jl to mimic the style is almost overhead-free: https://github.com/Moelf/UnROOT_RDataFrame_MiniBenchmark/issues/1#issuecomment-914232278
So I'm wondering if there's a chance we also get multi-threading in the last quadrant of the thred-query 2x2 matrix
| Single thread | Multi-threading | |
| For-loop | ✅ | ✅ |
| Query-style | ✅ | ❓ |
（check means Julia performance match or faster than C++)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Query.jl's getindex-based execution model and the linked FLoops.jl and ROOT RDataFrame examples. Compare the existing single-threaded Query-style path with the benchmark references and define how the missing Query-style multi-threading quadrant should work; done means a validated implementation with performance results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100