astronomy-commons / astronomy-commons/lsdb
Association cache
Open
enhancement
performance
- Dominant language
- Python
- Stars
- 55
- Forks
- 26
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 8
Description
Combine the notions of margin cache and association table to effectively co-partition the right side of a join with the left side, caching the rows that exist in neighboring partitions. This should reduce the total amount of data required to load joins at pipeline runtime.
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by locating the existing margin-cache and association-table implementations, then trace how joins load neighboring partitions; done means the two notions are combined and runtime joins require less data to load.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100