Use of LazyFrames for Polars implementation
- Dominant language
- Python
- Stars
- 42
- Forks
- 18
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
Polars has a lazy API, which allows for query optimization. Currently, AgentSetPolars and PolarsMixins are implemented with the eager API, which may lead to a significant performance loss. Collection could be done at each step.
References:
- https://docs.pola.rs/user-guide/concepts/lazy-vs-eager/
- https://stackoverflow.com/questions/76612163/what-are-the-advantages-of-a-polars-lazyframe-over-a-dataframe
Contributor guide
Research direction
Start with AgentSetPolars and PolarsMixins, then read the linked Polars lazy-versus-eager API reference to understand the intended query optimization. Compare the current eager operations with the proposed lazy workflow and review pull request #144 before starting. Done means the implementation uses LazyFrames where appropriate without changing its existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100