MeteoSwiss / MeteoSwiss/ampycloud

Could polars help speed up ampycloud even more ?

Open
#96 1 comment 0 reactions 0 assignees View on GitHub
enhancement help !!!
Dominant language
Python
Stars
6
Forks
2
Avg merge
4d 10h
Merged PRs (30d)
4

Description

**Describe the change**
There's a new DataFrame package called [polars](https://www.pola.rs/).

This looks like it could provide a significant speed gain over pandas, currently used by ampycloud. Speed is not an issue today with ampycloud, but any gain would be good to take.

@loforest: making an upgrade from pandas to polars would require some deep surgery ... but it could also be the perfect excuse to learn the code in all the details, if someone wanted to take ownership when I'll move to the great beyond (MCH).

I note that there may be good reasons why polars cannot work with ampycloud, but I have not checked in details yet.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating the pandas usage in ampycloud, then assess whether polars supports the same operations and measure any meaningful speed gain; done requires a compatibility and performance decision with the migration scope documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.