inefficient data fetching due to uncached time extraction
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 498
- Forks
- 178
- PR merge metrics
- No merged PRs in 30d
Description
https://twitter.com/JaredALee/status/1263311504469889031
A friend using this package to extract data across many wrf files was encountering huge slowdowns and found huge speedups switching to xarray. I dug a bit and I have a suspicion where the inefficiency is. There are two things about the linked line that concerns me. First, It is called before a possible short-circuit path that doesn't use its result, so perhaps this line can get moved to a few lines down? Second, I notice that this function call doesn't use the passed in _key. I don't fully understand the cache mechanism, but I wonder if this would benefit from it?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/wrf/util.py line 1609 and trace the surrounding short-circuit path and cache mechanism, including how the passed _key is used elsewhere. Compare the current data-fetching behavior across many WRF files, then define done as confirming and addressing the suspected uncached time extraction without changing paths that do not need its result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100