NCAS-CMS / NCAS-CMS/cf-python

Potentially slow iteration for setting condition value units

Open
#511 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dask performance
Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

As noted by @davidhassell in the review thread https://github.com/NCAS-CMS/cf-python/pull/427#discussion_r1040910208, the crux being:

if a large set is stored as a Data object, as opposed to a python built-in like a list, then iterating through it could be slow

relating to the code (currently on the lama-to-dask branch only):

https://github.com/NCAS-CMS/cf-python/blob/f17e08ee69b271c3b595131ef93d25f4bb3c2395/cf/query.py#L890-L896

with a suggested solution of:

For the Data object case we could just change the units without iterating through.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cf/query.py lines 890-896 and the related discussion in pull request 427, focusing on how condition values represented by Data objects are handled. Confirm the behavior on the lama-to-dask branch and preserve the existing unit-setting result while avoiding iteration over large Data objects.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.