NCAS-CMS / NCAS-CMS/PyActiveStorage
Multiple operations instead of means
@bnlawrence is already working on this.
Since Oct 27, 2022.
- Dominant language
- Python
- Stars
- 6
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
In working through the implications of implementing means in chunks, it is notable that once missing data is in play, we need to return two numbers from the reduce_chunk method: the sum, and the count, because means over chunks will be needed to be weighted by the actual number of values being meaned.
There are a number of ways we could implement this:
- Always return
(X, N), whereXis the expected operation, andNthe number of values contributing - Only return
(X, N)when required (e.g. for means) otherwise return(X,None)or(X,) - Return
X, except when it needs to be(X,N) - Something else.
The something else option could be slightly more interesting: do we think it's a smart idea to say we could chain a series of methods and expect a series of results, in a lightweight sort of caching?
Obvious use cases would be:
- mean = sum, count
- range = min, max
- sqmean = sum(squares), sum, count
This could be facilitated by handing not just "a method" but a list of 1.. many methods, and expect back a list of 1..many results.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.