Add mask_empty_buckets and mask_allnan_buckets methods for BucketResampler
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 385
- Forks
- 102
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 9
Description
Problem description
After #319 is merged, get_sum will return 0 for empty buckets. Therefore, a method like bucket_resampler.mask_empty_buckets(binned_data, fill_value) is needed to be able to differentiate between empty buckets and buckets with sum 0. This method can leverage self.counts being cached.
Moreover, buckets containing only NaN will also be 0 and have the same problem. Therefore, a method like bucket_resampler.mask_allnan_buckets(binned_data, fill_value) is also needed. Similarly, all-Nan buckets and empty buckets are both NaN in get_average, so the method can be used here as well.
For this to work, a variable like self.nancounts would need to be stored previously in the get_sum method. This is assuming that everything can be done lazily and does not increase the memory consumption. If this is not the case, something like bucket_resampler.mask_allnan_buckets(binned_data, original_data, fill_value) could be implemented.
Expected Output
Being able to differentiate between empty, all-NaN and sum=0 bins.
Actual Result, Traceback if applicable
Not possible at the moment. A workaround using get_count needs to be implemented manually by the user.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the BucketResampler entry points get_sum, get_average, and get_count, focusing on the cached counts and handling of NaN values. Define how mask_empty_buckets and mask_allnan_buckets should distinguish empty, all-NaN, and sum=0 buckets, then verify the methods work for both sum and average results without increasing memory use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100