Incorrect histogramdd behavior with masked arrays
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 32.8k
- Forks
- 12.8k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 197
Description
Summing over the histogram should always give the number of non-masked records when no weighting is used. As shown below, the sum is reduced, but not by the correct amount. Calling the "compressed()" method and reshaping before running the histogram produces correct results.
The following uses numpy 1.8.1.
x = ma.arange(300).reshape(100,3)
In [5]: print x[:,0].count()
100
In [6]: H,e = np.histogramdd(x)
In [7]: H.sum()
Out[7]: 100.0
In [8]: for i in range(10) :
...: x[i*i,:]=ma.masked
...:
In [9]: print x[:,0].count()
90
In [10]: H,e = np.histogramdd(x)
In [11]: H.sum()
Out[11]: 98.0
x = x.compressed().reshape(90,3)
H,e = np.histogramdd(x)
H.sum()
Out[14]: 90.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the masked-array example with numpy.histogramdd and compare its result with x.compressed().reshape(90,3). Check how masked values are handled during histogramdd and review the existing histogramdd tests, if available. Done means an unweighted histogram sums to the number of non-masked records.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100