numpy / numpy/numpy

numpy.mean along multiple axis gives wrong result for large arrays

Open
#8,869 16 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

00 - Bug component: numpy._core
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

A mean over an array containing only ones should obviously return only ones. However,

>>> import numpy as np
>>> test_array = np.ones((10000000, 4, 15), dtype = np.float32)
>>> print(test_array.mean(axis=(0,1)))
[ 0.4194304  0.4194304  0.4194304  0.4194304  0.4194304  0.4194304
  0.4194304  0.4194304  0.4194304  0.4194304  0.4194304  0.4194304
  0.4194304  0.4194304  0.4194304]

This returns the correct result:

>>> print(test_array.mean(axis=0).mean(axis=0))
[ 1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.]

I guess the reason for this problem is some overflow, because it does not appear when I use a test array with dtype = np.float64. I would expect numpy to either give the correct result or to at least give a warning whenever such an overflow happens.

surprisingly mean along all axis gives the correct result again:

>>> print(test_array.mean(axis=(0,1,2)))
1.0

(I used numpy-1.12.1 with python 3.5 on Ubuntu 16)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue provides no file or test entry point. Start by reproducing the float32 mean over the supplied large array and compare tuple-axis reduction with sequential reductions; done means the tuple-axis result is correct or the overflow behavior is explicitly addressed by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.