tfp.stats.histogram is not consistent in default beh... | tensorflow/probability

Nobody has claimed this yet.

Edit:

Inconsistent in out of bounds handling

Out of Lower Bound throws a Bincount error opposed to a meaningful error
Out of Upper Bound does not fail when data is provided out of bounds, instead ignores OOB values

Suggestion:
Truncate data OOB in the lower bound case, like in the upper bound case

~~This is due to tf.math.bincount not supporting negative values.~~

There's a pretty trivial fix which is just offsetting the value of bins to be non-negative and then them shifting back: https://github.com/tensorflow/probability/blob/b8ac98db180896dd938cf171b7b22a06e21e02ac/tensorflow_probability/python/stats/quantiles.py#L414
~~Before I implement this fix, I just wanted a sanity check, does this sound right?~~

Related: b/124015136

reproducing example:

x = tf.stack([tf.random.normal([1000]), tf.random.normal([1000])],
             axis=-1)

edges = [0., 0.5, 1.0, 1.5, 2.0]

tfp.stats.histogram(x, edges)

Stacktrace

InvalidArgumentError Traceback (most recent call last)
in ()
4 edges = [0., 0.5, 1.0, 1.5, 2.0]
5
----> 6 tfp.stats.histogram(x, edges)
7
8 tfp.stats.histogram(x, edges, axis=0)

5 frames
/usr/local/lib/python3.7/dist-packages/tensorflow_probability/python/stats/quantiles.py in histogram(x, edges, axis, weights, extend_lower_interval, extend_upper_interval, dtype, name)
422 maxlength=tf.shape(edges)[0] - 1,
423 axis=0,
--> 424 dtype=dtype or in_dtype)
425 n_edges = tf.compat.dimension_value(edges.shape[0])
426 if n_edges is not None:

/usr/local/lib/python3.7/dist-packages/tensorflow_probability/python/stats/quantiles.py in count_integers(arr, weights, minlength, maxlength, axis, dtype, name)
104 minlength=minlength,
105 maxlength=maxlength,
--> 106 dtype=dtype)
107
108 # Move dims in ~axis to the left, so we can tf.map_fn bincount over them,

/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/bincount_ops.py in bincount(arr, weights, minlength, maxlength, dtype, name, axis, binary_output)
139 return gen_math_ops.unsorted_segment_sum(weights, arr, output_size)
140 weights = constant_op.constant([], dtype)
--> 141 return gen_math_ops.bincount(arr, output_size, weights)
142
143 if not isinstance(arr, sparse_tensor.SparseTensor):

/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py in bincount(arr, size, weights, name)
1804 return _result
1805 except _core._NotOkStatusException as e:
-> 1806 _ops.raise_from_not_ok_status(e, name)
1807 except _core._FallbackException:
1808 pass

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
6939 message = e.message + (" name: " + name if name is not None else "")
6940 # pylint: disable=protected-access
-> 6941 six.raise_from(core._status_to_exception(e.code, message), None)
6942 # pylint: enable=protected-access
6943

/usr/local/lib/python3.7/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: Input arr must be non-negative! [Op:Bincount]

Start in tensorflow_probability/python/stats/quantiles.py at histogram and reproduce the provided example with out-of-bounds values. Check the existing extend_lower_interval and extend_upper_interval behavior; done means lower- and upper-bound out-of-range data are handled consistently without the cryptic Bincount failure.

Written by the indexing model from the issue text.

tfp.stats.histogram is not consistent in default behavior of out of bounds data and provides cryptic error

Description

Contributor guide

First steps

Research direction

Assessment