tensorflow / tensorflow/probability

Tanh bijector's fldj/ildj don't preserve dtype when numpy backend is used

Open
#861 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

TensorFlow version: 2.1.0, probability: 0.9.0
Nightly versions expose the same behaviour (TF: 2.2.0-dev20200402, TFP: 0.10.0-dev20200402)

In scope of https://github.com/tensorflow/probability/pull/800 I've added sinh bijector based on tanh one and it resulted in test failures in JohnsonSU distribution test added in the same PR. Failures are in dtype checking of dist methods. It happens with numpy backend only.

When bijector is implemented as inline one, ildj starts returning expected type.

Code to reproduce:

import numpy as np
import tensorflow as tf
import tensorflow_probability as tfp

tfp = tfp.experimental.substrates.numpy

tanh_bijector = tfp.bijectors.tanh.Tanh()

inline_bijector = tfp.bijectors.inline.Inline(
  forward_fn=tf.tanh,
  inverse_fn=tf.atanh,
  forward_log_det_jacobian_fn=lambda x: 2. * (np.log(2.) - x - tf.math.softplus(-2. * x)),
  forward_min_event_ndims=0,
  is_increasing=lambda: True,
  validate_args=True,
  name='tanh'
)

x = np.float32(0.5)
print(f'x={x}, x.dtype={x.dtype}')

for name, bijector in zip(('tanh', 'inline'), (tanh_bijector, inline_bijector)):
    y = bijector.forward(x)
    print(f'{name}: y={y}, y.dtype={y.dtype}')
    y = bijector.forward_log_det_jacobian(x, 0)
    print(f'{name}: y={y}, y.dtype={y.dtype}')
    y = bijector.inverse_log_det_jacobian(x, 0)
    print(f'{name}: y={y}, y.dtype={y.dtype}')

Output:

x=0.5, x.dtype=float32
tanh: y=0.46211716532707214, y.dtype=float32
tanh: y=-0.24022901391655516, y.dtype=float64
tanh: y=0.28768208236882487, y.dtype=float64
inline: y=0.46211716532707214, y.dtype=<dtype: 'float32'>
inline: y=-0.24022901391655516, y.dtype=float64
inline: y=0.2876819968223572, y.dtype=float32

Output if tfp = tfp.experimental.substrates.numpy is commented:

x=0.5, x.dtype=float32
tanh: y=0.46211716532707214, y.dtype=<dtype: 'float32'>
tanh: y=-0.24022895097732544, y.dtype=<dtype: 'float32'>
tanh: y=0.2876819968223572, y.dtype=<dtype: 'float32'>
inline: y=0.46211716532707214, y.dtype=<dtype: 'float32'>
inline: y=-0.24022895097732544, y.dtype=<dtype: 'float32'>
inline: y=0.2876819968223572, y.dtype=<dtype: 'float32'>

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the provided reproduction with the NumPy substrate and inspect the Tanh bijector entry point, focusing on forward_log_det_jacobian and inverse_log_det_jacobian. Compare the JohnsonSU distribution dtype-checking test behavior with the inline bijector; done means Tanh's log-Jacobian methods preserve the input dtype under the NumPy backend.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.