tensorflow / tensorflow/probability

bijector_fn argument does not work for bijector examples

Open
#839 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
No merged PRs in 30d

Description

print('tensorflow: ', tf.__version__)
print('tensorflow-probability: ', tfp.__version__)

tensorflow:  2.1.0. 
tensorflow-probability:  0.9.0

I want to make a flow with an autoregressive network. I understand from #448 #683 that I would have to pass the values of shift and log_scale to the bijector_fn, and use tfb.Shift and tfb.Scale as suggested because tfb.AffineScalar is deprecated. This is they way I though it works (but clearly does not)...

NN = tfb.AutoregressiveNetwork(
    params=2,
    event_shape=[10],
    hidden_units=[20,20],
)

shift, log_scale = tf.unstack(NN(tf.random.normal((30,10))), axis=-1)
bij = tfb.Shift(shift)(tfb.Scale(log_scale))

flow = tfb.MaskedAutoregressiveFlow(
    bijector_fn=bij,
    validate_args = True,
)
Error --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in 12 validate_args = True, 13 ) ---> 14 flow(tf.random.normal((30,10)))

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in call(self, value, name, **kwargs)
863 return chain.Chain([self, value], name=name, **kwargs)
864
--> 865 return self.forward(value, name=name or 'forward', **kwargs)
866
867 def _forward_event_shape_tensor(self, input_shape):

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in forward(self, x, name, **kwargs)
1001 NotImplementedError: if _forward is not implemented.
1002 """
-> 1003 return self._call_forward(x, name, **kwargs)
1004
1005 @classmethod

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in _call_forward(self, x, name, **kwargs)
975 if mapping.y is not None:
976 return mapping.y
--> 977 mapping = mapping.merge(y=self._forward(x, **kwargs))
978 # It's most important to cache the y->x mapping, because computing
979 # inverse(forward(y)) may be numerically unstable / lossy. Caching the

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/masked_autoregressive.py in _forward(self, x, **kwargs)
363 body=_loop_body,
364 loop_vars=(0, y0),
--> 365 maximum_iterations=static_event_size)
366 return y
367

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop_v2(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, maximum_iterations, name)
2476 name=name,
2477 maximum_iterations=maximum_iterations,
-> 2478 return_same_structure=True)
2479
2480

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name, maximum_iterations, return_same_structure)
2712 list(loop_vars))
2713 while cond(*loop_vars):
-> 2714 loop_vars = body(*loop_vars)
2715 if try_to_pack and not isinstance(loop_vars, (list, _basetuple)):
2716 packed = True

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in (i, lv)
2703 cond = lambda i, lv: ( # pylint: disable=g-long-lambda
2704 math_ops.logical_and(i < maximum_iterations, orig_cond(*lv)))
-> 2705 body = lambda i, lv: (i + 1, orig_body(*lv))
2706 try_to_pack = False
2707

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/masked_autoregressive.py in _loop_body(index, y0)
352 if vs.caching_device is None and not tf.executing_eagerly():
353 vs.set_caching_device(lambda op: op.device)
--> 354 bijector = self._bijector_fn(y0, **kwargs)
355 y = bijector.forward(x)
356 return index + 1, y

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/masked_autoregressive.py in _wrapper(x, **condition_kwargs)
1101 """A wrapper that validates bijector_fn."""
1102 bijector = bijector_fn(x, **condition_kwargs)
-> 1103 if bijector.forward_min_event_ndims != bijector.inverse_min_event_ndims:
1104 # Current code won't really work with this, but in principle we could
1105 # implement this.

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'forward_min_event_ndims'.

If I don't use the validate_args argument, this is the result:

flow = tfb.MaskedAutoregressiveFlow(
    bijector_fn=bij,
)
flow(tf.random.normal((30,10)))
Error --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in 2 bijector_fn=bij, 3 ) ----> 4 flow(tf.random.normal((30,10)))

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in call(self, value, name, **kwargs)
863 return chain.Chain([self, value], name=name, **kwargs)
864
--> 865 return self.forward(value, name=name or 'forward', **kwargs)
866
867 def _forward_event_shape_tensor(self, input_shape):

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in forward(self, x, name, **kwargs)
1001 NotImplementedError: if _forward is not implemented.
1002 """
-> 1003 return self._call_forward(x, name, **kwargs)
1004
1005 @classmethod

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/bijector.py in _call_forward(self, x, name, **kwargs)
975 if mapping.y is not None:
976 return mapping.y
--> 977 mapping = mapping.merge(y=self._forward(x, **kwargs))
978 # It's most important to cache the y->x mapping, because computing
979 # inverse(forward(y)) may be numerically unstable / lossy. Caching the

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/masked_autoregressive.py in _forward(self, x, **kwargs)
363 body=_loop_body,
364 loop_vars=(0, y0),
--> 365 maximum_iterations=static_event_size)
366 return y
367

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop_v2(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, maximum_iterations, name)
2476 name=name,
2477 maximum_iterations=maximum_iterations,
-> 2478 return_same_structure=True)
2479
2480

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name, maximum_iterations, return_same_structure)
2712 list(loop_vars))
2713 while cond(*loop_vars):
-> 2714 loop_vars = body(*loop_vars)
2715 if try_to_pack and not isinstance(loop_vars, (list, _basetuple)):
2716 packed = True

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_core/python/ops/control_flow_ops.py in (i, lv)
2703 cond = lambda i, lv: ( # pylint: disable=g-long-lambda
2704 math_ops.logical_and(i < maximum_iterations, orig_cond(*lv)))
-> 2705 body = lambda i, lv: (i + 1, orig_body(*lv))
2706 try_to_pack = False
2707

/tungstenfs/groups/gliberal/Users/pallgiov/prg/miniconda3/envs/tfp_scanpy/lib/python3.6/site-packages/tensorflow_probability/python/bijectors/masked_autoregressive.py in _loop_body(index, y0)
353 vs.set_caching_device(lambda op: op.device)
354 bijector = self._bijector_fn(y0, **kwargs)
--> 355 y = bijector.forward(x)
356 return index + 1, y
357 # If the event size is available at graph construction time, we can inform

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'forward'

Thanks a lot for the help,
Giovanni

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the notebook snippet with the reported TensorFlow and TensorFlow Probability versions, starting at tfb.AutoregressiveNetwork and tfb.MaskedAutoregressiveFlow. Check the documented bijector_fn contract and the related discussions in #448 and #683. Done means the bijector example's argument usage is valid and covered by a regression test or clarified example.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.