tensorflow / tensorflow/model-optimization
sparsity.prune_low_magnitude fails with mixed precision policy mixed_float16
@liyunlu0618 is already working on this.
Since Apr 16, 2021.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Describe the bug
When using tf.keras.mixed_precision.experimental.Policy("mixed_float16", loss_scale="dynamic") the sparsity.prune_low_magnitude fails in tensor type conversion with the error Tensor conversion requested dtype float32 for Tensor with dtype float16: <tf.Tensor 'pruning_ops/Cast_2:0' shape=(3, 3, 1, 12) dtype=float16>. Things work perfectly fine when precision is set to the default float32. Looks like some piece of code is not properly respecting the dtype.
System information
TensorFlow installed from (source or binary): pip3
TensorFlow version: 2.2.0
TensorFlow Model Optimization version:
Python version: 3.6
Describe the expected behavior
The prune_low_magnitude should work with layers using mixed_float16 policy.
Describe the current behavior
Throws error described above.
Code to reproduce the issue
See this colab link.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.