tensorflow / tensorflow/model-optimization

sparsity.prune_low_magnitude fails with mixed precision policy mixed_float16

Open
#409 6 comments 0 reactions 1 assignee View on GitHub

@liyunlu0618 is already working on this.

Since Apr 16, 2021.

feature request technique:pruning
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Describe the bug
When using tf.keras.mixed_precision.experimental.Policy("mixed_float16", loss_scale="dynamic") the sparsity.prune_low_magnitude fails in tensor type conversion with the error Tensor conversion requested dtype float32 for Tensor with dtype float16: <tf.Tensor 'pruning_ops/Cast_2:0' shape=(3, 3, 1, 12) dtype=float16>. Things work perfectly fine when precision is set to the default float32. Looks like some piece of code is not properly respecting the dtype.

System information

TensorFlow installed from (source or binary): pip3

TensorFlow version: 2.2.0

TensorFlow Model Optimization version:

Python version: 3.6

Describe the expected behavior

The prune_low_magnitude should work with layers using mixed_float16 policy.

Describe the current behavior

Throws error described above.

Code to reproduce the issue
See this colab link.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.