pytorch / pytorch/rl

[BUG]UserWarning: deterministic_sample wasn't found when queried in <class 'torch.distributions.categorical.Categorical'>. CompositeDistribution is falling back on mode instead. For better code quality and efficiency, make sure to either provide a distribution with a deterministic_sample attribute or to change the InteractionMode to the desired value.

Open
#2,458 0 comments 0 reactions 1 assignee View on GitHub

@vmoens is already working on this.

Since Sep 27, 2024.

bug
Dominant language
Python
Stars
3.6k
Forks
487
Avg merge
1d 1h
Merged PRs (30d)
207

Description

Describe the bug

When I proceeded further based on the issue https://github.com/pytorch/rl/issues/2402, I encountered this warning. I have pulled the GitHub projects torchrl and tensordict, and installed the latest libraries from both projects using python .\setup.py develop. I'm puzzled why it's not possible to perform deterministic_sample using Categorical within CompositeDistribution. Since CompositeDistribution can already compute log_prob, why can't it perform deterministic_sample?

To Reproduce

I have already used the code suggested in the answer of this issue: https://github.com/pytorch/rl/issues/2402

Expected behavior

I hope to eliminate this warning so that log_prob in CompositeDistribution can properly assist in performing deterministic_sample for discrete action spaces. Alternatively, could you tell me whether this warning has a negative impact on the convergence of my PPO model training process or the prediction process?

System info

win11 24h2
python 3.10.14

torch 2.4.0+cu118
torchaudio 2.4.0+cu118
torchrl 0.5.0+ca3a595
torchvision 0.19.0+cu118
tensordict 0.5.0+eba0769

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.