Score normalization in ReDo
- Dominant language
- Jupyter Notebook
- Stars
- 10.9k
- Forks
- 1.4k
- PR merge metrics
- No merged PRs in 30d
Description
I've been following the ReDo paper ("The Dormant Neuron Phenomenon in Deep Reinforcement Learning", https://arxiv.org/pdf/2302.12902.pdf) which describes the procedure to determine the dormant neurons in terms of their normalized scores. Attached the snapshot below:

As far as I can tell, this line implements equation (1) to compute the scores:
https://github.com/google/dopamine/blob/a6f414ca01a81e933359a4922965178a40e0f38a/dopamine/labs/redo/weight_recyclers.py#L314
However, my question is that while this code appears to implement the identical mathematical equation, it doesn't seem to match the text description afterwards, which says "We normalize the scores such that they sum to 1 within a layer". In order to implement this logic, shouldn't this line be:
```
score /= jnp.sum(score) + 1e-9
```
instead? I'm not sure how the scores sum to 1 after the normalization scheme as implemented by the original equation and the code.
Thank you!
Contributor guide
Assessment
This issue has not been assessed yet.