google / google/dopamine

Score normalization in ReDo

Open
#209 2 comments 5 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

I've been following the ReDo paper ("The Dormant Neuron Phenomenon in Deep Reinforcement Learning", https://arxiv.org/pdf/2302.12902.pdf) which describes the procedure to determine the dormant neurons in terms of their normalized scores. Attached the snapshot below:
![image](https://github.com/google/dopamine/assets/14655667/0d0aac15-4997-4ddd-a07b-3fb8186f9452)

As far as I can tell, this line implements equation (1) to compute the scores:
https://github.com/google/dopamine/blob/a6f414ca01a81e933359a4922965178a40e0f38a/dopamine/labs/redo/weight_recyclers.py#L314
However, my question is that while this code appears to implement the identical mathematical equation, it doesn't seem to match the text description afterwards, which says "We normalize the scores such that they sum to 1 within a layer". In order to implement this logic, shouldn't this line be:
```
score /= jnp.sum(score) + 1e-9
```
instead? I'm not sure how the scores sum to 1 after the normalization scheme as implemented by the original equation and the code.

Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.