anthony-wang / anthony-wang/CrabNet
Reason to use pe_scaler and ple_scaler
Open
- Dominant language
- Python
- Stars
- 132
- Forks
- 42
- PR merge metrics
- No merged PRs in 30d
Description
Could you explain the reason why you use the `pe_scaler` and `ple_scaler` in the forward pass of the `Encoder`-class in `kingcrab.py`?
In particular, why do you choose the form
`pe_scaler = 2**(1-self.pos_scaler)**2`
and
`ple_scaler = 2**(1-self.pos_scaler_log)**2`?
I don't really understand why one needs these two scalers (and also `self.emb_scaler`) in the first place and why you chose the above exponential forms for them.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.