google-deepmind / google-deepmind/rlax
Questions re: RLax Value Learning ?
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 109
- Avg merge
- 2h 15m
- Merged PRs (30d)
- 1
Description
Hi! I have several questions/requests regarding value learning https://github.com/deepmind/rlax/blob/master/rlax/_src/value_learning.py
1. If I want to use the `_quantile_regression_loss` without the Huber aspect, does setting `huber_param` equal to `0` accomplish this? That's my understanding, but I'd like to check :)
2. I'm interested in exploring expectile regression-naive DQN and expectile regression DQN, but code for these two related algorithms don't seem to exist. Is that correct? If code does exist, could you point me in the right direction?
3. If functions for expectile regression indeed do not exist, what would be the most straightforward way to implement them? If I just want expectile regression-naive, I'm thinking I would need to do the following:
a. Copy `_quantile_regression_loss()` to create `_expectile_regression_loss()`, replacing the quantile loss with expectile loss
b. Copy `quantile_q_learning()` to create `expectile_q_learning`, replacing the `_quantile_regression_loss()` call with a `_expectile_regression_loss()` call
Is this correct? If so, would you be open to PRs?
4. Expectile regression is a little trickier, due to its imputation strategy. Are you planning on implementing & releasing that? If not, how would you recommend implementing that?
Contributor guide
Assessment
This issue has not been assessed yet.