google-deepmind / google-deepmind/rlax

Questions re: RLax Value Learning ?

Open
#9 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
109
Avg merge
2h 15m
Merged PRs (30d)
1

Description

Hi! I have several questions/requests regarding value learning https://github.com/deepmind/rlax/blob/master/rlax/_src/value_learning.py

1. If I want to use the `_quantile_regression_loss` without the Huber aspect, does setting `huber_param` equal to `0` accomplish this? That's my understanding, but I'd like to check :)

2. I'm interested in exploring expectile regression-naive DQN and expectile regression DQN, but code for these two related algorithms don't seem to exist. Is that correct? If code does exist, could you point me in the right direction?

3. If functions for expectile regression indeed do not exist, what would be the most straightforward way to implement them? If I just want expectile regression-naive, I'm thinking I would need to do the following:

a. Copy `_quantile_regression_loss()` to create `_expectile_regression_loss()`, replacing the quantile loss with expectile loss
b. Copy `quantile_q_learning()` to create `expectile_q_learning`, replacing the `_quantile_regression_loss()` call with a `_expectile_regression_loss()` call

Is this correct? If so, would you be open to PRs?

4. Expectile regression is a little trickier, due to its imputation strategy. Are you planning on implementing & releasing that? If not, how would you recommend implementing that?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.