google-deepmind / google-deepmind/acme
Questions re: Custom Distributional Agents in ACME?
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 553
- PR merge metrics
- No merged PRs in 30d
Description
I have four questions towards creating custom distributional RL algorithms using Acme.
1. The only distributional agent I can find (in TF and jax) is D4PG. Is this correct?
2. When will other distributional agents (e.g. C51, Quantile Regression, Expectile Regression) be released?
3. If I want to implement my own distributional agent, I glanced at `d4pg.D4PGLearner` to get a sense of how this might be done. I'm a bit puzzled by a few things. Coming from PyTorch, I'm used to optimizers stepping. It looks like here, the Learner itself steps. Is this correct? What was the reason for this implementation choice?
4. If I want to implement expectile regression for discrete control, what would the recommended approach be? I imagine I'd need to start with DQN, change the output dimension and change the loss function, but how do I specify which output element (i.e. the output units corresponding to tau = 0.5) is used for control?
Contributor guide
Assessment
This issue has not been assessed yet.