google-deepmind / google-deepmind/acme

Questions re: Custom Distributional Agents in ACME?

Open
#58 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.1k
Forks
553
PR merge metrics
No merged PRs in 30d

Description

I have four questions towards creating custom distributional RL algorithms using Acme.

1. The only distributional agent I can find (in TF and jax) is D4PG. Is this correct?

2. When will other distributional agents (e.g. C51, Quantile Regression, Expectile Regression) be released?

3. If I want to implement my own distributional agent, I glanced at `d4pg.D4PGLearner` to get a sense of how this might be done. I'm a bit puzzled by a few things. Coming from PyTorch, I'm used to optimizers stepping. It looks like here, the Learner itself steps. Is this correct? What was the reason for this implementation choice?

4. If I want to implement expectile regression for discrete control, what would the recommended approach be? I imagine I'd need to start with DQN, change the output dimension and change the loss function, but how do I specify which output element (i.e. the output units corresponding to tau = 0.5) is used for control?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.