google-deepmind / google-deepmind/acme

The program always executes behavior clone when running CQLLearner.

Open
#305 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
4.1k
Forks
553
PR merge metrics
No merged PRs in 30d

Description

When I run the cql algorithm, I found the algorithm only execute behavior clone. I checked the config used. The training step is 100 and the 'num_bc_iters' is set to 50.
When I further dive to the source code of CQLLearner, I found the 'counts' in function 'step' has two keys "steps" and "walltime".
![image](https://github.com/deepmind/acme/assets/34134495/fcb4b72d-bcb8-4c42-ae1d-fbb966ddac0f)
However, in the inplementation of 'step', the key used is "learner_steps".
![image](https://github.com/deepmind/acme/assets/34134495/831672fb-2f5e-4b6e-a2f8-5a65873c404c)
The invalid key "learner_steps" makes the "cur_step" always be 0, thus causing the algorithm only execute behavior clone.
When I correct the key "learner_steps" to "steps", the problem is solved.
![image](https://github.com/deepmind/acme/assets/34134495/58bcc3e6-b8aa-4033-b089-c6d8eecfdd16)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.