[BUG] SACTrainer and CQLTrainer unnecessarily query collector environments
@aswanth-07 is already working on this.
Since Sep 14, 2026.
- Dominant language
- Python
- Stars
- 3.6k
- Forks
- 487
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 207
Description
Problem
SACTrainer and CQLTrainer read full_action_spec_unbatched from the collector whenever the loss has no explicit action_spec. That lookup is unnecessary when a numeric target_entropy already determines the entropy target or when the actor already carries the action spec.
Collectors that do not expose getattr_env therefore cannot construct either trainer even though the loss has all information it needs.
This is the SAC/CQL follow-up requested in the review of #4368.
Reproduction
A loss with an explicit target works independently:
loss = SACLoss(actor, qvalue, target_entropy=0.0)
assert loss.target_entropy == 0.0
Passing the same loss to SACTrainer with a collector that implements the ordinary trainer collector methods but has no getattr_env currently fails during construction:
AttributeError: 'Collector' object has no attribute 'getattr_env'
CQLTrainer fails at the equivalent lookup. The same failure occurs with target_entropy="auto" when the actor already has a valid action spec.
Expected behavior
Both trainers should use an already-resolved numeric target or the actor's action spec without querying the collector environment. The existing collector lookup should remain available when SAC uses automatic target entropy and neither the loss nor actor has an action spec.
Proposed scope
- Restrict the collector action-spec lookup to unresolved automatic target entropy.
- Cover numeric and actor-spec resolution for both trainers through their public constructors.
- Preserve and test SAC's collector fallback.
Checklist
- I checked open and closed issues and pull requests for overlapping SAC/CQL trainer, target entropy, action-spec, and
getattr_envchanges.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.