google-deepmind / google-deepmind/acme
Missleading errors when using discrete action space with `CanonicalSpecWrapper`
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 553
- PR merge metrics
- No merged PRs in 30d
Description
# Problem
Using an environment which uses a `DiscreteArray` as action's spec (e.g. gym's `CartPole`) with the `CanonicalSpecWrapper` (which, to the best of my knowledge, should be used with continuous action space only) returns the following error, which is **quite missleading**.
```
Traceback (most recent call last):
File "run_sac.py", line 86, in
app.run(main)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "run_sac.py", line 73, in main
config = build_experiment_config()
File "run_sac.py", line 47, in build_experiment_config
environment_spec = specs.make_environment_spec(environment)
File "/usr/local/lib/python3.8/dist-packages/acme/specs.py", line 46, in make_environment_spec
actions=environment.action_spec(),
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/single_precision.py", line 42, in action_spec
return _convert_spec(self._environment.action_spec())
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 54, in action_spec
return _convert_spec(self._environment.action_spec())
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 68, in _convert_spec
return tree.map_structure(_convert_single_spec, nested_spec)
File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in map_structure
[func(*args) for args in zip(*map(flatten, structures))])
File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in
[func(*args) for args in zip(*map(flatten, structures))])
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 63, in _convert_single_spec
return spec.replace(
File "/usr/local/lib/python3.8/dist-packages/dm_env/specs.py", line 146, in replace
return type(self)(**all_kwargs)
TypeError: __init__() got an unexpected keyword argument 'minimum'
```
# Expected behavior
Do not raise an error.
# Why is it important to solve ?
- I must admit that I have lost a bit of time (as newcomer to RL) to figure out the problem before finding this closed issue (#264).
- The closed issue also shows that I am not the only one who faced this issue.
- This change is pretty inexpensive.
# Solution
The following change should do the work, from:
https://github.com/deepmind/acme/blob/d1e69c92000079b118b868ce9303ee6d39c4a0b6/acme/wrappers/canonical_spec.py#L62
To:
```python
if type(spec) == specs.BoundedArray:
```
Contributor guide
Assessment
This issue has not been assessed yet.