google-deepmind / google-deepmind/acme

Missleading errors when using discrete action space with `CanonicalSpecWrapper`

Open
#294 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.1k
Forks
553
PR merge metrics
No merged PRs in 30d

Description

# Problem
Using an environment which uses a `DiscreteArray` as action's spec (e.g. gym's `CartPole`) with the `CanonicalSpecWrapper` (which, to the best of my knowledge, should be used with continuous action space only) returns the following error, which is **quite missleading**.

```
Traceback (most recent call last):
File "run_sac.py", line 86, in
app.run(main)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "run_sac.py", line 73, in main
config = build_experiment_config()
File "run_sac.py", line 47, in build_experiment_config
environment_spec = specs.make_environment_spec(environment)
File "/usr/local/lib/python3.8/dist-packages/acme/specs.py", line 46, in make_environment_spec
actions=environment.action_spec(),
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/single_precision.py", line 42, in action_spec
return _convert_spec(self._environment.action_spec())
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 54, in action_spec
return _convert_spec(self._environment.action_spec())
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 68, in _convert_spec
return tree.map_structure(_convert_single_spec, nested_spec)
File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in map_structure
[func(*args) for args in zip(*map(flatten, structures))])
File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in
[func(*args) for args in zip(*map(flatten, structures))])
File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 63, in _convert_single_spec
return spec.replace(
File "/usr/local/lib/python3.8/dist-packages/dm_env/specs.py", line 146, in replace
return type(self)(**all_kwargs)
TypeError: __init__() got an unexpected keyword argument 'minimum'

```

# Expected behavior
Do not raise an error.

# Why is it important to solve ?
- I must admit that I have lost a bit of time (as newcomer to RL) to figure out the problem before finding this closed issue (#264).
- The closed issue also shows that I am not the only one who faced this issue.
- This change is pretty inexpensive.

# Solution
The following change should do the work, from:
https://github.com/deepmind/acme/blob/d1e69c92000079b118b868ce9303ee6d39c4a0b6/acme/wrappers/canonical_spec.py#L62

To:
```python
if type(spec) == specs.BoundedArray:
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.