DLR-RM / DLR-RM/stable-baselines3
[Feature Request] Add support for gymnasium.spaces.Text
- Dominant language
- Python
- Stars
- 13.8k
- Forks
- 2.2k
- Avg merge
- 1h 35m
- Merged PRs (30d)
- 2
Description
### 🚀 Feature
Unless I'm mistaken, stable-baselines3 only supports Box, Discrete, MultiDiscrete, MultiBinary and Dict spaces from gymnasium.
It seems like a new _fundamental space_ has been introduced in Gymnasium ( [gymnasium.spaces.Text](https://gymnasium.farama.org/api/spaces/fundamental/#gymnasium.spaces.Text) ), which seems to be a more natural way to deal with text data.
### Motivation
This new support would ease the training of LLMs with RL.
### Pitch
_No response_
### Alternatives
One alternative is to use gymnasium.spaces.Box(low=0, high=vocab_size-1, dtype=np.int).
However this requires padding to support smaller texts.
### Additional context
_No response_
### Checklist
- [X] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [X] If I'm requesting a new feature, I have proposed alternatives
Contributor guide
Research direction
Start with the Gymnasium Text space documentation linked in the issue and compare it with the currently supported Box, Discrete, MultiDiscrete, MultiBinary, and Dict spaces. Determine the project areas and tests affected by accepting Text spaces; done means Text observations or actions are supported without the padding workaround and existing space support remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100