DLR-RM / DLR-RM/stable-baselines3

[Feature Request] Add support for gymnasium.spaces.Text

Open
#1,704 1 comment 2 reactions 0 assignees View on GitHub
check the checklist enhancement
Dominant language
Python
Stars
13.8k
Forks
2.2k
Avg merge
1h 35m
Merged PRs (30d)
2

Description

### 🚀 Feature

Unless I'm mistaken, stable-baselines3 only supports Box, Discrete, MultiDiscrete, MultiBinary and Dict spaces from gymnasium.
It seems like a new _fundamental space_ has been introduced in Gymnasium ( [gymnasium.spaces.Text](https://gymnasium.farama.org/api/spaces/fundamental/#gymnasium.spaces.Text) ), which seems to be a more natural way to deal with text data.

### Motivation

This new support would ease the training of LLMs with RL.

### Pitch

_No response_

### Alternatives

One alternative is to use gymnasium.spaces.Box(low=0, high=vocab_size-1, dtype=np.int).
However this requires padding to support smaller texts.

### Additional context

_No response_

### Checklist

- [X] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [X] If I'm requesting a new feature, I have proposed alternatives

Contributor guide

Open the contributing guide

Research direction

Start with the Gymnasium Text space documentation linked in the issue and compare it with the currently supported Box, Discrete, MultiDiscrete, MultiBinary, and Dict spaces. Determine the project areas and tests affected by accepting Text spaces; done means Text observations or actions are supported without the padding workaround and existing space support remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.