ray-project / ray-project/ray

[RLlib] MARWIL OfflinePreLearner stuck at 0.00 row/s when reading Parquet dataset with Dict action space

Open
#57,794 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

community-backlog P3 rllib stability
Dominant language
Python
Stars
43.9k
Forks
8.1k
PR merge metrics
PR metrics pending

Description

What happened + What you expected to happen

I'm running an offline MARWIL experiment using RLlib and a custom environment with a Dict action space and a Dict observation space generated using gymnasium and a parquet offline dataset generated with Ray Data.
After fixing a previous TypeError related to MultiDiscrete.from_jsonable, MARWIL now starts training — but it gets stuck indefinitely with messages like:

Running Dataset: dataset_3_0. Active & requested resources: 1/8 CPU, 256.0MB/196.9MB object store
0.00 row [00:13, ? row/s]
(MapWorker(MapBatches(OfflinePreLearner)) pid=27308) array([...])

The dataset progress stays at 0.00 row/s, suggesting that MARWIL / OfflinePreLearner is not reading any rows from the dataset

Environment: custom with Dict action and observation space

Dataset format: Parquet written with Ray Data

Repo with full reproduction:
https://github.com/IdairaRodYanez/RLlib-experiments/

Forum discussion (full context):
https://discuss.ray.io/t/marwil-with-gymnasium-dict-as-action-space/23236

Versions / Dependencies
  • Ray version: 2.48.0
  • Python version: 3.11.4
  • OS: Windows 11
  • Environment: Gymnasium 1.0.0
Reproduction script

https://github.com/[IdairaRodYanez/RLlib-experiments](https://github.com/IdairaRodYanez/RLlib-experiments/)/

Issue Severity

High: It blocks me from completing my task.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the linked RLlib-experiments reproduction with Ray 2.48.0, Python 3.11.4, the Dict action and observation spaces, and the Parquet dataset. Trace the RLlib OfflinePreLearner dataset path while it reports 0.00 row/s; done means the MARWIL run reads rows and progresses instead of remaining stuck.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.