Streaming Dataset Simulator stops working beyond 10,000 batches
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
Environment
- OS: Mac OS (Sequoia 15.2)
- Hardware (GPU, or instance type): Macbook (CPU Box)
- Streaming Version: 0.11.0
To reproduce
Enter in training duration == 100,000 batches. When training duration is > 10,000 (we tried setting training duration 100,000), the streaming simulator hangs. The terminal stops printing out progress per batch. Additionally, the plot in the UI hangs as well.
Here is what the terminal looks like when the simulator hangs:
Here's what the UI looks like when the simulator hangs:
he simulator hangs:
Expected behavior
I expect the simulator to work when the training duration is set at 100,000 like it does when the number of batches is 10,000 or less. Here is what the terminal looks like when the simulator is working.
Here is what the UI looks like when the simulator is working:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Streaming Dataset Simulator hang on Mac OS Sequoia 15.2 with Streaming 0.11.0, comparing 10,000 and 100,000 batches. Observe both terminal progress and the UI plot, then trace the simulator entry point responsible for batch progress and plotting. Done means 100,000 batches continue producing terminal progress and the UI plot without hanging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100