Out-of-memory crashes after sampling completes with high `tune` despite `save_warmup = False`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 207
- Forks
- 28
- Avg merge
- 7d 21h
- Merged PRs (30d)
- 1
Description
I have a model that returns samples ok with low values of tune, but when I bump to high values, python crashes just as the sampling completes, with my OS (Ubuntu) logs showing an OOM event. I don't need the warmup samples, so I tried setting save_warmup = False, but no dice.
Is it possible that the dropping of the warmup data is occurring unnecessarily late, ex. after conversion from arrow to InferenceData? That's the only thing I can think of to explain why higher values for tune cause crashes despite equal values for draws.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by tracing how warmup data is discarded and where Arrow data is converted to InferenceData, then reproduce with high tune, fixed draws, and save_warmup=False; done means the run completes without an out-of-memory crash and a regression test covers the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100