pymc-devs / pymc-devs/pytensor
Pytorch backend slow with pymc model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 644
- Forks
- 208
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 16
Description
Description
@ricardoV94 did a nice perf improvement in https://github.com/pymc-devs/pymc/pull/7578 to try to speedup jitted backends. I tried out torch as well. The model performed quite slow.
| mode | t_sampling (seconds) | manual measure (seconds) |
|---|---|---|
| NUMBA | 2.483 | 11.346 |
| PYTORCH (COMPILED) | 206.503 | 270.188 |
| PYTORCH (EAGER) | 60.607 | 64.140 |
We need to investigate why
- Torch is so slow
- Torch compile is slower than eager mode
When doing perf evaluations, keep in mind that torch does a lot of caching. If you want a truly cache-less eval, you can either add torch.compiler.reset() or set the env variable to disable the dynamo cache (google it).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file, test, or entry point is named. Reproduce the reported PyTorch eager and compiled benchmark, controlling caching with torch.compiler.reset() or the documented cache-disabling environment variable. Done means identifying why PyTorch is slow and why compilation is slower than eager mode, with supporting measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100