Feature request: a handy way of getting what operators are being used in TorchBench models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 346
- PR merge metrics
- No merged PRs in 30d
Description
Not sure if this feature should go here or elsewhere, but here it is:
As a library implementor, one question I have is, "what is the set of operations I need to add support for for my feature to get good coverage on torchbench models"?
We actually already have the mechanism to do this. Thanks to @ezyang's work on torch_function mode, we're able to, given a model, get out a list of PyTorch python APIs the model uses. @samdow's work on torch_dispatch mode lets us, given a model, get out a list of aten operators that the model uses.
Every time I want to do this, I have to:
- figure out how to install torchbench (and debug installation issues of the packages)
- write a script using the above mentioned mechanisms to do this
It would be cool if we autogenerated a list of operations used by the models and shoved them somewhere. Perhaps this is in scope for Operator HUD? https://docs.google.com/document/d/1esIWMJfy7behsupRxGVr4KEXNP0SRYS9pPoZbtqiXGA/edit
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing torch_function and torch_dispatch mechanisms described in the issue, along with the referenced Operator HUD proposal. Determine how TorchBench installation and model execution can produce an autogenerated list of Python APIs or aten operators, and where that list should be published. Done means the workflow is reproducible without each user writing a custom script.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100