Benchmark Channels Last
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 346
- PR merge metrics
- No merged PRs in 30d
Description
channels-last has an API already:
https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html
so it may be as simple as doing model.to(memory_format=torch.channels_last)
and making sure the same happens to inputs.
@jamesr66a I want to clarify the value prop of doing this at the benchmark infra level. The downside is in runtime cost to collect 2x the measurements and sift through 2x the data. The upside is you get some new signal that's potentially useful. Another (hidden) downside may be that we miss the change to incorporate channels-last as an optimization that we do automatically in our compiler. (granted, we don't have a very full story for using compiler techniques on training benchmarks so that's a gap right now).
I'm conflicted on adding this for the above reason. Thoughts?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the channels-last API described in the linked PyTorch memory-format tutorial, including model.to(memory_format=torch.channels_last) and applying the same treatment to inputs. Review the benchmark infrastructure and decide whether collecting these additional measurements provides enough signal; done means the scope and value of the benchmark change are resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100