Provide tiny wrapper over pytorch ThroughputBenchmark
Open
Nobody has claimed this yet.
enhancement
help wanted
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
🚀 Feature
PyTorch utils module provides ThroughputBenchmark since 1.2.0
>>> from torch.utils import ThroughputBenchmark
>>> bench = ThroughputBenchmark(my_module)
>>> # Pre-populate benchmark's data set with the inputs
>>> for input in inputs:
# Both args and kwargs work, same as any PyTorch Module / ScriptModule
bench.add_input(input[0], x2=input[1])
>>> # Inputs supplied above are randomly used during the execution
>>> stats = bench.benchmark(
num_calling_threads=4,
num_warmup_iters = 100,
num_iters = 1000,
)
>>> print("Avg latency (ms): {}".format(stats.latency_avg_ms))
>>> print("Number of iterations: {}".format(stats.num_iters))
It would be interesting to provide a tiny wrapper over this to simplify usage with ignite.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading PyTorch's linked torch.utils ThroughputBenchmark API and compare it with Ignite's existing utility or metric entry points. Define the smallest wrapper interface that fits Ignite usage, then verify that it supports the benchmark inputs and returns latency and iteration statistics as shown; the issue does not name a target file or test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100