mlcommons / mlcommons/endpoints
feat: Implement BURST and STEP load patterns
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 21
- Forks
- 28
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 13
Description
Problem
LoadPatternType.BURST and LoadPatternType.STEP are defined in the config schema enum but have no scheduler implementation:
# src/inference_endpoint/config/schema.py:72-73
BURST = "burst"
STEP = "step"
A user who selects either pattern in their YAML config will get no error, but also no expected behavior — the scheduler will silently fall through or raise an unhandled case.
Expected Behavior
BURST: Issue a configurable burst of N queries at a fixed interval (e.g. N queries every T seconds)STEP: Incrementally step up QPS in stages (useful for finding saturation point)
Files to Modify
src/inference_endpoint/config/schema.py— document the pattern parameterssrc/inference_endpoint/load_generator/scheduler.py— add scheduler implementationssrc/inference_endpoint/config/runtime_settings.py— ensure RuntimeSettings handles these patterns
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the LoadPatternType entries at src/inference_endpoint/config/schema.py:72-73, then inspect src/inference_endpoint/load_generator/scheduler.py and src/inference_endpoint/config/runtime_settings.py to understand existing pattern handling and configuration conventions. Done means BURST and STEP configurations are accepted and produce the documented query schedules, with coverage for both patterns and their parameters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100