vllm-project / vllm-project/aibrix
Does it support speculative decoding with a draft model that is not an ngram?
Open
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
Does it support speculative decoding with a draft model that is not an ngram?
If it is supported, how should the yaml be configured and is there any corresponding documentation?
Contributor guide
Research direction
Start by checking whether speculative decoding with a non-ngram draft model is supported and how its YAML configuration should be written. If supported, document the configuration and identify the corresponding documentation location; otherwise record the limitation and expected behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai-infra-agents
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100