microsoft / microsoft/MInference
[Bug]: Implement of TriangleMix
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 82
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
Describe the bug
When I try to apply tri_mix on qwen2.5-7B, I meet the error: Failed to apply TriangleMix: The attn_type tri_mix you specified is not supported.
I test with MInferenceConfig.get_available_attn_types(), but also miss the tri_mix.
How can I apply tri_mix on my LLM?
>>> from minference import MInferenceConfig
To benefit from fast kernel implementations, we recommend installing SGLang or vllm.
>>> supported_attn_types = MInferenceConfig.get_available_attn_types()
>>> supported_attn_types
['minference', 'vllm_minference', 'hf', 'vllm', 'dense', 'static', 'dilated1', 'dilated2', 'a_shape', 'tri_shape', 'vllm_a_shape', 'vllm_tri_shape', 'inf_llm', 'flexprefill', 'vllm_flexprefill', 'xattention']
>>>
Steps to reproduce
No response
Expected Behavior
No response
Logs
No response
Additional Information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at MInferenceConfig.get_available_attn_types() and trace the code path that rejects attn_type=tri_mix. Confirm the intended TriangleMix integration for qwen2.5-7B; done means tri_mix is accepted, listed as available, and can be applied without the reported error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100