modelscope / modelscope/ms-swift
Could you consider supporting quantization training for training the EXL3 (Exllamav3) quantized model?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Feature Request: Support Quantization Training for EXL3 (Exllamav3) Format
Describe the feature
Could you consider supporting quantization training for the EXL3 (Exllamav3) quantized model? The Exllamav3 format is highly optimized for consumer-level GPUs while maintaining excellent output quality. Integrating this would enable more efficient training and inference for users with limited hardware resources.
Paste any useful information
- Exllamav3 GitHub repository: https://github.com/turboderp-org/exllamav3
Additional context
Exllamav3 has gained significant support within the Hugging Face community, with many EXL3 models already available on the platform. Adding quantization training support for this format would align with the growing demand for efficient and accessible model deployment.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked Exllamav3 repository and ms-swift’s existing quantization and training support. Define the EXL3 integration scope, supported training workflow, and validation criteria before identifying the required implementation and tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100