Support for BitNet (Microsoft's native 1 bit LLM)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 32
- Forks
- 8
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 8
Description
How to use GitHub
- Please use the 👍 reaction to show that you are interested into the same feature.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Feature request
Microsoft has released Bitnet B1.58-2B-4T under the MIT license. This is a 1.58 bit model with 2 billion parameters, natively trained for that precision (instead of being created by quantization of larger models).
The implication is that BitNet is fairly small, yet powerful, and can run solely on a CPU with acceptable inference speed.
Furthermore, Microsoft provides an own inference framework named BitNet.cpp, based on llama.cpp. My understanding is that this would allow a rather straightforward integration in existing codebase.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names BitNet B1.58-2B-4T and BitNet.cpp but no repository files or tests. Start by reviewing BitNet.cpp's integration requirements and the project's existing model-loading path. Done means the requested BitNet model can run through the project with documented CPU inference behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100