1.58 bit implementation

Would it possible to implement 1.58 bit quantization on candle ? It was proposed in the following paper,

https://arxiv.org/pdf/2402.17764.pdf

The main inspiration behind using 1.58 bit implementation is that you could replace matrix multiplication with addition. If that is feasible, with apple accelerate framework's SIMD instructions, we could expect better training and inference on large language models.

A couple of Llama.cpp discussions here

https://github.com/ggerganov/llama.cpp/issues/5761
https://github.com/ggerganov/llama.cpp/pull/5999

There is also a training library which was released a couple of days ago,
https://github.com/rafacelente/bllama

Any thoughts ?

Start by reading the referenced paper, the linked llama.cpp discussion and pull request, and the bllama training library to understand the proposed 1.58-bit approach. No Candle files, tests, or entry points are named; the work would need a defined implementation scope and measurable training or inference outcome before completion can be assessed.

Written by the indexing model from the issue text.