Request for GGUF Quantised model

Hello team,

I would like to request a GGUF-quantized version of the Bagel model to enable efficient inference using CPU-friendly backends like llama.cpp and similar.

Why GGUF?
It enables running large models on low-resource hardware.

It is compatible with modern quantization methods (e.g., Q4_0, Q5_K_M).

It supports improved loading speed and memory usage compared to older formats.

Details:
Model requested: Bagel (please specify exact variant, if applicable)

Preferred quantization levels: Q4_0, Q5_K_M, or any available GGUF variant

Intended usage: Local inference, testing, and research via llama.cpp or other compatible runtimes

If a GGUF version already exists, kindly share the link or instructions to convert the model using transformers or any tooling you recommend.

Start by determining which Bagel model variant the request targets and whether a GGUF version already exists. Review the requested Q4_0 and Q5_K_M variants and the suggested llama.cpp or transformers tooling. Done means either sharing an existing GGUF link or documenting a supported conversion path and available quantized variants.

Written by the indexing model from the issue text.