huggingface / huggingface/candle

Quantization Error unsupported safetensor dtype F8_E4M3

Open
#2,875 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Using tensor-tools to quantize a flux finetine at little-lake-studios/demoncore-flux on hf hub
https://huggingface.co/little-lake-studios/demoncore-flux/

```
tensor-tools quantize demonCORESFWNSFW_fluxV13.safetensors --out-file Q8_0.gguf --quantization q8_0
```
gets an error
```
Error: unsupported safetensor dtype F8_E4M3
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the `tensor-tools quantize` command against the linked Flux model and trace where the safetensor dtype is handled. Identify the quantization path for `F8_E4M3`; done means the reported model can be quantized with `q8_0` without the unsupported-dtype error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.