model.safetensors header is NUL-padded and rejected by the reference safetensors loader (README claims it works)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 94.5k
- Forks
- 12.5k
- Avg merge
- 21h 27m
- Merged PRs (30d)
- 43
Description
Summary
model.safetensors in ruvnet/wifi-densepose-pretrained is not a spec-compliant safetensors file. Its header is padded with NUL bytes, which are not valid JSON whitespace, so the reference deserializer rejects it.
The README states this file works with the standard loader:
| Python training / evaluation / embedding extraction |
model.safetensors| ✅ Works — load withsafetensors.torch.load_file|
It does not.
Reproduction
from safetensors.numpy import load_file # same Rust backend as safetensors.torch
load_file("model.safetensors")
safetensors_rust.SafetensorError: Error while deserializing header:
invalid JSON in header: trailing characters at line 1 column 1462
Diagnosis
file size : 48,840 bytes
declared header : 1,464 bytes (u64 LE at offset 0)
valid JSON ends at char 1,461
trailing bytes : 3 -> b'\x00\x00\x00'
The header is padded from 1,461 to 1,464 to reach 8-byte alignment, but padded with 0x00. The safetensors format pads with spaces (0x20); NUL is not permitted JSON whitespace, so serde_json reports trailing characters.
Patching those three bytes to spaces (or truncating to 1,461) makes the file load, and the tensors are intact:
lora.A, lora.B, lora.scaling,
encoder.w1, encoder.b1, encoder.w2, encoder.b2,
encoder.bn1_gamma, encoder.bn1_beta, encoder.bn1_runMean, encoder.bn1_runVar,
encoder.bn2_gamma, encoder.bn2_beta, encoder.bn2_runMean, encoder.bn2_runVar,
presence_head.weights, presence_head.bias
So this is purely a serializer bug in whatever wrote the file, not corrupted weights.
csi-embed-v2.safetensors in the same repo loads fine with the reference loader, which suggests the two were produced by different writers.
Note on the Rust loader
sensing-server --model model.safetensors accepts the file:
INFO sensing_server: Model `model.safetensors` is safetensors weight file —
converting to RVF in-memory and loading (issue #894)
INFO sensing_server: Layer A ready: model=model vconverted-from-safetensors (3 segments)
Worth being aware that the in-tree reader is more lenient than the reference implementation — so this wouldn't be caught by testing through the server alone.
Suggested fix
Pad the header with 0x20 (or emit via the official safetensors writer) and re-upload. Alternatively, correct the README row for model.safetensors, since users following it hit an exception on the documented command.
Minor related observation while looking at this file: encoder.w1 and encoder.w2 are stored flat ((512,) and (8192,)) rather than as 2-D (64, 8) / (128, 64), so consumers must know the intended layout out of band.
Environment: safetensors 0.7.x on Python 3.13, Windows 11.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with model.safetensors and reproduce the failure using safetensors.numpy.load_file, then inspect the header padding described in the issue. Check the README row for the documented loader and compare against csi-embed-v2.safetensors. Done means the file loads with the reference safetensors loader and the README accurately describes its status.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100