huggingface / huggingface/candle
Regression: `layer_norm()`/`rms_norm()` always fails to find weight/bias tensors [main branch]
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Introduced by commit `b96522e6f2e59af87c12ea66e77417c195044dd9`
"fixes for BERT safetensors: weight/bias gamma/beta compatibility & BERT variable prefix (#1888)"
Before: `weight`/`bias` tensors were fetched directly with `vb.get_with_hints(...)`.
Bias was only fetched when `config.affine` was true.
After: a name is picked first via `vb.contains_tensor(name)`, checking `["weight", "gamma"]` and `["bias", "beta"]`. Missing tensor is a hard error (`"Failed to find weight tensor"`).
The bias-name lookup is unconditional. It runs even when `config.affine` is false. It also runs before the real `get_with_hints` call for both weight and bias.
**Bug 1: RmsNorm always fails**
`rms_norm()` calls `layer_norm()` with `affine: false`. RmsNorm has no bias tensor. The unconditional bias-name lookup finds neither `bias` nor `beta`.
Every `rms_norm()` call errors with `"Failed to find weight tensor"`.
This breaks nearly every RmsNorm-based model. RmsNorm has no bias by definition.
**Bug 2: `contains_tensor` breaks lazy/`VarMap` construction**
`contains_tensor(name)` returns false for a tensor a `VarMap`-backed `VarBuilder` has not yet materialized, even when it can be lazily created
via `Init`. So even the weight lookup fails for a fresh `VarMap`, regardless of `affine`.
This breaks any model built from `VarBuilder::from_varmap` with a fresh `VarMap` (e.g. tests, random init), for both affine and non-affine configs.
**Reproducer**
```rust
use candle_core::{DType, Device};
use candle_nn::{VarBuilder, VarMap};
let device = Device::Cpu;
let varmap = VarMap::new();
let vb = VarBuilder::from_varmap(&varmap, DType::F32, &device);
// Fails: "Failed to find weight tensor"
let _ = candle_nn::rms_norm(8, 1e-6, vb.pp("norm"));
```
Gate the bias-name lookup behind `config.affine`, matching the pre-#1888 behavior. Keep the weight/gamma and bias/beta aliasing for BERT compat, but only probe for tensor presence when actually needed, and don't let `contains_tensor` block legitimate lazy `Init`-based creation.
@Christof23
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the layer_norm() and rms_norm() entry points, then run the VarBuilder::from_varmap reproducer from the issue with a fresh VarMap. Done means rms_norm() works without a bias tensor, fresh lazy VarMap construction succeeds for affine and non-affine configurations, and weight/gamma and bias/beta compatibility remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100