Re-export normalise from NNlib to remove duplication
- Dominant language
- Julia
- Stars
- 124
- Forks
- 23
- PR merge metrics
- No merged PRs in 30d
Description
### Proposal
MLUtils and NNlib both define a `normalise(x; dims, ...)` (z-score standardization). Since MLUtils already depends on NNlib, MLUtils could drop its own definition and instead re-export `NNlib.normalise`, removing the duplication.
### Current state
- `MLUtils.normalise(x; dims=ndims(x), ϵ=ofeltype(x, 1e-5))` — [src/utils.jl](https://github.com/JuliaML/MLUtils.jl/blob/main/src/utils.jl)
- `NNlib.normalise(x; dims=ndims(x), eps=1f-5)` — added to NNlib in the normalization work ([FluxML/NNlib.jl#748](https://github.com/FluxML/NNlib.jl/pull/748)); it is the stateless building block behind `NNlib.layernorm`.
### The one difference to decide on
The two implementations place `eps`/`ϵ` differently:
| | formula | eps added to |
|---|---|---|
| `MLUtils.normalise` | `(x - μ) / (std + ϵ)` | standard deviation (outside sqrt) |
| `NNlib.normalise` | `(x - μ) / sqrt(var + eps)` | variance (inside sqrt) |
NNlib's inside-the-sqrt convention matches `layernorm`/cuDNN. Re-exporting would therefore be a (small) behavior change for MLUtils users, plus a keyword rename `ϵ` → `eps`. Both seem acceptable given the tiny numerical impact, but flagging it explicitly.
Also worth noting: MLUtils' current implementation carries a stale `# use this when Zygote#478 gets merged` comment and computes `std` without passing `mean=μ`; NNlib passes `mean=μ` to `var`, which is correct at first *and* second order (verified: Hessian is bit-identical to the recompute form).
### Options
1. Re-export `NNlib.normalise` (accept eps-placement change + `ϵ`→`eps` rename).
2. Keep a thin `MLUtils.normalise` that forwards to NNlib but preserves the `ϵ` outside-sqrt semantics for backward compat.
3. Leave as-is.
Happy to open a PR for whichever direction is preferred.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with MLUtils' implementation in src/utils.jl and compare it with NNlib.normalise from the normalization work in FluxML/NNlib.jl#748. Resolve whether to re-export, preserve MLUtils' existing semantics through a forwarding wrapper, or leave the duplication; done means the chosen eps behavior and keyword compatibility are explicit and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100