Stable Audio 3 VAE outputs broadband noise: encode→decode round-trip destroys audio (MPS and CPU)
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Summary
The Stable Audio 3 VAE produces broadband noise on Apple Silicon (MPS) and on CPU. A plain **encode → decode round-trip of a known-good WAV** — no diffusion, no text conditioning — destroys the audio. The same round-trip through the Stable Audio Open 1.0 VAE is clean, and the same SA3 weights generate correct audio through Stability AI's official `stable-audio-3` package on the same machine.
### Minimal reproduction (no sampling involved)
Workflow: `CheckpointLoaderSimple` → `LoadAudio` → `VAEEncodeAudio` → `VAEDecodeAudio` → `SaveAudio`
- checkpoint: `stable_audio_3_small_sfx.safetensors` (from `Comfy-Org/stable-audio-3`, sha256 `ed9cf1b6172f1a8c2921a9560c21109ff3239524563ced9dce6dcdef41e2f515`)
- input: any normal 44.1 kHz stereo WAV
Result: the output is broadband noise — energy spread evenly from DC to Nyquist for the entire duration, with no transient, no harmonics and no decay.
Control: the identical workflow with `stable-audio-open-1.0.safetensors` returns the input essentially unchanged.
Text-to-audio is affected the same way, which is expected if the decoder is the broken stage.
### What this is not
Each of the following was tested and ruled out:
| Suspected cause | Test | Result |
|---|---|---|
| Wrong graph | Compared verbatim against the official `audio_stable_audio_3_medium_base` template | Matches (`t5gemma_b_b_ul2` loaded via `CLIPLoader` with `type: stable_audio`) |
| Wrong sampler | Official settings `lcm` / `simple` / steps 50 / cfg 7 | No change |
| Prompt format | Official example prompts including the trailing `Length: X seconds` token | No change |
| Precision | `--force-fp32`, `--fp32-vae` | No change |
| Device | `--cpu` | No change (so this is not MPS-specific) |
| Partial loading of `weight_norm` layers | `--gpu-only --disable-smart-memory` | No change |
| Corrupt weights | sha256 of checkpoint and text encoder vs `Comfy-Org/stable-audio-3` | Exact match |
| Latent shape | SA3 is 256ch / 4096, `EmptyLatentAudio` emits 64ch / 2048 | `fix_empty_latent_channels()` rescales correctly; output duration matches the expected value |
| Model variant | `small_sfx` and `small_sfx_base` | Both fail identically |
| Stale checkout | 24 commits behind master | No audio/latent commits in the gap |
No `missing`/`unexpected key` warnings appear on load, and the log shows `Requested to load SAT5GemmaModel` and `Requested to load SA3AudioVAE`, so detection and weight loading look correct.
### Proof the weights are fine
The same `small-sfx` weights, on the same Apple Silicon machine, produce correct audio through Stability AI's own `stable-audio-3` inference package (MPS, 8 steps, cfg 1.0): clear transient, harmonic structure, decay to silence.
### Where I would look
`SoftNormBottleneck` in `comfy/ldm/audio/vae_sa3.py` applies per-channel `scaling_factor` and `running_std` normalisation, while `latent_formats.StableAudio3` in `comfy/latent_formats.py` only declares `latent_channels` / `latent_dimensions` / `temporal_downscale_ratio` and carries no `process_in` / `process_out` scaling. If those two do not agree, decoded output would look exactly like this. I have not confirmed this — it is a starting point, not a diagnosis.
### Environment
- ComfyUI 0.34.0, commit `0a33ed6c`
- macOS 26.6.2, Apple M4, 16 GB unified memory, PyTorch MPS backend
- Python 3.12
- Also reproduced with `--cpu` on the same machine
Contributor guide
Research direction
Reproduce the encode→decode workflow with stable_audio_3_small_sfx.safetensors, then compare it with stable-audio-open-1.0. Read SoftNormBottleneck in comfy/ldm/audio/vae_sa3.py alongside latent_formats.StableAudio3 in comfy/latent_formats.py, focusing on the scaling information mentioned in the report. Done means the Stable Audio 3 round-trip preserves the known-good WAV instead of producing broadband noise on CPU and MPS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- audio-video-rtc, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100