librosa / librosa/librosa

[Bug]: chroma_stft (via estimate_tuning/piptrack) leaves unreturned memory in the glibc arena, growing RSS across repeated calls

Open
#2,076 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.6k
Forks
1.1k
Avg merge
6d 8h
Merged PRs (30d)
4

Description

Bug summary

Repeated calls to librosa.feature.chroma_stft() cause process RSS (resident memory) to grow over time in a long-lived process, especially when input length varies call-to-call (as real audio files typically do). This is not a Python object leak or a growing cache — chroma_stft internally calls estimate_tuning()piptrack(), which allocates ~6 full-spectrogram-sized temporary arrays (including a redundant np.abs(S) copy, since the input power spectrogram is already non-negative) to compute a single scalar tuning offset. These are freed correctly at the Python level every call, but glibc's malloc does not return the freed memory to the OS under this allocation pattern. A single ctypes call to malloc_trim(0) reliably reclaims the excess, confirming this is arena fragmentation rather than a reference leak.

Code for reproduction
import ctypes
import numpy as np
import librosa

libc = ctypes.CDLL("libc.so.6")

def vmrss_mb():
    with open("/proc/self/status") as f:
        for line in f:
            if line.startswith("VmRSS:"):
                return int(line.split()[1]) / 1024

sr = 16000
rng = np.random.default_rng(0)
base_len = 3_964_032  # ~248s at 16kHz

# Fixed-length input: RSS should plateau quickly if this is just one-time setup cost.
y0 = rng.standard_normal(base_len).astype(np.float32)
librosa.feature.chroma_stft(y=y0, sr=sr, hop_length=256)  # warmup
print(f"[fixed length] warmup: {vmrss_mb():.1f} MB")
for i in range(1, 11):
    librosa.feature.chroma_stft(y=y0, sr=sr, hop_length=256)
    print(f"[fixed length] call {i}: {vmrss_mb():.1f} MB")

libc.malloc_trim(0)
print(f"[fixed length] after malloc_trim(0): {vmrss_mb():.1f} MB")

# Varying-length input, closer to real-world usage (real files are rarely identical length):
print()
for i in range(1, 21):
    length = base_len + rng.integers(-500_000, 500_000)
    y = rng.standard_normal(length).astype(np.float32)
    librosa.feature.chroma_stft(y=y, sr=sr, hop_length=256)
    print(f"[varying length] call {i} (len={length}): {vmrss_mb():.1f} MB")

libc.malloc_trim(0)
print(f"[varying length] after malloc_trim(0): {vmrss_mb():.1f} MB")
Actual outcome

Real output from the exact script above, on our machine:

[fixed length] warmup: 323.8 MB
[fixed length] call 1: 323.9 MB
[fixed length] call 2: 323.9 MB
...
[fixed length] call 10: 323.7 MB
[fixed length] after malloc_trim(0): 268.1 MB

[varying length] call 1 (len=3747151): 329.1 MB
[varying length] call 2 (len=4134037): 342.3 MB
[varying length] call 3 (len=4226114): 359.9 MB
...
[varying length] call 16 (len=4129722): 360.1 MB
[varying length] call 20 (len=4082139): 342.3 MB
[varying length] after malloc_trim(0): 285.2 MB

Fixed-length calls plateau almost immediately (~323.7–323.9 MB), yet malloc_trim(0) still recovers 55.6 MB that accumulated invisibly during warmup. Varying-length calls push the ceiling meaningfully higher (up to ~360 MB) before malloc_trim(0) recovers most of it. In both cases, none of this memory is referenced by Python — it's sitting in glibc's arena.

Expected outcome

RSS should stay close to a stable baseline across repeated calls, without requiring an explicit malloc_trim(0) to reclaim tens of megabytes — especially for a function whose actual return value here is a single scalar float per call. At minimum, we'd expect the growth to be negligible relative to the ~63MB scratch arrays being churned, not a meaningful fraction of it.

Additional information

Traced via bisection: isolated chroma_stft's four internal steps (STFT, estimate_tuning, filters.chroma filterbank lookup, final einsum contraction) — only estimate_tuning (via piptrack) shows this behavior; the others are flat. piptrack's implementation (librosa/core/pitch.py) computes S = np.abs(S) (redundant when S is already a non-negative power spectrogram), then allocates pitches = np.zeros_like(S), mags = np.zeros_like(S), avg = np.gradient(S, axis=-2), shift = _parabolic_interpolation(S, axis=-2), and dskew = 0.5 * avg * shift — 6 full-size temporary arrays just to return a scalar tuning estimate via estimate_tuning.

A possible fix: skip the redundant np.abs(S) copy when the caller can guarantee non-negative input (e.g. a power=2 spectrogram), and/or give estimate_tuning/chroma_stft a lighter-weight path that doesn't require full pitch/magnitude maps when only a scalar tuning value is needed. Happy to help prototype either approach if a maintainer can weigh in on which direction is preferred — didn't want to open a PR for a behavior/design change like this without checking first.

This may be related to #1286, which reported a similar RSS-growth pattern in librosa.load() — that thread concluded librosa's own code holds no cross-call state, consistent with what we found here: it's not a logic bug, it's an allocator-interaction side effect of allocating more scratch memory than the task strictly needs.

Operating system

Linux (Ubuntu-based, WSL2) — Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39

software versions
python: 3.12.13 | packaged by conda-forge | (main, Mar  5 2026, 16:50:00) [GCC 14.3.0]
librosa: 0.11.0
numpy: 2.2.6
scipy: 1.18.0
numba: 0.66.0
soundfile: 0.14.0
joblib: 1.5.3
Installation

conda

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in librosa/core/pitch.py and trace chroma_stft through estimate_tuning to piptrack, focusing on the temporary spectrogram-sized arrays described in the issue. Review the existing pitch and chroma tests before choosing an approach; done means repeated calls with varying input lengths avoid the reported RSS growth while preserving tuning and chroma results.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
audio-video-rtc, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.