huggingface / huggingface/candle

Running mamba seems to run single threaded on NixOs with AMD CPU

Open
#2,571 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

I am running cargo run --release --example mamba -- prompt "Mamba is the" on amd cpu without cuda on a fresh clone of candle. It seems to only use single threading.

Is this expected? I tried setting RAYON_NUM_THREADS but still single threaded. I installed rust toolchain using nixos and ensured I have all the required llvm dependencies, also using nixos packages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing `cargo run --release --example mamba -- prompt "Mamba is the"` on an AMD CPU under NixOS, with and without `RAYON_NUM_THREADS`. Determine whether the observed single-threaded behavior is expected, then use the result to define whether the issue needs documentation or an actionable fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.