cactus-compute / cactus-compute/cactus
transcribe() returns degenerate repeated tokens instead of speech — whisper-small, Android arm64
- Dominant language
- C++
- Stars
- 6k
- Forks
- 501
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
## Summary
`CactusSTT.transcribe()` returns long runs of a single repeated token ("an an an an…", "AAAA…", "'''''…", ". . . .") instead of a transcription. This reproduces with `whisper-small` downloaded from the registry, on clear English speech, via both the `number[]` and file-path input routes.
I've verified the audio, sample values, WAV encoding, and file location on my side — including playing the exact WAV back on a desktop — so I'm fairly confident the input is sound. I can't rule out something device-specific, so I'd appreciate confirmation of whether `whisper-small` is known-good on Android arm64 hardware of this class.
## Environment
| | |
| --- | --- |
| Device | Samsung Galaxy A52 5G (SM-A526U), Snapdragon 750G, 6GB RAM |
| Android | 14 (One UI 6.1) |
| ABI | arm64-v8a |
| `cactus-react-native` | 1.13.1 |
| `react-native-nitro-modules` | 0.33.9 |
| React Native | 0.86.0 |
| Expo SDK | 57 |
| Build | Expo development build (debug), `expo prebuild` + `expo run:android` |
## Reproduction
1. `const stt = new CactusSTT({ model: 'whisper-small' })`
2. `await stt.download()` then `await stt.init()` — both succeed
3. Capture mic audio via `useAudioStream` from `expo-audio` (`sampleRate: 16000, channels: 1, encoding: 'float32'`)
4. Write to a 16-bit PCM WAV, then `await stt.transcribe({ audio: wavPath, options: { useVad: false } })`
Speech is a short, clearly enunciated English sentence in a quiet room.
## Actual result
Degenerate repetition. Several runs, same setup:
```
'''''''''''''''''''''''''''''''''''''''''''''''''''
```
```
ananananananananananananan an an an an an an An An An
An An An A A A A A A AAAAAAAAAAAAAAAA
```
```
. . . . . . . . . . . . . . . . . .
```
The specific repeated token varies from run to run and doesn't appear to track anything about the input.
Also relevant:
- **`confidence` reports ~0.72–0.82** on all of these, so it doesn't reflect output quality.
- **Decode takes ~38–47s for 3–6s of audio** (~7–8× realtime), consistent with the decoder running to its token limit emitting filler.
## Expected result
A transcription of the spoken English.
## What I've ruled out
Each of these was verified rather than assumed.
- **Audio capture.** Samples are float32 in range (min ≈ −0.35, max ≈ +0.48, RMS ≈ 0.036), correct length for the recording duration, at 16000 Hz.
- **Playback.** I wrote the samples to a 16-bit PCM WAV, pulled it off the device, and played it on a desktop. Clean, audible speech.
- **WAV validity.** Header verified byte by byte: `RIFF`/`WAVE`/`fmt `, format 1 (PCM), 1 channel, `0x3E80` = 16000 Hz, 16 bit. File size matches sample count exactly (44 + 2×N).
- **Input route.** Fails as `number[]` and as a file path.
- **File location.** Fails from the app-internal cache dir and from `/sdcard/Android/data//files/models/` — the same directory the model loads from successfully.
- **Model.** Reproduces on the registry's `whisper-small`, not only on a custom converted model.
- **Language.** Reproduces on English.
- **Encoding.** `int16` and `float32` from `useAudioStream` both produce degenerate output — different repeated tokens, same failure shape.
- **Audio length.** Reproduces identically at ~3s and at 47.7s, so this isn't the 30-second windowing boundary.
- **Decoder prompt.** The binding's `defaultPrompt` is `<|startoftranscript|><|en|><|transcribe|><|notimestamps|>`, which looks correct, and passing it explicitly changes nothing.
## What I have not ruled out
- Whether `whisper-small` is known-good on a Snapdragon 750G, or on Android 14. Is there a reference Android device/result I could compare against? I'd be glad to run `cactus test --android` if that would produce something useful.
- Whether the VAD workaround below leaves the engine in a state that affects transcription.
## Two binding-level observations
Not the bug, but noticed while investigating, and possibly worth separate issues.
**1. `maxTokens` default diverges from the engine.** `CactusSTT.defaultTranscribeOptions` is a flat `{ maxTokens: 384 }`. The engine documents its own default as an audio-length estimate (`audio_sec × 20` for Whisper, min 100, capped to the 448-position limit). For a 3-second clip the engine would pick ~100; the binding lets it run to 384. This doesn't cause the garbage, but it's likely why the failure produces pages of repetition and ~40s decodes rather than a short wrong answer.
**2. `language` is unreachable from React Native.** The engine's transcribe options include `language`, but `CactusSTTTranscribeOptions` doesn't expose it — it appears only on `streamTranscribeStart`. Since the binding always supplies `prompt` (defaulting to one with a hardcoded `<|en|>`), and the engine ignores `language` when an explicit prompt is supplied, there's no way to set the language for a non-English model through the batch path. I'm working with a fine-tuned Igbo Whisper model, so this affects me directly, but it looks like a general gap.
## Setup note: VAD is required at init and must be placed manually
Part of my setup, so including it in case it's relevant.
`init()` fails with:
```
Cactus init failed: Failed to create VAD model - check VAD weights at: /vad
```
This happens regardless of `useVad: false` on the transcribe call — VAD appears to load eagerly at init. Since `CactusSTTDownloadParams` is only `{ onProgress }`, `download()` can't be pointed at a specific model, so I fetched VAD via a throwaway instance:
```ts
const vad = new CactusSTT({ model: 'silero-vad' });
await vad.download();
```
That places it at `files/cactus/models/silero-vad-int8`, not `/vad`, so I copied it into place with `adb shell run-as`. Init succeeds after that — but if there's an intended way to do this, I'd rather use it.
## Setup note: misleading error message
Placing a custom converted model under `/sdcard/Download/` produces:
```
Failed to create model - check config.txt exists at: /sdcard/Download/
```
`config.txt` was present. The real cause was Android scoped storage denying the read. A permissions-specific message would have saved some time here.
## Attached
- `temp.wav` — the exact audio producing the output above. 16 kHz mono 16-bit PCM, verified playable.
Happy to run any diagnostic that would help narrow this down.
[temp.wav](https://github.com/user-attachments/files/30865838/temp.wav)
Contributor guide
Research direction
Start with the CactusSTT.transcribe binding and its defaultTranscribeOptions, then compare the Android arm64 path with the engine behavior described in the report. Run the suggested `cactus test --android` diagnostic if available and reproduce using the attached temp.wav or equivalent input. Done means identifying the cause of the repeated-token output and confirming a normal transcription on the reported setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, cpp, react-native
- Domain
- ai, audio-video-rtc, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100