livepeer / livepeer/go-livepeer

audio-to-text rejects raw PCM WAV with "audio duration calculation failed"

Open
#3,918 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: triage
Dominant language
Go
Stars
586
Forks
226
Avg merge
1d 17h
Merged PRs (30d)
19

Description

## Summary

Gateway `/audio-to-text` endpoint returns `HTTP 400 {"error":{"message":"audio duration calculation failed"}}` for any raw PCM WAV input, before contacting any orchestrator. Re-encoding the same audio to MP3 succeeds.

## Repro

`livepeer 0.8.9`, gateway on arbitrum-one-mainnet with `--aiServiceRegistry`. A 22.5 s mono 16 kHz `pcm_s16le` WAV produced by `ffmpeg`:

```
$ file speech.wav
speech.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 16000 Hz

$ curl -X POST http://127.0.0.1:9935/audio-to-text \
-F "audio=@speech.wav" \
-F "model_id=openai/whisper-large-v3"
{"error":{"message":"audio duration calculation failed"}}
```

Returns in <1 ms. Gateway journal:

```
ai_process.go:1542 ... try=1 orch=https://lp-orch.j1v.co:8936 err=audio duration calculation failed
handlers.go:1718 ... Error with API code=400 err=audio duration calculation failed
```

`try=1` plus the sub-millisecond round-trip confirm the rejection is client-side at the gateway, not at the orchestrator.

## Expected

Same transcription that the MP3-encoded form of the same audio returns successfully:

```
$ ffmpeg -y -i speech.wav -ar 16000 -ac 1 -c:a libmp3lame -b:a 64k speech.mp3
$ curl -X POST http://127.0.0.1:9935/audio-to-text \
-F "audio=@speech.mp3;type=audio/mpeg" \
-F "model_id=openai/whisper-large-v3"
{"text":" Mary had a little lamb. Its fleece was as white as snow. ...","chunks":[...]}
```

## Root cause

`common/util.go:467-487` (`CalculateAudioDuration`):

```go
_, mediaFormat, err := ffmpeg.GetCodecInfoBytes(bytearr)
if err != nil {
return 0, errors.New("Error getting codec info")
}

duration := int64(mediaFormat.DurSecs)
if duration <= 0 {
return 0, ErrAudioDurationCalculation
}
```

For raw PCM WAV, lpms `GetCodecInfoBytes` returns successfully but with `DurSecs=0` — the WAV/RIFF format does not store duration as a container field, only as a derivable value (`data_chunk_size / (sample_rate × channels × bytes_per_sample)`). The `int64(0) <= 0` check then rejects the input.

A side note: even when `DurSecs` is populated as a float, the `int64()` truncation will reject any clip shorter than 1 second.

## Suggested fix

Two options:

1. **Real fix** — fall back for PCM WAV: when `DurSecs == 0`, derive duration from the RIFF data subchunk size and the format's sample rate / channels / bit-depth. WAV headers always carry these. PCM WAV is a perfectly reasonable input for an STT pipeline.
2. **Cheaper guard** — drop the `int64()` truncation and use a small float epsilon, so sub-second clips don't get rejected.

Option 1 also covers (2). Either way the workaround for now is to pre-transcode to MP3/M4A/Opus.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in common/util.go:467-487, especially CalculateAudioDuration, and reproduce the /audio-to-text request with the raw PCM WAV described in the issue. Trace how PCM WAV metadata reaches the duration check; done means valid PCM WAV input, including sub-second clips, is accepted while the existing MP3 behavior remains successful.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.