mudler / mudler/parakeet.cpp

Error when input file is longer than 5min

Open
#55 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
786
Forks
93
PR merge metrics
No merged PRs in 30d

Description

Transcription with parakeet is working ok with small files, but with files a longer than 5min it gives the following error: ErrorOutOfDeviceMemory.

I'm using the vulkan version.

Steps to reproduce

I have a 2hr+ recording that want to transcribe, gives error and did try with 10min, like this:

$ ffmpeg -t 00:10:00 -i in.wav -c copy p1.wav

$ ffprobe p1.wav
Input #0, wav, from 'p1.wav':
  Metadata:
    encoder         : Lavf62.12.102
  Duration: 00:10:00.00, bitrate: 705 kb/s
  Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, 1 channels, s16, 705 kb/s

$ parakeet-cli transcribe --model $HOME/00_M/tdt-0.6b-v3-q4_k.gguf --input p1.wav --json > out.json
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = NVIDIA GeForce GTX 1660 SUPER (NVIDIA) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: none
[parakeet] pk::Backend using device: Vulkan0
ggml_vulkan: Device memory allocation of size 900240064 failed.
ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
ggml_gallocr_reserve_n_impl: failed to allocate Vulkan0 buffer of size 16698365856
[parakeet] Backend::compute: ggml_gallocr_alloc_graph failed
Segmentation fault         (core dumped) parakeet-cli transcribe --model $HOME/00_M/tdt-0.6b-v3-q4_k.gguf --input p1.wav --json > out.json
  • I also did try the pipe example with ffmpeg, gives same error.
  • If I do the same, but with 5min, works.
  • I can do the the full 2hr+ audio with Whisper without any problem, but I'm trying parakeet now, because is faster.
  • I can chop the audio in many 5 min files, but is more like a workaround than a solution, also gives problems to join subtitles.
  • I did use frequently llama.cpp and sometimes sd.cpp, if it counts as experience.
  • My system is updated.
System

distro: Arch Linux x86_64
kernel: 6.18.25-1-lts
cpu: Intel i7-4790 (8) @ 3.600GHz
ram: 32 GB
gpu: NVIDIA GeForce GTX 1660 SUPER (6GB VRAM)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the parakeet-cli transcribe entry point and reproduce the difference between the working 5-minute input and the failing 10-minute input using the commands in the issue. Inspect the Vulkan allocation and compute path around the reported ggml_gallocr and Backend::compute failures. Done means longer audio no longer exhausts device memory or segfaults, while preserving transcription output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.