microsoft / microsoft/markitdown

Audio transcription sent to undeclared/test Google Account and not to the provided llm client

Open
#1,284 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
186k
Forks
13.7k
Avg merge
1d 4h
Merged PRs (30d)
49

Description

Transcribing audio content will use an unspecified Google API key for the transcription, and not as expected use the provided llm_client.

This is in part solved by #326 which at least provides an option to not route everything this way.


Instead of relying on the provided LLM llm_client markitdown will process audio via the SpeechRecognition library sr:
https://github.com/microsoft/markitdown/blob/da7bcea527ed04cf6027cc8ece1e1aad9e08a9a1/packages/markitdown/src/markitdown/converters/_transcribe_audio.py#L45-L49

In SpeechRecognition recognize_google is mapped to google_legacy:

https://github.com/Uberi/speech_recognition/blob/46e70560f605ed190b3b0c16f198ee34978de585/speech_recognition/__init__.py#L1288

The google_legacy method even comes with this warning (although does not declare where this key comes from and how the data may be used):

The Google Speech Recognition API key is specified by key. If not specified, it uses a generic key that works out of the box. This should generally be used for personal or testing purposes only, as it may be revoked by Google at any time.

https://github.com/Uberi/speech_recognition/blob/46e70560f605ed190b3b0c16f198ee34978de585/speech_recognition/recognizers/google.py#L225-L262

As it's using some unspecified API key
https://github.com/Uberi/speech_recognition/blob/46e70560f605ed190b3b0c16f198ee34978de585/speech_recognition/recognizers/google.py#L118-L119

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/markitdown/src/markitdown/converters/_transcribe_audio.py at the audio transcription call, then read SpeechRecognition's recognize_google and google_legacy implementations linked in the issue. Determine how transcription should use the provided llm_client instead of an unspecified Google API key, and verify that the resulting path no longer sends audio through that undeclared account.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.