WordPress / WordPress/php-ai-client

Add support for speech-to-text models

Open
#150 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
308
Forks
84
Avg merge
7d 21h
Merged PRs (30d)
2

Description

Models like OpenAI’s Whisper provide speech to text capabilities. This is useful if you want to create text form of audio content like podcasts or generate subtitles for videos.


The php-ai-client package currently provides capabilities and options for Text To Speech models, but not for Speech To Text.

Currently available:

Support for Speech To Text

To support Speech To Text models, I’d propose something like this:

Add:

  • CapabilityEnum::speechToTextConversion()
  • OptionEnum::inputMimeType()

Usage:

	    $sttCapabilities = [
		    CapabilityEnum::speechToTextConversion(),
	    ];
	    $sttOptions = [
		    new SupportedOption(OptionEnum::inputModalities(), [[ModalityEnum::audio()]]),
		    new SupportedOption(OptionEnum::inputMimeType(), [
			    'audio/mpeg',
			    'audio/ogg',
			    'audio/wav',
			    'audio/flac',
			    'audio/aac',
		    ]),
		    new SupportedOption(OptionEnum::outputModalities(), [[ModalityEnum::text()]]),
                    new SupportedOption(OptionEnum::outputMimeType(), ['text/plain', 'application/json']),
		    new SupportedOption(OptionEnum::customOptions()),
	    ];

(Inspired by OpenAI TTS settings).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/Providers/Models/Enums/CapabilityEnum.php and OptionEnum.php, then compare the existing metadata definitions in src/ProviderImplementations/OpenAi/OpenAiModelMetadataDirectory.php. Add the proposed speech-to-text capability and input MIME type consistently with the existing enums and metadata; done means speech-to-text models can advertise the listed input and output modalities and formats.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.