GoogleCloudPlatform / GoogleCloudPlatform/generative-ai

[Feat]: Add Support for Nigerian Languages or allow developers to finetune live-native-audio

Open
#2,585 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
17.7k
Forks
4.5k
Avg merge
12h 38m
Merged PRs (30d)
42

Description

### Is your feature request related to a problem? Please describe.

I implemented the gemini live API and I must say it is amazing, but it's not so good in terms of pronunciation of the African context. While it supports some languages like Swahili and Zulu, it does not support Hausa, Yoruba or Igbo which are the languages spoken by almost 200 million people in Africa.

### Describe the solution you'd like

I'm making a suggestion to add support for these languages, or allow us the ability to zero shot the live-native-audio with audio samples in the config along with the system_instruction, or allow us the ability to finetune this model within vertex AI.

At the moment, vertex ai does not support fine-tuning any of the audio models talk-less of the live models. It will be great to have this support, I'm willing to assist in anyway that I can.

### Describe alternatives you've considered

The easier alternative will be the zero shot approach, I'm eager to see how this will improve the performance of the models, but the best alternative will be the ability to finetune

PS: I have tried providing audio samples during a live session but the performance is still not there yet.

### Additional context

_No response_

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files, tests, or entry points. Start by checking Vertex AI live-native-audio documentation for supported Hausa, Yoruba, and Igbo languages and the stated audio-sample or fine-tuning options. Done would require a scoped implementation path and concrete acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.