[Feature]: Multi-Language Generation & Subtitles
- Dominant language
- TypeScript
- Stars
- 37.2k
- Forks
- 5.9k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 195
Description
### Problem or Motivation
OpenMAIC currently supports Chinese (zh-CN) and English (en-US) for the UI interface, but the classroom content generation is limited in language handling:
No language selector for content generation. If I write my prompt in English, the entire classroom is generated in English. There is no way to say "generate this lesson in French" or "generate in German" from the UI.
No subtitle/caption system. The AI teacher speaks through TTS, but there are no on-screen subtitles. For multilingual audiences (e.g., a European classroom with French, German, and English speakers), learners who don't speak the narration language are excluded.
No parallel generation. If I need the same lesson in 3 languages, I have to run the generation 3 separate times, manually translate or re-prompt, and manage 3 separate classrooms. This triples the cost and effort.
This is a critical gap for international and multilingual use cases. In the European Union alone, official documents and training materials often need to be available in multiple languages. In countries like Luxembourg, Belgium, or Switzerland, a single classroom might need French, German, and English simultaneously.
### Proposed Solution
3.1 — Language Selector at Generation Time
Add a language dropdown (or multi-select) on the generation input page:
Primary language: The main language for all generated content (slides, narration, quizzes). Dropdown with at least: English, Chinese, French, German, Spanish, Portuguese, Japanese, Korean, Arabic, Hindi, and an "Other (specify)" option.
Additional languages: Optional secondary languages for subtitle generation.
The selected language should control: the text on slides, the narration script sent to TTS, the quiz questions and answer options, the AI agent dialogue during discussions, and the interactive simulation UI labels.
3.2 — Real-Time Subtitles During Playback
Display synchronized subtitles at the bottom of the classroom view during narration:
Same-language subtitles: Show the narration text as the AI speaks (accessibility, noisy environments, hearing-impaired users)
Translated subtitles: Show a translated version while the AI speaks in the primary language
Toggle on/off: User can enable/disable subtitles and choose the subtitle language
Subtitle timing: Sync with TTS audio timestamps for accurate display
3.3 — Parallel Multi-Language Generation
Allow generating the same classroom in multiple languages in one operation:
User creates a lesson in English (primary)
Selects "Also generate in: French, German"
System generates the English version first, then uses it as a translation source to produce French and German versions
All three versions share the same structure (outline, scene types, images) but have localized text and narration
Each language version gets its own classroom URL
3.4 — Language-Aware TTS
The TTS system should automatically select the correct voice for the chosen language:
English lesson → English TTS voice
French lesson → French TTS voice
Mixed content (e.g., English with French technical terms) → primary language voice with best-effort pronunciation
Technical suggestion
The lib/i18n/ directory already handles UI internationalization with locale files. Content generation language could be passed as a parameter to the generation pipeline in lib/generation/, added to the LLM prompt as an explicit instruction ("Generate all content in French"). For subtitles, the narration scripts already exist as text in the scene data — displaying them as synced subtitles is primarily a frontend task in components/slide-renderer/. Translation could leverage the same LLM used for generation, or a dedicated translation API for accuracy.
### Alternatives Considered
Manual translation after export: Generate in English, export PPTX, translate manually in PowerPoint. Loses all interactive features and is extremely time-consuming for 20+ slide lessons.
Prompt engineering: Writing the input prompt in the target language and hoping the LLM generates consistently in that language. Works partially but LLMs sometimes mix languages, especially for technical terms. No subtitle support.
Browser auto-translate: Using Chrome's built-in translation on the classroom page. Translates UI labels but cannot translate TTS audio, cannot handle dynamic content properly, and breaks interactive elements.
External subtitle tools: Recording the classroom, uploading to a video platform, adding subtitles there. Extremely complex workflow that defeats the purpose of an interactive classroom.
None of these provide an integrated, native multilingual experience.
### Area
Classroom generation
### Additional Context
Real-world use case:
I am building educational content for a European startup operating in Luxembourg — a country with three official languages (French, German, Luxembourgish) plus English as a working language. When I create a training module about renewable energy cooperatives:
The board members prefer English
The local community members prefer French or German
Regulatory references mix EU English with French legal terminology
We need one source lesson that produces outputs in multiple languages
Currently I would need to generate and maintain 3 separate classrooms manually.
Priority ranking:
Language selector for generation (highest priority — low effort, high impact)
Same-language subtitles (accessibility compliance, very useful)
Translated subtitles (medium effort, huge value for multilingual audiences)
Parallel multi-language generation (higher effort, premium feature)
Languages to prioritize beyond zh-CN and en-US:
French, German, Spanish, Portuguese, Japanese, Korean, Arabic — these cover the largest potential user bases and align with major LLM capabilities.
Accessibility note: Subtitles are not just a multilingual feature — they are an accessibility requirement. Many educational institutions require captions for hearing-impaired learners. Adding subtitle support also prepares OpenMAIC for WCAG 2.1 compliance.
Contributor guide
Assessment
This issue has not been assessed yet.