Add AAC/M4A decode via platform-native decoders
- Dominant language
- Kotlin
- Stars
- 37
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
## Motivation
The sample app's file-transcription tab currently accepts only WAV / AIFF / AU
because those are the formats `AudioFileReader` parses (PCM container only — no
signal decoding). The most common consumer formats users will pick (.m4a from
voice memos, Zoom recordings, .mp3 podcasts) are rejected with
`AudioFileReadError.UnsupportedFormat`.
Issue prompt: a 22 MB `audio1420056308.m4a` from a Zoom recording could not be
transcribed. The file-upload pipeline now routes through the chunked
`OpenAIWhisperEngine.transcribe(audioFlow)` path, but it still needs PCM
`AudioFlow` input — i.e. the file has to be decoded first.
## Proposed architecture
Mirror the existing `kodio-core/src/commonMain/kotlin/space/kodio/core/io/files/`
package layout:
```
io/files/
wav/ AudioWav.kt (pure Kotlin, container-only — exists)
aiff/ AudioAiff.kt (pure Kotlin, container-only — exists)
au/ AudioAu.kt (pure Kotlin, container-only — exists)
m4a/ M4aDecoder.kt (NEW: expect/actual dispatch; signal decoding is platform-native)
mp3/ Mp3Decoder.kt (NEW: expect/actual dispatch; signal decoding is platform-native)
```
Each new package owns:
- Format detection (extension + magic-byte sniffing).
- Error types (re-using `AudioFileReadError.UnsupportedFormat` / `InvalidFile`).
- `expect fun decodeM4a(source: kotlinx.io.Source): AudioSource`
- `expect fun decodeMp3(source: kotlinx.io.Source): AudioSource`
Implementations per platform:
| Target | Decoder |
|---|---|
| `appleMain` (macos + ios) | `AVAudioFile` / `ExtAudioFile` (AVFoundation) |
| `androidMain` | `MediaExtractor` + `MediaCodec` |
| `jvmMain` (desktop) | `javax.sound.sampled.AudioSystem` with consumer-supplied SPI (e.g. JAAD for AAC, mp3spi for MP3). Library does not bundle SPIs; consumers add the SPI they need. |
| `webMain` (js + wasmJs) | Web Audio API `decodeAudioData` |
`AudioFileReader.read(...)` extends its `detectFormat` switch to dispatch to
these new decoders for `.m4a` / `.mp3` / `.mp4` / `.aac`.
## Why not pure-Kotlin commonMain decoders
Both AAC (M4A) and MP3 are full lossy codecs requiring significant DSP:
- MP3: bit reservoir, 32 Huffman tables, IMDCT, hybrid filter bank, 32-band
polyphase synthesis filter — reference impl (JLayer) is ~5,000 lines.
- AAC-LC: MP4 container parsing + Huffman, IMDCT (1024/128), TNS, PNS, M/S
stereo, intensity stereo — reference impl (JAAD) is ~5,000 lines.
A from-scratch port to commonMain would be ~10k LOC of carefully-tested DSP
code, far larger than the rest of `kodio-core` combined, and slower than
hardware-accelerated platform decoders. Out of scope.
## Acceptance criteria
- [ ] `AudioFileFormat` gains `M4a` and `Mp3` data objects.
- [ ] `kodio-core/src/commonMain/.../io/files/m4a/` and `.../mp3/` packages with `expect` decoder functions.
- [ ] Apple, Android, JVM, Web actuals.
- [ ] `AudioFileReader.read(bytes, fileName)` returns `AudioRecording` for `.m4a` / `.mp3` files on all platforms with a working actual.
- [ ] Sample app's file picker re-allows m4a/mp3 once landed.
- [ ] Tests for at least one short fixture per format on JVM (using a consumer-supplied SPI in jvmTest classpath).
## Out of scope
- HE-AAC / SBR / parametric stereo (basic AAC-LC is enough for the typical use case).
- MP3 free format / MPEG 2.5.
- Real-time streaming decode (decoders return whole `AudioRecording`; chunking happens downstream in `OpenAIWhisperEngine`).
## Related
- PR refactoring file transcription to chunked path (preceding this issue).
- `OpenAIWhisperEngine.transcribe(audioFlow)` already chunks PCM into ~10 s windows; once decoders land, the whole pipeline works for compressed inputs.
Contributor guide
Research direction
Start by reading AudioFileReader and the existing wav, aiff, and au packages under kodio-core/src/commonMain/kotlin/space/kodio/core/io/files/. Trace the platform source sets and run the existing file-reader tests before inspecting decoder APIs for each target. Done means the new m4a and mp3 formats dispatch through working platform actuals, pass JVM fixture tests, and restore compressed-file support in the sample picker.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, ios, java, javascript, kotlin, macos, wasm
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100