dosier / dosier/kodio

Add AAC/M4A decode via platform-native decoders

Open
#25 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Kotlin
Stars
37
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## Motivation

The sample app's file-transcription tab currently accepts only WAV / AIFF / AU
because those are the formats `AudioFileReader` parses (PCM container only — no
signal decoding). The most common consumer formats users will pick (.m4a from
voice memos, Zoom recordings, .mp3 podcasts) are rejected with
`AudioFileReadError.UnsupportedFormat`.

Issue prompt: a 22 MB `audio1420056308.m4a` from a Zoom recording could not be
transcribed. The file-upload pipeline now routes through the chunked
`OpenAIWhisperEngine.transcribe(audioFlow)` path, but it still needs PCM
`AudioFlow` input — i.e. the file has to be decoded first.

## Proposed architecture

Mirror the existing `kodio-core/src/commonMain/kotlin/space/kodio/core/io/files/`
package layout:

```
io/files/
wav/ AudioWav.kt (pure Kotlin, container-only — exists)
aiff/ AudioAiff.kt (pure Kotlin, container-only — exists)
au/ AudioAu.kt (pure Kotlin, container-only — exists)
m4a/ M4aDecoder.kt (NEW: expect/actual dispatch; signal decoding is platform-native)
mp3/ Mp3Decoder.kt (NEW: expect/actual dispatch; signal decoding is platform-native)
```

Each new package owns:
- Format detection (extension + magic-byte sniffing).
- Error types (re-using `AudioFileReadError.UnsupportedFormat` / `InvalidFile`).
- `expect fun decodeM4a(source: kotlinx.io.Source): AudioSource`
- `expect fun decodeMp3(source: kotlinx.io.Source): AudioSource`

Implementations per platform:

| Target | Decoder |
|---|---|
| `appleMain` (macos + ios) | `AVAudioFile` / `ExtAudioFile` (AVFoundation) |
| `androidMain` | `MediaExtractor` + `MediaCodec` |
| `jvmMain` (desktop) | `javax.sound.sampled.AudioSystem` with consumer-supplied SPI (e.g. JAAD for AAC, mp3spi for MP3). Library does not bundle SPIs; consumers add the SPI they need. |
| `webMain` (js + wasmJs) | Web Audio API `decodeAudioData` |

`AudioFileReader.read(...)` extends its `detectFormat` switch to dispatch to
these new decoders for `.m4a` / `.mp3` / `.mp4` / `.aac`.

## Why not pure-Kotlin commonMain decoders

Both AAC (M4A) and MP3 are full lossy codecs requiring significant DSP:
- MP3: bit reservoir, 32 Huffman tables, IMDCT, hybrid filter bank, 32-band
polyphase synthesis filter — reference impl (JLayer) is ~5,000 lines.
- AAC-LC: MP4 container parsing + Huffman, IMDCT (1024/128), TNS, PNS, M/S
stereo, intensity stereo — reference impl (JAAD) is ~5,000 lines.

A from-scratch port to commonMain would be ~10k LOC of carefully-tested DSP
code, far larger than the rest of `kodio-core` combined, and slower than
hardware-accelerated platform decoders. Out of scope.

## Acceptance criteria

- [ ] `AudioFileFormat` gains `M4a` and `Mp3` data objects.
- [ ] `kodio-core/src/commonMain/.../io/files/m4a/` and `.../mp3/` packages with `expect` decoder functions.
- [ ] Apple, Android, JVM, Web actuals.
- [ ] `AudioFileReader.read(bytes, fileName)` returns `AudioRecording` for `.m4a` / `.mp3` files on all platforms with a working actual.
- [ ] Sample app's file picker re-allows m4a/mp3 once landed.
- [ ] Tests for at least one short fixture per format on JVM (using a consumer-supplied SPI in jvmTest classpath).

## Out of scope

- HE-AAC / SBR / parametric stereo (basic AAC-LC is enough for the typical use case).
- MP3 free format / MPEG 2.5.
- Real-time streaming decode (decoders return whole `AudioRecording`; chunking happens downstream in `OpenAIWhisperEngine`).

## Related

- PR refactoring file transcription to chunked path (preceding this issue).
- `OpenAIWhisperEngine.transcribe(audioFlow)` already chunks PCM into ~10 s windows; once decoders land, the whole pipeline works for compressed inputs.

Contributor guide

Open the contributing guide

Research direction

Start by reading AudioFileReader and the existing wav, aiff, and au packages under kodio-core/src/commonMain/kotlin/space/kodio/core/io/files/. Trace the platform source sets and run the existing file-reader tests before inspecting decoder APIs for each target. Done means the new m4a and mp3 formats dispatch through working platform actuals, pass JVM fixture tests, and restore compressed-file support in the sample picker.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, ios, java, javascript, kotlin, macos, wasm
Domain
audio-video-rtc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.