mapbox / mapbox/mapbox-navigation-ios

SystemSpeechSynthesizer should use SSML on iOS 16

Open
#4,057 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

jira-sync-complete op-ex topic: voice
Dominant language
Swift
Stars
918
Forks
326
Avg merge
1h 16m
Merged PRs (30d)
3

Description

When falling back to the VoiceOver text-to-speech engine, we currently pass AVSpeechSynthesizer an AVSpeechUtterance created from the plain text representation of the spoken instruction, in some cases marked up with the IPA pronunciation of the road name. But the spoken instruction’s SSML representation contains more information about prosody, volume, embedded foreign languages, and how to interpret abbreviations and numbers, which we’re dropping on the floor.

iOS 16 introduces [`AVSpeechUtterance(ssmlRepresentation:)`](https://developer.apple.com/documentation/avfaudio/avspeechutterance/3566308-init) for creating an utterance directly from SSML source code. For iOS 16 and above, we would short-circuit the code that marks up the attributed string with IPA in favor of this initializer. I’m unsure how AVSpeechUtterance handles the proprietary Amazon Polly SSML attributes and tags that appear in the Mapbox Voice API output. We may need to strip them out using either a regular expression (fragile) or XMLDocument (slow). While we’re at it, we could check whether Apple has fixed some of the caveats we identified in #624: that the Alex voice doesn’t support attributed text and that the other voices don’t support the IPA symbols ɡ and ɹ.

https://github.com/mapbox/mapbox-navigation-ios/blob/4ad5f99b96f962709020ae5ce0d408fe30c6e533/Sources/MapboxNavigation/SystemSpeechSynthesizer.swift#L86 https://github.com/mapbox/mapbox-navigation-ios/blob/9e62f088e12da8a0cc2ab418b6bdf1e716e056be/Sources/MapboxNavigation/RouteVoiceController.swift#L8-L47

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Sources/MapboxNavigation/SystemSpeechSynthesizer.swift around line 86 and the related RouteVoiceController.swift code linked in the issue. Check how the spoken instruction’s SSML is passed to the iOS speech engine and how proprietary Amazon Polly markup and the caveats from #624 affect it. Done means iOS 16 and later can use the SSML representation without losing supported speech information.

Written by the indexing model from the issue text.

Assessment

Tech stack
ios, swift
Domain
mobile
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.