iOS adapter

This project was funded through the NGI0 Commons Fund, a fund established by NLnet. See the project page for more information.

Abstract

The aim of this project is to implement UIKit accessibility. UIKit is Apple's user interface framework powering iOS, iPadOS, tvOS, and visionOS. Once complete, this new backend will allow any user of the AccessKit library to get UI accessibility on iPhone, iPad, Apple TV, and the Apple Vision Pro without substantial change to their current implementation. The finished product will contain a generic component that can be plugged into any iOS application as well as a specialized component to support the popular winit windowing library. An existing UI toolkit written in Rust will be modified to showcase the feature.

Tasks

• Phase 1: Core iOS Adapter (#710)
  • Setup the low-level adapter skeleton following existing adapters structure
  • Traits, properties and states mapping for nodes
  • Post simple notifications to report UI events
  • Create the subclassing adapter to inject into a UIKit view
  • Write unit tests and update the CI to have them run
  • Document how to run the winit examples
  • Publish first version of accesskit_ios on crates.IO
• Phase 2: Integrate into the Slint UI toolkit (https://github.com/slint-ui/slint/pull/12750)
  • Validate the subclassing adapter through accesskit_winit with the Slint winit backend and make adjustments
  • Implement text editing capability using parley and validate on desktop platforms, iOS will be validated in the next milestone
  • Modify the Slint CI to ensure the accessibility tree can be seen in the simulator
• Phase 3: Integration Testing & Editable Text
  • Setup a test harness
  • Configure iOS simulator on GitHub CI
  • Write some integration tests to cover the key features (traits, properties and states exposed, changes detected)
  • Implement methods to support editable text (characters, selection), validate with a Slint application
  • Post required notifications for editable text (caret/selection move, text insertion/deletion), validate with a Slint application
  • Publish second version of accesskit_ios on crates.IO
• Phase 4: Finishing touches
  • Process feedback from security audit
  • Expose both low-level and subclassing adapters API in the C bindings
  • Update the Meson build script
  • Update the CI to generate pre-built packages
  • Integrate into the existing SDL example to demonstrate how to use
  • Publish new version of the C bindings with the new binaries for iOS
  • Write a blog post about the work done

Start with the existing accesskit_ios adapter and its existing adapter structure, then review the Phase 3 tasks for the test harness, iOS simulator CI, integration tests, and editable text. Use the Slint application through accesskit_winit to validate behavior. Done means the key traits, properties, states, changes, and editable-text notifications are covered and the second accesskit_ios release is published.