Accessibility: Add active accessibility resources into games
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 14.1k
- Forks
- 2.2k
- Avg merge
- 7h 35m
- Merged PRs (30d)
- 51
Description
First and foremost consider this:
- Only RetroArch bugs should be filed here. Not core bugs or game bugs
- This is not a forum or a help section, this is strictly developer oriented
Description
In Retroarch 1.8.2, accessibility mode was added so that blind users can now navigate the retroarch interface, enable the AI Service, and play games using sound and text. To break limits further, "accessibility packs" could be created, similar to Cheat/rumble codes, in order to actively read information, trigger sound, or vibrate the game pad to give further accessibility.
Here is how I envision it: We already have a cheat/rumble system, which can read memory locations, effect memory, and rumble the game pad. Accessibility packs, however, could focus on getting information in memory, from a project like This decompilation of Super Mario 64, like menus to be read as a user highlights menu options, health bars to be signaled through sound or vibration, maps and objects, so that a user can select something and be guided, using binaural sound, to the object, or in-game text, which can be automatically spoken on appearance by the accessibility system. This does not negate the need for the AI Service, as it will probably take a while before even many games have accessibility packs. Sound resources could either be managed by the pack, or a common set of sounds (foot step, hit wall, HP low, Limit break, etc., could be included within Retroarch to provide a common set of sounds for pack creators to work with.
For a great showcase of how this could work, see Pokemon Chrystal Access
Expected behavior
Retroarch could use accessibility packs to add active accessibility resources to games.
Actual behavior
For now, blind gamers use the AI Service to read in-game text.
Steps to reproduce the bug
N/A
Bisect Results
[Try to bisect and tell us when this started happening]
Version/Commit
You can find this information under Information/System Information
- RetroArch: [1.8.2]
Environment information
- OS:[MacOS 10.15.2
- Compiler: [In case you are running local builds]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No implementation files, tests, or entry points are named. The proposal needs a defined accessibility-pack format and implementation scope before a newcomer can start; done would mean RetroArch can use packs to provide active accessibility resources in games.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- accessibility
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100