libretro / libretro/RetroArch

Accessibility: Add active accessibility resources into games

Open
#9,920 1 comment 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

accessibility
Dominant language
C
Stars
14.1k
Forks
2.2k
Avg merge
7h 35m
Merged PRs (30d)
51

Description

First and foremost consider this:

  • Only RetroArch bugs should be filed here. Not core bugs or game bugs
  • This is not a forum or a help section, this is strictly developer oriented

Description

In Retroarch 1.8.2, accessibility mode was added so that blind users can now navigate the retroarch interface, enable the AI Service, and play games using sound and text. To break limits further, "accessibility packs" could be created, similar to Cheat/rumble codes, in order to actively read information, trigger sound, or vibrate the game pad to give further accessibility.

Here is how I envision it: We already have a cheat/rumble system, which can read memory locations, effect memory, and rumble the game pad. Accessibility packs, however, could focus on getting information in memory, from a project like This decompilation of Super Mario 64, like menus to be read as a user highlights menu options, health bars to be signaled through sound or vibration, maps and objects, so that a user can select something and be guided, using binaural sound, to the object, or in-game text, which can be automatically spoken on appearance by the accessibility system. This does not negate the need for the AI Service, as it will probably take a while before even many games have accessibility packs. Sound resources could either be managed by the pack, or a common set of sounds (foot step, hit wall, HP low, Limit break, etc., could be included within Retroarch to provide a common set of sounds for pack creators to work with.

For a great showcase of how this could work, see Pokemon Chrystal Access

Expected behavior

Retroarch could use accessibility packs to add active accessibility resources to games.

Actual behavior

For now, blind gamers use the AI Service to read in-game text.

Steps to reproduce the bug

N/A

Bisect Results

[Try to bisect and tell us when this started happening]

Version/Commit

You can find this information under Information/System Information

  • RetroArch: [1.8.2]
Environment information
  • OS:[MacOS 10.15.2
  • Compiler: [In case you are running local builds]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No implementation files, tests, or entry points are named. The proposal needs a defined accessibility-pack format and implementation scope before a newcomer can start; done would mean RetroArch can use packs to provide active accessibility resources in games.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
accessibility
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.