adobe / adobe/react-spectrum

useInteractionModality detects JAWS and NVDA as mouse in Chromium

Open
#10,310 6 comments 1 reaction 0 assignees View on GitHub
accessibility bug help wanted
Dominant language
TypeScript
Stars
15.9k
Forks
1.6k
Avg merge
3d 9m
Merged PRs (30d)
59

Description

### Provide a general summary of the issue here

Scenario: `useInteractionModality` reports "mouse" when screen reader user interacts
GIVEN a user has started their screenreader (JAWS or NVDA) and a chromium browser (Chrome)
WHEN the user interacts with the page clicking a button
THEN the hook `useInteractionModality` reports the modality as "mouse"

### 🤔 Expected Behavior?

`JAWS/NVDA + Chromium` interactions are detected as `virtual`

### 😯 Current Behavior

`JAWS/NVDA + Chromium` interactions are detected as `mouse`

### 💁 Possible Solution

We created a very simple 'codepen' to log the event information for every browser and screen reader, to check if there is any difference in terms of the received information.
https://es-d-95332884020260709-019f3c39-f927-7d3c-823e-4328eff26b40.codepen.dev/

We detected that the combination `JAWS/NVDA + Chrome` throws a `click`event and a `pointerdown` event after interacting with a button by pressing `Enter`. In addition, we receive `pointerType` as `mouse`.

`JAWS/NVDA + Firefox` and `VoiceOver` does not trigger a `pointerdown` event and the `pointerType` comes as an empty string, so we can detect the device is assistive technology.

We need another type of check for failing combinations to evaluate them as `virtual` interactions.

---
## Test scenario
1- Navigate to https://codepen.io/editor/jesusgonzaleza/pen/019f3c39-f927-7d3c-823e-4328eff26b40
2- Start your screen reader
3- Interact with the "Simulate click" button
4- Check the reported data from the event

### JAWS + Chrome
Insights: a `pointerdown` event is triggered and `pointerType` is `mouse`
Image

### JAWS + Firefox
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string

Image

### VoiceOver + Firefox
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string

Image

### VoiceOver + Chrome
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string

Image

### 🔦 Context

We detect if a user is a screen reader user in a SPA. In that case, we focus the title of the page in certain navigations so they are aware of changes.
We use `useInteractionModality` from `@react-aria/interactions` for some other component related complex accessibility logics.

### 🖥️ Steps to Reproduce

1- Start JAWS
2- Open Chrome
3- Interact with any button using the screen reader
4- Modality is reported as `mouse`

### Version

3.28.1

### What browsers are you seeing the problem on?

Chrome

### If other, please specify.

_No response_

### What operating system are you using?

Windows

### 🧢 Your Company/Team

_No response_

### 🕷 Tracking Issue

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at the useInteractionModality entry point in @react-aria/interactions and reproduce the reported behavior with JAWS or NVDA in Chrome. Compare the event data with the issue's Firefox and VoiceOver scenarios, then verify that these Chromium screen-reader interactions are reported as "virtual" without changing the existing correct cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
react, typescript
Domain
accessibility, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.