useInteractionModality detects JAWS and NVDA as mouse in Chromium
- Dominant language
- TypeScript
- Stars
- 15.9k
- Forks
- 1.6k
- Avg merge
- 3d 9m
- Merged PRs (30d)
- 59
Description
### Provide a general summary of the issue here
Scenario: `useInteractionModality` reports "mouse" when screen reader user interacts
GIVEN a user has started their screenreader (JAWS or NVDA) and a chromium browser (Chrome)
WHEN the user interacts with the page clicking a button
THEN the hook `useInteractionModality` reports the modality as "mouse"
### 🤔 Expected Behavior?
`JAWS/NVDA + Chromium` interactions are detected as `virtual`
### 😯 Current Behavior
`JAWS/NVDA + Chromium` interactions are detected as `mouse`
### 💁 Possible Solution
We created a very simple 'codepen' to log the event information for every browser and screen reader, to check if there is any difference in terms of the received information.
https://es-d-95332884020260709-019f3c39-f927-7d3c-823e-4328eff26b40.codepen.dev/
We detected that the combination `JAWS/NVDA + Chrome` throws a `click`event and a `pointerdown` event after interacting with a button by pressing `Enter`. In addition, we receive `pointerType` as `mouse`.
`JAWS/NVDA + Firefox` and `VoiceOver` does not trigger a `pointerdown` event and the `pointerType` comes as an empty string, so we can detect the device is assistive technology.
We need another type of check for failing combinations to evaluate them as `virtual` interactions.
---
## Test scenario
1- Navigate to https://codepen.io/editor/jesusgonzaleza/pen/019f3c39-f927-7d3c-823e-4328eff26b40
2- Start your screen reader
3- Interact with the "Simulate click" button
4- Check the reported data from the event
### JAWS + Chrome
Insights: a `pointerdown` event is triggered and `pointerType` is `mouse`
### JAWS + Firefox
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string
### VoiceOver + Firefox
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string
### VoiceOver + Chrome
Insights: it works correctly. a `pointerdown` event is not triggered and `pointerType` is an empty string
### 🔦 Context
We detect if a user is a screen reader user in a SPA. In that case, we focus the title of the page in certain navigations so they are aware of changes.
We use `useInteractionModality` from `@react-aria/interactions` for some other component related complex accessibility logics.
### 🖥️ Steps to Reproduce
1- Start JAWS
2- Open Chrome
3- Interact with any button using the screen reader
4- Modality is reported as `mouse`
### Version
3.28.1
### What browsers are you seeing the problem on?
Chrome
### If other, please specify.
_No response_
### What operating system are you using?
Windows
### 🧢 Your Company/Team
_No response_
### 🕷 Tracking Issue
_No response_
Contributor guide
Research direction
Start at the useInteractionModality entry point in @react-aria/interactions and reproduce the reported behavior with JAWS or NVDA in Chrome. Compare the event data with the issue's Firefox and VoiceOver scenarios, then verify that these Chromium screen-reader interactions are reported as "virtual" without changing the existing correct cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- react, typescript
- Domain
- accessibility, frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100