tiny-pilot / tiny-pilot/tinypilot

Handling unidentified/death keys in the keyboard history UI

Open
#1,747 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
3.5k
Forks
291
PR merge metrics
No merged PRs in 30d

Description

I noticed one quirk in the UI, regarding the keypress history in the status bar. Note: this only affects the UI, not the backend or HID interface!

When the client machine is configured to use a keyboard layout that doesn’t provide a mapping for certain keys or key combinations, the keyboard history shows inconsistent results across browsers. From the view point of the target machine, however, things work as expected. So this issue is specific to using non-EN-US keyboard layouts.

I tested this on a MacOS machine with a customised keyboard layout, which doesn’t define a mapping for the alt+L key combination. So pressing alt+L on the client machine wouldn’t produce any character on the client machine itself. However, on the target machine, the alt+L combination may be well-defined, e.g. if the target machine uses a German keyboard layout where alt+L maps to the @ character. In this scenario, TinyPilot would forward the alt+L combination to the target machine, and produce the @ there.

In the TinyPilot’s web UI, however, I see these results in the status bar:

As stated in the documentation, the key property of the KeyboardEvent object may be Unidentified or Dead:

  • If the KeyboardEvent represents the press of a dead key, the key value must be "Dead".
  • If the key cannot be identified, the returned value is Unidentified.

For the keyboard history, we rely on that key property, which is why we see this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at app/static/js/app.js line 72, where the keyboard history relies on KeyboardEvent.key. Reproduce the alt+L case with the documented MacOS layout across Chrome, Safari, and Firefox, then verify that handling for Unidentified and Dead keys produces consistent, useful history in the UI without changing backend or HID behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
frontend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.