H.264: Proposal for audio-related UI elements (status bar / menu item)

No files, tests, or entry points are named. Review issue #1276 and the current frontend audio and video UI architecture first; the proposal would be complete when the UI clearly distinguishes audio availability and mute state and provides a menu control, but the issue is currently on hold because its dependency is marked “won’t fix”.

This is blocked on https://github.com/tiny-pilot/tinypilot/issues/1276. (Update 2022-01-26: On hold for the time being, as that ticket is “won’t fix”.)

We could display some sort of indication in the UI whether or not audio is currently playing. This might otherwise be ambiguous, because when it’s silent, the user cannot tell whether that’s due to the target machine not emitting any audio, or due to the streaming connection being severed.

An additional benefit is that such an indicator would promote discoverability of the feature.

Proposal

I think the most natural place for such an indication would be in the status bar. We could distinguish 3 cases:

• Audio is generally available and on (unmuted)
• Audio is generally available, but off (muted)
• Audio is not available (e.g. in MJPEG mode, or in H.264 but without the appropriate hardware)

For the first two cases, the UI could look like below. For the third case, we could just hide that element altogether.

Whether or not audio is muted could be controlled via a menu item:

(Could also be phrased inversely, i.e. “Mute Audio”.)

Additional Notes

• In addition to the menu item, we could also consider making the status bar indicator itself interactive, i.e. letting the user mute and un-mute by clicking on it.
  • I think we should explore this option separately, though, because currently the status bar is only “informative” but not interactive, so we’d be introducing a slight conceptual change to the status bar UX. E.g., if we allow to mute/unmute, we might also have to re-consider whether we allow to toggle between H.264 and MJPEG in the adjacent video indicator. Generally speaking, I think the menu should always be the canonical way to do things, and potential status bar interactions should only provide redundant convenience.
• We need to keep in mind that the user can mute the entire tab. So if the user does that, the frontend still thinks it’s playing audio. As far as I know, this is not detectable from within the tab, so there might be nothing that we can do about it.