aws-samples / aws-samples/sample-collaborative-ai-dlc
[Feature]: User Experience Improvement
- Dominant language
- JavaScript
- Stars
- 75
- Forks
- 23
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 24
Description
### Description
## Personas
#### In scope — **Developer / Tech Lead**
Supervises AI coding agents end to end (intent → construction → review). Unblocks them, diagnoses failures, reviews code, and orchestrates the fleet from a single console.
Working context:
- Often interleaves Collaborative AI DLC with their main IDE / Figma / video meetings on a primary 4K screen.
- Reads code at ~50 cm reading distance, accepts technical jargon, and dictates more than they type.
#### In scope companion — **Tech Lead at a multi-screen workstation**
Same persona as above, but with a companion (Corsair Xeneon Edge 14.5" 32:9, peripherals like Stream Deck, Knob, TC001). The companion surfaces are **secondary, optional, always-on dashboards** — never the main place to act, and never required to use the product.
### Explored experiences
**two complementary modalities**: a primary **Touch + Voice** experience on desktop, touch screens, and an optional **hardware companion** layer that extends the cockpit onto physical peripherals. Companions are always opt-in: the product must remain fully usable without any of them.
#### A. Primary experience — Touch/click + Voice
The screen/touch screen is where actual work happens: triaging the queue, reading diffs, leaving voice feedback, unblocking sprints.
##### A.1 Voice review — Press & Hold (walkie-talkie)
- **Press & hold** on a document line = starts voice recording.
- **Release** = sends the feedback to the agent.
- **Short tap** on a line = selection / activation (no recording).
- **Haptic** on start ().
- Floating overlay at the bottom while recording: live waveform + timer + Cancel / Send buttons (safety net).
- Impossible to forget to stop: releasing = stop & send.
- ❌ **No Approve / Reject / Request changes action bar** in the review footer.All review decisions go through voice.
- ❌ No permanent floating mic button: the gesture lives on the line itself.
- ❌ No confirmation modal before sending: release = send.
**Why**: direct, immersive, zero permanent UI, . The user reads, holds, speaks, releases. Code stays readable at all times.
##### A.2 Agent location — Annotation threads
- **Fixed right-side panel**
- **One persistent thread per annotated line / block**.
- **Discreet badge in the left margin** of the document marking lines with a thread.
- The **document stays stable** on the left; threads live on the right.
- The agent replies **in text** in the matching thread (the dev speaks and listen music, the agent writes).
- Thread sort: pending first, then by line order.
- “Mark resolved” action available per thread.
##### A.3 Document rendering
- The document is rendered **line by line** (each line is a press-and-hold target).
- **Light markdown**: comments, inline-styled paragraphs.
- **Code blocks : mono font, neutral background, no wrap, horizontal scroll if needed.
- **Diffs (` ```diff `)**: per-line colouring:
- **Line numbers** shown on the right (≥ md), monospace, tabular-nums.
- **Code diffs MUST stay visible**: this is one of the central review cases. Known regression not to repeat.
##### A.4 Dashboard
**Role**: an operator cockpit for a developer. The first screen must immediately answer: what is blocked, which agent is working, which human action is expected?
**Useful content (and only what is useful)**
- **Header filters** (To do / Running / Shipped / Parking) with counters — this is the main navigation.
- **Operator queue**: list prioritised by impact + time lost. Errors and blocking decisions on top.
- **Active agents** (sidebar): progress, current step, duration.
- **Latest deliveries** (sidebar): what changed recently.
**Explicit refusals on the dashboard**
- ❌ No “Fleet state” block with aggregated progress bars.
**Action row (operator queue)**: status, phase, branch, wait time, agent question / error title, log excerpt on failure (last 4 lines), and direct actions without drill-down (voice review, quick replies, retry, inspect).
#### B. Companion experiences — optional hardware extensions
Companions are **side instruments**, not replacements. They make the cockpit ambient and tactile when the dev is at their full workstation. **Removing any companion must not degrade the touch/voice experience.**
##### Corsair Xeneon Edge — ultra-wide HUD
A 14.5" 32:9 (2560×720) capacitive touch panel sitting next to the main monitor. it runs there permanently as a horizontal cockpit.
https://www.corsair.com/fr/fr/p/monitors/CC-9011306-WW/xeneon-edge-14-5-lcd-touchscreen-cc-9011306-ww
- **Three dense columns**: Decisions Needed, Running, Shipped.
- High-contrast text and 56 px+ touch targets for ~50 cm reading distance.
- Live status ticker in the header (waiting / running / shipped counts).
- Status pulses on update
- Dark theme prioritised
Use case: the PO / Tech Lead glances at the Edge while coding or in meetings, taps to acknowledge or escalate, never has to leave their main screen.
##### Elgato Stream Deck XL + Bitfocus Companion — 1-tap actions
8×4 = 32 LCD keys driven by Companion
Covers:
- **A.x — 1-tap action**: focus a sprint, ack, retry, approve, open diff, voice, snooze, skip, contextual quick-replies derived from the focused sprint’s `quickReplies`.
- **B.x — persistent overview**: each key’s tone reflects sprint status (`running` / `waiting` / `failed` / `completed`); animations (`pulse`, `blink`, `dim`) mimic LCD brightness.
- **C.5 — passive notification**: focused-sprint white halo highlights context without stealing attention.
##### B.3 ESP32-S3 Knob Display — physical co-pilot
Round touch screen + rotary encoder + click. Peripheral halo = fleet status colour.
https://www.waveshare.com/esp32-s3-knob-touch-lcd-1.8.htm?srsltid=AfmBOorxJB4t_7kPfdSSpeA6xuydHIUy1uTbVMiQ7gQEYxqXoyqD-6zm
Two modes:
- **`nav` (default)**:
- Rotation → next/previous sprint in the NEEDS YOU queue.
- Short click → drill-in if `waiting` with > 1 quick-reply, otherwise direct ack.
- Long click → reserved.
- **`drill` (waiting sprint with multi-choice)**:
- Rotation → cycle available quick-replies.
- Short click → send the selected reply.
- Long click → leave drill mode without sending.
The halo uses the focused sprint’s semantic colour, full ring while drilling (context signal). Long-press threshold: 500 ms.
##### B.4 UlanZi TC001 — ambient passive display
8×32 LED matrix (Awtrix-like firmware) simulated by a dot grid. **Read-only**, no interaction.
Covers **C.5 — discreet visual notification** in hardware:
- Scrolling text scenes (≤ ~10 chars visible at a time): NEEDS counter, failed sprint, running fleet, latest done.
- Semantic tone per scene (`waiting` / `error` / `success` / `running`).
- Aimed at peripheral vision: you should _feel_ that something needs you without looking directly at it.
#### C. How the experiences combine
| Surface | Role | Required ? |
|---|---|---|
| Tablet (Touch + Voice) | Primary work surface — read, review, decide | ✅ Yes |
| Xeneon Edge | Always-on horizontal HUD next to the main screen | Optional |
| Stream Deck XL | 1-tap commands and persistent fleet overview | Optional |
| ESP32-S3 Knob | Physical triage of the NEEDS YOU queue | Optional |
| UlanZi TC001 | Ambient passive notification | Optional |
### Use case
### Area
Frontend
### Additional context
_No response_
Contributor guide
Research direction
No files, tests, or frontend entry points are named in the issue. Start by mapping the existing frontend review and dashboard flows against the requested touch, voice, and optional hardware experiences; completion criteria need to be narrowed before implementation can be considered done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100