[RFC] Background Voice Assistant: a complete keyboard-free interaction mode

## Summary

Make voice a complete, keyboard-free interaction mode for Maka rather than a microphone attached to the composer.

When Voice is active, a user should be able to start work, clarify scope, steer an in-progress task, answer approval requests, ask for status, cancel or resume work, and hear the result without returning to the keyboard. Maka should remain available for conversation while work runs in the background, and should proactively return only when something important happens.

The product promise is:

> A user can keep working with Maka by voice when typing is inconvenient or undesirable, without receiving a reduced version of the task, permission, or recovery model.

Voice and text remain two interaction surfaces over the same sessions, tasks, permissions, and results. Ending a voice conversation does not end background work, and reopening Voice restores a concise view of work that is still running or waiting for the user.

This RFC defines the product behavior and system boundaries required for that experience. It deliberately avoids provider-specific APIs and a file-by-file implementation plan.

## Problem

Voice input by itself solves only the first few seconds of a task. A useful background assistant has to support the entire work loop.

Today, a microphone can help a user produce an instruction, but the rest of the interaction still tends to assume that the user is watching the main window and can return to the keyboard:

- ambiguous requests need clarification;
- the user may need to narrow or change scope after execution starts;
- long-running work needs meaningful status, not a silent wait;
- approvals and blocked states require a decision;
- completed work needs a short spoken result and a durable detailed result;
- multiple active tasks need stable identity;
- network loss or an application restart must not duplicate important actions;
- stopping the microphone must not accidentally cancel background work.

Treating Voice as an isolated chat mode would create a second, weaker product:

- it would have its own transient memory rather than authoritative task state;
- it could report stale progress;
- it could lose approvals or completion events while disconnected;
- it could confuse the lifetime of a conversation with the lifetime of a task;
- it could bypass or weaken existing permission boundaries;
- it would force users back to the keyboard precisely when the workflow becomes important.

The missing capability is therefore not speech recognition. It is a persistent interaction contract that connects natural conversation to Maka's durable work model.

## Goals

### Complete keyboard-free task loop

A user can complete this loop without typing:

```text
state a goal
-> clarify material ambiguity
-> confirm the execution boundary
-> start background work
-> ask for status or change direction
-> answer an approval or unblock the task
-> receive a concise result
-> choose the next action
```

Text remains available for detail, evidence, and fallback, but it is not required for the primary flow.

### Conversation remains available during execution

Starting a task must not monopolize the voice conversation. While one task runs, the user can:

- ask what it is doing;
- add a constraint;
- pause, resume, or cancel it;
- start another task;
- ask an adjacent question;
- ask Maka to remain quiet and report later.

### Important events return to the user

The user should not need to repeatedly ask whether work has finished. Maka proactively surfaces:

- work accepted and started;
- a material phase change when useful;
- a blocker that requires information;
- an approval request;
- completion;
- failure or a retry decision.

Low-level activity remains visible in the text surface but is not read aloud.

### One authority across voice and text

Voice must read and mutate the same authoritative task, permission, and result state as the rest of Maka. A spoken status answer is a projection of actual task state, not a separate conversational guess.

### Recoverable and trustworthy behavior

Network interruption, application restart, audio failure, or reopening Voice must preserve acknowledged work, pending decisions, and important unread results without silently replaying consequential commands.

## Product principles

### Always reachable, never ambiguous

Voice has explicit listening, speaking, muted, reconnecting, and unavailable states. A user must be able to tell whether Maka heard them and whether an instruction was accepted.

### Speech first, screen available

The default response is suitable for listening: short, direct, and focused on what changed or what the user must decide. Detailed logs, diffs, evidence, and long reports remain in the session for later inspection.

### Background work is independent from the call

A voice conversation represents the period in which the user is talking to Maka. A task represents durable work. Their lifetimes are intentionally different:

- one voice conversation can create or manage multiple tasks;
- a task can outlive the voice conversation that created it;
- a completed task can be reported during a later conversation;
- ending Voice stops listening and speaking, not execution;
- cancelling all work is a separate, explicit action.

### Important uncertainty is confirmed

Maka can act directly on clear, low-risk, reversible requests. It confirms ambiguity that affects scope, external side effects, destructive actions, approval decisions, or which task the user means.

Confirmation should summarize the boundary rather than mechanically repeat the transcript:

> I will investigate and run tests, but I will not modify files.

### Proactive, not noisy

Maka speaks when the event changes the user's understanding or requires a decision. It does not narrate every file read, command, tool call, or internal retry.

## User experience

### Entry points

The product should support:

- an in-app Voice control for discovery and full status;
- a global shortcut for starting, stopping, or bringing Voice forward;
- a compact floating control that remains useful while the main window is behind other work.

The first version should not listen continuously by default. Starting microphone capture requires an explicit user action.

### Minimal visual surface

Even a keyboard-free mode needs a small trust surface showing:

- whether the microphone is listening;
- whether the user or Maka is currently speaking;
- the focused task;
- the number of other running tasks;
- pending approvals;
- the most recent transcript;
- mute, end Voice, and switch-to-text controls.

This surface is a status and recovery aid, not a requirement to watch the conversation.

### Interaction modes

| Mode | Behavior |
| --- | --- |
| Conversation | Continuous turn-taking; important task events may be spoken proactively. |
| Quiet | Listening and background work continue, but updates are displayed rather than spoken. |
| Push to talk | Audio is captured only while the user holds the shortcut or control. |
| Dictation | Speech populates the composer without entering a persistent voice conversation. |

Mode changes are explicit and visible. Maka must not silently begin background listening or change notification behavior.

### Starting a conversation

Voice begins with a brief acknowledgement:

> I'm listening.

If work from an earlier conversation is still relevant, Maka may add one short recovery prompt:

> Two tasks are still running. Do you want a status update first?

It should not automatically read a backlog before the user has a chance to speak.

### Starting work

Maka classifies an utterance as one of:

- ordinary conversation;
- a new task;
- a change to an existing task;
- a status query;
- pause, resume, or cancel;
- an approval response;
- a request to hear a result;
- a Voice mode change.

For a clear request, Maka acknowledges and starts quickly:

> Started. I will come back when there is a meaningful update.

For material ambiguity, Maka asks the smallest question that resolves the execution boundary.

### Steering work

The user can interrupt at any time with requests such as:

- "Pause that."
- "Only inspect the desktop side."
- "Do not make changes yet."
- "Run the tests before you tell me it is fixed."
- "Add this to the documentation task."

Maka confirms that a steering request reached the authoritative task before claiming that the task changed. If the target is ambiguous, it asks which task rather than guessing.

### Asking for status

A spoken status response should be synthesized from authoritative task facts:

1. current state;
2. most meaningful completed step;
3. what is happening next;
4. whether the user is needed.

It should not substitute recent conversation memory for actual runtime state.

### Hearing a result

On completion, Maka starts with a one-to-three-sentence spoken summary:

> I found the startup failure: the configuration points to an expired service endpoint. I did not modify any files. The evidence is saved with the task. Do you want me to fix it?

The durable text result contains the detail. The user can ask follow-up questions, request a document, or authorize further work by voice.

## Core model

### Voice session

A Voice session owns the conversational state required for natural interaction:

- listening/speaking/muted/reconnecting status;
- recent turns and a bounded conversational summary;
- focused task;
- notification mode;
- spoken-event acknowledgements;
- reconnection information.

It does not own the authoritative status or result of background work.

### Background task

Each task created or managed through Voice has:

- stable identity;
- a short human-friendly name;
- current state;
- last meaningful progress;
- whether it is waiting for the user;
- pending approval references;
- final result or failure;
- delivery state for important notifications.

The same task is visible and controllable from the text surface.

### Focused task

One task may be the conversational focus so that "continue", "pause it", or "how is it going?" can work naturally.

Focus changes when:

- the user starts a task;
- the user explicitly refers to another task;
- a task begins waiting for the user;
- the focused task completes and the conversation moves elsewhere.

Focus is a convenience, not identity. If more than one task plausibly matches an utterance, Maka asks.

### Task events

Background work produces durable events that Voice can project into user-facing updates. At minimum:

```text
TaskAccepted
TaskStarted
TaskProgressed
TaskBlocked
ApprovalRequested
ApprovalResolved
TaskCompleted
TaskFailed
TaskCancelled
```

Events carry task identity and an idempotent event identity. Voice may merge or summarize them for speech, but must not invent task facts that are absent from the underlying event or current task state.

### Speech notification queue

Important events enter a session-independent delivery queue. Each item records:

- source task and event;
- priority;
- whether user action is required;
- concise spoken form;
- full text reference;
- delivery state;
- whether interruption requires it to be offered again.

This queue prevents completion, failure, and approvals from disappearing when the user is already speaking, has selected Quiet mode, loses connectivity, or ends the Voice session.

## Event and notification policy

| Priority | Examples | Default behavior |
| --- | --- | --- |
| Action required | approval, destructive boundary, missing decision | Speak after the user's current turn; retain until resolved or dismissed. |
| Terminal | completed, failed, cancelled | Speak proactively when allowed; retain as unread if delivery is deferred. |
| Milestone | reproduced, root cause found, verification started | Coalesce and speak selectively. |
| Routine | file reads, individual commands, internal retries | Record in the text surface only. |

Before speaking proactively, Maka checks:

- whether the user is currently talking;
- whether Voice is in Quiet mode;
- whether a higher-priority item is pending;
- whether the same event has already been delivered;
- whether the content is appropriate and short enough for speech.

Multiple related events may be coalesced:

> The implementation is finished and all requested tests passed.

They must not be merged when doing so would hide a separate approval, failure, or task identity.

## Turn-taking and interruption

### End-of-turn behavior

Voice should tolerate natural pauses, corrections, and continuations rather than responding after every short silence. A turn is committed only when the system has enough evidence that the user has finished or the user explicitly submits it.

### User interrupts Maka

When the user begins speaking:

1. current audio output stops promptly;
2. the system starts listening;
3. ordinary conversational output may be discarded;
4. approvals, failures, and terminal results remain pending if their essential content was not delivered;
5. after the new utterance, Maka decides whether to continue the earlier topic, change task focus, or stay silent.

Stopping speech output is not the same as cancelling the underlying task.

### Maka interrupts the user

Maka should not talk over the user. Even action-required notifications wait for a safe turn boundary unless immediate physical safety is involved, which is outside the initial scope of Maka's software-task assistant.

## Approvals and permissions

Voice is another presentation and input surface for Maka's existing permission model. It must not grant broader authority.

An approval prompt explains:

1. what action is proposed;
2. why it is needed;
3. what it may affect;
4. the available choices.

A response is accepted only when it maps unambiguously to the pending approval and task. "Okay", partial transcripts, background speech, or an answer that could refer to more than one request must not authorize a high-impact operation.

Required invariants:

- an approval is bound to one task and one proposed action;
- approval identity survives Voice reconnection;
- the same answer is not applied twice;
- rejection or narrower scope is preserved;
- changing from voice to text does not create a second approval;
- ending Voice leaves the approval pending rather than approving or rejecting it.

## Reliability and recovery

### Delivery acknowledgement

Maka distinguishes:

- transcript observed;
- intent understood;
- command accepted;
- task state changed.

Only the last applicable acknowledgement allows Maka to say that an instruction was carried out. A network interruption after transcription but before command acceptance must not produce a false success message.

### Idempotency

Consequential commands and event delivery use stable identities so retries do not:

- start a task twice;
- apply the same steering request twice;
- answer an approval twice;
- announce the same completion repeatedly.

### Network interruption

If Voice connectivity is lost:

- listening/speaking state visibly changes;
- acknowledged background tasks keep running;
- unacknowledged utterances are not treated as executed;
- reconnection restores focused-task and pending-event summaries;
- retries do not duplicate work.

### Audio unavailable

If input or output audio becomes unavailable:

- background work continues;
- results and approvals remain in the text surface;
- the user receives a visible notification;
- Maka offers Dictation or text fallback when possible.

### Application restart

After restart:

- background task state is restored from the task authority;
- microphone listening does not resume automatically;
- unresolved approvals and unread important results remain available;
- the user can choose whether to resume the previous conversational topic;
- stale conversational guesses are discarded in favor of current task facts.

## Privacy and trust

- Microphone activity always has a persistent visible indicator.
- Listening starts and stops through explicit user control.
- Raw audio is not retained by default.
- Transcript retention is a user-visible setting.
- Voice does not expand task permissions.
- High-impact actions require explicit, scoped confirmation.
- The user can inspect running tasks and pending approvals at any time.
- "End Voice" and "cancel background work" are separate actions.
- Quiet mode changes speech delivery, not event durability.
- Reopening Voice does not silently resume microphone capture.

Long-term preferences may include language, voice, speaking rate, proactive-notification policy, confirmation policy, and transcript retention. Raw recordings are not treated as long-term memory.

## Proposed architecture

The design requires five cooperating responsibilities, independent of any specific provider:

1. **Realtime voice transport** owns audio input/output, turn detection, low-latency conversation, and interruption.
2. **Voice session coordinator** owns conversational focus, recent context, modes, reconnect, and delivery acknowledgement.
3. **Task gateway** maps spoken intent onto the same create, query, steer, pause, resume, and cancel operations used elsewhere in Maka.
4. **Task event projector and notification queue** converts authoritative task events into prioritized, deduplicated, speech-sized updates.
5. **Permission bridge** presents and resolves existing approvals without weakening their scope or identity.

```text
user speech
|
v
realtime voice transport
|
v
voice session coordinator <------> permission bridge
|
v
task gateway <-------------------> background task runtime
^ |
| v
speech notification queue <------ task event projector
|
v
spoken update + durable text result
```

The realtime conversational component may decide how to phrase a response, but task creation, state changes, approvals, and results remain authoritative outside that model.

## Rollout

### Phase 1: one complete voice-controlled task

Support one focused task from creation through clarification, execution, status, steering, cancellation, and spoken result.

Acceptance:

- the user completes the primary flow without a keyboard;
- conversation remains available while the task runs;
- user speech interrupts audio output promptly;
- completion is proactively offered;
- ending Voice does not cancel the task;
- text and Voice show the same task state.

### Phase 2: background assistant behavior

Add background availability, global entry, Quiet mode, durable important-event delivery, reconnect, and application-restart recovery.

Acceptance:

- Voice can be reached while the main window is not focused;
- reconnect never duplicates an accepted task;
- unread important events survive disconnect and restart;
- the user controls when proactive speech is allowed;
- microphone capture never resumes silently.

### Phase 3: multiple tasks and complete control

Add stable spoken task names, focus switching, task listing, multi-task notification coalescing, and voice approval handling.

Acceptance:

- "stop the test task" never affects a different plausible task without clarification;
- approvals remain bound to the correct task;
- multiple completions can be summarized without losing identity;
- ambiguous references trigger a question;
- the user can ask which tasks are running and act on any one of them.

### Phase 4: explicitly authorized environment context

Allow Voice to refer to the current window, selected text, images, or other visible context when the user has explicitly granted that access.

This phase is not required to validate the background-assistant model.

## Acceptance criteria for the first production release

1. A user can create a background task from idle using only voice.
2. The user can clarify, add, or narrow requirements while it runs.
3. The user can query, pause, resume, and cancel the task by voice.
4. The user can resolve a scoped approval by voice without weakening the permission boundary.
5. User speech promptly interrupts Maka's audio output.
6. Blocked, failed, and completed states return proactively when speech is allowed.
7. Ending Voice leaves accepted background work running.
8. Reopening Voice restores a concise, authoritative summary of unfinished work.
9. Network loss or restart does not duplicate a consequential command.
10. Important undelivered events remain available after Quiet mode, interruption, disconnect, or restart.
11. Detailed results remain durable and accessible from the text session.
12. The primary task loop does not require opening the composer or using the keyboard.

## Non-goals

The initial release does not attempt to:

- listen all day by default;
- narrate every background action;
- replace every visual surface;
- make users learn a command language;
- bypass permissions or accept ambiguous high-impact approvals;
- support an unlimited number of simultaneous voice-managed tasks;
- retain full raw call recordings by default;
- capture audio without a persistent status indicator;
- make the voice model an independent source of task truth;
- define provider-specific transport APIs in this RFC;
- prescribe a file-by-file implementation plan.

## Alternatives considered

### Add dictation only

Dictation is useful and should remain available, but it ends after text reaches the composer. It does not solve status, steering, approval, proactive completion, multi-task identity, or recovery.

### Keep Voice as an isolated conversation

This creates a second source of state and makes spoken answers unreliable once background work changes outside the Voice session. Voice should project Maka's existing task authority instead.

### Block the conversation until the task finishes

This is simple but removes the main benefit of a background assistant. Long-running work must not make the interaction surface unavailable.

### Speak every progress event

This produces noise, competes with the user, and exposes internal activity rather than useful outcomes. Durable event collection and selective speech delivery should be separate concerns.

### Cancel work when Voice ends

This conflates microphone privacy with task lifetime. Users need a safe way to stop listening immediately without losing accepted work.

## Open questions

1. What should be the default proactive-notification policy for first-time users: terminal events only, or terminal events plus blockers and approvals?
2. How should Maka generate stable, short spoken task names while avoiding collisions?
3. Which low-risk commands may execute without verbal boundary confirmation?
4. What interruption latency is required for the experience to feel trustworthy on supported hardware?
5. How long should unread terminal events remain eligible for proactive speech after the user returns?
6. Should transcript retention default to session-only or durable text history?
7. Which approval classes, if any, should require visual confirmation in addition to an unambiguous spoken answer?
8. How many simultaneous voice-managed tasks should the first multi-task phase support?
9. When a user asks a new question while a task is focused, what rules decide between ordinary conversation, task steering, and a new task?
10. What is the minimum offline or degraded behavior when realtime voice transport is unavailable but local dictation still works?

## Definition of success

The RFC is successful when Maka can be used as a background voice assistant rather than a speech input accessory:

- the user expresses goals and decisions by voice;
- Maka continues durable work in the background;
- conversation remains available while tasks execute;
- important events return without constant status polling;
- permissions, identity, and recovery remain trustworthy;
- the complete task state is shared with the rest of Maka.

The deciding test is simple: when a user does not want to use the keyboard, they can still use the complete Maka workflow rather than a small voice-only subset.

No implementation files, tests, or entry points are named in this RFC. Start by reading the Voice session, background task, focused task, event, and speech notification queue sections, then trace how existing text-surface task state is represented. Done requires a concrete implementation plan that preserves shared task authority, durable events, approvals, recovery, and keyboard-free interaction.

Written by the indexing model from the issue text.