feat: 外部物理控制器与语音触发的扩展点 / Extension point for external physical controllers & voice triggers
- Dominant language
- TypeScript
- Stars
- 2.7k
- Forks
- 395
- Avg merge
- 21h 48m
- Merged PRs (30d)
- 776
Description
大家好,我是 ReAI 的 Kang,我们在做 AI-Board——一块专为 Claude Code / Codex 用户设计的实体键盘(拨杆 + 可编程按键 + 麦克风,USB/蓝牙双模,Rust SDK)。
看到 Cindy 把 Claude Code 和 Codex 作为 harness 编排,且官网把 Voice 列为规划中的扩展能力,想提一个功能方向:**为外部物理设备开一个受控的扩展点**,让硬件可以:
1. **任务控制**:物理按键触发「确认 / 打断 / 继续」当前 Agent 任务——批任务时不用切窗口找按钮;
2. **模式切换**:拨杆映射 harness 的权限/模式状态(我们已为 Claude Code 的 Shift+Tab 模式环做过完整状态机,含从 hook 回读校正);
3. **语音入口**:PTT 按键 + 统一 16kHz 音频流,为将来的 Voice 能力提供物理触发和拾音链路。
我们能投入的:
- 已有的 Claude Code 联动实现可作参考(拨杆状态机、串行注入 worker、焦点门禁);
- Rust SDK 处理 USB/蓝牙连接、按键与模式事件、标准化音频,接入方不用碰设备层;
- 愿意基于你们的扩展机制写参考实现并提 PR,也可以提供样机给核心团队测试。
想先确认两个问题:
1. 这个方向是否符合你们的 roadmap(尤其 Voice 和插件市场的规划)?
2. 扩展点的形态你们倾向哪种——MCP 工具、插件市场里的 plugin,还是独立的本地 API?
如果方向不合适,直说即可,不会占用大家太多时间。
---
Hi, I'm Kang from ReAI. We build AI-Board — a physical keyboard designed for Claude Code / Codex users (mode dial + programmable keys + microphone, USB/Bluetooth, Rust SDK).
Since Cindy orchestrates Claude Code and Codex as harnesses, and Voice is listed as a planned capability on cindy.app, I'd like to propose **a controlled extension point for external physical devices**, enabling:
1. **Task control**: physical keys for confirm / interrupt / continue on the running agent task — no window-switching during task review;
2. **Mode switching**: a dial mapped to harness permission/mode states (we've built a full state machine for Claude Code's Shift+Tab mode cycle, including hook-based read-back correction);
3. **Voice entry**: a push-to-talk key plus a normalized 16kHz audio stream, providing the physical trigger and capture pipeline for the future Voice capability.
What we can contribute: our existing Claude Code integration as reference (dial state machine, serialized injection worker, focus gating); a Rust SDK that handles USB/BT connectivity, key/mode events, and normalized audio so integrators never touch the device layer; and we're happy to build a reference implementation as a PR once the extension mechanism lands — plus sample units for the core team.
Two questions first:
1. Does this direction fit your roadmap (especially Voice and the plugin marketplace)?
2. Which shape do you prefer for the extension point — an MCP tool, a marketplace plugin, or a standalone local API?
If this isn't a fit, a direct "not now" is totally fine.
Contributor guide
Research direction
No repository files, tests, or entry points are identified; start by reviewing the project roadmap and any existing extension or plugin entry points before scoping the proposal. Done would require an agreed extension shape, explicit boundaries for controller and voice integration, and defined acceptance criteria; this issue currently asks for direction rather than specifying a patch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, typescript
- Domain
- ai, api, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100