microsoft / microsoft/PowerToys

Typeless-style AI Voice-to-Structured Input for PowerToys

Open
#48,949 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Needs-Triage Product-PowerToys Run
Dominant language
C
Stars
139k
Forks
8.6k
PR merge metrics
PR metrics pending

Description

### Description of the new feature / enhancement

What is the expected behavior of the proposed feature?

Introduce a Typeless-like AI voice input system in PowerToys that converts natural speech directly into structured, actionable content instead of plain transcription.

Inspired by tools like Typeless, the feature should:

Convert continuous speech into structured outputs (e.g., GitHub issue format, notes, commands)
Automatically infer structure such as:
Title
Description
Steps to reproduce
Expected behavior
Allow natural “edit by voice” commands:
“make that shorter”
“replace the last sentence”
“add a bullet point”
Adapt formatting based on context (e.g., issue creation vs PowerToys Run input)
Provide a real-time draft preview before submission
Support interruption-free dictation (no need to manually switch fields)

The key idea is not transcription, but intent-to-structured-output transformation.

### Scenario when this would be used?

This feature would be used when:

Quickly creating GitHub issues without typing
Capturing bugs or ideas while coding or debugging
Multitasking (e.g., during meetings, screen sharing, or live development)
Users who prefer voice-first interaction over keyboard input
Accessibility scenarios where typing is slow or inconvenient

Why this matters for power users:

Removes friction of filling structured forms manually
Speeds up bug reporting and idea capture dramatically
Keeps developers in flow state without context switching
Improves accessibility and reduces cognitive load
Makes structured input feel conversational instead of mechanical

This turns issue creation into a hands-free, voice-driven drafting workflow.

### Supporting information

Tools like Typeless demonstrate a shift in interaction design:

Instead of:

Speaking → getting raw text → manually editing

They enable:

Speaking → AI interprets intent → structured, formatted output

Key advantages of this approach:

Higher-level semantic understanding (not just speech-to-text)
Automatic formatting into useful structures
Faster idea capture with minimal editing
Better support for “stream of thought” input
Reduces friction in professional workflows

For PowerToys, this would align with modern AI-native input paradigms while being tailored specifically for developer workflows (issues, commands, productivity tools).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or existing entry points. Start by reviewing the PowerToys architecture and locating the utilities responsible for voice or structured input, then determine how the proposed workflow would fit them. Done would require a defined scope, supported contexts, voice-edit behavior, draft preview, and acceptance tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
accessibility, ai, desktop
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.