microsoft / microsoft/AI-Engineering-Coach

feat: parse GitHub.copilot-chat/transcripts/*.jsonl event-stream format

Open
#64 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
4.2k
Forks
585
Avg merge
22h 5m
Merged PRs (30d)
16

Description

Summary

VS Code with the GitHub Copilot Chat extension writes session data in two distinct locations:

  1. workspaceStorage/<hash>/GitHub.copilot-chat/ — the existing format already parsed by the extension (per-session JSON blobs).
  2. GitHub.copilot-chat/transcripts/*.jsonl — a newer event-stream format written at the user-data level, alongside the workspace storage directory.

The extension currently only reads format 1. Format 2 is silently ignored, so sessions recorded there never appear in the dashboard.

The transcript format

Each .jsonl file is a newline-delimited stream of typed events:

type Meaning
session.start New conversation begins
user.message User turn
assistant.message AI response
tool.execution_start Tool call begins (includes toolName)
tool.execution_complete Tool call finishes

Who is affected

  • WSL / VS Code Server / Remote SSH / Dev Container users are most likely to encounter this because their workspaceStorage path is nested under ~/.vscode-server/ (fixed in #63), but the transcripts/ directory sits at ~/.vscode-server/data/User/GitHub.copilot-chat/transcripts/.
  • Desktop users writing to the standard user-data directory are equally affected if their sessions happen to be stored in the newer format.

Proposed solution

Add two functions to parser-vscode.ts:

  • listTranscriptFiles(dir: string): string[] — finds all *.jsonl files under a transcripts/ subfolder.
  • parseTranscriptFile(filePath: string): SessionData | null — reads the event stream and returns a SessionData object (same shape the rest of the pipeline expects), grouping events into turns and deduplicating tool names.

Wire both into the existing processWorkspaceEntry / processWorkspaceEntryAsync functions so the dashboard picks up sessions from either format transparently.

Acceptance criteria

  • parseTranscriptFile correctly maps event-stream turns to SessionData.requests[]
  • Tool names are deduplicated per turn
  • Empty/malformed files return null gracefully
  • Existing tests remain green
  • New unit tests cover: multi-turn grouping, empty session, tool deduplication, and full flow

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in parser-vscode.ts and trace processWorkspaceEntry and processWorkspaceEntryAsync to understand how existing workspace sessions become SessionData. Add transcript discovery and event-stream parsing there, then verify multi-turn grouping, empty or malformed files, per-turn tool deduplication, and the full flow with new unit tests while keeping existing tests green.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
developer-experience, tooling
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.