microsoft / microsoft/vscode-telemetry-extractor

Make duplicate event extraction deterministic

Open
#69 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
10
Forks
19
Avg merge
10d 8h
Merged PRs (30d)
2

Description

Problem

Duplicate telemetry event declarations can produce different telemetry.json output from identical source trees.

The extractor intentionally allows duplicate declarations to contribute disjoint properties. However, when overlapping declarations differ in emitted metadata such as event owner/comment or property comments, extraction currently succeeds and whichever file ripgrep returns first supplies the output. Ripgrep searches in parallel and does not guarantee result order, so repeated builds can select different declarations.

Why the duplicate check does not reject this

The current conflict signature projects each property down to only classification and purpose. The validator then compares only overlapping projected properties. This ignores output-relevant details including:

  • event owner, comment, and expiration
  • property comments, owners, expiration, endpoint, measurement, and column metadata
  • $tableInfo
  • properties present in only one compatible declaration

The parser also keeps a seenEvents set and discards every declaration after the first one. Consequently, declarations with identical privacy classifications but different emitted schemas pass validation, and the first ripgrep result wins.

Minimal reproduction

Create two source files declaring the same event. Give an overlapping property the same classification and purpose, but a different comment; optionally add another property to just one declaration. Run the extractor repeatedly while varying discovery order.

Expected:

  • overlapping output fields with different values are reported as a duplicate conflict
  • compatible declarations with disjoint fields are merged
  • file discovery is deterministic without disabling ripgrep's parallel traversal

Actual:

  • no conflict is reported
  • later compatible fields are discarded
  • output depends on ripgrep result order

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the parser's seenEvents handling, duplicate-conflict signature, and the ripgrep results that feed telemetry.json. Run the minimal reproduction repeatedly while varying discovery order. Done means overlapping output differences are reported as conflicts, disjoint fields are merged, and repeated extraction produces deterministic output.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.