posit-dev / posit-dev/positron

Experiments fully enabled by the TAS server are not enabled in Positron

Open
#3,465 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: core bug
Dominant language
TypeScript
Stars
4.3k
Forks
184
Avg merge
1d 9h
Merged PRs (30d)
195

Description

VSCode allows A/B experiments in extensions via the vscode-tas-client package. Some experiments are enabled for all users on the TAS server but disabled by default in the client code.

One example is the pythonTestAdapter experiment in the Python extension, which enables a full rewrite of test-related functionality. See https://github.com/microsoft/vscode-python/issues/17242 for a full list of the improvements. I don't believe the legacy test adapter has seen updates so we're effectively running unmaintained code.

Here's a tiny example, I'm sure there are worse side-effects. The legacy test adapter doesn't support colorized outputs:

Legacy:

image

Rewrite:

image

There may be other extensions running into a similar issue. I can think of two ways to deal with this:

  1. Let each extension deal with it separately, and enable any required experiments as needed.
  2. Override the module loader to patch vscode-tas-client to always return a "no-op" experimentation service that operates from a hard-coded list of enabled/disabled experiments (it could extend the TelemetryDisabledExperimentationService).
  3. Host our own TAS server and configure vscode-tas-client to use that.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing how Positron loads extensions and the vscode-tas-client package, including TelemetryDisabledExperimentationService.ts. Compare the three proposed approaches using the pythonTestAdapter experiment as the concrete case; done means experiments enabled by the TAS server work in Positron without requiring each extension to be handled separately.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
developer-experience, testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.