posit-dev / posit-dev/positron
Experiments fully enabled by the TAS server are not enabled in Positron
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 4.3k
- Forks
- 184
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 195
Description
VSCode allows A/B experiments in extensions via the vscode-tas-client package. Some experiments are enabled for all users on the TAS server but disabled by default in the client code.
One example is the pythonTestAdapter experiment in the Python extension, which enables a full rewrite of test-related functionality. See https://github.com/microsoft/vscode-python/issues/17242 for a full list of the improvements. I don't believe the legacy test adapter has seen updates so we're effectively running unmaintained code.
Here's a tiny example, I'm sure there are worse side-effects. The legacy test adapter doesn't support colorized outputs:
Legacy:
Rewrite:
There may be other extensions running into a similar issue. I can think of two ways to deal with this:
- Let each extension deal with it separately, and enable any required experiments as needed.
- Override the module loader to patch
vscode-tas-clientto always return a "no-op" experimentation service that operates from a hard-coded list of enabled/disabled experiments (it could extend theTelemetryDisabledExperimentationService). - Host our own TAS server and configure
vscode-tas-clientto use that.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing how Positron loads extensions and the vscode-tas-client package, including TelemetryDisabledExperimentationService.ts. Compare the three proposed approaches using the pythonTestAdapter experiment as the concrete case; done means experiments enabled by the TAS server work in Positron without requiring each extension to be handled separately.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- developer-experience, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100