microsoft / microsoft/playwright-dotnet

[Bug]: LaunchOptions from .runsettings are not applied when running tests via Visual Studio Test Explorer

Open
#3,287 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
3k
Forks
304
Avg merge
20h 47m
Merged PRs (30d)
6

Description

Version

1.58.0

Steps to reproduce
  1. Clone my repo at https://github.com/saalmi098/PlaywrightSlowMoRepro

  2. Test via command line (SlowMo works):

    dotnet test
    

    Expected: Browser opens with visible 2-second delays between each action
    Result: SlowMo works correctly

  3. Test via Visual Studio (SlowMo doesn't work):

    • Open PlaywrightSlowMoRepro.sln in Visual Studio
    • Open Test Explorer View
    • Run the existing test

    Expected: Browser opens with visible 2-second delays between each action
    Result: SlowMo is ignored - actions execute at normal speed

Expected behavior

The LaunchOptions (including SlowMo, Headless etc.) configured in the .runsettings file should be applied when running tests through Visual Studio Test Explorer, the same way they are applied when running via dotnet test.

Actual behavior
  • The .runsettings file is detected and loaded by Visual Studio
  • Environment variables from <RunConfiguration>/<EnvironmentVariables> in .runsettings work correctly in both VS Test Explorer and dotnet test
  • Playwright-specific settings from the <Playwright> section (like LaunchOptions) are completely ignored when running via VS Test Explorer --> no SlowMo delay between browser actions
  • The same .runsettings file works perfectly with dotnet test --> SlowMo delay works fine

Visual Studio Test Output confirms the runsettings are loaded:

Using runsettings file C:\Workspaces\Git\PlaywrightSlowMoRepro\tests.runsettings
Tests run settings for TestAssembly.dll:
<RunSettings>
  <Playwright>
    <BrowserName>chromium</BrowserName>
    <LaunchOptions>
      <SlowMo>1000</SlowMo>
      <Headless>false</Headless>
    </LaunchOptions>
  </Playwright>
</RunSettings>

Despite the settings being present in the processed runsettings, the browser launches at normal speed in VS Test Explorer.

Additional context

Looking at the PlaywrightSettingsProvider implementation, it implements ISettingsProvider and registers itself with:

[ExtensionUri("settings://playwright")]
[SettingsName("Playwright")]
public class PlaywrightSettingsProvider : ISettingsProvider
{    
    public static void LoadViaEnvIfNeeded()
    {
        if (_settings == null)
        {
            var settings = Environment.GetEnvironmentVariable("PW_INTERNAL_ADAPTER_SETTINGS");
            if (!string.IsNullOrEmpty(settings))
            {
                _settings = JsonSerializer.Deserialize<PlaywrightSettingsXml>(settings);
            }
            else
            {
                _settings = new PlaywrightSettingsXml();
            }
        }
    }

    public void Load(XmlReader reader)
    {
        _settings = new PlaywrightSettingsXml(reader);
        Environment.SetEnvironmentVariable("PW_INTERNAL_ADAPTER_SETTINGS", JsonSerializer.Serialize(_settings));
    }
}

The Load(XmlReader reader) method appears not to be invoked when running through Visual Studio's test platform, while it works correctly in the dotnet test execution path. This suggests the settings provider registration may not be properly recognized by Visual Studio's test adapter infrastructure.

Is there a specific registration step required for ISettingsProvider implementations to work with Visual Studio Test Explorer, or is this a known limitation with the xUnit v3 adapter integration?

Environment
- Operating System: Windows 11
- CPU: x64
- Browser: Chromium
- .NET Version (TFM): net10.0
- Visual Studio Professional 2026: v18.4.0
- Packages
  - Microsoft.Playwright: v1.58.0
  - Microsoft.Playwright.Xunit.v3: v1.58.0
  - xunit.v3: v3.2.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the difference between dotnet test and Visual Studio Test Explorer using the linked PlaywrightSlowMoRepro project and its .runsettings file. Inspect the PlaywrightSettingsProvider implementation, especially Load(XmlReader reader) and its ISettingsProvider registration, then compare the two execution paths. Done means Playwright LaunchOptions such as SlowMo and Headless are applied consistently in Visual Studio Test Explorer and dotnet test.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.