microsoft / microsoft/playwright-dotnet

[Bug]: Playwright MSTest.Sdk Uses MSTest Runner, Ignoring runsettings Configuration / Microsoft.Testing.Platform

Open
#3,124 10 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P3-collecting-feedback
Dominant language
C#
Stars
3k
Forks
304
Avg merge
20h 47m
Merged PRs (30d)
6

Description

Version

1.50.0

Steps to reproduce

When using Playwright version 1.50.0 with MSTest.Sdk version 3.8.1, the MSTest runner is used instead of the expected VSTest runner. As a result, the Playwright configuration section in the runsettings file is ignored. Key settings such as BrowserName, ExpectedTimeout, Headless, and Channel do not work.

However, when <UseVSTest>true</UseVSTest> is explicitly set in the .csproj file, and the VSTest runner is used, the Playwright settings in the runsettings file work as expected.

Expected behavior
  • Playwright should respect the runsettings file when running tests with MSTest.Sdk.
  • Playwright-specific settings such as BrowserName, ExpectedTimeout, Headless, and Channel should be applied correctly, regardless of whether MSTest or VSTest is used.
Actual behavior
  • With MSTest runner, the Playwright settings in runsettings are ignored.
  • With VSTest runner (<UseVSTest>true</UseVSTest>), the settings work as expected.
Additional context

This issue suggests that MSTest.Sdk is defaulting to the MSTest runner instead of the VSTest runner, which does not properly load the Playwright settings from runsettings. It would be helpful if Playwright could support runsettings consistently across different test runners.

Would appreciate any insights or fixes on this issue. Thanks!

Environment
- Playwright Version: 1.50.0
- MSTest.Sdk Version: 3.8.1
- .NET Version: net8.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with Playwright 1.50.0, MSTest.Sdk 3.8.1, and net8.0 using the .csproj and runsettings configuration described. Compare behavior with and without true, then trace how Playwright-specific settings are loaded under each runner. Done means BrowserName, ExpectedTimeout, Headless, and Channel are applied consistently with MSTest and VSTest.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.