posit-dev / posit-dev/vip

[Feature] Let the Workbench smoke suite run in parallel again

Open
#644 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement team: workbench
Dominant language
Python
Stars
8
Forks
1
Avg merge
18h 23m
Merged PRs (30d)
63

Description

workbench-smoke.yml:376 runs the suite with -n 0, disabling the -n auto --dist loadgroup default from pyproject.toml. The nightly's test step is the dominant cost of that workflow, measured between 475 and 686 seconds against a 14.3 minute median for the whole run, so restoring parallelism here is the largest single wall-clock win available in CI.

The comment above it explains the choice: the Docker Workbench returns "Temporary server error" when multiple workers attempt concurrent logins. That reasoning is now only half true. The -n 0 dates from the change that first enabled xdist. A cross-worker login lock landed later, in the change titled "parallelize tests instead of forcing serial", which is exactly the problem the comment describes.

The catch is that the lock does not cover this workflow's login path. oidc_login_lock is applied only inside _silent_sso_signin in src/vip_tests/workbench/conftest.py, which is the OIDC round-trip, while the smoke writes provider = "password" into its generated vip.toml. So the password login the smoke actually performs is genuinely unserialized, and simply deleting -n 0 today would likely reproduce the original error.

Suggested order of work: extend the existing lock to cover the password login path as well as the OIDC one, confirm on a dispatch run with the full tier that concurrent logins no longer error, and only then remove -n 0 and update the comment. Doing it in that order keeps a green nightly at every step.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with workbench-smoke.yml:376 and src/vip_tests/workbench/conftest.py, especially oidc_login_lock and _silent_sso_signin. Trace the generated vip.toml password login path, then use a full-tier dispatch run to verify concurrent logins before removing -n 0 and updating the comment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.