posit-dev / posit-dev/vip

[Feature] Populate [runtimes] in the CI Workbench config so test_runtime_versions runs

Open
#550 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
8
Forks
1
Avg merge
18h 23m
Merged PRs (30d)
63

Description

Child of #409. Found while measuring the Workbench suite for #539.

workbench/test_runtime_versions.py contributes three tests, and all three skip on every run — not because a Docker Workbench cannot support them, but because the generated config omits one section:

test_r_versions          SKIPPED — No expected R versions specified in vip.toml [runtimes]
test_r_version_in_session SKIPPED — No expected R versions specified in vip.toml [runtimes]
test_python_versions     SKIPPED — No expected Python versions specified in vip.toml [runtimes]

This is free coverage. workbench-smoke.yml writes vip.toml from scratch, so it can populate [runtimes] with whatever the container actually ships.

Scope

RuntimesConfig (src/vip/config.py:390) takes r_versions and python_versions as lists of version strings. The Workbench image installs runtimes under /opt/R and /opt/python, so the workflow can enumerate them from the running container and write them into the config before pytest starts.

Sketch, to be verified against the real image layout:

R_VERSIONS=$(docker exec "$CID" ls /opt/R 2>/dev/null | jq -R . | jq -sc .)
PY_VERSIONS=$(docker exec "$CID" ls /opt/python 2>/dev/null | jq -R . | jq -sc .)

Then emit a [runtimes] block into the generated vip.toml.

Watch out

Populating [runtimes] from the same container the tests then assert against makes the check partly tautological — it proves the versions Workbench reports in-session match what is installed on disk, which is a real check (test_r_version_in_session runs R and compares), but it cannot catch "the wrong runtimes were installed". That is the correct scope for a container smoke test; pinning expected versions is a customer-deployment concern.

These belong in the full tier at minimum. Whether they are fast enough for the PR gate depends on whether each spawns a session — test_r_version_in_session does, so budget roughly 30s for it.

Acceptance

  • workbench-smoke.yml writes a [runtimes] block derived from the running container.
  • The three test_runtime_versions tests execute rather than skip, verified in the smoke-results.xml artifact.
  • The tier placement (gate vs full) is chosen from the measured runtime, and recorded in the workflow.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read workbench-smoke.yml and RuntimesConfig in src/vip/config.py:390, then inspect workbench/test_runtime_versions.py. Verify the /opt/R and /opt/python layout in the running Workbench container, populate the generated vip.toml [runtimes] block, and choose the tier from measured runtime. Done means all three tests run rather than skip and their results appear in smoke-results.xml.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, python
Domain
ci-cd, devops, testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.