[Feature] Populate [runtimes] in the CI Workbench config so test_runtime_versions runs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8
- Forks
- 1
- Avg merge
- 18h 23m
- Merged PRs (30d)
- 63
Description
Child of #409. Found while measuring the Workbench suite for #539.
workbench/test_runtime_versions.py contributes three tests, and all three skip on every run — not because a Docker Workbench cannot support them, but because the generated config omits one section:
test_r_versions SKIPPED — No expected R versions specified in vip.toml [runtimes]
test_r_version_in_session SKIPPED — No expected R versions specified in vip.toml [runtimes]
test_python_versions SKIPPED — No expected Python versions specified in vip.toml [runtimes]
This is free coverage. workbench-smoke.yml writes vip.toml from scratch, so it can populate [runtimes] with whatever the container actually ships.
Scope
RuntimesConfig (src/vip/config.py:390) takes r_versions and python_versions as lists of version strings. The Workbench image installs runtimes under /opt/R and /opt/python, so the workflow can enumerate them from the running container and write them into the config before pytest starts.
Sketch, to be verified against the real image layout:
R_VERSIONS=$(docker exec "$CID" ls /opt/R 2>/dev/null | jq -R . | jq -sc .)
PY_VERSIONS=$(docker exec "$CID" ls /opt/python 2>/dev/null | jq -R . | jq -sc .)
Then emit a [runtimes] block into the generated vip.toml.
Watch out
Populating [runtimes] from the same container the tests then assert against makes the check partly tautological — it proves the versions Workbench reports in-session match what is installed on disk, which is a real check (test_r_version_in_session runs R and compares), but it cannot catch "the wrong runtimes were installed". That is the correct scope for a container smoke test; pinning expected versions is a customer-deployment concern.
These belong in the full tier at minimum. Whether they are fast enough for the PR gate depends on whether each spawns a session — test_r_version_in_session does, so budget roughly 30s for it.
Acceptance
workbench-smoke.ymlwrites a[runtimes]block derived from the running container.- The three
test_runtime_versionstests execute rather than skip, verified in thesmoke-results.xmlartifact. - The tier placement (gate vs full) is chosen from the measured runtime, and recorded in the workflow.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read workbench-smoke.yml and RuntimesConfig in src/vip/config.py:390, then inspect workbench/test_runtime_versions.py. Verify the /opt/R and /opt/python layout in the running Workbench container, populate the generated vip.toml [runtimes] block, and choose the tier from measured runtime. Done means all three tests run rather than skip and their results appear in smoke-results.xml.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, python
- Domain
- ci-cd, devops, testing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100