mitodl / mitodl/open-edx-plugins

Package plugin test suites + a [tests] extra so downstream builders can run them (lehrer plugin-regression)

Open
#830 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement project:open-edx
Dominant language
Python
Stars
10
Forks
6
Avg merge
3d 6h
Merged PRs (30d)
11

Description

Summary

Ship each maintained ol-openedx-* plugin's test suite inside its built
distribution
and declare a tests extra bundling its test-only dependencies,
so the tests can be installed and run against a real, already-built Open edX
platform via pip install "<dist>[tests]==<version>".

Why

lehrer added a plugin-regression runner (dagger call platform plugin-regression, lehrer#111)
that runs each installed plugin's own pytest suite inside a built cell image
(the real edx-platform version + the deployment's pinned plugin set + aqueduct
settings). It uses pytest discovery: it runs whatever tests are installed in
the image, resolving packages at runtime.

Today it discovers nothing, because:

  • published plugin wheels/sdists ship no test files (verified: the
    ol-openedx-logging==0.3.5 sdist contains only settings/test.py, no
    tests/), and
  • this monorepo has no per-package tags, so there's no way for a downstream
    builder to fetch "the tests at the exact installed version."

lehrer already installs "<dist>[tests]==<pinned-version>" opportunistically —
which is a safe no-op until the extra exists — so the moment a plugin ships its
tests + a tests extra, its suite starts running in that pipeline with no
lehrer change required
.

What to do (per maintained plugin)

  • Include the plugin's tests/ in the built distribution (package data /
    don't exclude tests), so pip install <dist> lands the tests in
    site-packages where pytest --pyargs <module> can discover them.
  • Declare a tests optional-dependency extra with the test-only deps the
    suite needs (pytest-mock, responses, factory_boy, etc.), matching what
    run_edx_integration_tests.sh installs today.
  • Confirm the suite runs under --ds=lms.envs.test (or the plugin's own
    standalone settings) against an installed edx-platform.

Roll out incrementally — one plugin unblocks real signal for that plugin.

Verify

Once at least one plugin ships the extra, from a lehrer checkout:

lehrer build plugin-regression --cell mit-ol/master/mitxonline \
  --custom-settings ./deployments/mit-ol/settings

should collect and run that plugin's suite (instead of "no tests discovered").

Tracked in the lehrer CI-verification project; companion CI-wiring task is
blocked on this.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by selecting one maintained ol-openedx-* plugin and inspect its packaging metadata, tests/ directory, and run_edx_integration_tests.sh to compare current test dependencies. Build and install that plugin with its tests extra, then run the documented lehrer build plugin-regression command and confirm pytest collects and runs the installed suite against the platform.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.