mitodl / mitodl/open-edx-plugins
Package plugin test suites + a [tests] extra so downstream builders can run them (lehrer plugin-regression)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10
- Forks
- 6
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 11
Description
Summary
Ship each maintained ol-openedx-* plugin's test suite inside its built
distribution and declare a tests extra bundling its test-only dependencies,
so the tests can be installed and run against a real, already-built Open edX
platform via pip install "<dist>[tests]==<version>".
Why
lehrer added a plugin-regression runner (dagger call platform plugin-regression, lehrer#111)
that runs each installed plugin's own pytest suite inside a built cell image
(the real edx-platform version + the deployment's pinned plugin set + aqueduct
settings). It uses pytest discovery: it runs whatever tests are installed in
the image, resolving packages at runtime.
Today it discovers nothing, because:
- published plugin wheels/sdists ship no test files (verified: the
ol-openedx-logging==0.3.5sdist contains onlysettings/test.py, no
tests/), and - this monorepo has no per-package tags, so there's no way for a downstream
builder to fetch "the tests at the exact installed version."
lehrer already installs "<dist>[tests]==<pinned-version>" opportunistically —
which is a safe no-op until the extra exists — so the moment a plugin ships its
tests + a tests extra, its suite starts running in that pipeline with no
lehrer change required.
What to do (per maintained plugin)
- Include the plugin's
tests/in the built distribution (package data /
don't exclude tests), sopip install <dist>lands the tests in
site-packages wherepytest --pyargs <module>can discover them. - Declare a
testsoptional-dependency extra with the test-only deps the
suite needs (pytest-mock, responses, factory_boy, etc.), matching what
run_edx_integration_tests.shinstalls today. - Confirm the suite runs under
--ds=lms.envs.test(or the plugin's own
standalone settings) against an installed edx-platform.
Roll out incrementally — one plugin unblocks real signal for that plugin.
Verify
Once at least one plugin ships the extra, from a lehrer checkout:
lehrer build plugin-regression --cell mit-ol/master/mitxonline \
--custom-settings ./deployments/mit-ol/settings
should collect and run that plugin's suite (instead of "no tests discovered").
Tracked in the lehrer CI-verification project; companion CI-wiring task is
blocked on this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by selecting one maintained ol-openedx-* plugin and inspect its packaging metadata, tests/ directory, and run_edx_integration_tests.sh to compare current test dependencies. Build and install that plugin with its tests extra, then run the documented lehrer build plugin-regression command and confirm pytest collects and runs the installed suite against the platform.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100