openedx / openedx/openedx-platform
Run tests on all shared folders using CMS settings
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.2k
- Forks
- 4.4k
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 42
Description
Background
We run lms/ tests with LMS settings and cms/ tests with CMS settings, as you'd imagine.
We intend to tests for all "shared" folders (xmodule/, openedx/, common/) under both LMS and CMS settings. However, it seems that with our current .github/workflows/unit-test-shards.json setup, we don't run the following tests with CMS settings (rationale unclear):
openedx/core/djangoapps/course_live/
openedx/core/djangoapps/notifications/
openedx/core/djangolib/
openedx/core/tests/
openedx/features/
openedx/testing/
This was discovered while rebalancing the test shards.
Please note: We also use the@skip_unless_lms in various "shared" tests. That's a much better way to skip test for CMS settings, because it's explicit and it's obvious to the developers who are working on the tests.
Tasks
- Make it so for every
shared-with-lms-Ntest shard, there's a correspondingshared-with-cms-Ntest shard with identical folders. - For every shared test that fails under CMS settings, do one of the following:
- Fix the test
- Move the folder to LMS to indicate that it's not a "shared" test
- Skip the test function or class using the
@skip_unless_lmsdecorator
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .github/workflows/unit-test-shards.json and compare each shared-with-lms-N shard with its CMS counterpart, including the listed openedx/ folders. Run the affected shared tests under CMS settings and track failures; done means every LMS shared shard has an identical CMS shard and each failure is fixed, moved to LMS, or explicitly skipped with @skip_unless_lms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100