opensafely-core / opensafely-core/opencodelists

Make better use of fixtures in the functional tests

Open
#2,543 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
60
Forks
16
Avg merge
4d 12h
Merged PRs (30d)
17

Description

Why are we doing this?

Note that this issue depends on #2403 and #2523: the decisions there may help solve this problem.

#2421 adds functional browser tests.

These use several pytest fixtures. Some of these were created specifically for the tests (in the conftest.py in /tests/functional/conftest.py). Others are those from the "universe" of fixtures.

This has some compromises:

  • The "universe" is only created once, so we need to use a workaround to change the fixture scope to avoid the coding system data getting wiped between test runs
  • The tests depend on the entire test "universe" when we need a smaller subset of this. This makes it difficult to fully understand the fixture dependency graph for the functional tests: the top-level conftest.py imports everything from the opencodelists/tests/fixtures.py.

The alternative would have been a clean break, duplicating the necessary fixtures, and using pytest's confcutdir option to only use the fixtures stored in the functional test directory. But this would require extra configuration duplicating as the top-level, global conftest.py does some other required database setup.

How will we know when it's done?

  • When the dependencies of the functional tests are better understood and specified
  • When we no longer need to toggle the fixture scope of the fixtures in opencodelists/tests/fixtures.py with the get_fixture_scope() function
  • When we are using a much smaller number of fixtures than we are now for the functional tests.

What are we doing?

Considering improvements to the use of fixtures. Two approaches might be to use

  • more isolated fixtures from the main fixtures available in the top-level conftest.py
  • specifically created fixtures for just these tests

There's also some possible duplication in opencodelists/tests/conftest.py with the way that users are currently created; the organization and non-organization users are created in essentially the same way, but just duplicated because they are very slightly different fixtures.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading /tests/functional/conftest.py, the top-level conftest.py, opencodelists/tests/fixtures.py, and opencodelists/tests/conftest.py, taking account of decisions in #2403 and #2523. Trace the functional tests' fixture dependencies and identify a smaller, explicit setup; this is done when get_fixture_scope() is no longer needed and the functional tests use fewer fixtures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.