opensafely-core / opensafely-core/opencodelists

Better validation of correctness for browser functional test states

Open
#2,541 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
60
Forks
16
Avg merge
4d 12h
Merged PRs (30d)
17

Description

Why are we doing this?

#2421 adds some functional browser tests.

One compromise there was to get the tests working by:

  1. The test author specifying a complete series of searches, along with associated concept selections to make; every concept that appears for a search must be made explicitly for each concept
  2. The tests can then apply the searches and complete concept selections via an existing method
  3. The concept selections are used to derive the final expected state of the codelist.

It is helpful that the correct final state is derived from the selections as it saves time and avoids the verbosity of writing this out.

However, this places limitations on the tests we can write:

  • We always have to specify every concept selection. This is regardless of whether it's really needed or not in the selection of interest, and even if it is overridden by a later search.
  • There are tests that we currently cannot write this way, because we always have to specify every selection. (For example, checking that implicitly included selections are correctly included as expected; we cannot do this, because we have to make a decision for every concept.)

What are we doing?

Considering one of the following solutions, and implementing it if possible:

  • Most likely: using a similar approach as the backend does to add searches, and the resulting codes include in those searches. Ideally, we could specify a search and collection of selected concepts as we do now.
  • Specifying the final expected states of the codelist explicitly without computing this at all.
  • Deriving the final state some other way, as we do now.

How will we know when it's done?

When we are able to write tests that:

  • do not need every concept specifying for every search
  • can cover other user interactions, such as implicit inclusion of concepts

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing issue #2421 and the existing functional browser test state setup. Compare the three proposed ways of deriving or specifying final codelist state, then verify that the chosen approach supports tests without every concept being specified and covers implicit inclusion interactions. Done means those test cases can be written and the resulting state is validated correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.