theupdateframework / theupdateframework/python-tuf

tests: test infrastructure (setup/teardown) improvements

Open
#1,348 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement testing
Dominant language
Python
Stars
1.7k
Forks
304
Avg merge
1d 2h
Merged PRs (30d)
17

Description

(Documenting test improvement ideas from #1346)

A lot (most?) of our tests require a test server (or several). This typically means moving metadata from sources to a temp directory, serving the temp directory and finally removing the temporary directories. Modified_Testcase handles parts of that but it seems obvious that A) it does not provide the required functionality (as tests re-implement a lot of it) and B) that it is hard to use correctly (as #1346 and many previous issues prove). The end result is that there are temp directories created inside tempdirectories and then those directories may or may not get removed (often in a fairly random order)

The common needs are these I think:

  • Test class must be able to serve a repository that all its tests run against
  • test instance must be able to serve it's own repository (because it wants to modify the contents)
  • the served repository paths must be exposed so the content can be modified during test

We could have a better Modified_Testcase that just always runs one test server per class, and provides helpers to "intitialize a new repo" (copy files from sources into a directory inside the served directory). This way both server startup/shutdown and tempdir management would be handled in one place: the actual test classes would only have to remember to chain up Setup*() and *Teardown() -- if they even need those when Modified_Testcase is better.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading Modified_Testcase and the test-infrastructure discussion from #1346 to map the existing setup and teardown patterns. Compare how tests create, serve, expose, and remove repositories and temporary directories. Done means the shared approach covers class- and instance-level repositories without repeated or unreliable cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.