theupdateframework / theupdateframework/python-tuf
ngclient: support `StorageBackendInterface`?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 304
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 17
Description
Description of issue or feature request:
Right now, tuf.ngclient is heavily tied to local system I/O: it assumes a metadata directory on disk that can be read/written. For example:
This is problematic in distributed worker setups like Warehouse (PyPI), where each worker has its own container/entire VM and thus can't easily share on-disk TUF repos. In particular, this causes both reliability and security concerns:
- Reliability: an unfortunate corruption in a single worker's TUF repo results in a hard-to-diagnose flaky worker, since each worker has its own copy of the repo.
- Security: each worker's TUF repo is independently stored on a (machine-local) disk, making them harder to audit.
This problem was noted a few years back, before tuf.ngclient was created: https://github.com/theupdateframework/python-tuf/issues/1009. The solution then was to add a filesystem abstraction to the tuf.metadata APIs, which was done via https://github.com/secure-systems-lab/securesystemslib/pull/232 and https://github.com/theupdateframework/python-tuf/issues/1009. However, this abstraction wasn't added to the ngclient APIs, only to the low-level metadata ones.
Current behavior:
tuf.ngclient currently assumes that it can perform persistent local I/O for its repository.
Expected behavior:
tuf.ngclient should support an I/O abstraction (such as the pre-existing StorageBackendInterface, if suitable) for persistent repo operations, enabling use in distributed deployments.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tuf/ngclient/updater.py around lines 293-312 and compare its persistent repository I/O with the existing StorageBackendInterface used by the low-level metadata APIs. Determine the API and integration scope needed for ngclient, then verify that repository operations can use non-local storage in distributed deployments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100