Initializing the remote caches as needed
- Dominant language
- Python
- Stars
- 140
- Forks
- 45
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 6
Description
Currently, remote caches are initialized pretty early in the buildstream start sequence. At least for `checkout` operations, this is often too early to know whether the caches are actually needed.
There are a few situations:
* target directory is not empty: Loading elements -> Resolving elements -> Initializing remote caches -> target directory is not empty
* this wastes time before reporting the error
* artifacts are cached locally: Loading elements -> Resolving elements -> Initializing remote caches -> Query cache -> Staging dependencies -> Integrating sandbox -> Checking out files
* We don't need remote caches here, so it also wastes time
* artifacts aren't cached: Loading elements -> Resolving elements -> Initializing remote caches -> Query cache -> Attempting to fetch missing or incomplete artifact(s) ...
* This is the case that needs initializing remote caches. However it makes more sense to do it after the "Attempting to fetch", not before.
Contributor guide
Research direction
Start by tracing the checkout startup sequence and the point where remote caches are initialized. Compare the listed loading, resolving, cache-query, fetch, and checkout steps; done means remote caches are initialized only when a missing or incomplete artifact must be fetched, not before target validation or local-cache use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100