bottlerocket-os / bottlerocket-os/twoliter
Smarter caching for kit OCI artifacts
- Dominant language
- Rust
- Stars
- 34
- Forks
- 43
- Avg merge
- 11h 13m
- Merged PRs (30d)
- 16
Description
### What's the problem
Sometimes the caching mechanism used for kits gets in the way of development and validation workflows.
I've noted that `cargo` will skip the build of a binary even when the compilation environment changed (e.g. if a different subset of flags were set). This is misleading since I would think that my new compilation environment is correct, to later on learn that a cached layer with an older binary was actually used in the build instead of my new binary.
To circumvent this problem, and fully trust that the binaries are compiled with the new configured environment, I have to delete `build/*`, `.cargo/*` and the `docker` cache. This experience isn't ideal if the only thing I want to test is a one-liner change in the flags used by the compilers. Additionally, nuking the `docker` cache impacts other workflows like building the Bottlerocket SDK since its builds are somewhat expensive.
I haven't been able to reproduce, but sometimes the wrong layers are used to put together a kit. This already impacted Bottlerocket releases (aka 1.24.0), and again nuking the `docker` cache slows down the development workflow.
It would be nice if `twoliter` could invalidate the `docker` cache but only for certain layers, so that we don't have to nuke the entire docker cache. Even nicer would be to detect which build would use artifacts from the cache (e.g. while building a package, Twoliter could decide to use or invalidate the cache depending on user's inputs).
Contributor guide
Research direction
No file, test, or entry point is named. Start by tracing twoliter's kit build and caching workflow, including cargo artifacts and Docker layers. Define done as invalidating only cache layers affected by changed build inputs while preserving unrelated Docker cache entries and validating that kits do not reuse stale artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100