Allow elements/artifacts/parts of artifacts to not be cached remotely
- Dominant language
- Python
- Stars
- 140
- Forks
- 45
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 6
Description
[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1297)
In GitLab by [[Gitlab user @coldtom]](https://gitlab.com/coldtom) on Apr 28, 2020, 15:27
## Background
[//]: # (Provide a background or the root/source that justifies this task or action.)
The cache is a major benefit to BuildStream, but sometimes it can be an issue when it's used without question. For some motivating use cases:
* Sometimes you have a very large blob in a single element (e.g. a massive debug file), it might be convenient to allow this to not be cached, but cache a filtered version
* Sometimes you may have a single artifact that is very large, for example if composing a disk image. Pushing this to a cache will likely be wasteful and intensive
* A final use case could be that you have certain files that should not end up in the cache due to licensing.
It would also allow for pipeline optimisation, as you may frequently be waiting for certain big artifacts to push.
## Task description
[//]: # (Short summary of the action to be executed)
This is _somewhat_ possible by crafting pipelines and projects a little, but it'd be nice for some way to configure this on an element level, perhaps `push: false` somewhere in configuration? From my understanding, doing this might be a little awkward due to the local caching mechanism, but I may well be wrong there.
Perhaps we could opt these elements out of pushing their artifacts?
Contributor guide
Research direction
No files, tests, or entry points are named; start by reviewing BuildStream’s artifact and remote-cache behavior, then compare the original GitLab issue for context. Define how element-level and partial-artifact opt-outs interact with local caching and pushing, and confirm completion with tests covering each requested scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100