apache / apache/buildstream

Local cache configuration is unclear and error prone

Open
#866 7 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/866)
In GitLab by [[Gitlab user @BenjaminSchubert]](https://gitlab.com/BenjaminSchubert) on Jan 15, 2019, 18:24

## Summary

[//]: # (Summarize the bug encountered concisely)

When setting a cache quota for the local cache, the quota doesn't take into account the sources' size.

I don't know what are the size of my sources and that means I need to remove them periodically to ensure I don't get an error with the cache.

## Steps to reproduce

[//]: # (How one can reproduce the issue - this is very important)
- Create a cache
- add {cache: {quota: 1MB}}
- Create an element that has a source that will exceed this size, but has no output artifacts
- bst build
- bst build

## What is the current bug behavior?

[//]: # (What actually happens)
BuildStream fails initializing the cache, because it doesn't have enough space

Error instantiating artifact cache: Your system does not have enough available space to support the cache quota specified.
You have specified a quota of 95% total disk space.
- The filesystem containing /home/bschubert/.cache/buildstream/artifacts only has: 49.2% of total disk space available.

## What is the expected correct behavior?

[//]: # (What you should see instead)

I would expect the size to be the total size of the cache, not just the artifacts.
I would also expect dangling sources to be removed when cleaning up the cache.

## Relevant logs and/or screenshots

[//]: # (Paste any relevant logs - please use code blocks ``` to format console output, logs, and code as it's hard to read otherwise. You can also add a Snippet and link it here. Check the markdown giude on Gitlab for further tips)

## Possible fixes

[//]: # (If you can, link to the line of code that might be responsible for the problem)

## Other relevant information

[//]: # (Add the BuildStream version. Substitute below the "x" by "1", "2" or the right version. Add more than one version if necessary)

* BuildStream version affected: /milestone %BuildStream_v1.x

----

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the configuration and repeated `bst build` steps described in the issue, then trace the local cache quota and cleanup behavior. Done means the quota accounts for source storage and cleanup removes dangling sources without the cache initialization failure described in the logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.