apache / apache/buildstream

The local cache quota is actually a CAS cache quota

Open
#1,022 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1022)
In GitLab by [[Gitlab user @jennis]](https://gitlab.com/jennis) on May 10, 2019, 10:36

## Summary
To set a local cache quota we need to specify something like this in our user configuration (typically at `~/.config/buildstream.conf`):

```yaml
cache:
quota: 100G
```

Now, having read our [documentation](https://buildstream.gitlab.io/buildstream/using_config.html#local-cache-expiry) regarding the cache quota, one would expect that this is a quota for the *entire* local cache, that is (typically) located at `~/.cache/buildstream`. However, it seems to me that our quota is *actually* a quota on `~/.cache/buildstream/cas`, the cas directory of the local cache.

## Discovery
* To discover this, I have two stack elements, which, when built *separately*, each leave `~/.cache/buildsteam` at 10G.
* I built the first stack element: `du -sh ~/.cache/buildstream` returns 11G
* I then set a cache quota to 18G
* I then built the second stack element (expecting a cleanup job), `du -sh ~/.cache/buildstream` returns 20G. And there was no cleanup job.
* `du -sh ~/.cache/buildstream/cas` returns 16G.

From looking at our [CASQuota class](https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_cas/cascache.py#L1135), it seems like the quota is *actually* for the local *CAS* cache, not the local cache.

In my opinion, this is a bug, naively, one would expect that a cache quota means that your entire cache should not exceed this quota. If it is intentional, then we need to make this clear in the documentation. If not, we need to ensure that the quota is applied to the *entire* local cache.

Contributor guide

Open the contributing guide

Research direction

Start with the CASQuota class in buildstream/_cas/cascache.py and the local-cache-expiry section of the configuration documentation. Reproduce the reported quota behavior using separate stack-element builds and compare ~/.cache/buildstream with its cas directory. Done means either the quota covers the entire local cache or the documentation clearly states that it only covers the CAS cache.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.