apache / apache/buildstream

Source files are read-only hard links with buildbox-run

Open
#1,218 26 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1218)
In GitLab by [[Gitlab user @juergbi]](https://gitlab.com/juergbi) on Nov 26, 2019, 09:36

With buildbox-run without FUSE, not only staged dependencies but also source files in the sandbox are hardlinked from CAS. This causes the following issues:
* Source files are read-only. This breaks a few tests and may break build commands or build systems that modify source files.
* The mtime of the source files is the mtime of the corresponding file in buildbox-casd's local cache. This breaks a few tests and may break reproducibility as the mtime may differ without affecting the cache key.

buildbox-run without FUSE is currently used either with the experimental buildbox-run sandboxing backend (#1177) or with remote execution if the server uses a buildbox worker.

A possible solution is to set the MTime NodeProperty to BuildStream's 'magic' timestamp (2011-11-11 11:11:11). This will require support by buildbox-casd to create copies with the requested mtime instead of hardlinks if the MTime NodeProperty is set. Support for this will anyway be required for incremental workspace builds.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the buildbox-run path used without FUSE, then inspect how buildbox-casd handles the MTime NodeProperty and hardlinks from the CAS. Done means sandbox source files are writable and receive BuildStream's magic timestamp without changing the cache key; check the affected tests and incremental workspace build behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.