apache / apache/buildstream

Incomplete sources with remote execution

Open
#1,066 0 comments 0 reactions 0 assignees View on GitHub
optimization
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1066)
In GitLab by [[Gitlab user @raoul]](https://gitlab.com/raoul).hidalgocharman on Jul 5, 2019, 14:00

## Background

[//]: # (Provide a background or the root/source that justifies this task or action.)
Follow up to #1043, sources also don't need to be complete in remote execution but this requires more work to get functional.

## Task description

[//]: # (Short summary of the action to be executed)

* [ ] Add argument to source cache pull that only fetches the directory tree
* [ ] Figure out which scenarios we only want the directory tree and use this when pulling sources from a source cache
* [ ] Ensure that commands such as source checkout fail cleanly and maybe have a `--pull` option or similar
* [ ] Add tests for this.

## Acceptance Criteria

[//]: # (Acceptance criteria should follow the S.M.A.R.T. principle https://en.wikipedia.org/wiki/SMART_criteria )
With sources in a remote source cache, we should be able to build elements without having to pull source files locally.

----
[//]: # (To review information about possible relevant labels for this issue please view the list of labels: https://gitlab.com/BuildStream/buildstream/labels)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the source cache pull path and the remote execution scenarios described in the task checklist. Identify which cases need only the directory tree, verify how source checkout behaves with incomplete sources, and add tests covering builds from a remote source cache without local source files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.