apache / apache/buildstream

We need to think about temporary vs. permanent failures in source mirroring

Open
#522 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/522)
In GitLab by [[Gitlab user @jonathanmaw]](https://gitlab.com/jonathanmaw) on Jul 27, 2018, 13:38

## Background

[//]: # (Provide a background or the root/source that justifies this task or action.)

The Source Mirroring implementation currently treats any kind of BstError raised in `Source.fetch()`, `Source.track()` or SourceFetcher.fetch()` as non-critical, and only if all fetch/track calls fail does it raise the last exception.
This is going to have unexpected behaviour when the final call was a temporary error, but the previous ones were not.

## Task description

[//]: # (Short summary of the action to be executed)

* Check whether the below plan makes sense.
* When the error is temporary, try the next call.
* When the error is not temporary, raise it.

## Acceptance Criteria

[//]: # (Acceptance criteria should follow the S.M.A.R.T. principle https://en.wikipedia.org/wiki/SMART_criteria )

* When a non-temporary error is raised by `Source.fetch()`, `Source.track()` or `SourceFetcher.fetch()`, buildstream aborts.
* When a temporary error is raised by the above methods, but not every call, buildstream proceeds as normal.
* When a temporary error is raised every time the above methods are called, buildstream aborts.

----

Contributor guide

Open the contributing guide

Research direction

Start by tracing source-mirroring calls to Source.fetch(), Source.track(), and SourceFetcher.fetch(), then inspect how BstError failures are classified and propagated. Verify the behavior against the acceptance criteria: non-temporary errors abort immediately, temporary errors allow later calls, and failure of every call still aborts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.