apache / apache/buildstream

Add support for running tests in BuildStream

Open
#776 8 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/776)
In GitLab by [[Gitlab user @chiaratolentino]](https://gitlab.com/chiaratolentino) on Nov 22, 2018, 14:26

## Background

[//]: # (Provide a background or the root/source that justifies this task or action.)
From the [discussion on the mailing list](https://mail.gnome.org/archives/buildstream-list/2018-October/msg00058.html):

> As software integrators, we would like to run tests as part of BuildStream
> builds and be able to validate (at a later stage in build) that the tests for a
> given element have passed. Ideally, such tests should have the following
> properties:
>
> 1.) Tests should not block reverse dependency builds.
>
> 2.) Strong association between an element and its tests.
>
> 3.) Additional dependencies for testing an element should not be required
> while building it.
>
> 4.) Elements should be able to assert that the tests for their dependencies
> have passed.
>
> 5.) Test elements are not necessarily pass-through elements - they can have
> meaningful output such as coverage reports.

## Task description

[//]: # (Short summary of the action to be executed)

The proposed solution is to introduce the concepts of 'conditions' and 'abstract dependencies'. For more detail, see the [full proposal from the mailing list](https://mail.gnome.org/archives/buildstream-list/2018-November/msg00038.html)

* [ ] Add a new ```conditions``` field to the bst element file with subfields ```provides``` and ```requires```, e.g.

```yaml
conditions:
provides:
tested:
- app-test.bst
packaged:
- app-pkg.bst
requires:
- tested
```
* [ ] Add abstract dependencies (https://gitlab.com/BuildStream/buildstream/issues/777)

* [ ] Add functionality to ```bst show``` to check the ```conditions``` fields of an element (for possible sample outputs, see the examples under part 3 of the mailing list proposal)

* [ ] Expose a Python API for staging an element's build tree (see part 4 of the mailing list proposal for the justification for this)
* [ ] Provide a method to guarantee that the build tree we want to stage (for the purposes of running tests) exists in the dependency artifact

## Acceptance Criteria

[//]: # (Acceptance criteria should follow the S.M.A.R.T. principle https://en.wikipedia.org/wiki/SMART_criteria )

Users can run tests as part of the build of an element such that the test properties outlined above are satisfied.

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are named. Start with the linked mailing-list proposal and issue 777, then locate the element-file parsing, bst show, and build-tree staging entry points. Done means users can run and validate tests while satisfying the five listed test properties and the acceptance criterion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.