Feature: Artifact Checkout Default Behavior
- Dominant language
- Python
- Stars
- 140
- Forks
- 45
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 6
Description
Hi, me again. (If there's a better way for me to submit things like this, let me know. There isn't really a clear "how to contact BuildStream Dev team")
When doing a "bst artifact checkout", the default argument to "--deps" is "run". Can this be made a user-configurable option so we can set it to "none" by default via https://docs.buildstream.build/master/using_config.html or as a project.conf value?
Reasoning:
The components we're cross-compiling have these base dependencies as "build and runtime" dependencies:
```
depends:
- base/base.bst
- toolchain/cross-compiler.bst
```
When running "bst artifact checkout", the artifact list is as I would expect (using cross-compiled zlib as an example):
```
$ bst -o target_arch aarch64 -o platform rpi artifact list-contents components/target/zlib.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
components/target/zlib.bst:
robs
robs/staging
robs/staging/image
robs/staging/image/usr
robs/staging/image/usr/lib
robs/staging/image/usr/lib/libz.so
robs/staging/image/usr/lib/libz.so.1
robs/staging/image/usr/lib/libz.so.1.2.12
robs/staging/sysroot
robs/staging/sysroot/usr
robs/staging/sysroot/usr/include
robs/staging/sysroot/usr/include/zconf.h
robs/staging/sysroot/usr/include/zlib.h
robs/staging/sysroot/usr/lib
robs/staging/sysroot/usr/lib/libz.a
robs/staging/sysroot/usr/lib/libz.so
robs/staging/sysroot/usr/lib/libz.so.1
robs/staging/sysroot/usr/lib/libz.so.1.2.12
robs/staging/sysroot/usr/share
robs/staging/sysroot/usr/share/man
robs/staging/sysroot/usr/share/man/man3
robs/staging/sysroot/usr/share/man/man3/zlib.3
robs/staging/sysroot/usr/share/pkgconfig
robs/staging/sysroot/usr/share/pkgconfig/zlib.pc
```
However, this is inconsistent with what "bst artifact checkout" does, which outputs the artifacts as well as the entire sandbox (...why?). I would only be expecting the "robs" directory inside "components/target/zlib/":
```
$ bst -o target_arch aarch64 -o platform rpi artifact checkout components/target/zlib.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
[--:--:--][bd873de7][ main:components/target/zlib.bst ] START Staging dependencies
[00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Staging dependencies
[--:--:--][bd873de7][ main:components/target/zlib.bst ] START Integrating sandbox
[00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Integrating sandbox
[--:--:--][bd873de7][ main:components/target/zlib.bst ] START Checking out files in '/components/target/zlib'
[00:00:01][bd873de7][ main:components/target/zlib.bst ] SUCCESS Checking out files in '/components/target/zlib'
$ ls components/target/zlib/
bin dev etc lib lib64 proc robs sbin tmp usr
```
So we initially thought this was an issue with how we were handling dependencies, so had changed the base 'depends' to 'build-depends':
```
build-depends:
- base/base.bst
- toolchain/cross-compiler.bst
```
And this fixed "artifact checkout" to be what we'd expect. However, it broke "bst shell" since then the base sandbox wasn't there:
```
$ bst -o target_arch aarch64 -o platform rpi shell components/target/zlib.bst
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
[--:--:--][bd873de7][ main:components/target/zlib.bst ] START Staging dependencies
[00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Staging dependencies
[--:--:--][bd873de7][ main:components/target/zlib.bst ] START Integrating sandbox
[00:00:00][bd873de7][ main:components/target/zlib.bst ] SUCCESS Integrating sandbox
[--:--:--][bd873de7][ main:components/target/zlib.bst ] STATUS Running command
sh -i
Error launching shell: Staged artifacts do not provide command 'sh'
```
It was only **_after_** all this that I finally noticed that "artifact checkout" has a "--deps" option, which is "run" by default: https://docs.buildstream.build/master/using_commands.html#cmdoption-bst-artifact-checkout-d
Which I'm sure there's a reason for, but the inconsistency between "checkout" and "list-contents" was confusing for multiple people.
So, being able to change this default value in some config file would be nice, since our workflow would have us only ever really using that style. Also, the command line is getting quite long with the options we need (related #1770). For example: `$ bst -o target_arch aarch64 -o platform rpi -o debug true artifact checkout --deps none components/target/zlib.bst`, which is kinda... yeah.
Thanks :)
Contributor guide
Research direction
Start with the artifact checkout command and its --deps option, then read the using_config.html and using_commands.html documentation referenced in the issue. Check how project.conf and command defaults are described, and define done as allowing users to configure --deps to default to none without requiring the command-line option.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100