When using S3 for artifacts, `artifact-agent upload` behaves differently from `artifact_paths` in `pipeline.yaml`
- Dominant language
- Go
- Stars
- 1.1k
- Forks
- 378
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 74
Description
I discovered an interesting undocumented feature.
The following configuration works.
```
steps:
- label: ':docker: / :node: - Build'
key: build
plugins:
- ecr#v2.5.0:
login: true
region: 'us-east-1'
- docker#v3.9.0:
image: '.dkr.ecr.us-east-1.amazonaws.com/buildkite/docker/library/node:17'
environment:
- NPM_CONFIG__AUTH
- NPM_CONFIG_REGISTRY
commands:
- 'cd /workdir'
- 'npm install && npm run build'
- 'tar --gzip -cf mock.tar.gz build'
- buildkite-agent artifact upload mock.tar.gz
- label: ':amazon-s3: - Publish'
key: publish
depends_on: build
commands:
- 'buildkite-agent artifact download mock.tar.gz .'
- 'tar xmf mock.tar.gz'
- './scripts/publish.sh build'
plugins:
- cultureamp/aws-assume-role#v0.2.0:
role: arn:aws:iam:::role/cross-acct
```
This configuration does not work.
```
steps:
- label: ':docker: / :node: - Build'
key: build
plugins:
- ecr#v2.5.0:
login: true
region: 'us-east-1'
- docker#v3.9.0:
image: '.dkr.ecr.us-east-1.amazonaws.com/buildkite/docker/library/node:17'
environment:
- NPM_CONFIG__AUTH
- NPM_CONFIG_REGISTRY
commands:
- 'cd /workdir'
- 'npm install && npm run build'
- 'tar --gzip -cf mock.tar.gz build'
artifact_paths:
- mock.tar.gz
- label: ':amazon-s3: - Publish'
key: publish
depends_on: build
commands:
- 'buildkite-agent artifact download mock.tar.gz . --step build'
- 'tar xmf mock.tar.gz'
- './scripts/publish.sh build'
plugins:
- cultureamp/aws-assume-role#v0.2.0:
role: arn:aws:iam:::role/cross-acct
```
Debugging the agent showed me that when using `artifact_paths`, the artifact is retrieved via the S3 client. If the step has a cross-account role without access to the artifact bucket, the step cannot access the artifact.
When creating an artifact use `buildkite-agent artifact upload`, the s3 uploader creates a presigned link which is accessible from the `publish` step using an assumed role.
This concerns me because the functionality is not documented and therefore subject to change.
Contributor guide
Assessment
This issue has not been assessed yet.