argoproj / argoproj/argo-workflows
Simplify configuration of artifact repository
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 138
Description
# Summary
This is a (feature) request to simplify the configuration of the artifact repository.
Currently a configuration map like this is required:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: artifact-repository
data:
default: |
s3:
bucket: artifacts
endpoint: minio.company.com:9000
insecure: false
accessKeySecret:
name: my-minio-cred
key: accesskey
secretKeySecret:
name: my-minio-cred
key: secretke
```
See https://argoproj.github.io/argo-workflows/artifact-repository-ref/
I have several issues with this configuration layout:
1. The configuration key ("default") contains a complex multi-line value. That value is structured in a way that it looks like a fragment of a Kubernetes deployment. I had a lot of trouble to understand how this is supposed to work and it made my head hurt. It looks like a mix of specific configuration (e.g. a host name) and a Kubernetes deployment fragment to me. Yes, technically it is possible to put anything into such a configuration item but I don't think it's a good idea to do that if only a few parts of that value needs to be configurable by themself (like the name of the s3 host or the bucket). I think this should be simplified.
2. The structured value of the configuration key ("default") contains several environment-specific elements: "bucket", "endpoint", "insecure". Right now I don't see a good way to adjust just(!) these values to a specific deployment environment and leaving the rest of it untouched because the entire multi-line structure is just on single value.
3. The configuration element "s3.insecure" leads to redundancies. For some reason the configuration element "s3.endpoint" doesn't support a fully qualified host name. That means, even if I find a way to put everything into a config map with simple key-value pairs, since I'm using the same S3 endpoint for some tasks in my workflow as well (which actually require a fully qualified S3 endpint), I am now forced to define the "endpoint" twice: 1 time with "https://" and 1 time without it. I would love to see that I could just put the full-qualified S3 endpoint into that configuration without the need to deal with the "s3.insecure" element.
My goal (in general) is to keep a Kubernetes deployment as simple as possible and have an environment-independent workflow deployment and everything that is environment-specific is supposed to be contained in a config map like this:
```
[...]
ENDPOINT=https://minio.company.com:9000
BUCKET:artifacts
[...]
```
Right now, I don't know how I could do that with Argo (specifically with the artifact repository configuration).
In case I missed something, please enlighten me :) However, I would appreciate if you seriously consider these suggestions.
# Use Cases
In every workflow that uses artifacts and also needs to be generic to be deployed in different environments.
---
**Message from the maintainers**:
Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.
Contributor guide
Research direction
Start with the artifact repository reference linked in the issue and the Kubernetes ConfigMap example shown there. Define the desired configuration shape from the stated use case, including environment-specific bucket and endpoint values and support for a fully qualified S3 endpoint; done requires an agreed design and corresponding implementation coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100