argoproj / argoproj/argo-workflows

Simplify configuration of artifact repository

Open
#9,023 0 comments 1 reaction 0 assignees View on GitHub
area/artifacts
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

This is a (feature) request to simplify the configuration of the artifact repository.

Currently a configuration map like this is required:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: artifact-repository
data:
default: |
s3:
bucket: artifacts
endpoint: minio.company.com:9000
insecure: false
accessKeySecret:
name: my-minio-cred
key: accesskey
secretKeySecret:
name: my-minio-cred
key: secretke
```
See https://argoproj.github.io/argo-workflows/artifact-repository-ref/

I have several issues with this configuration layout:

1. The configuration key ("default") contains a complex multi-line value. That value is structured in a way that it looks like a fragment of a Kubernetes deployment. I had a lot of trouble to understand how this is supposed to work and it made my head hurt. It looks like a mix of specific configuration (e.g. a host name) and a Kubernetes deployment fragment to me. Yes, technically it is possible to put anything into such a configuration item but I don't think it's a good idea to do that if only a few parts of that value needs to be configurable by themself (like the name of the s3 host or the bucket). I think this should be simplified.

2. The structured value of the configuration key ("default") contains several environment-specific elements: "bucket", "endpoint", "insecure". Right now I don't see a good way to adjust just(!) these values to a specific deployment environment and leaving the rest of it untouched because the entire multi-line structure is just on single value.

3. The configuration element "s3.insecure" leads to redundancies. For some reason the configuration element "s3.endpoint" doesn't support a fully qualified host name. That means, even if I find a way to put everything into a config map with simple key-value pairs, since I'm using the same S3 endpoint for some tasks in my workflow as well (which actually require a fully qualified S3 endpint), I am now forced to define the "endpoint" twice: 1 time with "https://" and 1 time without it. I would love to see that I could just put the full-qualified S3 endpoint into that configuration without the need to deal with the "s3.insecure" element.

My goal (in general) is to keep a Kubernetes deployment as simple as possible and have an environment-independent workflow deployment and everything that is environment-specific is supposed to be contained in a config map like this:

```
[...]
ENDPOINT=https://minio.company.com:9000
BUCKET:artifacts
[...]
```

Right now, I don't know how I could do that with Argo (specifically with the artifact repository configuration).

In case I missed something, please enlighten me :) However, I would appreciate if you seriously consider these suggestions.

# Use Cases

In every workflow that uses artifacts and also needs to be generic to be deployed in different environments.

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Start with the artifact repository reference linked in the issue and the Kubernetes ConfigMap example shown there. Define the desired configuration shape from the stated use case, including environment-specific bucket and endpoint values and support for a fully qualified S3 endpoint; done requires an agreed design and corresponding implementation coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.