argoproj / argoproj/argo-workflows
Enable Custom Metadata for S3 Output Artifacts
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 138
Description
# Summary
I propose adding a feature to Argo Workflows that allows setting object metadata (user metadata, headers like content-disposition, content-encoding, etc) for output artifacts being stored on S3-compatible storage. The `minio-go` sdk used by Argo [already supports these attributes in `PutObject`](https://min.io/docs/minio/linux/developers/go/API.html#PutObject:~:text=multipart%20upload%20operation.-,minio.PutObjectOptions,-Field)
## Use Cases
This feature would be particularly useful in the following scenarios:
1. **Compliance and Security**: Custom metadata can be used to store information related to data classification, compliance, or security requirements.
2. **Data Management**: Users can store versioning information, timestamps, or other relevant data in metadata, improving data management and retrieval processes.
3. **Integration with Other Systems**: When integrating with other systems that rely on specific metadata keys, this feature would facilitate smoother interoperability.
4. **Tagging and Categorization**: Users can tag or categorize artifacts with metadata, aiding in organization and searchability within large datasets.
---
**Message from the maintainers**:
Love this feature request? Give it a 👍. We prioritise the proposals with the most 👍.
Contributor guide
Research direction
The issue names no repository files, tests, or configuration entry points. Start by locating the S3 output-artifact upload path and the minio-go PutObject call, then determine how custom metadata and headers should be configured; done means S3 artifacts accept and preserve the requested metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100