argoproj / argoproj/argo-workflows

support s3fs mount

Open
#9,708 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

I want to mount all objects to my local container. If objects are added to bucket, the container should update in real time.

# Use Cases

Recently, I use minio as storage backend. when the drone sends picture to the minio, I get callback and process the picture. If there is only one picture, I can mount the object in the local path in the fellowing way.
image

BUT, we have a lot of picture and when workflow starts, the pictures is still uploaded to the minio. We cannot start a workflow for each picture, because processing the current picture may depend on the pervious pictures.

We are now using the method of downloading pictures through http in real time.

I found s3fs can mount buckets locally. So, i want argo to support s3fs.

other question, Is there other solution for argo to resolve my problem?

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

No source files, tests, or entry points are named. First clarify how Argo Workflows should expose an s3fs-mounted MinIO bucket in a local container, then review the Kubernetes volume-mount behavior and existing artifact or storage integration; done means newly added objects become available to the workflow as requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
cloud, data-engineering, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.