argoproj / argoproj/argo-workflows

FileName is a directory appears when downloading artifacts from minio

Open
#9,908 3 comments 0 reactions 0 assignees View on GitHub
P3
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I can confirm the issues exists when I tested with `:latest`
- [X] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/master/docs/CONTRIBUTING.md))

### What happened/what you expected to happen?

Downloading artifacts from minio when retring no longer appears on fileName is a directory.

As you can see from the logs of the init container, the connection reset will be retried. the spec.templates[1].inputs.artifacts[0].s3.key is the directory.From the source code https://github.com/argoproj/argo-workflows/blob/6b086368f6480a2de5e2d43eec73514de0ad01ac/workflow/artifacts/s3/s3.go#L84 ,we can see that the s3cli.GetFile method would throws a fileName is a directory error if path is an existing folder. This folder is created the first time s3cli.GetDirectory is called.

### Version

v3.3.8

### Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
apiVersion: argoproj.io/v1alpha1
kind: Workflow # new type of k8s spec
metadata:
generateName: devops-test- # name of the workflow spec
spec:
entrypoint: devops-cicd-java # invoke the whalesay template
templates:
- name: devops-cicd-java # name of the template
dag:
tasks:
- name: compilecode
template: compilecode-task

- name: compilecode-task
inputs:
artifacts:
- name: devops-codecompile-source-s3
path: '{{inputs.parameters.work-dir}}'
s3:
endpoint: xxxxxx
bucket: xxxxxx
insecure: true
key: '827398648978857985/checkoutcode-task-0ee4a9f6-409b-413f-9ab8-efb24b446cf9/src/code'
accessKeySecret:
name: xxxxxx
key: xxxxxx
secretKeySecret:
name: xxxxxx
key: xxxxx
container:
image: nginx
```

### Logs from the workflow controller

workflow controller logs don't matter

### Logs from in your workflow's wait container

time="2022-10-25T06:25:52.899Z" level=info msg="Start loading input artifacts..."
time="2022-10-25T06:25:52.899Z" level=info msg="Downloading artifact: codebuildimage-s3"
time="2022-10-25T06:25:52.899Z" level=info msg="S3 Load path: /argo/inputs/artifacts/codebuildimage-s3.tmp, key: xxxxxxx/src/code/outputs"
time="2022-10-25T06:25:52.899Z" level=info msg="Creating minio client using static credentials" endpoint="xxxxxx"
time="2022-10-25T06:25:52.899Z" level=info msg="Getting file from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:52.911Z" level=info msg="Getting directory from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:52.911Z" level=info msg="Listing directory from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs
time="2022-10-25T06:25:53.733Z" level=info msg="Transient error: read tcp 172.40.62.4:51698->10.48.196.248:9000: read: connection reset by peer"
time="2022-10-25T06:25:55.097Z" level=info msg="S3 Load path: /argo/inputs/artifacts/codebuildimage-s3.tmp, key: xxxxxxx/src/code/outputs"
time="2022-10-25T06:25:55.097Z" level=info msg="Creating minio client using static credentials" endpoint="xxxxxx"
time="2022-10-25T06:25:55.097Z" level=info msg="Getting file from s3" bucket=xxxxxx endpoint="xxxxxx" key=xxxxxxx/src/code/outputs path=/argo/inputs/artifacts/codebuildimage-s3.tmp
time="2022-10-25T06:25:55.097Z" level=warning msg="Non-transient error: fileName is a directory."
time="2022-10-25T06:25:55.097Z" level=error msg="executor error: artifact codebuildimage-s3 failed to load: failed to get file: fileName is a directory."
time="2022-10-25T06:25:55.097Z" level=info msg="Alloc=6222 TotalAlloc=11034 Sys=18642 NumGC=3 Goroutines=3"
time="2022-10-25T06:25:55.097Z" level=fatal msg="artifact codebuildimage-s3 failed to load: failed to get file: fileName is a directory."

Contributor guide

Open the contributing guide

Research direction

Start with workflow/artifacts/s3/s3.go at the linked GetFile code and trace how GetFile falls back to GetDirectory after a connection reset. Reproduce the retry with the supplied workflow and log pattern, then verify that the artifact download succeeds without the existing temporary directory causing a “fileName is a directory” failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.