OCI runtime create failed: runc create failed: unable to start container process

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I can confirm the issues exists when I tested with `:latest`
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/master/docs/CONTRIBUTING.md))

### What happened/what you expected to happen?

I use artifacts as inputs to start container with workingDir "/sandbox", but throw error "StartError (exit code 128): failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: mkdir /src: not a directory: unknown"

artifacts:
- name: argo-source
path: /sandbox
azure:
endpoint: "{{inputs.parameters.azure_sa_endpoint}}"
container: "sandbox-content-file"
blob: "{{workflow.parameters.user}}/{{workflow.parameters.artifactId}}.tar.gz"
accountKeySecret:
name: azure-storage-credentials
key: account-access-key

I'm not sure what's wrong

### Version

v3.4.6

### Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
artifacts:
- name: argo-source
path: /sandbox
azure:
endpoint: "{{inputs.parameters.azure_sa_endpoint}}"
container: "sandbox-content-file"
blob: "{{workflow.parameters.user}}/{{workflow.parameters.artifactId}}.tar.gz"
accountKeySecret:
name: azure-storage-credentials
key: account-access-key

container:
image: xxx
imagePullPolicy: "Always"
command: ["/bin/bash", "-c"]
args: [xxx]
workingDir: /sandbox
```

### Logs from the workflow controller

```text
time="2023-06-14T15:15:11.924Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:14.744Z" level=info msg="Updated phase -> Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:14.744Z" level=info msg="Steps node sandbox-freestyle-exec-rnhsw initialized Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:14.744Z" level=info msg="StepGroup node sandbox-freestyle-exec-rnhsw-895115467 initialized Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:14.745Z" level=info msg="Pod node sandbox-freestyle-exec-rnhsw-1605168834 initialized Pending" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:15.165Z" level=info msg="Created pod: sandbox-freestyle-exec-rnhsw[0].fetch-options (sandbox-freestyle-exec-rnhsw-fetch-options-1605168834)" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:15.165Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-895115467 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:15.165Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:15.167Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:15.385Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277039 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.925Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.926Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.926Z" level=info msg="node changed" namespace=iops new.message= new.phase=Running new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-1605168834 old.message= old.phase=Pending old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.927Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-895115467 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.927Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:21.927Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:22.538Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277109 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.540Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.541Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.541Z" level=warning msg="workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.541Z" level=info msg="node changed" namespace=iops new.message= new.phase=Succeeded new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-1605168834 old.message= old.phase=Running old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.542Z" level=info msg="Step group node sandbox-freestyle-exec-rnhsw-895115467 successful" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.542Z" level=info msg="node sandbox-freestyle-exec-rnhsw-895115467 phase Running -> Succeeded" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.542Z" level=info msg="node sandbox-freestyle-exec-rnhsw-895115467 finished: 2023-06-14 15:15:32.542501465 +0000 UTC" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.542Z" level=info msg="StepGroup node sandbox-freestyle-exec-rnhsw-4183572718 initialized Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.542Z" level=info msg="SG Outbound nodes of sandbox-freestyle-exec-rnhsw-1605168834 are [sandbox-freestyle-exec-rnhsw-1605168834]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.548Z" level=info msg="Steps node sandbox-freestyle-exec-rnhsw-153584188 initialized Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.548Z" level=info msg="StepGroup node sandbox-freestyle-exec-rnhsw-1277426710 initialized Running" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:32.549Z" level=info msg="Pod node sandbox-freestyle-exec-rnhsw-790435683 initialized Pending" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.753Z" level=info msg="Created pod: sandbox-freestyle-exec-rnhsw[1].freestyle-shell[0].step-execute-script-shell (sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683)" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.753Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-1277426710 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.753Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-4183572718 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.753Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.753Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.986Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277195 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:33.992Z" level=info msg="cleaning up pod" action=labelPodCompleted key=iops/sandbox-freestyle-exec-rnhsw-fetch-options-1605168834/labelPodCompleted
time="2023-06-14T15:15:43.957Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.957Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.957Z" level=info msg="node changed" namespace=iops new.message= new.phase=Running new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-790435683 old.message= old.phase=Pending old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.959Z" level=info msg="SG Outbound nodes of sandbox-freestyle-exec-rnhsw-1605168834 are [sandbox-freestyle-exec-rnhsw-1605168834]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.965Z" level=info msg="cleaning up pod" action=terminateContainers key=iops/sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683/terminateContainers
time="2023-06-14T15:15:43.966Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-1277426710 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.966Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-4183572718 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.966Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.966Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:43.967Z" level=info msg="https://api.argo-q.iops-pg.internal.live.k8s.ondemand.com:443/api/v1/namespaces/iops/pods/sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683/exec?command=%2Fvar%2Frun%2Fargo%2Fargoexec&command=kill&command=15&command=1&container=log-collector&stderr=true&stdout=true&tty=false"
time="2023-06-14T15:15:44.573Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277273 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:47.413Z" level=info msg="signaled container" container=log-collector error="" namespace=iops pod=sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683 stderr= stdout="killing 1 with terminated\n"
time="2023-06-14T15:15:47.414Z" level=info msg="https://api.argo-q.iops-pg.internal.live.k8s.ondemand.com:443/api/v1/namespaces/iops/pods/sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683/exec?command=%2Fvar%2Frun%2Fargo%2Fargoexec&command=kill&command=15&command=1&container=wait&stderr=true&stdout=true&tty=false"
time="2023-06-14T15:15:50.733Z" level=info msg="signaled container" container=wait error="" namespace=iops pod=sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683 stderr= stdout="killing 1 with terminated\n"
time="2023-06-14T15:15:59.600Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.600Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.600Z" level=info msg="Pod failed: StartError (exit code 128): failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: mkdir /src: not a directory: unknown" displayName=step-execute-script-shell namespace=iops pod=sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683 templateName=execute-script-shell workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.600Z" level=warning msg="workflow uses legacy/insecure pod patch, see https://argoproj.github.io/argo-workflows/workflow-rbac/" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.600Z" level=info msg="node changed" namespace=iops new.message="StartError (exit code 128): failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: mkdir /src: not a directory: unknown" new.phase=Failed new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-790435683 old.message= old.phase=Running old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.601Z" level=info msg="SG Outbound nodes of sandbox-freestyle-exec-rnhsw-1605168834 are [sandbox-freestyle-exec-rnhsw-1605168834]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.603Z" level=info msg="Running OnExit handler" lifeCycleHook="&LifecycleHook{Template:exit-handler,Arguments:Arguments{Parameters:[]Parameter{Parameter{Name:job_status,Default:nil,Value:*{{steps.step-execute-script-shell.status}},ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},Parameter{Name:error_info,Default:nil,Value:*None,ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},},Artifacts:[]Artifact{Artifact{Name:combined_log,Path:,Mode:nil,From:{{steps.step-execute-script-shell.outputs.artifacts.combined_log}},ArtifactLocation:ArtifactLocation{ArchiveLogs:nil,S3:nil,Git:nil,HTTP:nil,Artifactory:nil,HDFS:nil,Raw:nil,OSS:nil,GCS:nil,Azure:nil,},GlobalName:,Archive:nil,Optional:false,SubPath:,RecurseMode:false,FromExpression:,ArtifactGC:nil,Deleted:false,},},},TemplateRef:nil,Expression:,}" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:15:59.610Z" level=info msg="Pod node sandbox-freestyle-exec-rnhsw-1783598922 initialized Pending" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:00.434Z" level=info msg="Created pod: sandbox-freestyle-exec-rnhsw[1].freestyle-shell[0].step-execute-script-shell.onExit (sandbox-freestyle-exec-rnhsw-exit-handler-1783598922)" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:00.434Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-1277426710 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:00.434Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-4183572718 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:00.434Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:00.434Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:01.040Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277395 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:01.046Z" level=info msg="cleaning up pod" action=labelPodCompleted key=iops/sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683/labelPodCompleted
time="2023-06-14T15:16:10.634Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.634Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.635Z" level=info msg="node changed" namespace=iops new.message= new.phase=Running new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-1783598922 old.message= old.phase=Pending old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.635Z" level=info msg="SG Outbound nodes of sandbox-freestyle-exec-rnhsw-1605168834 are [sandbox-freestyle-exec-rnhsw-1605168834]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.638Z" level=info msg="Running OnExit handler" lifeCycleHook="&LifecycleHook{Template:exit-handler,Arguments:Arguments{Parameters:[]Parameter{Parameter{Name:job_status,Default:nil,Value:*{{steps.step-execute-script-shell.status}},ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},Parameter{Name:error_info,Default:nil,Value:*None,ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},},Artifacts:[]Artifact{Artifact{Name:combined_log,Path:,Mode:nil,From:{{steps.step-execute-script-shell.outputs.artifacts.combined_log}},ArtifactLocation:ArtifactLocation{ArchiveLogs:nil,S3:nil,Git:nil,HTTP:nil,Artifactory:nil,HDFS:nil,Raw:nil,OSS:nil,GCS:nil,Azure:nil,},GlobalName:,Archive:nil,Optional:false,SubPath:,RecurseMode:false,FromExpression:,ArtifactGC:nil,Deleted:false,},},},TemplateRef:nil,Expression:,}" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.639Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-1277426710 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.639Z" level=info msg="Workflow step group node sandbox-freestyle-exec-rnhsw-4183572718 not yet completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.639Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:10.639Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:11.253Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277469 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:20.734Z" level=info msg="cleaning up pod" action=killContainers key=iops/sandbox-freestyle-exec-rnhsw-execute-script-shell-790435683/killContainers
time="2023-06-14T15:16:22.302Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.302Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.302Z" level=info msg="node changed" namespace=iops new.message= new.phase=Succeeded new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-1783598922 old.message= old.phase=Running old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.303Z" level=info msg="SG Outbound nodes of sandbox-freestyle-exec-rnhsw-1605168834 are [sandbox-freestyle-exec-rnhsw-1605168834]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="Running OnExit handler" lifeCycleHook="&LifecycleHook{Template:exit-handler,Arguments:Arguments{Parameters:[]Parameter{Parameter{Name:job_status,Default:nil,Value:*{{steps.step-execute-script-shell.status}},ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},Parameter{Name:error_info,Default:nil,Value:*None,ValueFrom:nil,GlobalName:,Enum:[],Description:nil,},},Artifacts:[]Artifact{Artifact{Name:combined_log,Path:,Mode:nil,From:{{steps.step-execute-script-shell.outputs.artifacts.combined_log}},ArtifactLocation:ArtifactLocation{ArchiveLogs:nil,S3:nil,Git:nil,HTTP:nil,Artifactory:nil,HDFS:nil,Raw:nil,OSS:nil,GCS:nil,Azure:nil,},GlobalName:,Archive:nil,Optional:false,SubPath:,RecurseMode:false,FromExpression:,ArtifactGC:nil,Deleted:false,},},},TemplateRef:nil,Expression:,}" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="Step group node sandbox-freestyle-exec-rnhsw-1277426710 deemed failed: child 'sandbox-freestyle-exec-rnhsw-790435683' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="node sandbox-freestyle-exec-rnhsw-1277426710 phase Running -> Failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="node sandbox-freestyle-exec-rnhsw-1277426710 message: child 'sandbox-freestyle-exec-rnhsw-790435683' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="node sandbox-freestyle-exec-rnhsw-1277426710 finished: 2023-06-14 15:16:22.30560775 +0000 UTC" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="step group sandbox-freestyle-exec-rnhsw-1277426710 was unsuccessful: child 'sandbox-freestyle-exec-rnhsw-790435683' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.305Z" level=info msg="Outbound nodes of sandbox-freestyle-exec-rnhsw-790435683 is [sandbox-freestyle-exec-rnhsw-790435683]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Outbound nodes of sandbox-freestyle-exec-rnhsw-153584188 is [sandbox-freestyle-exec-rnhsw-790435683]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-153584188 phase Running -> Failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-153584188 message: child 'sandbox-freestyle-exec-rnhsw-790435683' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-153584188 finished: 2023-06-14 15:16:22.306026455 +0000 UTC" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Checking daemoned children of sandbox-freestyle-exec-rnhsw-153584188" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Step group node sandbox-freestyle-exec-rnhsw-4183572718 deemed failed: child 'sandbox-freestyle-exec-rnhsw-153584188' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-4183572718 phase Running -> Failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-4183572718 message: child 'sandbox-freestyle-exec-rnhsw-153584188' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw-4183572718 finished: 2023-06-14 15:16:22.306263357 +0000 UTC" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="step group sandbox-freestyle-exec-rnhsw-4183572718 was unsuccessful: child 'sandbox-freestyle-exec-rnhsw-153584188' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Outbound nodes of sandbox-freestyle-exec-rnhsw-153584188 is [sandbox-freestyle-exec-rnhsw-790435683]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Outbound nodes of sandbox-freestyle-exec-rnhsw is [sandbox-freestyle-exec-rnhsw-790435683]" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw phase Running -> Failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw message: child 'sandbox-freestyle-exec-rnhsw-153584188' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="node sandbox-freestyle-exec-rnhsw finished: 2023-06-14 15:16:22.306732662 +0000 UTC" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Checking daemoned children of sandbox-freestyle-exec-rnhsw" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.306Z" level=info msg="Running OnExit handler: exit-handler" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:22.307Z" level=info msg="Pod node sandbox-freestyle-exec-rnhsw-4011934118 initialized Pending" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:23.117Z" level=info msg="Created pod: sandbox-freestyle-exec-rnhsw.onExit (sandbox-freestyle-exec-rnhsw-exit-handler-4011934118)" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:23.528Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277577 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:23.534Z" level=info msg="cleaning up pod" action=labelPodCompleted key=iops/sandbox-freestyle-exec-rnhsw-exit-handler-1783598922/labelPodCompleted
time="2023-06-14T15:16:33.304Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.304Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.304Z" level=info msg="node changed" namespace=iops new.message= new.phase=Running new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-4011934118 old.message= old.phase=Pending old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.305Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.305Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.305Z" level=info msg="Running OnExit handler: exit-handler" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:33.913Z" level=info msg="Workflow update successful" namespace=iops phase=Running resourceVersion=44277655 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.304Z" level=info msg="Processing workflow" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.304Z" level=info msg="Task-result reconciliation" namespace=iops numObjs=0 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.304Z" level=info msg="node changed" namespace=iops new.message= new.phase=Succeeded new.progress=0/1 nodeID=sandbox-freestyle-exec-rnhsw-4011934118 old.message= old.phase=Running old.progress=0/1 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="TaskSet Reconciliation" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg=reconcileAgentPod namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="Running OnExit handler: exit-handler" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="Updated phase Running -> Failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="Updated message -> child 'sandbox-freestyle-exec-rnhsw-153584188' failed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="Marking workflow completed" namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.305Z" level=info msg="Checking daemoned children of " namespace=iops workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:44.311Z" level=info msg="cleaning up pod" action=deletePod key=iops/sandbox-freestyle-exec-rnhsw-1340600742-agent/deletePod
time="2023-06-14T15:16:44.922Z" level=info msg="Workflow update successful" namespace=iops phase=Failed resourceVersion=44277734 workflow=sandbox-freestyle-exec-rnhsw
time="2023-06-14T15:16:45.131Z" level=info msg="cleaning up pod" action=labelPodCompleted key=iops/sandbox-freestyle-exec-rnhsw-exit-handler-4011934118/labelPodCompleted
```

### Logs from in your workflow's wait container

```text
kubectl logs -n iops -c wait -l workflows.argoproj.io/workflow=${sandbox-freestyle-exec-rnhsw},workflow.argoproj.io/phase!=Succeeded

No resources found in iops namespace.
```

Start with the supplied artifact and container workingDir configuration and reproduce the failure using the minimal workflow. Trace pod and container creation around the logged `/src` and `/sandbox` paths, then determine why the OCI runtime rejects the setup. Done means the workflow starts successfully with the artifact and workingDir, with a regression test for the reproduced case.

Written by the indexing model from the issue text.