S3 Artifact error on ARM

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I have tested with the `:latest` image tag (i.e. `quay.io/argoproj/workflow-controller:latest`) and can confirm the issue still exists on `:latest`. If not, I have explained why, **in detail**, in my description below.
- [X] I have searched existing issues and could not find a match for this bug
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/main/docs/CONTRIBUTING.md))

### What happened? What did you expect to happen?

When running a working workflow template on a ARM based architecture node, vs a x86 node, it will fail to retrieve logs and fail the workflow with the error:
```
Error (exit code 1): failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors
```
The workflow will launch and run successfully but when the logs are attempting to be placed into s3 it fails.

NOTE: The works 100% fine with x86 architectures. I tried it with no configuration changes between arm and x86 (amd64) node architectures.

To me this appears like it's an issue with the configuration in argoexec or whatever is running the logic to assume a aws role to read and write log files to a target s3 bucket.

Logs from the pod itself:
```
time="2024-09-11T20:03:12.557Z" level=info msg="capturing logs" argo=true
WORKS!
time="2024-09-11T20:03:13.558Z" level=info msg="sub-process exited" argo=true error=""
```

### Version(s)

Helm chart: "0.42.1", quay.io/argoproj/argocli:v3.5.10, quay.io/argoproj/argocli@sha256:ccc55a32c9739f6d3e7649fe8b896ea90980fae95c4d318a43610dc80d20ddf9

### Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
apiVersion: v1
kind: ServiceAccount
metadata:
name: argo-workflows-testing-role
namespace: argo-workflows
automountServiceAccountToken: true

---

apiVersion: v1
kind: Secret
metadata:
name: argo-workflows-testing-role.service-account-token
namespace: argo-workflows
annotations:
kubernetes.io/service-account.name: argo-workflows-testing-role
type: kubernetes.io/service-account-token

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: argo-workflows-testing-role
rules:
- apiGroups: # https://argo-workflows.readthedocs.io/en/latest/workflow-rbac/
- argoproj.io
resources:
- workflowtaskresults
verbs:
- create
- patch
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- patch

---

# Will attach the first defined service account to the above role.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: argo-workflows-testing-rb
subjects:
- kind: ServiceAccount
name: argo-workflows-testing-role
namespace: argo-workflows
roleRef:
kind: ClusterRole
name: argo-workflows-testing-role
apiGroup: rbac.authorization.k8s.io

---

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: trigger-test
spec:
ttlStrategy:
secondsAfterCompletion: 86400
secondsAfterFailure: 3600
entrypoint: trigger
templates:
- name: trigger
serviceAccountName: argo-workflows-testing-role
metadata:
annotations:
kubectl.kubernetes.io/default-container: main
labels:
role: argo-workflows-testing-role
script:
image: "bash:latest"
command: [bash]
source: |
echo "WORKS!"
```

### Logs from the workflow controller

kubectl logs deployment/argo-workflows-workflow-controller --tail=100 | grep trigger-test

```
time="2024-09-11T20:02:47.002Z" level=info msg="Processing workflow" Phase= ResourceVersion=1982565562 namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.016Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.016Z" level=info msg="Updated phase -> Running" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.017Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.017Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.017Z" level=info msg="Pod node trigger-test-bnzpr initialized Pending" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.037Z" level=info msg="Created pod: trigger-test-bnzpr (trigger-test-bnzpr)" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.037Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.037Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:47.048Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=1982565566 workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.038Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=1982565566 namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.039Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=0 workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.054Z" level=info msg="node changed" namespace=argo-workflows new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=trigger-test-bnzpr old.message= old.phase=Pending old.progress=0/1 workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.054Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.054Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:02:57.063Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=1982565775 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=1982565775 namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=1 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg="task-result changed" namespace=argo-workflows nodeID=trigger-test-bnzpr workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg="node changed" namespace=argo-workflows new.message= new.phase=Running new.progress=0/1 nodeID=trigger-test-bnzpr old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.210Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.216Z" level=info msg="cleaning up pod" action=terminateContainers key=argo-workflows/trigger-test-bnzpr/terminateContainers
time="2024-09-11T20:03:20.220Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Running resourceVersion=1982566292 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:20.232Z" level=info msg="https://172.20.0.1:443/api/v1/namespaces/argo-workflows/pods/trigger-test-bnzpr/exec?command=%2Fvar%2Frun%2Fargo%2Fargoexec&command=kill&command=15&command=1&container=wait&stderr=true&stdout=true&tty=false"
time="2024-09-11T20:03:23.738Z" level=info msg="signaled container" container=wait error="Internal error occurred: error executing command in container: failed to exec in container: failed to create exec \"fea15772efec59bbc64b0cd77d695139bc02e71933228e8f7f016252c6190948\": cannot exec in a deleted state: unknown" namespace=argo-workflows pod=trigger-test-bnzpr stderr="" stdout=""
time="2024-09-11T20:03:34.266Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=1982566292 namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.267Z" level=info msg="Task-result reconciliation" namespace=argo-workflows numObjs=1 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.268Z" level=info msg="task-result changed" namespace=argo-workflows nodeID=trigger-test-bnzpr workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.269Z" level=info msg="Pod failed: Error (exit code 1): failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors" displayName=trigger-test-bnzpr namespace=argo-workflows pod=trigger-test-bnzpr templateName=trigger workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.269Z" level=info msg="node changed" namespace=argo-workflows new.message="Error (exit code 1): failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors" new.phase=Error new.progress=0/1 nodeID=trigger-test-bnzpr old.message= old.phase=Running old.progress=0/1 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.269Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.269Z" level=info msg=reconcileAgentPod namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.269Z" level=info msg="Updated phase Running -> Error" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.270Z" level=info msg="Updated message -> Error (exit code 1): failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.270Z" level=info msg="Marking workflow completed" namespace=argo-workflows workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.280Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Error resourceVersion=1982566645 workflow=trigger-test-bnzpr
time="2024-09-11T20:03:34.281Z" level=info msg="Queueing Error workflow argo-workflows/trigger-test-bnzpr for delete in 24h0m0s due to TTL"
time="2024-09-11T20:03:34.305Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-workflows/trigger-test-bnzpr/labelPodCompleted
time="2024-09-11T20:03:53.738Z" level=info msg="cleaning up pod" action=killContainers key=argo-workflows/trigger-test-bnzpr/killContainers
```

### Logs from in your workflow's wait container

kubectl logs -n argo-workflows trigger-test-bnzpr -c wait
```
time="2024-09-11T21:55:51.385Z" level=info msg="Starting Workflow Executor" version=v3.5.10
time="2024-09-11T21:55:51.388Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2024-09-11T21:55:51.388Z" level=info msg="Executor initialized" deadline="2024-09-11 23:54:58 +0000 UTC" includeScriptOutput=false namespace=argo-workflows podName=trigger-test-bnzpr templateName=trigger version="&Version{Version:v3.5.10,BuildDate:2024-08-01T05:12:26Z,GitCommit:25829927431d9a0f46d17b72ae74aedb8d700884,GitTag:v3.5.10,GitTreeState:clean,GoVersion:go1.21.12,Compiler:gc,Platform:linux/arm64,}"
time="2024-09-11T21:55:51.404Z" level=info msg="Starting deadline monitor"
time="2024-09-11T21:56:31.445Z" level=info msg="Main container completed" error=""
time="2024-09-11T21:56:31.445Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2024-09-11T21:56:31.445Z" level=info msg="No output parameters"
time="2024-09-11T21:56:31.445Z" level=info msg="No output artifacts"
time="2024-09-11T21:56:31.445Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: trigger-test-bnzpr/trigger-test-bnzpr/main.log"
time="2024-09-11T21:56:31.454Z" level=info msg="Creating minio client using assumed-role credentials" roleArn="arn:aws:iam:::role/my-artifact-role"
time="2024-09-11T21:56:34.613Z" level=warning msg="Non-transient error: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"
time="2024-09-11T21:56:34.613Z" level=info msg="Save artifact" artifactName=main-logs duration=3.168001265s error="failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors" key=trigger-test-bnzpr/trigger-test-bnzpr/main.log
time="2024-09-11T21:56:34.613Z" level=error msg="executor error: failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"
time="2024-09-11T21:56:34.630Z" level=info msg="Alloc=10955 TotalAlloc=17092 Sys=24421 NumGC=4 Goroutines=8"
time="2024-09-11T21:56:34.645Z" level=fatal msg="failed to create new S3 client: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"
```

k logs deployment/argo-workflows-server --tail=300 | grep trigger-test
```
time="2024-09-11T20:02:47.227Z" level=info duration=6.731845ms method=GET path=/api/v1/workflows/argo-workflows/trigger-test-bnzpr size=9044 status=0
time="2024-09-11T20:02:47.388Z" level=info duration=9.271235ms method=GET path=/api/v1/workflows/argo-workflows/trigger-test-bnzpr size=9044 status=0
time="2024-09-11T20:02:47.448Z" level=info duration=6.964976ms method=GET path=/api/v1/workflows/argo-workflows/trigger-test-bnzpr size=9044 status=0
```

Start with the argoexec wait container and workflow controller logs, comparing S3 artifact credential initialization on the reported linux/arm64 and x86 paths. Reproduce the provided WorkflowTemplate with the listed v3.5.10 images; done means the ARM workflow completes and captures logs to S3 without the NoCredentialProviders error.

Written by the indexing model from the issue text.