Temporal assumes the credentials of storage service used for archival will stay the same through out the life cycle of the application.

Expected Behavior

1. We use AWS S3 to store archived workflows.
2. Authentication is via AWS Keys. The keys are pulled every 8 hours and written to the credential file. This is done as our keys are valid only for 8 hours.
3. I expect temporal to use the latest keys from the credential file each and every time when connecting to AWS S3.

Actual Behavior

1. Archiver object which does the archiving (which containers the AWS session object) is created only once for the first call and is cached. There after for subsequent calls, the same archiver object is used.
2. This means that the AWS Keys present at the first call (at the start of the application) is used for the whole life cycle.
3. This will result in archival failure after 8 hours and the keys expire after 8 hours.

I feel the root of the problem is that temporal assumes that the credentials will stay the same for the whole lifecycle of the application which isn't true.

Steps to Reproduce the Problem

1. Setup AWS S3 archival
2. Configure the AWS Keys to expire , say 4 hours from creation.
3. Archival will stop working after 4 hours.

This has been discussed here -> https://community.temporal.io/t/aws-keys-updated-every-10-hours-in-credential-file/3612

Specifications

• Version: 1.1.12
• Platform: Java

Start by tracing the archival path that creates and caches the Archiver and its AWS session, then inspect how credentials are read from the credential file. Reproduce the failure with AWS S3 keys that expire after a few hours; done means later archival attempts use the refreshed credentials instead of the initial keys.