AWS Profiles not properly resolving the .aws/config file

AWS Profile not correctly resolving .aws/config file

When attempting to use a profile in a .aws/config file, sops does not properly resolve the credentials. Example sops and .aws config files are provided below. I believe I have a fix working, with a PR coming shortly.

SOPS file

hello: ENC[...]
creds:
    user: ENC[...]
    password: ENC[...]
sops:
    kms:
    -   arn: arn:aws:kms:us-east-1:<account_id>:key/<key_id>
        created_at: '2020-06-09T16:55:32Z'
        enc: "..."
        aws_profile: my-account # This profile exists in .aws/config
    gcp_kms: []
    azure_kv: []
    lastmodified: '2020-06-09T16:57:40Z'
    mac: "..."
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.5.0

.aws/config file

Here we're defining a profile where we assume a role into a separate account. While assuming roles is possible with sops using the kms_arn+role_arn configuration, our organization uses dozens of roles to manage AWS access. Different teams accessing the same KMS key are likely to have different roles. Ideally this is solved through normal behavior of the AWS sdk, rather than be reliant on sops' implementation.

[profile my-account]
source_profile = my-main-account
role_arn = arn:aws:iam::<account_id>:role/my-automation-role

Existing problems

SOPS does not currently resolve the profile correctly. This prevents standard usage where profiles defined in the .aws/config file assume AWS credentials before executing commands. The following command works with the aws cli, setting a profile through the environment variables:

# Properly assumes role through .aws/config and prints buckets
AWS_PROFILE=my-account aws s3 ls

In sops, the following is returned:

AWS_PROFILE=my-accounts sops my-file.yaml
# ...
Group 0: FAILED
  arn:aws:kms:us-east-2:<account_id>:key/<key_id>: FAILED
    - | Error decrypting key: SharedCredsLoad: failed to get profile

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Similarly, if no AWS_PROFILE is provided, the one from the sops file is used. This similarly fails to resolve:

sops my-file.yml
# ...
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:us-east-2:668652621010:key/c24e651c-578c-4602-ab4b-30d85770d963: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      |         For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Cause

The above problems seem to be caused by improperly instantiating an AWS session:

func (key MasterKey) createSession() (*session.Session, error) {
	re := regexp.MustCompile(`^arn:aws[\w-]*:kms:(.+):[0-9]+:(key|alias)/.+$`)
	matches := re.FindStringSubmatch(key.Arn)
	if matches == nil {
		return nil, fmt.Errorf("No valid ARN found in %q", key.Arn)
	}

	config := aws.Config{Region: aws.String(matches[1])}

    // Does not respect the .aws/config file. Forces the profile to be in .aws/credentials, preventing standard assume roles though the ./aws/config file.
	if key.AwsProfile != "" {
		config.Credentials = credentials.NewSharedCredentials("", key.AwsProfile)
	}

    // Not providing a SharedConfigState option makes .aws/config ignored by default
    // This can be manually set with AWS_SDK_LOAD_CONFIG, but most CLI tools have this enabled by default to just work
    // Can also pass the profile in here, which respects the .aws/config file
	opts := session.Options{
		Config:                  config,
		AssumeRoleTokenProvider: stscreds.StdinTokenProvider,
	}
	sess, err := session.NewSessionWithOptions(opts)
	if err != nil {
		return nil, err
	}
	if key.Role != "" {
		return key.createStsSession(config, sess)
	}
	return sess, nil
}

Start at MasterKey.createSession, especially the AWS session and profile handling shown in the issue. Reproduce the difference between AWS_PROFILE with the AWS CLI and with sops using a profile in .aws/config. Done means sops can use that profile to assume its configured role and decrypt the KMS-wrapped data key.