Error decrypting key: AccessDeniedException with AWS KMS
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.1k
- Forks
- 1.1k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 9
Description
Hello,
I've encountered an issue/QoL deficit which I haven't been able to find among other issues.
When I encrypt something as follows:
sops --aws-profile <profile> --kms <key arn> -e --output-type yaml <secret file> > secrets.encrypted.yaml
As part of the output, the following is generated:
sops:
kms:
- arn: arn:aws:kms:xxx
created_at: "2022-06-08T14:21:13Z"
enc: xxx
aws_profile: xxx
When running this with FluxCD, and service account has permissions with an annotation to a role, it fails with an error similar to:
Error decrypting key: AccessDeniedException: The ciphertext
| refers to a customer master key that does not exist, does
| not exist in this region, or you are not allowed to access.
| status code: 400
despite permissions being otherwise in order.
It appears to me that sops attempts to use the aws_profile used during encryption as generated at the encryption time.
Simply removing this part of the generated encrypted file will allow sops running inside of FluxCD to decrypt objects successfully.
This is also verifiable on CLI. Given no configured profiles, attempting to decrypt a file which has a sops.kms configuration with a specified aws_profile will fail with a similar error as above.
The QoL part of the issue is that encrypting a file with an aws_profile will require further manual edits on the encrypted file to ensure it can be consumed by Flux, also decrypting the file on CLI with:
sops --aws-profile <profile> --kms <key id> -d secrets.encrypted.yaml will not work unless aws_profile is specified on the file as well, even though we are passing the --aws-profile on CLI.
In summary, one has to be very careful with encrypting/decrypting files with sops if they are using an aws profile and have to manually modify the files after the fact for FuxCD consumption. This issue has personally cost me hours of debugging before I realized why sops on Flux was failing to decrypt my secrets.
My suggestion would be to not assume that the entity encrypting and the entity decrypting have equal credential configuration. I think SOPS should drop the aws_profile parameter it bakes into encrypted files and instead use default credentials, default should still be overridable by .sops config file.
This means that if you encrypt a file using an aws profile, and attempt to decrypt it without using one, it should be down to your default credentials / .sops config
If you attempt to decrypt it while specifying --aws-profile then it should take that with highest priority.
Let me know if you need any further clarification.
Cheers!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing AWS KMS handling for the sops.kms.aws_profile metadata and the --aws-profile CLI option. Reproduce the shown encryption and decryption commands with and without configured profiles, then verify behavior through FluxCD-style credentials. Done means credential precedence is consistent and encrypted files do not require manual aws_profile removal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cli, cloud, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100