atlassian / atlassian/escalator

[Question] How to assume IAM role inside the escalator pod? Getting 403 despite instructions

Open
#231 10 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Go
Stars
685
Forks
68
Avg merge
47m
Merged PRs (30d)
1

Description

Hello guys,
It's a wonderful project and I've almost got it working. Having followed Readme instructions in (https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md)
I have those ticked off:
- IAM role created
- IAM policy created as per (https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md#permissions)
- --aws-assume-role-arn flag passed in the command in escalator-deployment-aws.yaml as below:
```
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: escalator
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: escalator
role: escalator
template:
metadata:
# I'm really not sure all three are required as below: https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md#deployment
annotations:
iam.amazonaws.com/role: arn:aws:iam::XXX:role/bitbucket-pipelines-escalator-role
labels:
app: escalator
role: escalator
spec:
serviceAccountName: escalator
containers:
- image: atlassian/escalator
command:
- ./main
- --nodegroups
- /opt/conf/nodegroups/nodegroups_config.yaml
- --cloud-provider
- aws
# this bit: https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md#sts-assume-role
- --aws-assume-role-arn
- arn:aws:iam::XXX:role/bitbucket-pipelines-escalator-role
name: escalator
ports:
- containerPort: 8080
env:
# this bit: https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md#aws-credentials
- name: AWS_ROLE_ARN
value: arn:aws:iam::XXX:role/bitbucket-pipelines-escalator-role
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: AWS_REGION
value: eu-west-1
volumeMounts:
- name: escalator-nodegroups
mountPath: /opt/conf/nodegroups
readOnly: true
```
- AWS_ROLE_ARN and AWS_REGION passed into env in above deployment, as mentioned in discovery by default credential chain in (https://github.com/atlassian/escalator/blob/master/docs/deployment/aws/README.md#aws-credentials)
- finally annotations:iam.amazonaws.com/role with role arn as mentioned in the kube2iam part of Deployment instructions.

Given all that I'm still getting 403 on attempt to assume role.
`
AccessDenied: User: arn:aws:sts::XXX:assumed-role/eksctl-bitbucketpipelines-nodegro-NodeInstanceRole-XXX is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::XXX:role/bitbucket-pipelines-escalator-role\n\tstatus code: 403
`

1) I am missing something? Is the documentation complete?
2) Other sources suggest creating OIDC Provider for the cluster (https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html) I've done that with eksctl and it has no impact on it's own
3) Is there a specific trust relationship on the IAM role required before the escalator pod can assume it please?

Any pointers would be much appreciated. Thank you!

Contributor guide

Open the contributing guide

Research direction

Start with docs/deployment/aws/README.md, especially the Permissions, Deployment, STS AssumeRole, and AWS Credentials sections, then compare them with the deployment manifest and its kube2iam and EKS OIDC references. Clarify the required IAM trust relationship and credential setup for the reported 403, and ensure the documented deployment steps are complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, documentation, security
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.