Lack of guidance on how to succeed with Service-Linked-Roles Using UI to create a FSX/Lustre S3 backed cluster
- Dominant language
- Python
- Stars
- 888
- Forks
- 314
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 43
Description
**Required Info:**
- AWS ParallelCluster version: 3.8.0
- Full cluster configuration without any credentials or personal data.: mmm, nope.
- Cluster name: REDACTED
- Output of `pcluster describe-cluster` command.: Nope.
- [Optional] Arn of the cluster CloudFormation main stack: Nope
**Bug description and how to reproduce:**
Using the PCluster User Interface API-GW/Lambda and when providing an S3 bucket config for cluster creation w/attached FSX/Lustre S3 file system you will get an error
```
Amazon FSx is unable to create Service-Linked-Role to access the S3 bucket.
Ensure the IAM role or user you are using has the required permissions.
For more details, visit https://docs.aws.amazon.com/fsx/latest/LustreGuide/setting-up.html#fsx-adding-permissions-s3.
(Service: AmazonFSx; Status Code: 400; Error Code: BadRequest; Request ID: eb6ce2e0-6c1f-4c6f-bb7c-d71cad7ea27c; Proxy: null)
```
The documentation in that link is far to generic to be useful for how to succeed at creating this kind of cluster.
-- and --
The user doesn't know which role to select to make the modifications to...
-- and --
The magic role ( _magic to the user_ ) that needs the additional policy statement is full of it's maximum 10 policies (see my other issue related to max policies https://github.com/aws/aws-parallelcluster/issues/6114)
**How To Get Past That Issue (but still fail with another)**
Since I had no idea which of the several roles that PCluster3' CFT creates, to change; I went hunting through thousands of cloud trail entries (not knowing what I was looking for until I found the "not authorized" statement ) until I found the entry which stated what the role was and gave this error message,
```
arn:aws:sts::REDACTED:assumed-role/REDACTEDPRFIXParallelClusterLambdaRole-REDACTEDSUFFIX/pcluster3-ui-cft-stack-3e3-ParallelClusterFunction-REDACTEDSUFFIX is not authorized to perform: iam:PutRolePolicy
on resource: arn:aws:iam::REDACTED:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/AWSServiceRoleForFSxS3Access_fs-REDACTED
```
That combined with the documentation link mentioned above in the error got me to realize how to proceed.
To continue you cannot add an additional policy attachment (see the bug I linked) which would be the cleaner way, you have to scavage amongst the 10 attached polices (or perhaps add an inline?, I didn't think about that), some of the 10 policies are near their maximum policy size so if you have to pick a policy that is very small with enough room to add the statement (not the whole policy)
```
{
"Effect": "Allow",
"Action": [
"iam:CreateServiceLinkedRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/*"
}
```
Now, you have a solution to that one issue, albeit ugly.
The next mystery is why the import fails...
```
Amazon FSx is unable to import objects from the linked data repository. Please file a ticket at https://console.aws.amazon.com/support/home#/. While filing your ticket, please include your file system ID and name of the linked data repository.
```
Contributor guide
Research direction
Start with the FSx for Lustre S3 permissions documentation linked in the report and trace the ParallelCluster UI API-GW/Lambda cluster-creation flow described here. Document which role and permissions are required for the service-linked role, including the policy-limit constraint, and clarify the separate data-repository import failure. No repository file or test is named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, documentation, security
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100