airbytehq / airbytehq/airbyte

[source-dynamodb] connector does not work in AWS cross-account setting with Role based authentication

未關閉
#60,944 4 則留言 7 個 reaction 已指派 0 人 在 GitHub 檢視
area/connectors community connectors/source/dynamodb team/db-dw-sources type/bug
主要語言
Python
星號
22.1k
分支
5.4k
平均合併
5 小時
30 天內合併 PR
671

描述

### Connector Name

source-dynamodb

### Connector Version

0.3.9

### What step the error happened?

Configuring a new connector

### Relevant information

Hello

I have the following setup:

Airbyte is hosted in AWS account A.
DynamoDB tables are hosted in AWS account B.

I am trying to set up a dynamodb source in airbyte, using role based authentication.
The role belongs to account A, and has permissions to perform dynamodb actions on account B.

Account B tables have a policy which allows account A role to access them, according to the documentation.
https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/configure-cross-account-access-to-amazon-dynamodb.html#configure-cross-account-access-to-amazon-dynamodb-summary

When trying to set up the source, the first action taken is trying to list the tables in the same account, which is not the desired behaviour, since I'm trying to sync from a different account. This also fails, since the role in account A does not have permissions to list tables in account A, and it's irrelevant since the tables are in account B.

I also tried to set up [AWS PrivateLink for DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/privatelink-interface-endpoints.html), hoping it would "force" the client to perform the actions in the appropriate account, but to no avail.

As a possible solution, I would like to enter the full source ARN, so that the SDK tries to query the correct table in the correct account.
This is the suggested solution in the example provided by AWS:
https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/configure-cross-account-access-to-amazon-dynamodb.html#configure-cross-account-access-to-amazon-dynamodb-additional

Thank you!

### Relevant log output

```shell
2025-05-28 14:04:29 info INFO i.a.i.s.d.DynamodbUtils(createDynamoDbClient):44 Using Role Based Access
2025-05-28 14:04:29 error ERROR i.a.i.s.d.DynamodbSource(check):64 Error while listing Dynamodb tables with reason: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: User: arn:aws:sts::account_A:assumed-role/AccountARole/aws-sdk-java-** is not authorized to perform: dynamodb:ListTables on resource: arn:aws:dynamodb:ap-southeast-2:account_A:table/* because no identity-based policy allows the dynamodb:ListTables action (Service: DynamoDb, Status Code: 400, Request ID:***)
```

### Contribute

- [ ] Yes, I want to contribute

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。