airbytehq / airbytehq/airbyte

[source-dynamodb] connector does not work in AWS cross-account setting with Role based authentication

Aperta
#60,944 4 commenti 7 reazioni 0 assegnatari Vedi su GitHub
area/connectors community connectors/source/dynamodb team/db-dw-sources type/bug
Lingua principale
Python
Stelle
22.1k
Fork
5.3k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

### Connector Name

source-dynamodb

### Connector Version

0.3.9

### What step the error happened?

Configuring a new connector

### Relevant information

Hello

I have the following setup:

Airbyte is hosted in AWS account A.
DynamoDB tables are hosted in AWS account B.

I am trying to set up a dynamodb source in airbyte, using role based authentication.
The role belongs to account A, and has permissions to perform dynamodb actions on account B.

Account B tables have a policy which allows account A role to access them, according to the documentation.
https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/configure-cross-account-access-to-amazon-dynamodb.html#configure-cross-account-access-to-amazon-dynamodb-summary

When trying to set up the source, the first action taken is trying to list the tables in the same account, which is not the desired behaviour, since I'm trying to sync from a different account. This also fails, since the role in account A does not have permissions to list tables in account A, and it's irrelevant since the tables are in account B.

I also tried to set up [AWS PrivateLink for DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/privatelink-interface-endpoints.html), hoping it would "force" the client to perform the actions in the appropriate account, but to no avail.

As a possible solution, I would like to enter the full source ARN, so that the SDK tries to query the correct table in the correct account.
This is the suggested solution in the example provided by AWS:
https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/configure-cross-account-access-to-amazon-dynamodb.html#configure-cross-account-access-to-amazon-dynamodb-additional

Thank you!

### Relevant log output

```shell
2025-05-28 14:04:29 info INFO i.a.i.s.d.DynamodbUtils(createDynamoDbClient):44 Using Role Based Access
2025-05-28 14:04:29 error ERROR i.a.i.s.d.DynamodbSource(check):64 Error while listing Dynamodb tables with reason: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: User: arn:aws:sts::account_A:assumed-role/AccountARole/aws-sdk-java-** is not authorized to perform: dynamodb:ListTables on resource: arn:aws:dynamodb:ap-southeast-2:account_A:table/* because no identity-based policy allows the dynamodb:ListTables action (Service: DynamoDb, Status Code: 400, Request ID:***)
```

### Contribute

- [ ] Yes, I want to contribute

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.