awslabs / awslabs/amazon-kinesis-client-nodejs
KCL nodejs with dynamodb local
- Dominant language
- JavaScript
- Stars
- 304
- Forks
- 198
- Avg merge
- 36m
- Merged PRs (30d)
- 2
Description
### Configuration
KCL nodejs version: `2.3.1`
- Env variables used:
```
AWS_ACCESS_KEY_ID=fake
AWS_SECRET_ACCESS_KEY=fake
```
- `.properties` configuration
```
executableName=/dist/kclClient/handler.js
streamName=arn:aws:dynamodb:ddblocal:000000000000:table/local_Dynamo-table/stream/2025-04-28T22:52:06.529
applicationName=KCL_local_Dynamo-table
initialPositionInStream=LATEST
dynamodbEndpoint=http://localhost:6007 // dynamodblocal instance runs on 6007
AwsCredentialsProvider=DefaultCredentialsProvider
```
- command to run
```
-e -p --java /usr/bin/java
```
### Issue
- The node-js client doesn't seem to working when using StreamARN that is created on DynamoDBlocal.
- Noticing that the error logs point to security token be not being valid.
```
2025-04-29 17:16:55,655 [multi-lang-daemon-0000] ERROR s.a.kinesis.coordinator.Scheduler [NONE] - Caught exception when initializing LeaseCoordinator
software.amazon.kinesis.leases.exceptions.DependencyException: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: The security token included in the request is invalid. (Service: DynamoDb, Status Code: 400, Request ID: TCIHPG78RU1KGM97NVMUHSGM33VV4KQNSO5AEMVJF66Q9ASUAAJG)
at software.amazon.kinesis.leases.dynamodb.DynamoDBLeaseRefresher.createTableIfNotExists(DynamoDBLeaseRefresher.java:355)
at software.amazon.kinesis.leases.dynamodb.DynamoDBLeaseRefresher.createLeaseTableIfNotExists(DynamoDBLeaseRefresher.java:294)
at software.amazon.kinesis.leases.dynamodb.DynamoDBLeaseCoordinator.initialize(DynamoDBLeaseCoordinator.java:279)
at software.amazon.kinesis.coordinator.Scheduler.initialize(Scheduler.java:372)
at software.amazon.kinesis.coordinator.Scheduler.run(Scheduler.java:346)
at software.amazon.kinesis.multilang.MultiLangDaemon$MultiLangRunner.call(MultiLangDaemon.java:96)
at software.amazon.kinesis.multilang.MultiLangDaemon$MultiLangRunner.call(MultiLangDaemon.java:87)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
```
### Followup
- I did check this issue mentioned on KCL repo
- [[Issue]( https://github.com/awslabs/amazon-kinesis-client/issues/652)]: However, this is very specific to `localstack` and not specific to `dynamodblocal`
- I did also checkout the following article about DynamoDB Streams Kinesis adapter [link](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.KCLAdapter.html). Does this mean, the kcl node.js library cannot be used for processing dynamo streams? If so, would appreciate if you could point the dynamo stream adapter for node.js(I know this is currently only available in java, are there any plans that we could expect in future)
- Could the issue thread be specific to node.js implementation 🙏.
Readers:
- 👍 if you've experienced something similar
- 👎 if you don't find this relevant — I'd appreciate hearing your thoughts either way
Contributor guide
Research direction
Start by reproducing the KCL bootstrap command with the shown .properties file, then inspect the failure at DynamoDBLeaseRefresher.createTableIfNotExists and the Node.js handler at /dist/kclClient/handler.js. Determine whether DynamoDB Local stream ARNs are supported by this client configuration and document the required setup or confirmed limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java, javascript, nodejs
- Domain
- backend, databases, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100