awslabs / awslabs/amazon-kinesis-client-nodejs
Shutdown vs leaseLost and shardEnded
- Dominant language
- JavaScript
- Stars
- 304
- Forks
- 198
- Avg merge
- 36m
- Merged PRs (30d)
- 2
Description
Hi Team,
Version 1 of this library required a shutdown function to be exposed, which v2 replaced with `leaseLost` and `shardEnded`. Having migrated to v2, I went through the process of implementing the new `leaseLost` and `shardEnded` functions and deployed the code (a few months ago). All went well until i had to split the shards which caused more than an hour of downtime while hunting for reasons why we weren't able to get beyond the shard split.
Turns out, v2 appears to be checking for the shutdown function and, if present, invokes it before doing any sort of checks for the new functions first. In my case, I rerouted the new leaseLost and shardEnded functions into the shutdown function I had before the v2 migration (with slight modifications):
```ts
leaseLost(_leaseLostInput: kcl.LeaseLossInput, completeCallback: kcl.Callback): void {
return this.shutdown('LeaseLost', undefined, completeCallback);
}
shardEnded(shardEndedInput: kcl.ShardEndedInput, completeCallback: kcl.Callback): void {
return this.shutdown('ShardEnded', shardEndedInput.checkpointer, completeCallback);
}
shutdown(reason: string, checkpointer: kcl.Checkpointer, completeCallback: kcl.Callback): void {
...
}
```
Having a `shutdown` function on my processor resulted in the kcl not invoking the `shardEnded` function and instead called my `shutdown` function, with my `reason` actually being a checkpointer, and my `checkpointer` was actually the completeCallback function. This failed horribly.
I suggest reworking this part of the codebase so that it attempts to call the new functions first, and if the new functions aren't present, then revert to attempting the `shutdown` function.
Contributor guide
Research direction
No file or test is named in the issue. Start by locating the callback dispatch for leaseLost, shardEnded, and shutdown; verify which handler is selected during shard splitting. Done means the newer callbacks are preferred, shutdown remains a fallback, and the callback arguments are passed to the selected handler correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100