serverless / serverless/serverless
AWS streams: Introduce `dynamodb` and `kinesis` events, deprecate `stream`
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 46.9k
- Forks
- 5.7k
- Avg merge
- 10h 7m
- Merged PRs (30d)
- 57
Description
Initially discussed here: https://github.com/serverless/serverless/issues/8117#issuecomment-680727895
Use case description
To avoid ambiguity it'll be nice to handle both stream types with two distinct event types.
Benefits of that change:
- Event source type is clear upfront at event defitnition
- Internally we do not have to resolve event source type from ARN (and possiblities of that are limited)
- Allows to improve property naming (e.g.
batchWindowandmaximumRecordAgeInSecondsdoesn't seem to follow same convention) - Allows to address inconsistencies and bugs as:
- In
streamevent defaultbatchSizeto10while on AWS grounds default is100 In(fixed with https://github.com/serverless/serverless/pull/8202)stream0is not recognized value forbatchWindow, while it is a supported value o AWS grounds
- In
Proposed solution
- Deprecate currently implemented
streamevent - Introduce
kinesisanddynamodbevents, but configure them with sameAWS::Lambda::EventSourceMappingresource generator.
As we need to leavestreamevent working as is, I believe this should be introduced as new implementation.
List of properties to be supported by those events (note new naming in some cases):
batchSize: optional, maps toBatchSizebisectBatchOnFunctionError: optional, maps toBisectBatchOnFunctionErroronFailureDestination: optional, maps toDestinationConfig.OnFailurearn: required, maps toEventSourceArnmaximumBatchingWindow: optional, maps toMaximumBatchingWindowInSecondsmaximumRecordAge: optional, maps toMaximumRecordAgeInSecondsmaximumRetryAttempts: optional, maps toMaximumRetryAttemptsparallelizationFactor: optional, maps toParallelizationFactorstartingPosition: optional (but required in AWS), maps toStartingPositionand we should map toTRIM_HORIZONas default (as in case ofstreamevent)
- New events should be backed by new tests, configured with
runServerlessutil (follow: https://github.com/serverless/serverless/tree/master/tests#unit-tests). Integration tests should be also introduced (we may reuse those already configured for those events, but it'll also be good to keep testngstreamevent) - Documentation should be updated to document
dynamodbandkinesisas separate events (it's fine to remove documentation forstreamevent)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the existing stream event implementation and its shared AWS::Lambda::EventSourceMapping resource generator. Read the unit-test guidance around runServerless, then inspect the existing integration tests for stream events. Done means separate dynamodb and kinesis events, compatibility handling for stream, updated unit and integration coverage, and documentation for the new event names.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, javascript
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100