serverless / serverless/serverless

AWS streams: Introduce `dynamodb` and `kinesis` events, deprecate `stream`

Open
#8,137 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

breaking bug/design cat/aws-event-stream deprecation enhancement needs feedback
Dominant language
JavaScript
Stars
46.9k
Forks
5.7k
Avg merge
10h 7m
Merged PRs (30d)
57

Description

Initially discussed here: https://github.com/serverless/serverless/issues/8117#issuecomment-680727895

Use case description

To avoid ambiguity it'll be nice to handle both stream types with two distinct event types.

Benefits of that change:

  • Event source type is clear upfront at event defitnition
  • Internally we do not have to resolve event source type from ARN (and possiblities of that are limited)
  • Allows to improve property naming (e.g. batchWindow and maximumRecordAgeInSeconds doesn't seem to follow same convention)
  • Allows to address inconsistencies and bugs as:
Proposed solution
  1. Deprecate currently implementedstream event
  2. Introduce kinesis and dynamodb events, but configure them with same AWS::Lambda::EventSourceMapping resource generator.
    As we need to leave stream event working as is, I believe this should be introduced as new implementation.
    List of properties to be supported by those events (note new naming in some cases):
  • batchSize: optional, maps to BatchSize
  • bisectBatchOnFunctionError: optional, maps to BisectBatchOnFunctionError
  • onFailureDestination: optional, maps to DestinationConfig.OnFailure
  • arn: required, maps to EventSourceArn
  • maximumBatchingWindow: optional, maps to MaximumBatchingWindowInSeconds
  • maximumRecordAge: optional, maps to MaximumRecordAgeInSeconds
  • maximumRetryAttempts: optional, maps to MaximumRetryAttempts
  • parallelizationFactor: optional, maps to ParallelizationFactor
  • startingPosition: optional (but required in AWS), maps to StartingPosition and we should map to TRIM_HORIZON as default (as in case of stream event)
  1. New events should be backed by new tests, configured with runServerless util (follow: https://github.com/serverless/serverless/tree/master/tests#unit-tests). Integration tests should be also introduced (we may reuse those already configured for those events, but it'll also be good to keep testng stream event)
  2. Documentation should be updated to document dynamodb and kinesis as separate events (it's fine to remove documentation for stream event)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the existing stream event implementation and its shared AWS::Lambda::EventSourceMapping resource generator. Read the unit-test guidance around runServerless, then inspect the existing integration tests for stream events. Done means separate dynamodb and kinesis events, compatibility handling for stream, updated unit and integration coverage, and documentation for the new event names.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, javascript
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.