crate / crate/commons-codec

DMS: Serialization flaw: DDL/DML events are processed out of order

Open
#101 13 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
2
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## Problem

We are using an AWS Lambda for processing DMS events from a Kinesis stream, and observed out-of-order processing behaviour. In our case, there seems to be a race condition at the beginning of the load where `INSERT` is attempted before `CREATE TABLE`.

![Image](https://github.com/user-attachments/assets/c7476742-24b0-4b06-adbf-7257e80837dd)

## Thoughts

@hammerhead said:

> This makes me think if we have any issue with the serialization of CDC events. For plain INSERTs, the order in which we receive those CDC events from Kinesis and execute them doesn’t matter. But as soon as we have let’s say two UPDATE statements changing the same record, we need to be sure to execute them in the right order (latest should win). Not sure yet how Lambda behaves in terms of concurrency and if Kinesis maintains any order of events.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.