julien-duponchelle / julien-duponchelle/python-mysql-replication

Potential parser desynchronization and lack of boundary validation in binlog event post-header extraction

Open
#644 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
2.4k
Forks
690
PR merge metrics
No merged PRs in 30d

Description

### Description
While reviewing the event processing loop within pymysql-replication, I observed that the event deserialization classes (such as QueryEvent, RowEvent, and others) do not utilize the post_header_lengths array declared dynamically by the server inside the FORMAT_DESCRIPTION_EVENT (FDE).

Instead, the library assumes static, hardcoded post-header sizes for individual event types. This exposes the parser to two distinct risks: a forward-compatibility failure if the underlying databases expand event formats, and a potential runtime truncation error due to missing length validation guards during the extraction of the FDE schema array itself.

### Forward compatibility
The MySQL/MariaDB replication protocol guarantees that replicas can parse unknown server variants by reading the explicit layout lengths from the FDE array. If an event type increases its post-header length in a newer server release, a compliant client must dynamically jump past the server-defined length (payload_begin = post_header_start + post_header_len) rather than assuming static byte offsets.

E.g. in QueryEvent the parsing logic reads a static 13-byte block (4 + 4 + 1 + 2 + 2) explicitly:

```
# Post-header (Hardcoded 13 bytes assumption)
self.slave_proxy_id = self.packet.read_uint32()
self.execution_time = self.packet.read_uint32()
self.schema_length = struct.unpack("!B", self.packet.read(1))[0]
self.error_code = self.packet.read_uint16()
self.status_vars_length = self.packet.read_uint16()

# Payload
status_vars_end_pos = self.packet.read_bytes + self.status_vars_length

```

If the server specifies that the QueryEvent post-header is actually 15 or 19 bytes, the packet reader cursor will stop premature of the actual payload boundaries. It will mistakenly read the remaining post-header metadata bytes as the payload data (status_vars), resulting in immediate deserialization failures, incorrect metadata tracking, or data stream corruption.

### Lack of Minimum Length Verification on FDE Initialization

When parsing the FormatDescriptionEvent, the client must unpack the dynamic event type length array. If a malformed or corrupted network packet reports an unnaturally shortened event length configuration, or if the stream provides fewer bytes than the type array footprint requires, the parser will throw unexpected Python runtime errors (such as struct.error: unpack requires a buffer of X bytes) rather than gracefully catching a MalformedPacket condition. This needs to be checked for both MySQL and MariaDB since their post_header_lengths differ for several events.

Contributor guide

Open the contributing guide

Research direction

Start at the event processing loop and compare QueryEvent, RowEvent, and FormatDescriptionEvent handling for MySQL and MariaDB. Trace how the server-defined post_header_lengths array is extracted and how event payload boundaries are computed; done means malformed or extended layouts are handled without truncation or unexpected struct errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.