[Packetbeat] Mysql message parser doesn't follow sequence numbers
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
Hello,
The mysql message parser doesn't follow the mysql specification and some messages can be mistreated by the packbeat parser.
More precisely, it doesn't handle sequence numbers. The mysql specification states that :
> Data between client and server is exchanged in packets of max 16MByte size [...] The sequence-id is incremented with each packet and may wrap around. It starts at 0 and is reset to 0 when a new command begins in the [Command Phase](https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_command_phase.html).
- https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_packets.html
- https://mariadb.com/kb/en/0-packet/
Here is the current Packetbeat code:
```go
if s.isClient {
// starts Command Phase
if m.seq == 0 && isRequest(m.typ) {
// parse request
m.isRequest = true
m.start = s.parseOffset
s.parseState = mysqlStateEatMessage
} else {
// ignore command
m.ignoreMessage = true
s.parseState = mysqlStateEatMessage
}
} else if !s.isClient {
// parse response
m.isRequest = false
if hdr[4] == 0x00 || hdr[4] == 0xfe {
logp.Debug("mysqldetailed", "Received OK response")
m.start = s.parseOffset
s.parseState = mysqlStateEatMessage
m.isOK = true
} else if hdr[4] == 0xff {
logp.Debug("mysqldetailed", "Received ERR response")
m.start = s.parseOffset
s.parseState = mysqlStateEatMessage
m.isError = true
} else if m.packetLength == 1 {
logp.Debug("mysqldetailed", "Query response. Number of fields %d", hdr[4])
m.numberOfFields = int(hdr[4])
m.start = s.parseOffset
s.parseOffset += 5
s.parseState = mysqlStateEatFields
} else {
// something else. ignore
m.ignoreMessage = true
s.parseState = mysqlStateEatMessage
}
}
```
https://github.com/elastic/beats/blob/main/packetbeat/protos/mysql/mysql.go#L306
As described in the Mysql specification, a query/response larger than 16MB is split into multiple packets. Those packets has a 4 bytes header with an incremented `sequence_id`.
The problem appears if the bits error sequence `0xFF` is seen in the payload of split response packages with `sequence_id > 1`. Packetbeat will mark them as error messages except they are not. There is a similar problem for queries as a big query can be split on multiple packets and with the current code, those packets will be ignored.
Until `sequence_id` can properly be supported in the Packetbeat parser, I propose this quick fix for the problem with split server responses:
```go
} else if m.seq == 1 && hdr[4] == 0xff {
logp.Debug("mysqldetailed", "Received ERR response")
m.start = s.parseOffset
s.parseState = mysqlStateEatMessage
m.isError = true
}
```
This solved the issue we had for a client with a latin1 (yes I know) database.
There is no real specification of the maximum size of a mysql error message but it should be safe to assume it will not exceed 16MB. In the Mysql code we can see [MYSQL_ERRMSG_SIZE: 512](https://dev.mysql.com/doc/dev/mysql-server/latest/mysql__com_8h.html#a3f5f3eab30894e1dfa8d5bd977a889be) anyway.
Contributor guide
Research direction
Start in packetbeat/protos/mysql/mysql.go around line 306 and trace how packet headers, sequence IDs, and parser state are handled for client requests and server responses. Reproduce split packets larger than 16 MB, then verify that sequence IDs are followed and payload bytes such as 0xff are not mistaken for error headers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- databases, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100