binlog streamer throwing error: unsupport type 141 in binlog and don't know how to handle
- Dominant language
- Go
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 2h 31m
- Merged PRs (30d)
- 4
Description
Hello,
We are evaluating gh-ost tool for doing online schema migration for our database system. While testing the tool on pre-production environment we seem to have encountered an issue with this tool which looks like a bug. Upon further investigation we found ghost (looks really to be go-mysql lib issue) throws out errors when binlog events contains write event record related to any unrelated table having compression enabled at column level. It seems that the go-mysql lib is unable to parse event for such scenarios.
I am able to replicate the issue both in percona mysql and mariadb 10.4.13/10.4.18.
Following is the test scenario:
```
create table test123 (
auto_seq bigint(20) unsigned NOT NULL AUTO_INCREMENT,
name VARCHAR(512) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT utc_timestamp(),
PRIMARY KEY (auto_seq),
) ENGINE=InnoDB DEFAULT CHARSET=binary;
```
insert records:
```
insert into test123 (name)
values
("hello"), ("world"), ("test123"), ("john doe"), ("blah");
```
Create another table comp_test with one column in compressed format
```
create table comp_test (
auto_seq bigint(20) unsigned NOT NULL AUTO_INCREMENT,
name VARCHAR(512) COMPRESSED CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT utc_timestamp(),
PRIMARY KEY (auto_seq),
) ENGINE=InnoDB DEFAULT CHARSET=binary;
```
Run below ghost command for doing online schema migration for table test123
```
./gh-ost --max-load=Threads_running=25 --critical-load=Threads_running=1000 --chunk-size=5000 --max-lag-millis=15000 --conf="database.cnf" --host=localhost --allow-on-master --database="testdb" --table="test123" --verbose --alter="ENGINE=INNODB" --switch-to-rbr --allow-master-master --cut-over=default --exact-rowcount --concurrent-rowcount --default-retries=240 --panic-flag-file=testdb.test123.panic.flag --postpone-cut-over-flag-file=testdb.test123.postpone.flag --serve-socket-file=testdb.test123.sock --initially-drop-ghost-table --execute
```
Now while ghost is running, if there is a write event record for the above **comp_test** table, we observe that ghost tool would start complaining with below error
[2021/07/29 01:45:45] [info] binlogsyncer.go:354 begin to sync binlog from position (ON.000003, 4)
[2021/07/29 01:45:45] [info] binlogsyncer.go:203 register slave for master server localhost:3306
[2021/07/29 01:45:45] [info] binlogsyncer.go:723 rotate to (ON.000003, 4)
2021-07-29 01:45:45 INFO rotate to next log from ON.000003:0 to ON.000003
[2021/07/29 01:45:45] [error] binlogstreamer.go:77 close sync with err: Header &replication.EventHeader{Timestamp:0x6102572d, EventType:0x17, ServerID:0x1, EventSize:0x4b1, LogPos:0x3ccd4, Flags:0x0}, Data "some bunch of data in binary format", Err: unsupport type 141 in binlog and don't know how to handle
We can replicate the above error by issuing below insert command while ghost is actively doing migration or waiting for cut-over:
```
insert into comp_test (name) values ("danger");
```
I am curious if anyone faced this issue. Many thanks in advance!!
Contributor guide
Assessment
This issue has not been assessed yet.