github / github/gh-ost

gh-ost does not copy data on tables with high write load

Open
#1,150 4 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Go
Stars
13.6k
Forks
1.4k
Avg merge
2h 31m
Merged PRs (30d)
4

Description

MySql v8.0.23

I created a table using sysbench with ~68M rows. To simulate a production load, I am running following sysbench script to generate continuous load on machine

`sysbench --db-driver=mysql --mysql-user=xxxxx --mysql_password=xxxxx --mysql-db=sbtest --mysql-host=xxxxx --mysql-port=3306 --tables=1 --threads=20 --time=0 --events=0 --rate=1 --report-interval=10 --rate= /usr/share/sysbench/oltp_read_write.lua run`

When there is continuous data generated like above, gh-ost is unable to migrate by copying existing data and is stuck on handling the newer writes only.

```
Copy: 0/67930960 0.0%; Applied: 49272370; Backlog: 1000/1000; Time: 15h24m30s(total), 15h24m30s(copy); streamer: binlog.000179:773080899; Lag: 0.22s, HeartbeatLag: 46161.22s, State: migrating; ETA: N/A
Copy: 0/67930960 0.0%; Applied: 49298550; Backlog: 1000/1000; Time: 15h25m0s(total), 15h25m0s(copy); streamer: binlog.000179:786810019; Lag: 0.02s, HeartbeatLag: 46187.02s, State: migrating; ETA: N/A
```

What exactly is causing this?
Is there a specific config I need to use allow copy?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.