Simplify the chunk range condition generate logic, reduce the risk of using wrong execute plan(enhancement)
- 主要言語
- Go
- スター
- 13.6k
- フォーク
- 1.4k
- 平均マージ
- 2時間 31分
- マージ済み PR(30日)
- 4
説明
> We meet a performance case when use gh-ost to do ddl
### Problem Description
When we do a ddl on a table which has a composed primary key, active session arise quickly and the database hang。 But when we do the same ddl operation using pt-online-schema-change, it worked。 It is very confusing。 So we want to do a deep analysis to get the reason。
### The reason We got
when we compare the chunk sql of pt-osc and gh-ost . The Range condition has some different like below:
The table's primary key compose with three columns : shard_key, bt_id, object
for the same chunk
start : (973,50,.trash/9d16-4b95-bb50-xxxxxxxxxx155 )
end : (973,50,.trash/9d16-4b95-bb50-xxxxxxxxxx2ss )
#### the range condition from gh-ost
```
WHERE (`shard_key` > _binary '973'
OR (`shard_key` = _binary '973'
AND `bt_id` > _binary '50')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` > _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx155'))
AND (`shard_key` < _binary '973'
OR (`shard_key` = _binary '973'
AND `bt_id` < _binary '50')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` < _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx155')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` = _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx2ss'))
```
#### The range condition from pt-online-schema-change
```
WHERE (`shard_key` > '973'
OR (`shard_key` = '973'
AND `bt_id` > '50')
OR (`shard_key` = '973'
AND `bt_id` = '50'
AND `object` >= '.trash/9d16-4b95-bb50-xxxxxxxxxx155'))
AND (`shard_key` < '973'
OR (`shard_key` = '973'
AND `bt_id` < '50')
OR (`shard_key` = '973'
AND `bt_id` = '50'
AND `object` <= '.trash/9d16-4b95-bb50-xxxxxxxxxx2ss'))
```
We Can see that the condition of gh-ost is more complex than pt-osc. when we check the sql execution plan, they are all using primary key .
pt-osc'sql scan only 499 rows , but gh-ost scan 20 millions rows and it take 30 seconds to accomplish .
We think the query condition is more complex, the database optimizer has more risk to using a wrong plan .
### Optimization
We change the chunk query condition generate logic to make gh-ost generate a chunk sql query like pt-osc does . Making the condition more simple and readable , We also do more test about data consistency, It worked well now . So we want give this pr to gh-ost .
> Thank you!
コントリビューションガイド
調査の方向性
まず、チャンクの範囲条件を生成するロジックを見つけ、その出力を issue にある pt-online-schema-change の例と比較します。報告された複合主キーのケース、データの一貫性、行数、実行計画に対して、修正した条件を検証します。完了の条件は、過剰なスキャンを伴わない、より単純な生成 SQL になることです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- go, mysql
- 領域
- databases
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100