github / github/gh-ost

Simplify the chunk range condition generate logic, reduce the risk of using wrong execute plan(enhancement)

オープン
#855 コメント 5 件 リアクション 0 件 担当者 0 名 GitHub で見る
enhancement
主要言語
Go
スター
13.6k
フォーク
1.4k
平均マージ
2時間 31分
マージ済み PR(30日)
4

説明

> We meet a performance case when use gh-ost to do ddl

### Problem Description

When we do a ddl on a table which has a composed primary key, active session arise quickly and the database hang。 But when we do the same ddl operation using pt-online-schema-change, it worked。 It is very confusing。 So we want to do a deep analysis to get the reason。

### The reason We got

when we compare the chunk sql of pt-osc and gh-ost . The Range condition has some different like below:
The table's primary key compose with three columns : shard_key, bt_id, object
for the same chunk
start : (973,50,.trash/9d16-4b95-bb50-xxxxxxxxxx155 )
end : (973,50,.trash/9d16-4b95-bb50-xxxxxxxxxx2ss )

#### the range condition from gh-ost

```
WHERE (`shard_key` > _binary '973'
OR (`shard_key` = _binary '973'
AND `bt_id` > _binary '50')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` > _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx155'))
AND (`shard_key` < _binary '973'
OR (`shard_key` = _binary '973'
AND `bt_id` < _binary '50')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` < _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx155')
OR (`shard_key` = _binary '973'
AND `bt_id` = _binary '50'
AND `object` = _binary '.trash/9d16-4b95-bb50-xxxxxxxxxx2ss'))
```

#### The range condition from pt-online-schema-change

```
WHERE (`shard_key` > '973'
OR (`shard_key` = '973'
AND `bt_id` > '50')
OR (`shard_key` = '973'
AND `bt_id` = '50'
AND `object` >= '.trash/9d16-4b95-bb50-xxxxxxxxxx155'))
AND (`shard_key` < '973'
OR (`shard_key` = '973'
AND `bt_id` < '50')
OR (`shard_key` = '973'
AND `bt_id` = '50'
AND `object` <= '.trash/9d16-4b95-bb50-xxxxxxxxxx2ss'))
```
We Can see that the condition of gh-ost is more complex than pt-osc. when we check the sql execution plan, they are all using primary key .
pt-osc'sql scan only 499 rows , but gh-ost scan 20 millions rows and it take 30 seconds to accomplish .

We think the query condition is more complex, the database optimizer has more risk to using a wrong plan .

### Optimization

We change the chunk query condition generate logic to make gh-ost generate a chunk sql query like pt-osc does . Making the condition more simple and readable , We also do more test about data consistency, It worked well now . So we want give this pr to gh-ost .


> Thank you!

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

まず、チャンクの範囲条件を生成するロジックを見つけ、その出力を issue にある pt-online-schema-change の例と比較します。報告された複合主キーのケース、データの一貫性、行数、実行計画に対して、修正した条件を検証します。完了の条件は、過剰なスキャンを伴わない、より単純な生成 SQL になることです。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
go, mysql
領域
databases
issue の種類
機能追加
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。