github / github/gh-ost

injectHeartbeat writing failed on teardown

未關閉
#1,663 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Go
星號
13.6k
分支
1.4k
平均合併
2 小時 31 分鐘
30 天內合併 PR
4

描述

Hi all!

Environment:
Azure Mysql Flexible Sever, 8.0.44-azure
gh-ost 1.1.8

We have an instance with a number of partitioned tables and a large schema metadata. With this setup, the dry run check fails with the following errors (from the first to the last):

```code
Closed streamer connection. err=
Dropping table `_ghc`
Table dropped
Error 1146 (42S02): Table '_ghc' doesn't exist
...
Error 1146 (42S02): Table '_ghc' doesn't exist
...
injectHeartbeat writing failed 61 times, last error: Error 1146 (42S02): Table '_ghc' doesn't exist
```

It appears that on slow disks/thousands of tables, [finalCleanup](https://github.com/github/gh-ost/blob/v1.1.8/go/logic/migrator.go#L1723) is executed too early and drops the table before [teardown](https://github.com/github/gh-ost/blob/v1.1.8/go/logic/migrator.go#L373) (where injectHeartbeat actually stops). And deleting a '_ghc' table takes longer than `default-retries` (60) * `heartbeat-interval-millis` (100 ms).

Currently, this can be fixed by increasing `default-retries` or `heartbeat-interval-millis` (or both), but this seems like a workaround.

Is it possible to fix the order of execution?

Thanks in advance!

貢獻指南

開啟貢獻指南

研究方向

Start in go/logic/migrator.go at finalCleanup and teardown, then trace where injectHeartbeat is stopped during the dry-run flow. Reproduce with a slow cleanup or large schema and verify that heartbeat writes stop before the _ghc table is dropped, without the reported teardown errors.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
go, mysql
領域
database
Issue 類型
缺陷
難度
3/5
預估耗時
1-2 天
活躍度
冷清
描述清晰度
描述清楚
新手友好度
58/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。