ERROR Timeout while waiting for events up to lock.
- Dominant language
- Go
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 2h 31m
- Merged PRs (30d)
- 4
Description
command:
gh-ost --allow-on-master --assume-rbr --initially-drop-ghost-table --initially-drop-old-table --chunk-size=1500 --hooks-path=/tmp/hook --exact-rowcount --user=root--password="123.com" --host=10.10.1.202 --port=3306 --assume-master-host=10.10.1.202:3306 --alter='xxx' -execute
The error:
```shell
2019-01-23 11:27:44 INFO Looking for magic cut-over table.
2019-01-23 11:27:44 INFO Will now proceed to drop magic table and unlock tables.
2019-01-23 11:27:44 INFO Dropping magic cut-over table.
2019-01-23 11:27:44 INFO Releasing lock from `broker`.`b_invoice`, `broker`.`_b_invoice_del`.
2019-01-23 11:27:45 INFO Tables unlocked.
2019-01-23 11:27:45 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 28s(total), 7s(copy); streamer: mysql-bin.005986:88395447; State: migrating; ETA: due.
2019-01-23 11:27:46 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 29s(total), 7s(copy); streamer: mysql-bin.005986:88551253; State: migrating; ETA: due.
2019-01-23 11:27:46 DEBUG checking for cut-over postpone.
2019-01-23 11:27:46 DEBUG checking for cut-over postpone: complete.
2019-01-23 11:27:46 INFO Grabbing voluntary lock: gh-ost.8877.lock.
2019-01-23 11:27:46 INFO Setting LOCK timeout as 6 seconds.
2019-01-23 11:27:46 INFO Looking for magic cut-over table.
2019-01-23 11:27:46 INFO Creating magic cut-over table `broker`.`_b_invoice_del`.
2019-01-23 11:27:46 INFO Magic cut-over table created.
2019-01-23 11:27:46 INFO Locking `broker`.`b_invoice`, `broker`.`_b_invoice_del`.
2019-01-23 11:27:47 INFO Tables locked.
2019-01-23 11:27:47 INFO Session locking original & magic tables is 8877.
2019-01-23 11:27:47 INFO Writing changelog state: AllEventsUpToLockProcessed:1548214067024736192.
2019-01-23 11:27:47 INFO Waiting for events up to lock.
2019-01-23 11:27:47 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 30s(total), 7s(copy); streamer: mysql-bin.005986:88671411; State: migrating; ETA: due.
2019-01-23 11:27:48 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 31s(total), 7s(copy); streamer: mysql-bin.005986:88958928; State: migrating; ETA: due.
2019-01-23 11:27:49 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 32s(total), 7s(copy); streamer: mysql-bin.005986:89309509; State: migrating; ETA: due.
2019-01-23 11:27:50 ERROR Timeout while waiting for events up to lock.
2019-01-23 11:27:50 ERROR 2019-01-23 11:27:50 ERROR Timeout while waiting for events up to lock.
goroutine 1 [running]:.
runtime/debug.Stack(0x5f, 0x100, 0xc42001e1e0).
/usr/local/go/src/runtime/debug/stack.go:24 +0xa7.
runtime/debug.PrintStack().
/usr/local/go/src/runtime/debug/stack.go:16 +0x22.
github.com/github/gh-ost/vendor/github.com/outbrain/golib/log.logErrorEntry(0x2, 0x93ed00, 0xc4204dcf00, 0x0, 0x100000001).
/home/shlomi-noach/go/src/github.com/github/gh-ost/vendor/github.com/outbrain/golib/log/log.go:178 +0xfd.
github.com/github/gh-ost/vendor/github.com/outbrain/golib/log.Errore(0x93ed00, 0xc4204dcf00, 0xc4204dcf00, 0x1).
/home/shlomi-noach/go/src/github.com/github/gh-ost/vendor/github.com/outbrain/golib/log/log.go:224 +0x3e.
github.com/github/gh-ost/go/logic.(*Migrator).atomicCutOver(0xc420122090, 0x0, 0x0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:634 +0xa48.
github.com/github/gh-ost/go/logic.(*Migrator).cutOver(0xc420122090, 0x2, 0xc420240b80).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:524 +0x286.
github.com/github/gh-ost/go/logic.(*Migrator).(github.com/github/gh-ost/go/logic.cutOver)-fm(0x3b9aca00, 0xc420192620).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:413 +0x2a.
github.com/github/gh-ost/go/logic.(*Migrator).retryOperation(0xc420122090, 0xc4202905d0, 0x0, 0x0, 0x0, 0x1, 0xc4202905d0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:140 +0x75.
github.com/github/gh-ost/go/logic.(*Migrator).(github.com/github/gh-ost/go/logic.retryOperation)-fm(0xc4202905d0, 0x0, 0x0, 0x0, 0x1, 0xc4200181b0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:411 +0x52.
github.com/github/gh-ost/go/logic.(*Migrator).Migrate(0xc420122090, 0x0, 0x0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:413 +0x873.
main.main().
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/cmd/gh-ost/main.go:252 +0x1da0.
```
What is the cause? please, thanks
Contributor guide
Assessment
This issue has not been assessed yet.