ERROR Timeout while waiting for events up to lock.
- Lingua principale
- Go
- Stelle
- 13.6k
- Fork
- 1.4k
- Merge medio
- 2h 31m
- PR unite (30g)
- 4
Descrizione
command:
gh-ost --allow-on-master --assume-rbr --initially-drop-ghost-table --initially-drop-old-table --chunk-size=1500 --hooks-path=/tmp/hook --exact-rowcount --user=root--password="123.com" --host=10.10.1.202 --port=3306 --assume-master-host=10.10.1.202:3306 --alter='xxx' -execute
The error:
```shell
2019-01-23 11:27:44 INFO Looking for magic cut-over table.
2019-01-23 11:27:44 INFO Will now proceed to drop magic table and unlock tables.
2019-01-23 11:27:44 INFO Dropping magic cut-over table.
2019-01-23 11:27:44 INFO Releasing lock from `broker`.`b_invoice`, `broker`.`_b_invoice_del`.
2019-01-23 11:27:45 INFO Tables unlocked.
2019-01-23 11:27:45 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 28s(total), 7s(copy); streamer: mysql-bin.005986:88395447; State: migrating; ETA: due.
2019-01-23 11:27:46 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 29s(total), 7s(copy); streamer: mysql-bin.005986:88551253; State: migrating; ETA: due.
2019-01-23 11:27:46 DEBUG checking for cut-over postpone.
2019-01-23 11:27:46 DEBUG checking for cut-over postpone: complete.
2019-01-23 11:27:46 INFO Grabbing voluntary lock: gh-ost.8877.lock.
2019-01-23 11:27:46 INFO Setting LOCK timeout as 6 seconds.
2019-01-23 11:27:46 INFO Looking for magic cut-over table.
2019-01-23 11:27:46 INFO Creating magic cut-over table `broker`.`_b_invoice_del`.
2019-01-23 11:27:46 INFO Magic cut-over table created.
2019-01-23 11:27:46 INFO Locking `broker`.`b_invoice`, `broker`.`_b_invoice_del`.
2019-01-23 11:27:47 INFO Tables locked.
2019-01-23 11:27:47 INFO Session locking original & magic tables is 8877.
2019-01-23 11:27:47 INFO Writing changelog state: AllEventsUpToLockProcessed:1548214067024736192.
2019-01-23 11:27:47 INFO Waiting for events up to lock.
2019-01-23 11:27:47 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 30s(total), 7s(copy); streamer: mysql-bin.005986:88671411; State: migrating; ETA: due.
2019-01-23 11:27:48 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 31s(total), 7s(copy); streamer: mysql-bin.005986:88958928; State: migrating; ETA: due.
2019-01-23 11:27:49 DEBUG Getting nothing in the write queue. Sleeping....
Copy: 24339/24339 100.0%; Applied: 0; Backlog: 0/1000; Time: 32s(total), 7s(copy); streamer: mysql-bin.005986:89309509; State: migrating; ETA: due.
2019-01-23 11:27:50 ERROR Timeout while waiting for events up to lock.
2019-01-23 11:27:50 ERROR 2019-01-23 11:27:50 ERROR Timeout while waiting for events up to lock.
goroutine 1 [running]:.
runtime/debug.Stack(0x5f, 0x100, 0xc42001e1e0).
/usr/local/go/src/runtime/debug/stack.go:24 +0xa7.
runtime/debug.PrintStack().
/usr/local/go/src/runtime/debug/stack.go:16 +0x22.
github.com/github/gh-ost/vendor/github.com/outbrain/golib/log.logErrorEntry(0x2, 0x93ed00, 0xc4204dcf00, 0x0, 0x100000001).
/home/shlomi-noach/go/src/github.com/github/gh-ost/vendor/github.com/outbrain/golib/log/log.go:178 +0xfd.
github.com/github/gh-ost/vendor/github.com/outbrain/golib/log.Errore(0x93ed00, 0xc4204dcf00, 0xc4204dcf00, 0x1).
/home/shlomi-noach/go/src/github.com/github/gh-ost/vendor/github.com/outbrain/golib/log/log.go:224 +0x3e.
github.com/github/gh-ost/go/logic.(*Migrator).atomicCutOver(0xc420122090, 0x0, 0x0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:634 +0xa48.
github.com/github/gh-ost/go/logic.(*Migrator).cutOver(0xc420122090, 0x2, 0xc420240b80).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:524 +0x286.
github.com/github/gh-ost/go/logic.(*Migrator).(github.com/github/gh-ost/go/logic.cutOver)-fm(0x3b9aca00, 0xc420192620).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:413 +0x2a.
github.com/github/gh-ost/go/logic.(*Migrator).retryOperation(0xc420122090, 0xc4202905d0, 0x0, 0x0, 0x0, 0x1, 0xc4202905d0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:140 +0x75.
github.com/github/gh-ost/go/logic.(*Migrator).(github.com/github/gh-ost/go/logic.retryOperation)-fm(0xc4202905d0, 0x0, 0x0, 0x0, 0x1, 0xc4200181b0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:411 +0x52.
github.com/github/gh-ost/go/logic.(*Migrator).Migrate(0xc420122090, 0x0, 0x0).
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/logic/migrator.go:413 +0x873.
main.main().
/home/shlomi-noach/go/src/github.com/github/gh-ost/go/cmd/gh-ost/main.go:252 +0x1da0.
```
What is the cause? please, thanks
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia con il comando e lo stack trace segnalati, poi leggi go/logic/migrator.go intorno ad atomicCutOver alla riga 634 e i suoi chiamanti cutOver e retryOperation. Riproduci il timeout, se possibile, e traccia l’attesa degli eventi fino al lock; il lavoro è completato quando viene identificata e documentata o risolta una causa supportata, con un test di regressione se il repository ne fornisce uno.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- go, mysql
- Ambito
- databases
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 20/100