pingcap / pingcap/tiup

tiup playground scale-in panics

Open
#1,314 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/playground type/bug
Dominant language
Go
Stars
466
Forks
338
Avg merge
3d 7h
Merged PRs (30d)
8

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
tiup playground nightly --db 0
tiup playground scale-out --db 1
tiup playground scale-in --pid xxxxx
  1. What did you expect to see?

The TiDB node added by scale-out is stopped.

  1. What did you see instead?

The scane-in command's output:

Error: Post "http://127.0.0.1:9527/command": EOF
Error: run `/home/tidb/.tiup/components/playground/v1.4.1/tiup-playground` (wd:/home/tidb/.tiup/data/SUj1ZcK) failed: exit status 1

tiup playground's output:

receive command: scale-in
2021-04-16 06:51:32.365126 I | http: panic serving 127.0.0.1:45502: runtime error: invalid memory address or nil pointer dereference
goroutine 193 [running]:
net/http.(*conn).serve.func1(0xc00040a000)
        net/http/server.go:1824 +0x153
panic(0x5618ed0b1f80, 0x5618ed7a27a0)
        runtime/panic.go:971 +0x499
github.com/pingcap/tiup/components/playground/instance.(*TiFlashInstance).Pid(0xc0001a7ad0, 0x16241)
        <autogenerated>:1 +0x34
main.(*Playground).handleScaleIn.func1(0x5618eca761b7, 0x7, 0x5618ed20b9d8, 0xc0001a7ad0, 0x0, 0x0)
        github.com/pingcap/tiup/components/playground/playground.go:251 +0x52
main.(*Playground).WalkInstances(0xc000168000, 0xc001415988, 0xc0001b4060, 0x1a)
        github.com/pingcap/tiup/components/playground/playground.go:574 +0x46e
main.(*Playground).handleScaleIn(0xc000168000, 0x5618ed1ed1f8, 0xc0003240e0, 0x16241, 0xc001415a08, 0x1)
        github.com/pingcap/tiup/components/playground/playground.go:250 +0xb4
main.(*Playground).handleCommand(0xc000168000, 0xc0001e2310, 0x5618ed1ed1f8, 0xc0003240e0, 0xc0003240e0, 0x0)
        github.com/pingcap/tiup/components/playground/playground.go:469 +0x177
main.(*Playground).commandHandler(0xc000168000, 0x5618ed2044c0, 0xc0003240e0, 0xc0008fc000)
        github.com/pingcap/tiup/components/playground/playground.go:503 +0x334
net/http.HandlerFunc.ServeHTTP(0xc0004e7190, 0x5618ed2044c0, 0xc0003240e0, 0xc0008fc000)
        net/http/server.go:2069 +0x46
net/http.(*ServeMux).ServeHTTP(0x5618ed7bfb40, 0x5618ed2044c0, 0xc0003240e0, 0xc0008fc000)
        net/http/server.go:2448 +0x1ad
net/http.serverHandler.ServeHTTP(0xc000325a40, 0x5618ed2044c0, 0xc0003240e0, 0xc0008fc000)
        net/http/server.go:2887 +0xa5
net/http.(*conn).serve(0xc00040a000, 0x5618ed2065f0, 0xc000188180)
        net/http/server.go:1952 +0x8cd
created by net/http.(*Server).Serve
        net/http/server.go:3013 +0x3ad
  1. What version of TiUP are you using (tiup --version)?

Didn't understand what actual version it is.

~$ tiup --version
v1.3.2 tiup
Go Version: go1.13
Git Branch: release-1.3
GitHash: 2d88460
~$ tiup playground --version
Starting component `playground`: /home/tidb/.tiup/components/playground/v1.4.1/tiup-playground --version
tiup version 1.4.1 tiup
Go Version: go1.16.3
Git Ref: v1.4.1
GitHash: cd19b75b6418f627d121d43d4b1e41af673526cf

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with components/playground/playground.go at handleScaleIn around lines 250-251 and inspect WalkInstances around line 574, using the reported TiFlashInstance.Pid nil-pointer stack trace. Reproduce with the listed playground scale-out and scale-in commands. Done means scale-in stops the added TiDB node without panicking or returning the HTTP EOF error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.