pingcap / pingcap/tiproxy

go client reports `invalid connection` after idle for wait_timeout

Open
#327 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

wontfix
Dominant language
Go
Stars
73
Forks
41
Avg merge
21h 3m
Merged PRs (30d)
21

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
dbt.mustExec("SET @@SESSION.wait_timeout = 2")

// wait for TiDB to close our connection
time.Sleep(3 * time.Second)

tx, err := dbt.db.Begin()
if err != nil {
	dbt.Fatal(err)
}
2. What did you expect to see? (Required)

The driver logs packets.go:122: closing bad idle connection: EOF 3 times but dbt.db.Begin doesn't report any errors because it creates a new connection internally and then retry.

3. What did you see instead (Required)

The driver logs packets.go:37: unexpected EOF and dbt.db.Begin reports invalid connection.

4. What is your version? (Required)

master.

The reason:
TiProxy doesn't close the connection immediately after wait_timeout and still receives requests. After it receives the request, it finds that TiDB has disconnected and then closes the client connection. So the client encounters an ErrInvalidConn in readPacket instead of ErrBadConn in writePacket.
The client will retry by creating a new connection if encounters ErrBadConn but won't retry if encounters ErrInvalidConn, see https://github.com/golang/go/blob/master/src/database/sql/sql.go#L1536-1546
I think there's no way to solve it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the Go reproduction in the issue, then inspect packets.go:37 and packets.go:122 alongside the database/sql retry behavior at the linked sql.go lines 1536-1546. The issue does not specify a proposed code change; progress would require determining whether the invalid-connection path can be changed so the observed wait_timeout behavior is handled correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.