tikv / tikv/pd

Connection won't be re-established if DNS has a delay

Open
#9,894 0 comments 0 reactions 0 assignees View on GitHub
affects-8.5 component/mcs severity/major type/bug type/cse
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Bug Report

### What did you do?

We have a cache to manage the mapping between URL and gRPC connection:
https://github.com/tikv/pd/blob/e2ce37be70eb478137bd22b34843c5fff8481d28/client/servicediscovery/tso_service_discovery.go#L124

But if the DNS has a delay, the gRPC connection won't be re-established. If the pod IP can be reused, there is a possibility that the request is sent to the wrong place. We have encountered this issue in the serverless environment, which causes the tso request to be sent to the tso server, which is not the primary. And the client keeps trying at the same connection, which is wrong until the TiDB is restarted.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.