Keyspace related HTTP API may encounter error or performance issue when there are too many keyspaces
- Dominant language
- Go
- Stars
- 1.2k
- Forks
- 783
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 36
Description
## Bug Report
### What did you do?
* Create 100K keyspaces by posting to `/pd/api/v2/keyspaces` HTTP API
* Get keyspace meta by getting from `/pd/api/v2/keyspaces` HTTP API
* Iterate all keyspaces in PD's code
### What did you expect to see?
Everything works fine
### What did you see instead?
* Create 100K keyspaces by posting to `/pd/api/v2/keyspaces` HTTP API
* With 10 concurrency in the client, the performace slows down as the progress proceeds, and it takes about 7-8 hours to finish creating all the keyspaces.
* The reason might be the `UpdateKeyspaceForGroup` step.
* Get keyspace meta by getting from `/pd/api/v2/keyspaces` HTTP API
* ```
$ curl 192.168.178.11:3250/pd/api/v2/keyspaces
"rpc error: code = ResourceExhausted desc = trying to send message larger than max (11983608 vs. 2097152)"
```
* Iterate all keyspaces in PD's code
* It takes about 90s to finish iterating 100K keyspaces.
### What version of PD are you using (`pd-server -V`)?
master, with unrelated modifications in `GCStateManager` (see: https://github.com/tikv/pd/pull/10498)
Contributor guide
Assessment
This issue has not been assessed yet.