CodisLabs / CodisLabs/codis

proxy的压测遇到的问题

Open
#1,323 18 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
13.2k
Forks
2.7k
PR merge metrics
No merged PRs in 30d

Description

## 配置文件
```
##################################################
# #
# Codis-Proxy #
# #
##################################################

# Set Codis Product Name/Auth.
product_name = "codis_test3"
product_auth = ""

# Set auth for client session
# 1. product_auth is used for auth validation among codis-dashboard,
# codis-proxy and codis-server.
# 2. session_auth is different from product_auth, it requires clients
# to issue AUTH before processing any other commands.
session_auth = ""

# Set bind address for admin(rpc), tcp only.
admin_addr = "0.0.0.0:11082"

# Set bind address for proxy, proto_type can be "tcp", "tcp4", "tcp6", "unix" or "unixpacket".
proto_type = "tcp4"
proxy_addr = "0.0.0.0:19200"

# Set jodis address & session timeout
# 1. jodis_name is short for jodis_coordinator_name, only accept "zookeeper" & "etcd".
# 2. jodis_addr is short for jodis_coordinator_addr
# 3. proxy will be registered as node:
# if jodis_compatible = true (not suggested):
# /zk/codis/db_{PRODUCT_NAME}/proxy-{HASHID} (compatible with Codis2.0)
# or else
# /jodis/{PRODUCT_NAME}/proxy-{HASHID}
jodis_name = ""
jodis_addr = ""
jodis_timeout = "20s"
jodis_compatible = false

# Set datacenter of proxy.
proxy_datacenter = ""

# Set max number of alive sessions.
proxy_max_clients = 10000

# Set max offheap memory size. (0 to disable)
proxy_max_offheap_size = "1024mb"

# Set heap placeholder to reduce GC frequency.
proxy_heap_placeholder = "256mb"

# Proxy will ping backend redis (and clear 'MASTERDOWN' state) in a predefined interval. (0 to disable)
backend_ping_period = "5s"

# Set backend recv buffer size & timeout.
backend_recv_bufsize = "128kb"
backend_recv_timeout = "30s"

# Set backend send buffer & timeout.
backend_send_bufsize = "128kb"
backend_send_timeout = "30s"

# Set backend pipeline buffer size.
backend_max_pipeline = 1024

# Set backend never read replica groups, default is false
backend_primary_only = false

# Set backend parallel connections per server
backend_primary_parallel = 1
backend_replica_parallel = 1

# Set backend tcp keepalive period. (0 to disable)
backend_keepalive_period = "75s"

# Set number of databases of backend.
backend_number_databases = 16

# If there is no request from client for a long time, the connection will be closed. (0 to disable)
# Set session recv buffer size & timeout.
session_recv_bufsize = "128kb"
session_recv_timeout = "30m"

# Set session send buffer size & timeout.
session_send_bufsize = "64kb"
session_send_timeout = "30s"

# Make sure this is higher than the max number of requests for each pipeline request, or your client may be blocked.
# Set session pipeline buffer size.
session_max_pipeline = 10000

# Set session tcp keepalive period. (0 to disable)
session_keepalive_period = "75s"

# Set session to be sensitive to failures. Default is false, instead of closing socket, proxy will send an error response to client.
session_break_on_failure = false

# Set metrics server (such as http://localhost:28000), proxy will report json formatted metrics to specified server in a predefined period.
metrics_report_server = ""
metrics_report_period = "1s"

# Set influxdb server (such as http://localhost:8086), proxy will report metrics to influxdb.
# metrics_report_influxdb_server = ""
# metrics_report_influxdb_period = "1s"
# metrics_report_influxdb_username = ""
# metrics_report_influxdb_password = ""
# metrics_report_influxdb_database = ""

# Set statsd server (such as localhost:8125), proxy will report metrics to statsd.
metrics_report_statsd_server = ""
metrics_report_statsd_period = "1s"
metrics_report_statsd_prefix = ""
```

## 压测命令
./redis-benchmark -p 19200 -h 10.100.90.20 -n 100000000 -r 10000000000 -c 512 -d 100 -t get,set,mset -q

## 压测
### 场景1
在两台机器同时向一个proxy压,并行启动两个redis-benchmark。
其中一个运行正常,另一个会报如下错误
```
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
Writing to socket: Connection reset by peer
All clients disconnected... aborting.
```

### 场景2
在同一台机器(codis proxy所在机器)启动3个redis-benchmark。
前两个不会报错,第三个会报如下错误。
```
All clients disconnected... aborting.
```

## 问题
这个是否是配置有问题导致的还是proxy升级后有的问题。以前测试codis2.0的时候是没有这个问题的。

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.