vllm-project / vllm-project/aibrix
Infinistore failed to work in the scale testing
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🐛 Describe the bug


### Steps to Reproduce
8 vllm instances, 960 GB Infinistore single instance
### Expected behavior
it should work as expected
### Environment
- AIBrix version: v0.3.0-rc.1
```
apiVersion: v1
kind: Pod
metadata:
name: infinistore
namespace: default
labels:
app: infinistore
spec:
hostNetwork: true
containers:
- name: infinistore
image: aibrix-cn-beijing.cr.volces.com/aibrix/infinistore:20250506
command: ["infinistore"]
args:
- "--manage-port=8088"
- "--dev-name=mlx5_0" # if mlx5_0 is not available, it will fallbacks to other devs
- "--service-port=12345"
- "--link-type=Ethernet"
# - "--hint-gid-index=7"
- "--log-level=debug"
- "--prealloc-size=950"
ports:
- containerPort: 8088
- containerPort: 12345
securityContext:
privileged: true
resources:
requests:
cpu: "10"
memory: "960Gi"
limits:
cpu: "10"
memory: "960Gi"
restartPolicy: Never
```
```
env:
- name: VLLM_RPC_TIMEOUT
value: "1000000"
- name: VLLM_USE_V1
value: "0"
- name: AIBRIX_KV_CACHE_OL_L1_CACHE_ENABLED
value: "0"
- name: AIBRIX_KV_CACHE_OL_L2_CACHE_BACKEND
value: "infinistore"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_HOST_ADDR
value: "192.168.3.35"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_SERVICE_PORT
value: "12345"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_CONNECTION_TYPE
value: "RDMA"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_IB_PORT
value: "1"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_LINK_TYPE
value: "Ethernet"
- name: AIBRIX_KV_CACHE_OL_INFINISTORE_VISIBLE_DEV_LIST
value: "mlx5_1:7,mlx5_2:7,mlx5_3:7,mlx5_4:7"
```
Contributor guide
Assessment
This issue has not been assessed yet.