deepseek-ai / deepseek-ai/3FS

2 * 100Gbps RDMA network cards, single client write throughput can only reach 6GiB/s ???

Open
#245 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
10.2k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Deploy 6 storage nodes(24 * nvmeSSD per node), all with 2 * 100Gbps network cards, network card model
0000:17:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
0000:17:00.1 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]

Client fio can only run at 6GiB/s, the fio command is as follows:
```
/root/3fs/fio --name=500M_1M_bigio_write -numjobs=100 -fallocate=none \
-ioengine=external:/usr/lib64/hf3fs_usrbio.so -direct=1 -rw=write -bs=1MB \
--group_reporting -filesize=500MB --nrfiles=10 -iodepth=1 \
--directory=/3fs/stage/500M_1M_bigio_write -mountpoint=/3fs/stage -ior_depth=1
```

![Image](https://github.com/user-attachments/assets/851d89b3-49d9-48da-a171-06f4e535a214)

Does this capability meet 3FS expectations? Are there other optimization parameters that are not enabled?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.