deepseek-ai / deepseek-ai/3FS

Share more about the file system design tailored to the characteristics of LLM tasks

Open
#40 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
10.2k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Dear community,

Firstly thanks for your contribution to the open source.

> The Fire-Flyer File System (3FS) is a high-performance distributed file system designed to address the challenges of AI training and inference workloads.

I understand 3FS is mainly designed for AI or LLM workloads and it has high performance by leveraging modern SSDs and RDMA networks. However, from the README and [design notes](https://github.com/deepseek-ai/3FS/blob/main/docs/design_notes.md), I can't see the design dedicated to AI workloads clearly.

I can see some parts as following:

![Image](https://github.com/user-attachments/assets/afebc140-ba48-44b5-b40a-ddd1b6a4382b)

![Image](https://github.com/user-attachments/assets/8e835b33-50ab-4e41-98e9-ec4fe5d839b8)

Could you share more about the difference between your file system design compared with a general distributed file system?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.