baidu / baidu/bfs

shutdown: handle writing files

Open
#378 0 comments 0 reactions 0 assignees View on GitHub
2pt new feature
Dominant language
C++
Stars
2.8k
Forks
551
PR merge metrics
No merged PRs in 30d

Description

批量下机器中,正在写的文件的处理

对于正在写的block,所处的位置为:

ChunkServerManager中,cs->block的map
BlockMapping中,block->incomplete_replica的set

避免出现单副本的下线方案中,会先将待下线cs中的block进行预恢复,考虑正在写的block,如果此时将其恢复,则在PushBlock过程中,仍可能正在写入数据,最后可能导致丢失写入的数据(不管有没有sync)。

所以下线前,不仅要保证待下线机器上没有新增block的写入流量,还要保证待下线机器上现有的正在写的block被安全关闭,切断一切写入流量。

写入过程中如果有机器宕机,则会在BlockReport时将其返回给cs进行关闭,但是这个动作是异步的,没有对close的完成时间有任何保证,不太好复用到这里,所以看起来,需要给cs加一个同步关闭指定block的接口?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the shutdown and single-replica recovery flow around ChunkServerManager, BlockMapping, PushBlock, and BlockReport. Trace how in-progress blocks are tracked and how BlockReport closes blocks after a failure. Done means shutdown can synchronously stop writes to existing in-progress blocks before recovery begins, without losing pending data.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.