shutdown: handle writing files
- Dominant language
- C++
- Stars
- 2.8k
- Forks
- 551
- PR merge metrics
- No merged PRs in 30d
Description
批量下机器中,正在写的文件的处理
对于正在写的block,所处的位置为:
ChunkServerManager中,cs->block的map
BlockMapping中,block->incomplete_replica的set
避免出现单副本的下线方案中,会先将待下线cs中的block进行预恢复,考虑正在写的block,如果此时将其恢复,则在PushBlock过程中,仍可能正在写入数据,最后可能导致丢失写入的数据(不管有没有sync)。
所以下线前,不仅要保证待下线机器上没有新增block的写入流量,还要保证待下线机器上现有的正在写的block被安全关闭,切断一切写入流量。
写入过程中如果有机器宕机,则会在BlockReport时将其返回给cs进行关闭,但是这个动作是异步的,没有对close的完成时间有任何保证,不太好复用到这里,所以看起来,需要给cs加一个同步关闭指定block的接口?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the shutdown and single-replica recovery flow around ChunkServerManager, BlockMapping, PushBlock, and BlockReport. Trace how in-progress blocks are tracked and how BlockReport closes blocks after a failure. Done means shutdown can synchronously stop writes to existing in-progress blocks before recovery begins, without losing pending data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100