tarantool / tarantool/tarantool
Raft: decide on worker fiber vs async writes
Nobody has claimed this yet.
- Dominant language
- Lua
- Stars
- 3.7k
- Forks
- 419
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 88
Description
Initially in Raft implementation it was decided to use ev_check/ev_prepare watchers to dump Raft state on disk when it changes. This didn't work, because these watchers are invoked from a scheduler fiber, where it is not possible to yield.
Need to check if this is true - perhaps it is not possible to yield only in ev_check because it is last. And ev_prepare would work.
Also Raft has another watcher - ev_timer to track when leader dies, and when election times out.
For dumping currently Raft has a worker fiber to perform state dumps, as a workaround.
So in total, Raft uses both fiber and watcher to perform "async" work. That is inconsistent. Either ev_timer should be removed, and the worker fiber should do the waiting, or the fiber should be removed, and the dumping should be done using async WAL writes with a completion callback.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the Raft implementation by verifying whether yielding is possible from ev_check and ev_prepare, then trace the ev_timer, worker fiber, and async WAL write paths. The issue names no files or tests; done means choosing and documenting one consistent approach for waiting and state dumps, with the unused mechanism removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100