use file-based barrier for keyval fence and pids file
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 122
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
I see we still have a sleep(10) as our fence for the file system key/value store:
To add a true barrier here, we could use a scheme where we use a write flock and have each server add a line (or a byte), then have all servers wait until that file reaches a certain line count or size with stat. For proper locking, this would likely require a file system like Lustre or GPFS.
Anyone know of problems with that approach?
Similarly, I think we could refactor creation of the "pids" file.
I believe all this really requires is for rank 0 to create a file when it knows for sure that all servers have published their client-server rpc address, which we can do if we have a true barrier between servers from the key value store. The current sync logic waits for the file to hit a certain line count, but it doesn't need to know actual server pids:
https://github.com/LLNL/UnifyFS/blob/dev/util/unifyfs/src/unifyfs-rm.c#L221-L229
If we do need to collect pid values, that could also be done through the key value exchange. We could likely simplify that to just test for existence of the file at all.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the key/value fence in common/src/unifyfs_keyval.c around line 800, then inspect pids-file creation in server/src/unifyfs_server.c around line 404 and sync logic in util/unifyfs/src/unifyfs-rm.c around lines 221-229. Done would mean replacing the sleep-based coordination with a confirmed barrier and simplifying pids-file handling without losing server startup synchronization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- distributed-systems, operating-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100