llnl / llnl/UnifyFS

Potential deadlock caused by concurrent sync calls

Open
#810 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
122
Forks
34
PR merge metrics
No merged PRs in 30d

Description

### Describe the problem you're observing

I'm observing some TIMEOUT errors when trying to stage-in many files simultaneously.
It seems that concurrent `unifyfs_sync()` may cause deadlock on the server side.
After some investigations, I found the server side is blocking at the `process_pending_sync` call in this case:

client A on server 0 --> write/sync file 1 --> owner is server 1
client B on server 1 --> write/sync file 2 --> owner is server 0

https://github.com/LLNL/UnifyFS/blob/58ece4441716678f5111a6dbff9baadd6188c2b6/server/src/unifyfs_service_manager.c#L1479-L1483

@MichaelBrim Is this the cause? Any idea how to fix this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in server/src/unifyfs_service_manager.c around lines 1479-1483 and inspect the process_pending_sync call. Reproduce staging many files with concurrent unifyfs_sync() calls across servers, then verify the change prevents the described cross-server deadlock and TIMEOUT errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
distributed-systems, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.