hpc / hpc/mpifileutils

Inefficiency in dsync when restarting a failed dsync

Open
#422 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
200
Forks
85
Avg merge
3d 21h
Merged PRs (30d)
2

Description

As far as I can judge, a dsync that is aborted for some reason and then restarted will delete all files and make new copies.
The reason being that owner/permission/mod-time doesn't match the source.

If the meta-data on files is set directly after the data has been copied, instead of in a separate loop after coping all data, this would not be a problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing dsync's restart path and the separate data-copy and metadata-setting loops described in the issue. Confirm how an interrupted run handles owner, permission, and modification-time mismatches; done means a restarted failed sync preserves already copied files instead of deleting and recreating them.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.