hpc / hpc/mpifileutils

Non-zero return code when copying a file from another user which is already present

Open
#669 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
200
Forks
85
Avg merge
3d 21h
Merged PRs (30d)
2

Description

When I do something like dsync file destfile; echo $?; dsync file destfile; echo $? - both the echos show 0. But, if I do dsync ~another_user/file destfile; echo $?; dsync ~another_user/file destfile; echo$? - the first echo will be 0, and the second will be 1. This build is compiled with xattrs, lustre, etc. But, it does not appear to be lustre related. If I do -X none, it still returns 1.

I've tried a build with XATTRS off, but the same non-zero return code is provided.

The output from dsync is identical when copied from another user or my own space.

[2026-02-04T03:31:15] Walked 1 items in 0.001 secs (859.593 items/sec) ...
[2026-02-04T03:31:15] Walked 1 items in 0.001 seconds (835.713 items/sec)
[2026-02-04T03:31:15] Comparing file sizes and modification times of 1 items
[2026-02-04T03:31:15] Started   : Feb-04-2026, 03:31:15
[2026-02-04T03:31:15] Completed : Feb-04-2026, 03:31:15
[2026-02-04T03:31:15] Seconds   : 0.000
[2026-02-04T03:31:15] Items     : 1
[2026-02-04T03:31:15] Item Rate : 1 items in 0.000154 seconds (6507.841738 items/sec)
[2026-02-04T03:31:15] Updating timestamps on newly copied files
[2026-02-04T03:31:15] Completed updating timestamps
[2026-02-04T03:31:15] Completed sync

Given this was a success, it should be returning a 0 status.

A quick look, and it appears to be the int tmp_rc = dsync_strmap_compare area. That is returning a tmp_rc < 0 (actual value of -1). Specifically, it is this function

              /* copy metadata values from source to destination, if needed */
              tmp_rc = mfu_flist_file_sync_meta(src_list, src_index, dst_list,
                                                dst_index, mfu_dst_file);

Looking at that function, my guess would be

     /* get owner and group ids */
      uid_t src_uid = (uid_t) mfu_flist_file_get_uid(src_list, src_index);
      gid_t src_gid = (gid_t) mfu_flist_file_get_gid(src_list, src_index);

      uid_t dst_uid = (uid_t) mfu_flist_file_get_uid(dst_list, dst_index);
      gid_t dst_gid = (gid_t) mfu_flist_file_get_gid(dst_list, dst_index);

      /* update ownership on destination if needed */
      if ((src_uid != dst_uid) || (src_gid != dst_gid)) {
          tmp_rc = mfu_copy_ownership(src_list, src_index, dest_path, mfu_file);
          if (tmp_rc < 0) {
              rc = -1;
          }
      }

This is trying to force ownership to be identical. I do not understand why as people surely would want to copy data from other users? I can't see a way to disable this either.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the two dsync commands with a file from another user and compare the return codes. Start at dsync_strmap_compare and mfu_flist_file_sync_meta, especially the mfu_copy_ownership call, to trace why an already-successful sync returns -1. Add or update coverage for this case so a completed sync reports status 0, while preserving the intended metadata behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.