Problems with dcp/dsync with ACLs
Open
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 200
- Forks
- 85
- Avg merge
- 3d 21h
- Merged PRs (30d)
- 2
Description
Using version 0.10.1
Simple test case:
b-cn0123 [ake]$ ls -lR dcp-test
dcp-test:
total 4
dr-xr-x---+ 3 ake folk 4096 Oct 2 21:02 q/
dcp-test/q:
total 4
dr-xr-x---+ 2 ake folk 4096 Oct 2 20:30 r/
dcp-test/q/r:
total 0
-r--r-x---+ 1 ake folk 0 Oct 2 20:30 file*
b-cn0123 [ake]$ getfacl -R dcp-test/
# file: dcp-test/
# owner: ake
# group: folk
user::r-x
user:yyy:r-x
group::r-x
mask::r-x
other::---
# file: dcp-test//q
# owner: ake
# group: folk
user::r-x
user:yyy:r-x
group::---
mask::r-x
other::---
# file: dcp-test//q/r
# owner: ake
# group: folk
user::r-x
user:yyy:r-x
group::---
mask::r-x
other::---
# file: dcp-test//q/r/file
# owner: ake
# group: folk
user::r--
user:yyy:r-x
group::---
mask::r-x
other::---
And this is the result:
b-cn0123 [stor10]$ mpirun -n 1 dcp --preserve /pfs/nobackup/home/a/ake/dcp-test /pfs/stor10/proj-test
[2020-10-02T21:13:03] Preserving file attributes.
[2020-10-02T21:13:03] Walking /pfs/nobackup/home/a/ake/dcp-test
[2020-10-02T21:13:03] Walked 4 items in 0.008278 secs (483.180546 items/sec) ...
[2020-10-02T21:13:03] Walked 4 items in 0.008357 seconds (478.636938 items/sec)
[2020-10-02T21:13:03] Copying to /pfs/stor10/proj-test
[2020-10-02T21:13:03] Items: 4
[2020-10-02T21:13:03] Directories: 3
[2020-10-02T21:13:03] Files: 1
[2020-10-02T21:13:03] Links: 0
[2020-10-02T21:13:03] Data: 0.000 B (0.000 B per file)
[2020-10-02T21:13:03] Creating directories.
[2020-10-02T21:13:03] level=6 min=1 max=1 sum=1 rate=118.951962/sec secs=0.008407
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:866] ERROR: Create `/pfs/stor10/proj-test/dcp-test/q' mkdir() failed (errno=13 Permission denied)
[2020-10-02T21:13:03] level=7 min=1 max=1 sum=1 rate=1242.445929/sec secs=0.000805
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:866] ERROR: Create `/pfs/stor10/proj-test/dcp-test/q/r' mkdir() failed (errno=2 No such file or directory)
[2020-10-02T21:13:03] level=8 min=1 max=1 sum=1 rate=26178.695777/sec secs=0.000038
[2020-10-02T21:13:03] level=9 min=0 max=0 sum=0 rate=0.000000/sec secs=0.000000
[2020-10-02T21:13:03] Created 3 directories in 0.009315 seconds (322.064200 items/sec)
[2020-10-02T21:13:03] Creating files.
[2020-10-02T21:13:03] level=6 min=0 max=0 sum=0 rate=0.000000 secs=0.000002
[2020-10-02T21:13:03] level=7 min=0 max=0 sum=0 rate=0.000000 secs=0.000000
[2020-10-02T21:13:03] level=8 min=0 max=0 sum=0 rate=0.000000 secs=0.000000
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:1113] ERROR: File `/pfs/stor10/proj-test/dcp-test/q/r/file' mknod() failed (errno=2 No such file or directory)
[2020-10-02T21:13:03] level=9 min=1 max=1 sum=1 rate=21728.266302 secs=0.000046
[2020-10-02T21:13:03] Created 1 items in 0.000194 seconds (5160.331500 items/sec)
[2020-10-02T21:13:03] Copying data.
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:1926] ERROR: Failed to open output file `/pfs/stor10/proj-test/dcp-test/q/r/file' (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:2059] ERROR: Failed to copy `/pfs/nobackup/home/a/ake/dcp-test/q/r/file' to `/pfs/stor10/proj-test/dcp-test/q/r/file'
[2020-10-02T21:13:03] Copy data: 0.000 B (0 bytes)
[2020-10-02T21:13:03] Copy rate: 0.000 B/s (0 bytes in 0.002185 seconds)
[2020-10-02T21:13:03] Syncing data to disk.
[2020-10-02T21:13:03] Sync completed in 0.000652 seconds.
[2020-10-02T21:13:03] Setting ownership, permissions, and timestamps.
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:328] ERROR: Failed to change ownership on `/pfs/stor10/proj-test/dcp-test/q/r/file' lchown() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:354] ERROR: Failed to change permissions on `/pfs/stor10/proj-test/dcp-test/q/r/file' chmod() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:491] ERROR: Failed to change timestamps on `/pfs/stor10/proj-test/dcp-test/q/r/file' utime() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:328] ERROR: Failed to change ownership on `/pfs/stor10/proj-test/dcp-test/q/r' lchown() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:354] ERROR: Failed to change permissions on `/pfs/stor10/proj-test/dcp-test/q/r' chmod() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:491] ERROR: Failed to change timestamps on `/pfs/stor10/proj-test/dcp-test/q/r' utime() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:328] ERROR: Failed to change ownership on `/pfs/stor10/proj-test/dcp-test/q' lchown() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:354] ERROR: Failed to change permissions on `/pfs/stor10/proj-test/dcp-test/q' chmod() (errno=2 No such file or directory)
[2020-10-02T21:13:03] [0] [/scratch/eb-buildpath/mpifileutils/0.10.1/gompi-2020a/mpifileutils-0.10.1/src/common/mfu_flist_copy.c:491] ERROR: Failed to change timestamps on `/pfs/stor10/proj-test/dcp-test/q' utime() (errno=2 No such file or directory)
[2020-10-02T21:13:03] Updated 4 items in 0.008529 seconds (468.992447 items/sec)
[2020-10-02T21:13:03] Syncing directory updates to disk.
[2020-10-02T21:13:03] Sync completed in 0.000095 seconds.
[2020-10-02T21:13:03] Started: Oct-02-2020,21:13:03
[2020-10-02T21:13:03] Completed: Oct-02-2020,21:13:03
[2020-10-02T21:13:03] Seconds: 0.021
[2020-10-02T21:13:03] Items: 1
[2020-10-02T21:13:03] Directories: 1
[2020-10-02T21:13:03] Files: 0
[2020-10-02T21:13:03] Links: 0
[2020-10-02T21:13:03] Data: 0.000 B (0 bytes)
[2020-10-02T21:13:03] Rate: 0.000 B/s (000 bytes in 0.021 seconds)
The same test with a directory without ACLs works perfectly.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported dcp --preserve run from version 0.10.1 with the ACL-bearing directory and compare it with the directory without ACLs. Inspect src/common/mfu_flist_copy.c around the reported lines 328, 354, 491, 866, 1113, 1926, and 2059. Done means the ACL test creates the complete directory tree and file while preserving the reported attributes without the mkdir and follow-on errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100