uutils / uutils/coreutils

`dd` fails on gnu test `tests/dd/direct.sh` - dynamic removal of `O_DIRECT` missing

Open
#6,078 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

U - dd
Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

I started fixing remaining issues from the GNU test suite that are related to dd as this was the last util I was working on.

I investigated what the issue is by manual tests and using strace.
Summary:
The test tries to do file copy with the direct input and output mode.
direct mode is special as it has some restrictions regarding the buffer locations and its size:
While a direct read apparently succeds in the case when it reads the last chunk from a file that is irregularily sized,
a direct write fails if we try to write a buffer that doesn't match the blocksize.

I used strace to figure out how GNU is solving it.
Aparently it gets rid of the O_DIRECT status flag before writing the last chunk by using the fcntl system call.

GNU:

write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 511
read(0, "", 512)                        = 0
fcntl(1, F_GETFL)                       = 0xc001 (flags O_WRONLY|O_DIRECT|O_LARGEFILE)
fcntl(1, F_SETFL, O_WRONLY|O_LARGEFILE) = 0
lseek(1, 0, SEEK_CUR)                   = 7680
fadvise64(1, 4096, 0, POSIX_FADV_DONTNEED) = 0
write(1, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 511) = 511
fsync(1)                                = 0
fadvise64(1, 4096, 0, POSIX_FADV_DONTNEED) = 0
close(0)                                = 0
close(1)                                = 0
write(2, "15+1 records in\n15+1 records out"..., 3315+1 records in
15+1 records out
) = 33
write(2, "8191 bytes (8.2 kB, 8.0 KiB) cop"..., 598191 bytes (8.2 kB, 8.0 KiB) copied, 0.00712461 s, 1.1 MB/s) = 59
write(2, "\n", 1
)                       = 1
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

uutils:

write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 511
read(3, "", 512)                        = 0
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 511) = -1 EINVAL (Invalid argument)
close(4)                                = 0
futex(0x5dca26e81db8, FUTEX_WAKE_PRIVATE, 1) = 1
close(3)                                = 0
write(2, "dd", 2dd)                       = 2
write(2, ": ", 2: )                       = 2
write(2, "IO error: Invalid input", 23IO error: Invalid input) = 23
write(2, "\n", 1
)                       = 1
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
munmap(0x79831cf07000, 12288)           = 0
exit_group(1)                           = ?
+++ exited with 1 +++

I'm alread working on a solution

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the GNU test at tests/dd/direct.sh and reproducing the failure under strace, then inspect the dd implementation's handling of direct input and output. Done means copying an irregularly sized final chunk succeeds in direct mode and the test passes without the EINVAL error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.