EINVAL in mfu_lseek() for very large files (>16TB)
Open
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 200
- Forks
- 85
- Avg merge
- 3d 21h
- Merged PRs (30d)
- 2
Description
Hi there!
We're trying to dsync a set of very large files (32TB) and the copy fails with the following error:
[709] [/dev/shm/mpifileutils-0.9.1/src/common/mfu_flist_copy.c:1262] ERROR: Couldn't seek in destination path `/oak/stanford/orgs/kipac/users/swmclau2/Darksky/ds14_a_1.0000' (errno=22 Invalid argument)
Both source and destination filesystems are Lustre.
The resulting copied file's size is only 16TB, so we suspect some kind of overflow in mfu_lseek().
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with mfu_lseek() and the failure at mfu_flist_copy.c:1262, then trace how dsync represents offsets for large files. Reproduce the seek or copy against the reported Lustre setup with a file larger than 16TB. Done means the operation no longer returns EINVAL and the destination reaches the source size.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100