LinearTapeFileSystem / LinearTapeFileSystem/ltfs
How should upper-layer applications handle permanent write errors and extent cleanup?
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 352
- Forks
- 110
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 2
Description
I performed a test in which I wrote four files to a tape using four parallel threads. The four threads use a lock around the `write()` system call to ensure that the drive is accessed serially. However, operations such as `open()`, `fsync()`, and `close()` for each file are not protected by the lock.
After all four jobs completed, I called `ltfs.sync()` once.
During the writing of the second file, a "Cannot write block" error occurred. The tape entered a permanent write error state, and LTFS immediately performed an extent cleanup. The relevant LTFS log is as follows:
```text
Sep 15 01:54:53
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 64 LTFS11337I Update index-dirty
flag (1) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS12045E Cannot write block:
backend call failed (-20301). Dropping to read-only mode.
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11072E Cannot write blocks:
failed to write to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11077E Cannot write: failed
to write blocks to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.warn ltfs[98]: 65 LTFS13014W Data partition
writer: failed to write data to the tape (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13024I Clean up extents and
append index at index partition (-20301).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17292I Current position is
(1, 10246), Error position is (1, 10117).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13025I Truncate extents
larger than position (1, 10117), block size = 524288.
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11334I Remove extent :
utdm.0.0.1613966600.2147720233 (9407, 439877632).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11343I Try to write an index
on the IP on NO_BARCODE because of a permanent write error on the DP..
Sep 15 01:55:49
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17235I Writing index of
NO_BARCODE to a (Reason: Write perm, 5 files) 11C9D0D0D3.
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17236I Wrote index of
NO_BARCODE (Gen = 2, Part = a, Pos = 8, 11C9D0D0D3).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11337I Update index-dirty
flag (0) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 8c LTFS14022E Cannot flush file data
to the medium. Data might be lost (ltfs_fuse_flush).
```
On the application side, I observed that the `write()` calls for the second, third, and fourth files returned `errno = 2`. Then `ltfs.sync()` returned `-1`.
In this situation, LTFS performed an extent cleanup after the permanent write error. My questions are:
1. **Can the contents of the first file, for which some `write()` calls had already returned success, be partially or completely affected by the cleanup?**
In other words, is it possible that some data from the first file that had previously been reported as successfully written by `write()` was truncated or removed because of the subsequent permanent write error?
2. **How can an upper-layer application reliably detect that the tape has entered the permanent write error ("write-perm") state?**
Is there an LTFS API, ioctl, status interface, return code, or other mechanism that an application can use to detect this state without parsing the LTFS log?
3. **How can the upper-layer application determine that LTFS has performed an extent cleanup or rollback?**
Is there any API, event, status, or other mechanism that exposes this information to the application?
4. **What should the upper-layer application do after `ltfs.sync()` returns `-1`?**
What checks are recommended to determine whether previously successful writes are still valid and whether any data may have been lost or truncated?
5. More generally, **what guarantees does LTFS provide for a successful `write()` in the presence of a subsequent permanent write error and extent cleanup?**
6. It is quite possible that the "Cannot write block" error was caused by a drive-related problem. In that case, how can we reliably determine whether the root cause was the drive itself, rather than the tape medium or the I/O path (e.g., HBA/FC/backend)?
I would like to understand the expected semantics in this situation and the recommended way for an upper-layer application to detect and safely handle a permanent write error and the resulting extent cleanup.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the application-visible behavior of write(), fsync(), close(), and ltfs.sync() against the permanent write error and extent-cleanup log entries in the issue. Done means the semantics of successful writes, cleanup, error detection, and recommended handling after ltfs.sync() returns -1 are defined, including how drive, medium, and I/O-path causes can be distinguished.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- backend, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100