iOfficeAI / iOfficeAI/OfficeCLI
[BUG] xlsx: batch not atomic under IPC failure at scale — partial write left on disk after reported failure
- Dominant language
- C#
- Stars
- 30.7k
- Forks
- 2.1k
- Avg merge
- 9d 8h
- Merged PRs (30d)
- 5
Description
Version: 1.0.138 (Homebrew build, --no-self-contained), macOS 26.5.2 arm64.
Summary: docs for `batch` say it's "atomic by default: any failed item rolls back the whole batch." At large scale (tens of thousands of items in one `batch` call), we saw the opposite: the CLI process reported failure while the resident kept processing in the background and wrote most (but not all) of the changes to disk — leaving the workbook in a partially-modified, inconsistent state despite the CLI reporting total failure.
## What we did
A real-world xlsx (6 sheets, 51,824 formula cells with empty cached values — `...` with no ``) needed those formulas re-evaluated. We enumerated every formula cell per sheet with:
```bash
officecli query file.xlsx "SheetName!cell:has(formula)" --json
```
built one combined batch of 51,824 `set` items (re-setting each cell's own formula to trigger write-time auto-evaluation), and ran:
```bash
officecli batch file.xlsx --input fix_all.json --json
```
## Observed
The command failed:
```
Resident for file.xlsx is running but the batch could not be delivered (main pipe busy or unresponsive). Retry, or run 'officecli close file.xlsx' and try again.
```
(subprocess exit code 3)
The resident process (`__resident-serve__ file.xlsx` in `ps`) kept running at 100%+ CPU for several minutes after the CLI process had already exited with the error, then disappeared on its own. Checking the file afterward, per sheet:
| Sheet | Cells fixed | Total formulas |
|---|---|---|
| A | 1824 | 1896 |
| B | 1822 | 1896 |
| C | 4224 | 4424 |
| D | 8970 | 9480 |
| E | 5394 | 5688 |
| F | 26520 | 28440 |
| **Total** | **48754** | **51824** |
~94% of the batch had actually been written to disk, despite the CLI reporting total failure and no rollback occurring.
Separately, we noticed per-item cost within a single `batch` call grows superlinearly with batch size — 200 / 2,000 / 10,000-item synthetic batches averaged 1.19ms / 1.91ms / 4.29ms per item respectively, while a single unrelated `set` on files pre-populated to those same sizes cost a near-constant ~0.4–0.5s regardless of file size (ruling out file size as the driver). This growth is likely what pushes very large batches past whatever timeout produces the "pipe busy or unresponsive" failure in the first place.
## Expected
One of:
- `batch` actually honors "atomic by default" — on any failure (including this IPC/timeout scenario), the file on disk is unchanged from before the batch started, or
- if true atomicity isn't feasible when the CLI loses its connection to the resident mid-batch, the CLI should not report unqualified failure — it should reflect the partial-completion state (e.g. "N/total succeeded before connection was lost") instead of an error implying nothing happened.
## Notes
We don't have a small, reliably-reproducing repro yet — this surfaced on a real ~52k-cell/6-sheet workbook. It may reproduce with fewer items on a slower machine, or with more cross-sheet references driving up per-item cost. Happy to share the synthetic benchmark generator if useful.
Contributor guide
Research direction
Start at the `batch` CLI path and the resident process handling `__resident-serve__`; reproduce the reported connection-loss case with a large multi-sheet batch or the author's synthetic benchmark generator. Trace what happens after the CLI reports that the pipe is busy or unresponsive, and verify completion by checking whether the workbook is unchanged or clearly reports partial progress.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100