canonical / canonical/cloud-init
atomic_helper.write_file missing os.fsync() leaves 0-byte files on sudden reboot/power-loss, causing fatal JSONDecodeError in stages.py
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 1.1k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 18
Description
https://github.com/canonical/cloud-init/blob/494f7dece8223d42ad6c1a171033ecd77cf760a5/cloudinit/atomic_helper.py#L78
When a system crashes or is forcefully rebooted (e.g. `SIGKILL`, hard reset) right after `cloud-init` writes instance cache files, `network-config.json` can become a 0-byte file on reboot.
On the next boot, `cloud-init` crashes on an unhandled `JSONDecodeError` and fails to initialize networking.
Root Cause
1. **Missing `fsync` before `rename`**: `cloudinit/atomic_helper.py` writes to a tempfile and calls `os.rename()` without `tf.flush()` or `os.fsync()`. On filesystems like ext4 (with delayed allocation), `rename` commits directory metadata to the journal while dirty data remains in volatile RAM. A sudden crash before background writeback leaves an empty file (`Size: 0, Blocks: 0`).
2. **Missing error handling**: `cloudinit/stages.py:448` calls `util.load_json()` without `try...except`, raising unhandled `json.decoder.JSONDecodeError` on empty/corrupted cache.
Suggested Fix
1. cloudinit/atomic_helper.py: Flush and sync before rename:
tf.write(content)
tf.flush()
try:
os.fsync(tf.fileno())
except OSError:
pass
os.chmod(tf.name, mode)
os.rename(tf.name, filename)
2. cloudinit/stages.py: Handle corrupted/empty JSON gracefully
if os.path.isfile(net_cfg_fname):
try:
content = util.load_text_file(net_cfg_fname)
if content:
return util.load_json(content)
except (json.decoder.JSONDecodeError, ValueError) as exc:
LOG.warning("Failed to parse cached network config at %s: %s", net_cfg_fname, exc)
Contributor guide
Research direction
Read cloudinit/atomic_helper.py around line 78 and cloudinit/stages.py around line 448, then trace how instance cache files are written and how network-config.json is loaded. Verify the write path handles sudden interruption and the load path handles empty or corrupted JSON without an unhandled exception. Done means cache writes are durable and a damaged cache does not prevent networking initialization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100