canonical / canonical/cloud-init

atomic_helper.write_file missing os.fsync() leaves 0-byte files on sudden reboot/power-loss, causing fatal JSONDecodeError in stages.py

Open
#7,078 0 comments 0 reactions 0 assignees View on GitHub
bug networking
Dominant language
Python
Stars
3.8k
Forks
1.1k
Avg merge
2d 23h
Merged PRs (30d)
18

Description

https://github.com/canonical/cloud-init/blob/494f7dece8223d42ad6c1a171033ecd77cf760a5/cloudinit/atomic_helper.py#L78

When a system crashes or is forcefully rebooted (e.g. `SIGKILL`, hard reset) right after `cloud-init` writes instance cache files, `network-config.json` can become a 0-byte file on reboot.
On the next boot, `cloud-init` crashes on an unhandled `JSONDecodeError` and fails to initialize networking.

Root Cause
1. **Missing `fsync` before `rename`**: `cloudinit/atomic_helper.py` writes to a tempfile and calls `os.rename()` without `tf.flush()` or `os.fsync()`. On filesystems like ext4 (with delayed allocation), `rename` commits directory metadata to the journal while dirty data remains in volatile RAM. A sudden crash before background writeback leaves an empty file (`Size: 0, Blocks: 0`).
2. **Missing error handling**: `cloudinit/stages.py:448` calls `util.load_json()` without `try...except`, raising unhandled `json.decoder.JSONDecodeError` on empty/corrupted cache.

Suggested Fix
1. cloudinit/atomic_helper.py: Flush and sync before rename:

tf.write(content)
tf.flush()
try:
os.fsync(tf.fileno())
except OSError:
pass
os.chmod(tf.name, mode)
os.rename(tf.name, filename)

2. cloudinit/stages.py: Handle corrupted/empty JSON gracefully

if os.path.isfile(net_cfg_fname):
try:
content = util.load_text_file(net_cfg_fname)
if content:
return util.load_json(content)
except (json.decoder.JSONDecodeError, ValueError) as exc:
LOG.warning("Failed to parse cached network config at %s: %s", net_cfg_fname, exc)

Contributor guide

Open the contributing guide

Research direction

Read cloudinit/atomic_helper.py around line 78 and cloudinit/stages.py around line 448, then trace how instance cache files are written and how network-config.json is loaded. Verify the write path handles sudden interruption and the load path handles empty or corrupted JSON without an unhandled exception. Done means cache writes are durable and a damaged cache does not prevent networking initialization.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.