bottlerocket-os / bottlerocket-os/bottlerocket
BOTTLEROCKET_PRIVATE can lose/corrupt data on power loss
- Dominant language
- Rust
- Stars
- 9.7k
- Forks
- 586
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 11
Description
In a very specific situation I have a bunch of devices that are getting power cycled _immediately_ after first boot is complete. In many cases the device then fails to boot a second time. Upon inspection of the BOTTLEROCKET_PRIVATE filesystem of a device that fails to boot the datastore is incomplete/corrupted and fails to boot the second time due to not being able to synthesize a proper system configuration. If I put in a sync/drop caches command in a bootstrap container I notice that the settings never become corrupted.
**Image I'm using:**
Custom **metal-dev** variant
**What I expected to happen:**
The system will boot multiple times in a row even if there is a power loss
**What actually happened:**
The system only boots the first time when there is a power loss immediately after first boot
**How to reproduce the problem:**
This one is a bit tough. I have a screen that is displaying information on an HDMI port from a bootstrap container. Once I see that output I pull power from the system. Boot a second time and notice how the settings are corrupted and the system won't boot.
I'm speculating at this point due to testing with a hard **sync** right before the power loss that this is due to a file write-cache issue. The settings are in flight in the write cache but don't get synced out to disk. Although this is a bit of a corner case does it make sense to mount BOTTLEROCKET_PRIVATE without a write-cache so writes are guaranteed to hit the disk since that partition is a low performance target anyway?
Contributor guide
Assessment
This issue has not been assessed yet.