ctengel / ctengel/simpler-objects
Complete the sleeping-disk timeout sweep on the write & replication paths
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Follow-up to #72. The fix there raised the timeouts on the **bucket-listing read path** (`list_bucket` / `head_bucket` fan-out and the replicator's `auto_replica` / `get_bucket_contents` client legs). But the same sleeping-disk root cause — an HDD takes ~15s to spin up, "even 15 seconds is normal" — still bites two other places that were left at `timeout=1`/`timeout=2`. Both run a real risk because the replicator runs off-hours, exactly when drives are spun down.
(The two single-object correctness analogues are tracked separately: #75 read false-404, #76 write duplicate.)
## 1. `add_object` bucket-existence check → spurious 507
`locator_api.py:120`, the final write-candidate stage:
```python
async def check_bucket(server):
try:
result = await client.head(server + bucket + "/", timeout=1)
result.raise_for_status()
return server, True
except httpx.HTTPError:
return server, False
```
A sleepy candidate's bucket HEAD times out at 1s → `ok=False` → the candidate is pruned. If every candidate is pruned this way the locator returns **507 (no space)** even though space exists — a write fails purely because a disk was asleep.
## 2. Replicator direct object-server HEADs → false "no space" / aborted run
The replicator talks straight to object servers (no locator in front), so it hits cold disks directly:
- `async_replicate.py:21` (`find_space`), `httpx.head(server + bucket + "/", timeout=1)` — a sleepy replication target times out, gets popped, and you get a false `No space to replicate object` warning + non-zero exit.
- `async_replicate.py:33` (`get_object_size`), `httpx.head(obj, timeout=2)` — a sleepy source or destination times out and raises, aborting the replication of that object (and the run).
Because replication runs during off-hours when drives are spun down, this is the **most likely** real-world trigger of the whole sleeping-disk class.
## Suggested fix
Raise these to a spin-up-tolerant timeout (~16s, matching what #72 used for the locator fan-out; keep the ladder client ≥ locator-internal ≥ ~15s), or adopt whichever distinguish-asleep-from-down strategy comes out of #72/#75/#76. Out of scope here: `get_object_server_health` (`locator_api.py:39`, `timeout=1`) is left short on purpose so genuinely-down nodes fail fast for candidate selection, and `statvfs` doesn't spin a disk anyway.
## Related
- #72 — read-path fix this completes
- #75 — read-path false 404 (single object)
- #76 — write-path duplicate (single object)
- #49 — data-path streaming timeout
- #63 — partial results; #1, #3 — metadata caching
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.