block / block/buzz

A3 conformance probe can wedge relay boot without a deadline or early health listener

Open
#7,292 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Problem

At `7a9a523`, relay startup runs the A3 git object-store conformance probe as a hard boot gate. The default probe uses 32 writers across 3 rounds against blob storage. Individual requests have timeouts, but the probe has no overall deadline.

On a homelab MinIO deployment, the probe wedged silently for more than an hour during `if_none_match_race` round 0. The container remained `running` with zero restarts and emitted no further logs. Restarting the container fixed the issue; a normal probe completes in about five seconds.

The health listener on port 8080, including `/_liveness` and `/_readiness`, starts only after the boot gates complete. During the wedge, the Compose healthcheck received connection refusals until its full budget expired and marked the container unhealthy. Orchestrators cannot distinguish a relay that is still booting from one stuck in the probe.

## Suggested fixes

- Add an overall deadline to the A3 probe and fail startup with an explicit error when it expires.
- Bind the health listener before long-running boot gates so liveness and readiness remain observable during startup.

Contributor guide

Open the contributing guide

Research direction

Read the relay startup path around the A3 git object-store conformance probe, then inspect the health listener endpoints `/_liveness` and `/_readiness` and the Compose healthcheck behavior. Done means the probe cannot wedge startup indefinitely, expiration reports an explicit error, and health remains observable while boot gates run.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, git, rust
Domain
backend, devops, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.