oxidecomputer / oxidecomputer/crucible
crucible reconcile is sending Nexus too many notifications
Open
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 260
- Forks
- 34
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 8
Description
During a recent reconciliation:
23:05:09.775Z INFO propolis-server (vm_state_driver): sending reconcile request ReconciliationId(3434)
= downstairs
client = 0
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.783Z INFO propolis-server (vm_state_driver): notified Nexus of reconcile progress
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.783Z INFO propolis-server (vm_state_driver): new DNS resolver
addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53]
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.791Z INFO propolis-server (vm_state_driver): notified Nexus of reconcile progress
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.791Z INFO propolis-server (vm_state_driver): new DNS resolver
addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53]
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.798Z INFO propolis-server (vm_state_driver): notified Nexus of reconcile progress
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.798Z INFO propolis-server (vm_state_driver): new DNS resolver
addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53]
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:09.805Z INFO propolis-server (vm_state_driver): notified Nexus of reconcile progress
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:10.286Z INFO propolis-server (vm_state_driver): new DNS resolver
addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53]
job = notify
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
23:05:10.288Z INFO propolis-server (vm_state_driver): sending reconcile request ReconciliationId(3438)
= downstairs
client = 0
session_id = 54ea0ad5-3e3d-4080-b354-ed0767898da6
It seems like we are sending 4 notified Nexus of reconcile progress between each actual reconciliation request.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Rust reconciliation path that emits “notified Nexus of reconcile progress” and compare those notifications with the actual reconcile requests shown in the issue. Reproduce the log pattern if possible; done means redundant progress notifications are no longer sent between reconciliation requests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100