agent-substrate / agent-substrate/substrate

[Bug]: Lifecycle finalizers can overwrite a concurrent worker-deletion crash

Offen
#1,443 0 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @EItanya Auf GitHub ansehen
area/api-machinery area/reliability kind/bug
Vorherrschende Sprache
Go
Sterne
1.8k
Forks
316
Ø Merge
2 T. 43 Min.
Gemergte PRs (30 T.)
287

Beschreibung

### What happened?

When a worker disappears during ResumeActor or SuspendActor, DeleteWorker marks the actor
CRASHED and clears its worker assignment. The lifecycle workflow can subsequently re-read that
updated actor and unconditionally finalize it:

- Resume changes CRASHED to RUNNING, leaving a running actor without a worker.
- Suspend changes CRASHED to SUSPENDED, potentially without a valid snapshot.

Optimistic version checks do not prevent this because the finalizers read and update the new
CRASHED version.

### Expected Behavior

finalization should require the actor to remain RESUMING or SUSPENDING. If worker deletion has already crashed it, the workflow should preserve CRASHED and return FailedPrecondition.

### Steps to Reproduce

Regression tests can reproduce both cases by deleting the worker from the fake atelet
immediately after restore/checkpoint completes.

### Sandbox Runtime

Both / Runtime Agnostic

### Agent Substrate Version / Commit SHA

main

### Kubernetes Version & Environment

_No response_

### Host OS & Architecture

_No response_

### Relevant Logs and Diagnostic Output

```shell

```

### Additional Context

_No response_

### Confirmation

- [x] I have searched existing issues and verified that this is not a duplicate.
- [x] I have verified that this issue occurs on the latest commit on `main`.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.