openworkflowdev / openworkflowdev/openworkflow

Flaky parallel crash-recovery test needs explicit step synchronization

Offen
#710 1 Kommentar 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

@yqin0512 arbeitet bereits daran.

Seit 13.9.2026.

bug
Vorherrschende Sprache
TypeScript
Sterne
1.3k
Forks
66
Ø Merge
1 T. 1 Std.
Gemergte PRs (30 T.)
69

Beschreibung

Description

recovers from crashes during parallel step execution is nondeterministic because it assumes the successful step-a branch has persisted its completion before the failing step-b branch rejects Promise.all. There is no synchronization enforcing that ordering.

Source: https://github.com/openworkflowdev/openworkflow/blob/1b819b106b020242ff4c01b823860eb90b2a2c01/packages/openworkflow/worker/worker.test.ts#L380-L429

Observed failure
AssertionError: expected { a: 'x', b: 'b', attempts: 2 } to deeply equal { a: 'a', b: 'b', attempts: 2 }

- Expected
+ Received

  {
-   "a": "a",
+   "a": "x",
    "attempts": 2,
    "b": "b",
  }

The test passes most runs but fails intermittently under concurrent/cloud execution.

Race sequence
  1. step-a and step-b start concurrently in Promise.all.
  2. step-b throws Simulated crash.
  3. The workflow is rescheduled and releases worker ownership before step-a durably records its successful output.
  4. On the second workflow attempt, step-a is absent from the completed-step cache.
  5. Its callback runs again and deliberately returns "x" because attemptCount > 1.

The assertion expects "a", so correctness currently depends on promise/database scheduling.

Suggested fix

Add explicit synchronization so step-b does not throw until step-a has completed its durable step write. For example, expose a deferred signal resolved after the step-a promise completes, and await it in the step-b branch before throwing. Synchronizing only inside the step-a callback is insufficient because persistence happens after that callback returns.

Alternatively, poll the backend for a completed step-a attempt before allowing step-b to fail. This would preserve the intended assertion: completed parallel work is cached across workflow retry.

Expected behavior

The test should deterministically establish that step-a is completed before triggering the simulated crash, then verify that replay reads "a" from durable history rather than executing its callback again.

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.