Order completion rollup: all-cancelled and partially-failed orders both report 'completed' — intended?
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 42/100
Research direction
Start with allFulfillmentsFailed() and the completion rollup in run(), then replay the testsuite scenarios using traces/s8_all_unavailable_amend_completes_empty.ndjson and traces/s7_partial_shipment_failure.ndjson. Confirm the intended status semantics with maintainers; done means an agreed behavior is covered by the relevant workflow checks or documented on allFulfillmentsFailed().
Written by the indexing model from the issue text.
Description
Summary
While running a formal-verification exercise over this reference app's workflows (details below, and previously referenced in #121), the model checker surfaced two reachable end states of the Order workflow that a reader of the order status alone would probably not expect. Both follow from the same line of logic, both are easy to reproduce, and both may well be intentional demo simplifications — filing this as a question about intent rather than a bug report.
The root is allFulfillmentsFailed() combined with the completion rollup in run():
status := OrderStatusCompleted
if wf.allFulfillmentsFailed() { // failures >= 1 && failures == len(fulfillments)
status = OrderStatusFailed
}
Only FulfillmentStatusFailed counts toward failure — cancelled does not — so:
Observation 1 — an order can complete with nothing fulfilled and nothing charged
Shortest path (two customer-visible steps):
- Customer places an order in which every item is unavailable → status
customerActionRequired, all fulfillmentsunavailable. - Customer sends the
amendaction (as the flow invites) →cancelUnavailableFulfillments()cancels all of them → each fulfillment'sprocess()returns immediately → zero failures → final statuscompleted, fulfillments allcancelled.
No charge was made, nothing shipped, and the order reports success. A Charge activity is never invoked on this path.
Observation 2 — partial failure reports as plain completed
With two fulfillments: one charge is declined (fulfillment → failed), the other fulfillment ships and delivers (→ completed). Since not all fulfillments failed, the order's final status is completed, with no partial marker at the status level. A caller reading only OrderStatus.Status (e.g. the orders list in the web UI) cannot distinguish this from full success; the per-fulfillment detail is present but requires drilling into the fulfillments array.
Questions for maintainers
- Are these the intended semantics for the demo? (Entirely plausible — a partial-success status adds UI and API surface a teaching app may not want.)
- If a change would be welcome, two minimal options, happy to PR either:
- count
cancelledfulfillments toward the "nothing succeeded" check, so the all-cancelled case endscancelled/failedrather thancompleted; and/or - introduce a
partiallyCompleted-style status (or a boolean onOrderStatus) when fulfillment outcomes are mixed.
- count
- If the current behavior is intended, would you take a short doc comment on
allFulfillmentsFailed()noting that cancelled ≠ failed and that mixed outcomes reportcompleted? That would make the intent legible to people using this app as a reference, which is presumably its job.
How this was found / how to reproduce
The workflows were driven unmodified through the SDK's testsuite (no server) to capture execution traces; several independently LLM-derived specs were replayed against those traces to establish faithfulness; then an exhaustive model check ran against intent invariants such as "completed implies some fulfillment completed." Both observations come with machine-found shortest counterexamples and captured trace witnesses:
- write-up + traces + counterexamples: https://github.com/jdubray/polygraph/tree/main/examples/polygraph-oms-go
- trace witnesses:
traces/s8_all_unavailable_amend_completes_empty.ndjson(Observation 1),traces/s7_partial_shipment_failure.ndjson(Observation 2)
Reproducing Observation 1 by hand in the running app: add only out-of-stock items to an order, then choose "amend" when prompted — the order lands on completed.
Related: #121 (carrier-status validation on the Shipment workflow, from the same exercise).
- Dominant language
- Go
- Stars
- 84
- Forks
- 18
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100