microsoft / microsoft/duroxide-pg

Add upgrade tests for persisted serde format (Event, WorkItem, KvEntry)

Open
#14 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
44
Forks
25
Avg merge
1d 14h
Merged PRs (30d)
2

Description

Problem

The provider persists several duroxide Rust types to PostgreSQL via serde_json, but there is no test that catches an incompatible change to that on-disk format. A change to these types (or their serde derives), or in principle a change in serialization behavior pulled in via a dependency bump, could silently break deserialization of already-persisted rows — i.e. existing instances fail to replay after an upgrade — and nothing in CI would flag it.

What is actually persisted (all in src/provider.rs)

Type Storage Notes
duroxide::Event (Vec<Event> / single) history.event_data orchestration history; long-lived, must deserialize across many versions
duroxide::providers::WorkItem worker_queue / orchestrator_queue payloads (incl. completion) wide enum (ActivityExecute/Completed/Failed, StartOrchestration, TimerFired, ExternalRaised, CancelInstance, ContinueAsNew, QueueMessage, …); the largest format surface
duroxide::providers::KvEntry kv_store KV snapshot

Serialize sites: serde_json::to_string/to_value around L1134, L1157/1164, L1419, L1467, L1676, L1906.
Deserialize sites: serde_json::from_value::<Vec<Event>> L1001, Vec<WorkItem> L1016, KvEntry L1024, Event L1384/2029/2256, WorkItem L1576.

Why the contract lives here, not in duroxide

duroxide core knows nothing about its providers or whether/how its types are persisted — its serde representation is, from its own perspective, an internal detail it may change. The durability contract ("rows written by an older version must still deserialize") is created by this provider's choice to serde_json these types into durable columns. So the compatibility guard belongs at the provider boundary. A nice property of putting it here: a breaking duroxide change can only surface via this repo's duroxide dependency bump, so a provider-side compat test fires at exactly the right moment.

Existing coverage (insufficient)

  • corrupt_instance_history writes {"garbage": true} to verify graceful failure, not format compatibility.
  • version-related tests cover orchestration/code versioning and capability/pinned-version filtering — unrelated to serde wire format.
  • No golden/fixture of an older serialized blob is checked in; nothing asserts current code can read prior-version Event / WorkItem / KvEntry.

Proposed work

  1. Golden-fixture round-trip tests: check in serialized fixtures (representative Event, every WorkItem variant, KvEntry) captured under prior versions, and assert the current provider deserializes them. Regenerate-on-purpose, fail-on-accidental-change.
  2. Fixture replay / in-flight straddle test: persist history + queue rows under an older build (or fixtures), then drive them to completion with the current binary — covers both long-lived history and short-lived in-flight queue payloads across a binary swap.
  3. Wire these into CI so a duroxide dependency bump that changes any persisted type's representation fails loudly.

Context: surfaced while reviewing pg_durable's upgrade testing, whose Scenario A explicitly excludes the duroxide schema and delegates this format-compat concern to the provider.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/provider.rs with the listed serde_json serialization and deserialization sites, then inspect existing provider tests and CI configuration. Define representative older fixtures for Event, every WorkItem variant, and KvEntry, plus the history and queue replay scenario. Done means current code reads the fixtures, completes the straddle test, and CI catches an incompatible persisted-format change.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, rust
Domain
database, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.