hardbyte / hardbyte/awa

Transactional job completion: commit handler output atomically with the completion (Rust in-tx API + SQL contract for Python et al)

Open
#401 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Rust
Stars
32
Forks
5
Avg merge
15h 21m
Merged PRs (30d)
21

Description

## Summary

Let handlers commit their business output **atomically with the job's completion** in the same Postgres transaction — upgrading at-least-once execution to **exactly-once state effects within the application's database**.

Requested-shape example: a handler writes rows to the app's tables and the job's `completed` transition commits with them, or neither happens; a crash before commit means a clean retry, and the existing `run_lease` stale-completion guard already refuses a rescued attempt's late commit — no double-apply.

## Why the pieces are already here

- ADR-029 commits follow-up **enqueues** atomically with completion — the transactional-finalize machinery exists.
- `fail_terminal_in_tx` / `cancel_by_unique_key_tx` (#358) established the `&mut PgConnection` lifecycle-op pattern.
- `run_lease` rejection is exactly the guard that makes user-owned completion safe.

## Design constraints (for the ADR)

1. **Python (and any non-Rust consumer) must be inverted.** Don't pass a live sqlx transaction across FFI — expose completion as a **SQL contract** (`awa.complete_job_compat(job_id, run_lease, …)` family mirroring `insert_job_compat`) that the user calls inside *their own* transaction with *their own* driver (asyncpg/SQLAlchemy/psycopg). The handler then returns a new `JobResult::AlreadyFinalized` so the executor skips its completion. This belongs inside #342's SQL contract — completion, not just enqueue.
2. **Opt-in bypass of the completion batcher.** Queue-storage batches completions (#352); a user tx can't join a batch carrying other jobs' completions. Atomic completions write their own single terminal evidence in the user's tx — the pre-batching shape, bounded to opted-in jobs. Throughput-sensitive queues don't opt in.
3. Rust gets the ergonomic in-process variant: `store.complete_in_tx(&mut tx, job_id, run_lease, …)` + the same result variant. ADR-029 follow-ups join the same tx via existing `_tx` inserts.
4. Failure paths compose: user-tx conflict → handler error → normal retry; `fail`/DLQ variants already have `_in_tx` shapes.

## Deliverables

ADR (touches ADR-026/029), Rust API + queue-storage `complete_in_tx` path, the SQL completion contract + cross-driver examples and conformance coverage (with #342), tests including a rescue-races-user-commit witness (TLA+ candidate: completion-vs-rescue with a user-held transaction), docs positioning ("exactly-once state effects in your Postgres").

Sequencing: M4, alongside #342. From the roadmap-gap review; requested in discussion of worker-side atomicity for Rust/Python clients.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ADR-026 and ADR-029, then read the SQL contract work in #342 and the existing fail_in_tx, cancel_by_unique_key_tx, and transactional-finalize entry points. Map the Rust and SQL completion paths, queue-storage batching, and rescue guard before defining the API. Done means the ADR, Rust and SQL contracts, cross-driver examples, conformance and race tests, and positioning documentation are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python, rust, sql
Domain
api, backend, database, distributed-systems, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.