Add `ExactlyOnceEvaluator`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 44
- Forks
- 7
- Avg merge
- 17h 16m
- Merged PRs (30d)
- 13
Description
Add an evaluator that enforces exactly-once execution semantics for CallableModels that perform external side effects (sending notifications, posting webhooks, writing to external systems, kicking off downstream jobs, etc.).
This is distinct from durable caching. Durable caching is an optimization — a missing entry simply means recompute. Exactly-once is a correctness guarantee — a missing ledger entry means "the side effect has not happened, run it now," and a present entry means "do not run it again under any circumstances." Clearing the ledger has real-world consequences and should not be casual.
Behavior
- Maintain a durable ledger of completed
(model, context)invocations. - On invocation, check the ledger; if present, skip execution; if absent, execute and record completion atomically.
- Pluggable ledger backend.
Open questions
- Return value on skip. When a call is skipped because the ledger already records completion, what does the evaluator return? Options include a stored minimal acknowledgment, a typed sentinel, or raising. Each has tradeoffs for downstream composition.
- Atomicity. The side effect and the ledger write are not generally co-transactional. How do we minimize the window where the side effect has happened but the ledger has not yet been updated, and what's the recommended user-side pattern when this matters?
- Relationship to durable caching. Should the two evaluators share a storage abstraction, or stay deliberately separate to keep the semantics from blurring?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the CallableModel evaluator entry points and the existing durable-caching implementation referenced in the issue. Before implementation, resolve the return value, atomicity, and storage-abstraction questions; done means the semantics and pluggable ledger behavior are specified well enough to test exactly-once execution and skipped invocations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100