microsoft / microsoft/durabletask-python

Consider lazy durable entity state serialization semantics

Open
#234 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
40
Forks
33
Avg merge
2d 2h
Merged PRs (30d)
6

Description

Summary

Durable entity state is currently serialized eagerly when set_state() is called. This snapshots the value immediately, so mutations made to the same Python object after set_state() do not affect the state eventually persisted at the end of the operation.

state = Point(1)
self.set_state(state)
state.value = 99

The persisted state is Point(1), not Point(99).

This may surprise Python users who expect ordinary reference semantics and could introduce subtle bugs when entity code mutates an object after passing it to set_state().

Scope

The behavior is implemented in the core durabletask entity StateShim, so it affects core durabletask entities and providers built on it, including durabletask-azuremanaged and azure-functions-durable.

Current rationale

Eager serialization was introduced intentionally to:

  • surface serialization errors inside the operation that called set_state();
  • preserve per-operation rollback behavior in an entity batch;
  • retain the original serialized wire payload when state is unmodified; and
  • prevent mutations to values returned by get_state() from implicitly changing persisted state without a corresponding set_state().

Design question

Should set_state() retain the live value and defer serialization until successful operation completion, so mutations made after set_state() but before the operation returns are persisted?

If so, the design needs to define:

  • whether get_state() after set_state() returns the same pending live object or a reconstructed copy;
  • where serialization occurs so serialization failures still fail and roll back the current operation;
  • behavior across multiple operations in one entity batch;
  • compatibility implications of changing set_state() from snapshot-at-call semantics to retain-reference-until-commit semantics; and
  • consistency with Durable Entity semantics in other language SDKs.

Suggested investigation

  • Compare mutation semantics with other Durable Entity SDKs.
  • Prototype deferred serialization at the operation commit boundary.
  • Add tests for post-set_state() mutation, serialization failure, rollback, deletion, unmodified wire payloads, and multiple operations in one batch.
  • Document the current requirement to perform mutations before the final set_state() call unless the behavior changes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the core durabletask entity StateShim and trace how set_state(), get_state(), operation completion, rollback, and deletion currently handle serialized state. Compare mutation semantics with other Durable Entity SDKs before deciding whether deferred serialization is appropriate. Add coverage for post-set_state mutation, serialization failure, rollback, deletion, unchanged wire payloads, and multiple operations in one batch.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.