aws / aws/aws-durable-execution-sdk-python

[Feature]: Deduplicate SerDes deserialization within an invocation

Open
#678 0 comments 0 reactions 0 assignees View on GitHub
enhancement parity pkg:sdk
Dominant language
Python
Stars
53
Forks
25
Avg merge
1d 19h
Merged PRs (30d)
40

Description

## What would you like?

Deduplicate repeated SerDes deserialization within a single Lambda invocation.

Context-dependent SerDes implementations may perform filesystem or network I/O. Repeated access to the same checkpointed result or exception currently invokes the configured SerDes again, even when the serialized input and durable payload identity are unchanged.

## Possible Implementation

- Add an invocation-scoped deserialization cache around SDK-managed SerDes calls.
- Key entries by SerDes instance identity, durable execution ARN, operation/entity identity, payload kind when available, and a digest of the serialized input.
- Cache successful values, including `None`, but never cache failures.
- Bound completed entries with an LRU or weak-reference policy so large deserialized values are not retained indefinitely.
- If SDK-managed child/concurrency threads can request the same value concurrently, share one in-flight result rather than issuing duplicate external reads.
- Clear all entries at the end of the Lambda invocation; do not use a process-global cache across replays.
- Add tests for cache hits, distinct contexts/data, `None`, failures, eviction, and concurrent callers.

## Is this a breaking change?

No.

## Does this require an RFC?

No.

## Additional Context

This is primarily valuable for filesystem and other external-storage SerDes implementations. The Java SDK work in aws/aws-durable-execution-sdk-java#648 uses an invocation-scoped bounded cache and in-flight deduplication.

Contributor guide

Open the contributing guide

Research direction

Start by locating the SDK-managed SerDes calls and the Lambda invocation lifecycle where an invocation-scoped cache could be established and cleared. Add tests covering cache hits, distinct contexts and data, None values, failures, eviction, and concurrent callers; done means repeated matching requests are deduplicated without caching failures or crossing invocation boundaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.