microsoft / microsoft/pg_durable

Keep large HTTP transfer payloads out of Duroxide history

Open
#376 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
2.8k
Forks
80
Avg merge
1d 22h
Merged PRs (30d)
32

Description

Currently, large HTTP payload bytes end up in the Duroxide history. In theory, (I think) we only need to persist the things necessary to reproduce whatever orchestration decisions are necessary, and resume work. That means we shouldn't need to persist HTTP payloads, and doing so causes actual problems (performance, security, data retention, data control, etc).

It's possible that some situations will require keeping the body in the history (particularly where the bodies are small, replay/orchestration needs the body's exact value, it contains no credentials/secrets/potential PII/otherwise sensitive info, ...). That's probably fine. The precise set of conditions when these need to be kept needs to be hammered out as part of this work.

This needs some further design too.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by mapping where HTTP payloads enter and are written to Duroxide history, then define which payload conditions are needed for replay and orchestration. Done means large or sensitive payloads are kept out of history while any explicitly justified small payloads remain available when required.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.