temporalio / temporalio/features

Research and implement a way to test and make sure history is still compatible across SDK versions

Open
#598 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
32
Forks
28
Avg merge
1d 9h
Merged PRs (30d)
7

Description

Need a more formal way to test that history is compatible across SDK versions and will continue to be. Notes:

  • The features repo today has a concept of capturing history as a JSON file when it runs using --generate-history. This history, if present on future runs, will be replayed. For example, here's some captured history for Go/Java at https://github.com/temporalio/features/tree/main/features/activity/retry_on_error/history that is now tested on each features run. But this capture/replay is not supported in all languages and it doesn't make sense for every feature. We could expand use of this.
  • Some SDK tests, especially in Go and Java, have a decent amount of old histories they replay during unit testing. We could do this more in other SDKs.
  • The omes repo has a "fuzzer" that is capable of generating all sorts of workflow histories. This could be leveraged to generate thousands of histories to be run via replayer across newer SDK versions.
  • Some SDKs such as Python rely on standard library code to decide coroutines. There is a concern that newer language runtimes may affect things like coroutine order. So ideally any such replay testing also includes histories of workflows using asynchronous utilities from within the language that people use.

So long as the basic goal of confirming older histories are compatible on newer SDKs is accomplished, we can take one of several approaches.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the features repo's --generate-history capture/replay behavior and the existing Go and Java history examples under features/activity/retry_on_error/history. Review the replay histories already used by SDK unit tests and the omes fuzzer's generated histories. Done means a formal approach confirms older workflow histories remain compatible on newer SDK versions, including relevant asynchronous-language cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java, python
Domain
backend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.