zigflow / zigflow/helpers

Explore a generic Go workflow step runner with hooks

Open
#2 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1
Forks
0
Avg merge
10m
Merged PRs (30d)
4

Description

Summary

Thread

Explore a small Go helper for Temporal workflows that wraps logical workflow steps and applies reusable checks/hooks before and after executing them.

The initial motivation is making Continue-As-New easier to adopt for workflows that are approaching Temporal's recommended history limits, without forcing developers to manually thread CAN checks and resume logic throughout their workflow code.

This is similar to behaviour Zigflow already gets naturally because workflows are represented as an ordered sequence of tasks with state carried explicitly between them.

Rough shape

Something along these lines:

err := Step(
    ctx,
    "fetch-account",
    func() error {
        return workflow.ExecuteActivity(
            ctx,
            FetchAccount,
            accountID,
        ).Get(ctx, &account)
    },
    Checkpoint(&state),
    ContinueAsNewWhenSuggested(),
    ObserveSteps(),
)

The public API should remain small and idiomatic Go:

func Step(
    ctx workflow.Context,
    id string,
    fn func() error,
    hooks ...Hook,
) error

Avoid separate pre and post hook arguments. A hook should be able to participate before, after or both.

For example:

type Hook interface {
    Before(ctx workflow.Context, step Step) Decision
    After(ctx workflow.Context, step Step, result Result) Decision
}

A no-op adapter could make hooks that only implement one side easy to write.

Execution model

Conceptually:

run Before hooks
    -> proceed
    -> skip step
    -> Continue-As-New
    -> error

execute Temporal operation

run After hooks
    -> proceed
    -> request Continue-As-New
    -> error

return

Post-hook semantics need some thought. In particular, hook ordering should ideally not become part of correctness. For example, a CAN request from one post hook probably should not prevent another hook from recording state or updating visibility.

Initial built-in hooks

Checkpoint / resume

Carry explicit workflow state containing the current logical position.

For example:

type State struct {
    ContinueAsNewStep StepID

    // business state...
}

Each logical step needs a stable identifier.

A checkpoint hook can:

  • skip steps that were completed in a previous Run
  • mark a step as completed after successful execution
  • allow a new Run created via Continue-As-New to resume from the correct logical position

Prefer explicit stable IDs such as:

"fetch-account"
"charge-account"

rather than randomly generated UUIDs. Step identifiers become part of durable workflow state and should remain deterministic and stable across replay and Continue-As-New.

The internal mechanism may be better thought of as a cursor, while the overall pattern is checkpointing.

Continue-As-New when suggested

Before starting a new logical step, check Temporal's Continue-As-New recommendation.

If CAN is recommended, return a Continue-As-New result carrying the workflow state and current checkpoint.

This allows CAN to happen automatically at safe logical boundaries rather than requiring checks to be scattered throughout workflow code.

Observability

Potentially provide hooks for things such as:

  • logging logical step transitions
  • recording the current step
  • upserting visibility/search attributes
  • measuring logical step duration

These should remain optional and should not introduce surprising workflow behaviour.

Testing hooks

One particularly useful extension is deterministic fault/control injection in workflow tests.

Examples:

FailBefore("charge-account")
ContinueAsNewBefore("charge-account")
RecordExecution(...)

This should make it straightforward to test behaviour such as:

fetch-account -> execute
Continue-As-New
fetch-account -> skip
charge-account -> execute

This provides a useful way to prove that checkpoint/resume behaviour actually works rather than only testing CAN in the happy path.

Extensibility

The library should deliberately expose the same hook API used by built-in behaviour so users can provide their own policies:

Step(
    ctx,
    "charge-account",
    fn,
    Checkpoint(&state),
    ContinueAsNewWhenSuggested(),
    MyCompanyPolicy(),
)

The goal is not to build a large framework. The useful abstraction is simply:

Apply reusable policy at logical workflow-step boundaries.

Possible customer-specific uses might include validation, observability, testing controls or other deterministic workflow-local checks.

Non-goals

Avoid turning this into generic Temporal magic.

Initially do not:

  • automatically heartbeat Activities
  • silently mutate retry policies
  • silently mutate timeouts
  • silently change Task Queues
  • try to restore arbitrary Go call stacks across Continue-As-New
  • hide non-deterministic behaviour behind hooks

Validation of configuration may be useful. Silently rewriting workflow behaviour is probably not.

The runner should operate at explicit resumable step boundaries.

Pattern name

Possible Temporal Design Patterns name:

Checkpointed Continue-As-New

Intent:

Transparently Continue-As-New at safe execution boundaries while preserving progress through a multi-step Workflow.

The reusable Go primitive underneath it is broader and could be described as a Workflow Step Runner or Workflow Step Middleware.

A possible package/API name is Resumable Steps.

Why explore this

There is a real customer use case behind this.

A Go customer has workflows whose histories are already large enough for Temporal to recommend Continue-As-New, but they have been reluctant to implement CAN because of the additional workflow bookkeeping and complexity.

A small helper could provide something concrete to put in front of them:

How are you getting on with Continue-As-New, and would this make it easier?

That makes this a good opportunity to validate the abstraction against a real workflow rather than designing it entirely in isolation.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by validating the proposed Step API and Before/After hook execution model against a real Go Temporal workflow, including checkpointing and Continue-As-New behavior. Done requires an agreed small API, defined hook ordering and decisions, and deterministic tests covering resume, skip, fault injection, and Continue-As-New.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.