Archival: configurable workflow-status filter (only archive non-successful workflows)

Archival: configurable workflow-status filter (only archive non-successful workflows)

Background

Temporal's archival is currently all-or-nothing at the namespace level: when archival is enabled, every closed workflow is written to the configured archiver regardless of its final status. For installations where the operational interest is heavily skewed toward debugging failures, this means the vast majority of archive volume (and cost) is workflows that nobody ever reads again.

At scale, this is the dominant component of archive cost. For a 50M-workflows/day deployment with a typical 5–15% failure rate, archiving every workflow generates ~7.5B PUT requests/month against the configured S3-compatible store (5 PUTs per workflow: 1 history blob + 4 visibility index entries). At AWS S3 list pricing that's ~$37,500/month; ~90% of those requests are for successful workflows that exit retention without ever being inspected.

The structural shape of the cost — 5 PUTs per workflow — doesn't change with payload size, compression, or codec. It scales linearly with workflow completion rate.

Describe the solution you'd like

A configurable filter at the archiver provider level that decides which workflow statuses to archive. All existing implementations of archiver.VisibilityArchiver and archiver.HistoryArchiver (s3store, filestore, gcloud) would short-circuit Archive() calls for workflows whose final status isn't in the configured allowlist — benefits aren't backend-specific.

Config shape (sketched, not prescriptive):

archival:
  history:
    state: enabled
    enableRead: true
    statusFilter:                                   # NEW
      - WORKFLOW_EXECUTION_STATUS_FAILED
      - WORKFLOW_EXECUTION_STATUS_TIMED_OUT
      - WORKFLOW_EXECUTION_STATUS_TERMINATED
      - WORKFLOW_EXECUTION_STATUS_CANCELED
    provider:
      s3store:
        region: us-east-1
        # ... existing s3store config unchanged
  visibility:
    state: enabled
    enableRead: true
    statusFilter:                                   # same shape
      - WORKFLOW_EXECUTION_STATUS_FAILED
      - WORKFLOW_EXECUTION_STATUS_TIMED_OUT
    provider:
      s3store:
        # ...

When statusFilter is omitted, behavior is unchanged (archive everything — backwards compatible).

The filter applies independently to history and visibility so deployments can keep richer visibility (all failures + cancellations) while keeping history archival narrower (only true failures), or vice versa.

Implementation sketch

The filter naturally lives at the archiver provider layer because the cost (request volume) is incurred at the archiver-PUT path. Two places need to check the filter:

VisibilityArchiver.Archive() — archiverspb.VisibilityRecord already carries Status directly. A few lines of code at the top of each provider's Archive():

if !cfg.StatusFilter.Allows(record.GetStatus()) {
    return nil
}

HistoryArchiver.Archive() — ArchiveHistoryRequest doesn't carry status. The cleanest path is to read the workflow's last event from the bootstrap container's ExecutionManager and inspect its EventType:

resp, err := h.executionManager.ReadHistoryBranch(ctx, &persistence.ReadHistoryBranchRequest{
    ShardID:     request.ShardID,
    BranchToken: request.BranchToken,
    MinEventID:  request.NextEventID - 1,
    MaxEventID:  request.NextEventID,
    PageSize:    1,
})
// inspect resp.HistoryEvents[0].GetEventType()

One extra persistence read per archive attempt — negligible vs. the S3 PUT volume saved.

The wrapper-archiver pattern (intercepting the existing provider rather than modifying it) is one option; a flat config field on each provider is another. Both work — maintainers should pick whichever fits the codebase better. Happy to share a reference implementation if useful.

Behavior on filter-out

The archiver returns nil (success) without writing. Temporal's retention timer treats it as a successful archive, persistence deletion proceeds normally. From the cluster's perspective, the workflow was archived; from the operator's perspective, only the filtered statuses ever land in S3.

Behavior on filter-determination failure

If the history archiver can't determine status (e.g., transient persistence read failure), it should fall open — delegate to the inner archiver and write anyway. Over-archiving on a transient failure is preferable to silently dropping data the operator might need.

Describe alternatives I've considered

Wrapping archiver in user code, requires forking temporal-server. Works as a proof-of-concept and the wrapper code is small (~270 lines against a stable API), but every Temporal user who needs this feature ends up maintaining their own fork — duplicated effort and a non-trivial operational commitment (rebase against upstream tags, security-patch SLAs). Operating Temporal forks at scale isn't a commitment many teams are in a position to take on, which is what motivates landing this upstream.

Sidecar / external completion observer that polls Temporal and copies only failures to S3. Doesn't require Temporal changes, but has its own problems: needs a durable cursor across restarts, can't easily match Temporal's own retention-timer semantics, adds a second source of truth, and reimplements much of the archival pipeline. Practical workaround but architecturally worse than a config field.

Multiple namespaces by archival policy. Theoretically possible — one namespace for archived-by-default, one for never-archived. Doesn't work in practice because the archival decision needs to be made at workflow-close time based on outcome, not at workflow-start time when only the namespace is known.

S3 lifecycle policies that delete successful workflows after a short period. Still pays the full PUT cost up front — doesn't address the dominant cost component. Also operationally complex (per-object metadata for status, lifecycle filter expressions).

Post-process janitor that deletes successful workflows from the archive. Same problem as lifecycle — still incurs the PUT cost. And introduces a second async system that can fall behind.

Additional context

Scale data

Empirical observations from a POC stack running Temporal 1.27.2 against MinIO with s3store archival enabled:

• Per workflow: 5 PUTs. One history blob at <namespace>/history/<wfID>/<runID>/<version>/<batchIdx>, plus four visibility index files at <namespace>/visibility/{workflowID|workflowTypeName}/<value>/{startTimeout|closeTimeout}/<timestamp>/<runID>. Verified by listing the bucket after archival.
• Archival timing: visibility AND history archive immediately on workflow close, NOT at retention expiry. Retention only governs when workflows are deleted from live persistence; archival fires within seconds of the workflow closing.
• 5 PUTs × 50M workflows/day × 30 days = 7.5B PUTs/month. Independent of payload size or compression.

Why this matters more for OSS users than Temporal Cloud users

Temporal Cloud customers don't see this cost because archival is part of their bundled price. OSS users self-hosting on AWS S3 see the full PUT cost, and at high volumes (50M+ workflows/day) it dominates the total Temporal-related infrastructure spend.

Existing related discussions

These long-standing UI bugs in temporalio/ui make the current "archive everything" behavior even less useful for OSS users:

• temporalio/temporal#5624 — ListArchivedWorkflowExecutions errors get swallowed by the UI, rendering an empty "no workflows" state
• temporalio/temporal#6193 — clicking into an archived workflow 404s because the UI calls DescribeWorkflowExecution, which is not archive-aware

These aren't blockers for the RFC but speak to the broader "archived data is hard to use" experience that makes the cost-vs-value tradeoff worse for archive-everything.

Backward compatibility

Adding statusFilter as an optional config field is fully backward compatible — omitting it preserves current behavior. No existing deployments need to change anything when upgrading to a release that includes this feature.

Open to discussing

If maintainers are open to this in principle, happy to discuss design choices before anyone writes code:

• whether the filter belongs in the archiver provider config (per-provider) or somewhere more central (per-namespace, per-cluster)
• whether to expose just status filtering or a more general predicate API
• naming (statusFilter? archiveStatuses? excludeStatuses?)
• whether the wrapping-archiver pattern or a flat config field is the better fit for the codebase