temporalio / temporalio/features

[Feature Request] Endtoend latency metric for cron executions

Open
#175 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
32
Forks
28
Avg merge
1d 9h
Merged PRs (30d)
7

Description

Currently workflow_endtoend_latency SDK metric uses WorkflowExecutionStarted event time:
https://github.com/temporalio/sdk-go/blob/master/internal/internal_task_handlers.go#L1580
https://github.com/temporalio/sdk-java/blob/master/temporal-sdk/src/main/java/io/temporal/internal/replay/ReplayWorkflowExecutor.java#L114

for calculation of the exec"start time".
For cron workflows this can lead to very large endtoend latencies as the first workflow task is not added to tq until firstWorkflowTaskBackoff timer fires.

Request here is to either subtract the firstWorkflowTaskBackoff duration from reported workflow_endtoend_latency metric if that is indeed its intention (to measure actual time worker processes workflow exec).
If not, then to add a new metric maybe that denotes this, for example workflow_firsttask_to_close_latency or something similar.
Other ideas:
distinguish via:
workflow_starttoclose_latency (would be as current workflow_endtoend_latency)
workflow_scheduletoclose_latency (would be workflow_endtoend_latency - firstWorkflowTaskBackoff)

or workflow_cron_execution_latency

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the workflow_endtoend_latency handling at internal/internal_task_handlers.go:1580 in the Go SDK and temporal-sdk/src/main/java/io/temporal/internal/replay/ReplayWorkflowExecutor.java:114 in the Java SDK. Clarify whether cron backoff should be excluded or represented by a new metric, then validate the chosen semantics with the repository's behavior and history compatibility tests; done means the metric behavior and naming are agreed across SDKs.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.