temporalio / temporalio/sdk-rust
[Bug] Sometimes `temporal_workflow_endtoend_latency` does not appear in metrics
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 523
- Forks
- 161
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 67
Description
Describe the bug
When testing Python, we had a test that ran a simple workflow and continually tried to get the /metrics Prometheus endpoint every 200ms for 10s trying to find temporal_workflow_endtoend_latency. Locally it does appear and it also appears in CI about half the time, but the other half of the time it never shows up, even after 10s and even though metrics like temporal_workflow_completed do show. I am afraid there is a racy/flaky situation where this metric is not recorded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Python end-to-end test that polls the /metrics Prometheus endpoint every 200ms for 10 seconds. Compare runs where temporal_workflow_endtoend_latency appears with runs where temporal_workflow_completed appears without it; done means the latency metric is reliably exposed after the simple workflow completes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100