open-telemetry / open-telemetry/shared-workflows

Bound dashboard delivery batches before shortening the publisher lease

Open
#382 0 comments 0 reactions 1 assignee View on GitHub

@trask is already working on this.

Since Sep 16, 2026.

Dominant language
No language data
Stars
7
Forks
18
Avg merge
22h 1m
Merged PRs (30d)
75

Description

The pull request dashboard publisher holds a repository-scoped Git lease while it delivers side effects and updates the dashboard issue. Most publisher jobs finish in under a minute, but several delivery paths can process all accumulated repository work, so the current 50-minute job timeout and 60-minute lease cannot safely be reduced based on typical runtime alone.

Add explicit, durable per-run bounds for the currently unbounded delivery paths:

  • Slack notifications across open non-draft pull requests
  • prepared author nudges
  • Copilot review requests
  • dashboard command replies
  • dashboard issue rendering or any other work whose cost grows with the repository's open pull request count

Preserve pending work for later publishers, as the status-comment rollout already does with its 50-item batch. Define bounds using API cost and worst-case runtime, not only pull request count, and ensure retries cannot lose or duplicate external effects.

Once all lock-held work has a reliable upper bound, reduce the publisher job timeout and publisher-lock lease together. The job must always be terminated before its lease can expire, while a crashed publisher should stall its repository for substantially less than the current one-hour maximum.

This follows up on the publisher-starvation work in #347. Acceptance criteria:

  • Every delivery path executed while holding the publisher lock has an explicit per-run bound.
  • Deferred work is durable and is drained by subsequent targeted or hourly publishers.
  • Tests cover partial batches, retries, new work arriving during delivery, and completion across multiple runs.
  • Production runtime data supports a shorter job timeout and lease with documented safety margin.
  • The stale-lock recovery window is reduced without allowing a live publisher to outlast its lease.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.