denoland / denoland/deploy_feedback

Concurrent CLI deploys to the same app intermittently fail with internal errors

Open
#948 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
79
Forks
5
PR merge metrics
No merged PRs in 30d

Description

> **Disclosure:** this report was written by Claude (an AI coding assistant) while diagnosing CI failures on behalf of Kyle June, who reviewed and submitted it.

## Summary

When two or more `deno deploy` CLI uploads target the same app at roughly the same time, the backend intermittently fails one of them — either with `An unexpected internal error occurred… trace id: …` during the diffsync upload, or by closing the `revisions.watchUntilReady` stream without ever reporting missing hashes (which the CLI then surfaces as an unrelated-looking crash — filed separately). The same commit deploys fine when it's the only deploy in flight.

## Environment

- `@deno/deploy` CLI 0.0.9904 via `deno deploy --org --app [--prod]`, Deno 2.x, GitHub Actions (ubuntu-latest)
- App: `udibo/udibo` on console.deno.com (CLI source uploads, ~1360 files, diffsync)

## Evidence (2026-07-23 → 2026-07-24, UTC)

Across ~8 deploys of the same app we observed a clean split — every failure coincided with another deploy of the same app in flight; every solo deploy succeeded, including re-runs of the exact commits that had failed:

- 07-23 09:41 — deploy racing another; failed: `trace id: 9b39654553e29bd80e8c2a719cf3dba4`. A quiet re-run of this exact commit on 07-24 succeeded.
- 07-24 03:16 + 03:19 — two pushes three minutes apart → both preview deploys failed (revision created, `watchUntilReady` closed without reporting; e.g. build `s4t2f9z2wsfp`).
- 07-24 08:08 — re-run racing two other deploys of the app; failed mid-diffsync: `trace id: b78273b98b2ce551239b268b5add7d4a` (build `vge2d3be6s20`), right after `Found 1298 already uploaded files`.
- 07-24 ~09:30 — same commit re-run with nothing else deploying: success, no changes.

We also bisected the upload content via four probe deploys (new directory names, added workspace member, lockfile changes, `.gitmodules` changes) — all passed solo, ruling out content as the trigger.

## Expected

Concurrent revisions for one app should queue or fail with a clear retryable error (e.g. "another build is in progress"), not an opaque 5xx mid-upload.

## Workaround

We now serialize deploys per app with a GitHub Actions `concurrency` group, which resolved the failures on our side.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing two `deno deploy` uploads to the same app from the issue's CLI command and compare the diffsync and `revisions.watchUntilReady` outcomes with a solo deploy. Review the reported trace IDs and build IDs, then determine whether concurrent revisions can be queued or given a clear retryable error; done means the race no longer produces an opaque internal error or silent stream close.

Written by the indexing model from the issue text.

Assessment

Tech stack
deno, github-actions
Domain
backend, cli, cloud
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.