firebase / firebase/firebase-tools
Cloud Functions v2: firebase deploy sends UpdateFunction twice in one deploy; duplicate fails with 409 ABORTED "unable to queue the operation"
- Dominant language
- TypeScript
- Stars
- 4.5k
- Forks
- 1.3k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 84
Description
### [REQUIRED] Environment info
- **firebase-tools:** 15.22.3
- **Platform:** Linux (GitHub Actions, ubuntu-24.04), Node.js 22
- **Auth:** service account via `GOOGLE_APPLICATION_CREDENTIALS` / ADC (non-interactive CI)
### [REQUIRED] Test case / Steps to reproduce
1. Have a deployed 2nd-gen (Cloud Functions v2 / Cloud Run) HTTPS function.
2. Make a real change so the function is not skipped.
3. Run `firebase deploy --only functions` (non-interactive, service-account auth).
### [REQUIRED] Expected behavior
A single `UpdateFunction` operation per changed function, and the deploy reports success.
### [REQUIRED] Actual behavior
The CLI issues `google.cloud.functions.v2.FunctionService.UpdateFunction` **twice for the same function, ~2 seconds apart, within a single `firebase deploy`**. The first call is accepted and runs the deploy operation to completion; the second (duplicate) call is rejected by the API with `code 10 ABORTED` / HTTP 409 `unable to queue the operation`, because the first operation is still in flight.
The CLI surfaces the *second* call's 409 and reports the deploy as failed — **even though the function actually deployed successfully** (it reaches `ACTIVE` with the new source hash, and a new Cloud Run revision goes live).
A subsequent re-run reports `Skipping the deploy of unchanged functions` / `No changes detected` (because the first call already deployed), which masks the problem and makes it look like a transient/flaky 409.
#### CLI output
```
i functions: updating Node.js 22 (2nd Gen) function MY_FUNCTION(us-central1)...
⚠ functions: Request to https://cloudfunctions.googleapis.com/v2/projects/MY_PROJECT/locations/us-central1/functions/MY_FUNCTION?updateMask=name%2C...%2Clabels had HTTP Error: 409, unable to queue the operation
⚠ functions: failed to update function projects/MY_PROJECT/locations/us-central1/functions/MY_FUNCTION
```
#### Evidence (Cloud Audit Logs)
All entries are from the same principal, same caller IP, and the same `FirebaseCLI/15.22.3` user agent — i.e. **one CLI invocation**. Project/SA identifiers redacted.
| Time (UTC) | `methodName` | Result |
|---|---|---|
| `02:43:37.582` | `…v2.FunctionService.UpdateFunction` (**request A**) | accepted → starts `operation-…-6eff80c7` (`operation.first = true`, empty `status`) |
| `02:43:39.523` | `…v2.FunctionService.UpdateFunction` (**request B**) | `status.code: 10`, `"unable to queue the operation"` (severity ERROR) |
| `02:44:25.434` | operation `…-6eff80c7` completes | function `state: ACTIVE`, new hash live, `operation.last = true` |
Supporting `run.googleapis.com` system events show the underlying Cloud Run revision rolling out and reporting **"Deploying revision succeeded in 6.56s"** at `02:44:23`, confirming request A's operation deployed the function correctly.
Key point: requests **A** and **B** are identical `UpdateFunction` calls (same `update_mask`, same `function` body, same `firebase-functions-hash`) ~1.9s apart from the same CLI process. **B is a redundant duplicate.**
### Impact
- Every real-change deploy of a 2nd-gen function spuriously fails in CI, despite the deploy actually succeeding.
- The failure looks transient (a re-run says "No changes detected"), so it is easy to misdiagnose as a Google-side or IAM issue. (`cloudfunctions.functions.update` was `granted: true` on both calls — it is not a permissions problem.)
### Likely cause
Appears to be a client-side retry re-sending `UpdateFunction` after the first request was already accepted server-side, with no idempotency guard — so the retry collides with its own in-flight operation and gets `ABORTED`.
### Workaround
Wrap the deploy and retry on `unable to queue the operation`; the retry observes "No changes detected" (because the first call already deployed) and the deploy reports success.
### Related
- #10661 (`firebase deploy exits 0 after "409 unable to queue the operation"`)
- #6989 (`firebase deploy returns exit code 0 even if a function failed to be deployed`)
These cover the exit-code / 409-handling symptoms; this report focuses on the **duplicate `UpdateFunction` submission** that produces the 409 in the first place.
Contributor guide
Research direction
Start by reproducing `firebase deploy --only functions` for a changed Cloud Functions v2 function and trace the `FunctionService.UpdateFunction` request and retry path. Compare the two identical requests and their timing in the audit evidence; done means one update operation is submitted and the deploy reports success without a duplicate 409.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, typescript
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100