koala73 / koala73/worldmonitor
fix(vercel-ignore): same-commit redeploy is always skipped, so env var changes cannot reach production
- Dominant language
- TypeScript
- Stars
- 86.6k
- Forks
- 13.1k
- Avg merge
- 8h 4m
- Merged PRs (30d)
- 825
Description
## Summary
An environment variable added or rotated in Vercel cannot reach production on `main` by redeploying, because `scripts/vercel-ignore.sh` cancels every same-commit redeploy. The standard recovery for a credential change — "set the value, then redeploy" — silently does nothing.
This extended the #8208 outage. Checkout, customer-portal and notification-channels stayed down for ~45 minutes *after* the correct secret was in Vercel, across two redeploy attempts.
## Reproduction
1. Set or rotate any env var in Vercel production.
2. Redeploy the current `main` commit (dashboard Redeploy, or `vercel --prod`).
3. The deployment is canceled after ~7s and production keeps serving the old build, which does not have the value.
```
Running "bash scripts/vercel-ignore.sh"
Skipping: no web-relevant changes on main
The deployment was canceled because the Ignored Build Step command returned exit code 0.
```
Observed on deployments `worldmonitor-5z2wvbamw` (02:29Z) and `worldmonitor-jofyfh6f2` (02:51Z) on 2026-09-16.
## Cause
`scripts/vercel-ignore.sh:6-20` decides on a path diff:
```bash
WEB_CHANGES=$(git diff --name-only "$VERCEL_GIT_PREVIOUS_SHA" HEAD -- 'src/' 'api/' ...)
[ -z "$WEB_CHANGES" ] && echo "Skipping: no web-relevant changes on main" && exit 0
```
On a same-commit redeploy `VERCEL_GIT_PREVIOUS_SHA` is `HEAD`, so the diff is necessarily empty and the skip is unconditional. The check is correct for its actual purpose (don't rebuild for a docs-only merge) — it just has no notion of "the build inputs changed even though the code did not".
Two things make it expensive to diagnose:
- The deployment reports as **`Canceled`**, which reads like a transient infra failure rather than a deliberate skip. Nothing distinguishes it from a real cancellation in `vercel ls`.
- Vercel binds env vars at **build** time, so the symptom is a config-missing branch (`api/create-checkout.ts:164` → 503) in code that looks correctly written.
## Proposed fix
Options, roughly in order of preference:
1. **Escape hatch in the script.** Honour an explicit override so a redeploy can force a build — e.g. build unconditionally when `VERCEL_GIT_PREVIOUS_SHA == HEAD` (a same-commit redeploy is by definition a deliberate manual action, never a merge-train event). This makes the dashboard button work as everyone expects.
2. **Document the real procedure** in `docs/relay-credentials.md` and anywhere else that says "set the variable and redeploy": the working command is `vercel --prod --force --scope eliewm` (`--force` bypasses the Ignored Build Step).
3. Make the skip legible — the current message says *why* it skipped but not that env changes will not land.
(1) and (2) are not exclusive; (2) is worth doing regardless since it is the recovery path under incident pressure.
## Diagnostic worth keeping
Variable newer than the newest READY production build ⇒ production does not have it:
```
vercel env ls production --scope eliewm --project worldmonitor # prints each var's age
vercel ls worldmonitor --scope eliewm --prod # prints each deployment's age + status
```
Credential-free liveness probe for the gateway env (returns `misconfigured` if any required var is missing in the serving build, `invalid_state` if all are present):
```
curl -A "worldmonitor-ops-diagnostic/1.0" \
"https://www.worldmonitor.app/api/discord/oauth/callback?code=probe&state=nonexistent"
```
Note the project lives under the `eliewm` team and the repo has no `.vercel/` link, so both CLI commands need explicit `--scope`/`--project`.
Context: #8208, and the validation comment https://github.com/koala73/worldmonitor/issues/8208#issuecomment-5691556926
Contributor guide
Research direction
Start with scripts/vercel-ignore.sh:6-20 and review how Vercel sets VERCEL_GIT_PREVIOUS_SHA for same-commit redeploys. Check docs/relay-credentials.md for the current recovery procedure, then validate the chosen behavior with a forced or same-commit deployment; done means environment-variable changes can reach production and the documented command works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash
- Domain
- build-system, cloud, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100