Set up a development server for the API

EJAM-API Development/Staging Server Setup Plan

Date: 2026-07-08

Target outcome: create a development/staging version of EJAM-API at https://apidev.ejanalysis.com, backed by its own Google Cloud Run service and its own Artifact Registry image stream, so API changes can be tested before production deploys.

Recommended architecture: separate Cloud Run service + Artifact Registry + GitHub Actions.

Scope split:

• The owner of the EJAM-API repository / Google Cloud Run setup should handle the EJAM-API code, Artifact Registry, GitHub Actions, and Cloud Run work in this plan.
• @ejanalysis should handle Cloudflare separately after the Cloud Run dev service URL exists.

Confirmed decisions:

• Add a /status endpoint as soon as possible to both production and development API deployments.
• apidev.ejanalysis.com should be public.
• Do not use caching for the dev API server.
• Dev should run with min-instances=0 for now.
• Use manual GitHub Actions workflows only for now.
• After the dev deploy flow works, add automated smoke tests and image-digest promotion to production.

1. Current State

Production is the public EJAM-API service:

• Friendly public URL: https://api.ejanalysis.com
• Cloud Run origin documented in Public-Environmental-Data-Partners/EJAM-API: https://ejamapi-84652557241.us-central1.run.app
• API framework: R + plumber
• Container runtime: Docker image running Rscript main.r
• App port: 8080
• Current Docker image source appears to be ericnost/ejamapi:latest
• Current repo has no GitHub Actions deployment workflow; deploys appear to be manual build, push, and Cloud Run redeploy steps.

The new staging service should not share the production Cloud Run service. It should be a separate Cloud Run service named ejamapi-dev.

2. Desired End State

The development API should look like this:

GitHub repo
Public-Environmental-Data-Partners/EJAM-API
        |
        | GitHub Actions manual workflow
        v
Artifact Registry
us-central1-docker.pkg.dev/<GCP_PROJECT_ID>/ejamapi/ejamapi-dev:<git-sha>
us-central1-docker.pkg.dev/<GCP_PROJECT_ID>/ejamapi/ejamapi-dev:staging
        |
        | Cloud Run deploy
        v
Cloud Run service
ejamapi-dev, region us-central1
        |
        | default Cloud Run URL
        v
https://ejamapi-dev-<project-number>.us-central1.run.app
        |
        | separate @ejanalysis Cloudflare alias/proxy work
        v
https://apidev.ejanalysis.com

Production remains separate:

https://api.ejanalysis.com -> production Cloud Run service ejamapi

3. Why This Option

This is the best setup because:

• Dev and production are isolated at the Cloud Run service level.
• A bad dev deployment cannot receive production traffic by accident.
• Staging has a stable URL for EJAM and EJScreen integration tests.
• GitHub Actions can build and deploy the image, so local Docker knowledge is not required.
• Artifact Registry is the Google-native image store for Cloud Run and avoids relying on Docker Hub for the main deploy path.
• Production promotion can later deploy the exact same image digest that passed dev tests.
• A /status endpoint will make it clear which API code, image, EJAM version, and data version are actually deployed.

Avoid this as the main staging setup:

• Cloud Run production revision tags or traffic splits. They are useful for final production canaries, but they are not a clean staging environment.
• Docker Hub latest as the deploy source. It is too easy to lose track of what is running.
• Local-only Docker. Useful for debugging, but not enough for hosted integration testing.

4. Naming Conventions

Use these names unless there is a Google Cloud naming conflict:

GCP_PROJECT_ID="<actual-google-cloud-project-id>"
GCP_PROJECT_NUMBER="84652557241"
GAR_LOCATION="us-central1"
GAR_REPOSITORY="ejamapi"
CLOUD_RUN_DEV_SERVICE="ejamapi-dev"
CLOUD_RUN_REGION="us-central1"
DEV_RUNTIME_SERVICE_ACCOUNT="ejamapi-dev-runner"
DEPLOYER_SERVICE_ACCOUNT="github-ejam-api-deployer"
WIF_POOL_ID="github-actions"
WIF_PROVIDER_ID="github"
GITHUB_REPO="Public-Environmental-Data-Partners/EJAM-API"
DEFAULT_EJAM_VERSION="v3.2022.1"

The project number 84652557241 is visible in the current production Cloud Run URL. The project ID still needs to be confirmed in Google Cloud Console.

5. ASAP First API Change: /status and Dev No-Cache

Before or alongside the first dev Cloud Run deployment, add a lightweight status endpoint to EJAM-API and deploy it to both production and dev.

This is the highest-priority code change because it answers the operational question that is otherwise hard to answer:

What code, image, EJAM version, data version, and environment is this API URL actually serving?

Add:

GET /status

The endpoint should return JSON like:

{
  "service": "ejamapi",
  "environment": "dev",
  "api_git_sha": "787bcccc7a...",
  "image": "us-central1-docker.pkg.dev/.../ejamapi-dev:787bcccc7a...",
  "ejam_version_build_arg": "v3.2022.1",
  "ejam_package_version": "3.2022.1",
  "ejamdata_marker": "v3.2022.0",
  "built_at_utc": "2026-07-08T00:00:00Z"
}

The same endpoint should exist in production, with:

{
  "environment": "prod"
}

as the environment value.

Implementation notes:

• Pass API_GIT_SHA, IMAGE_TAG, BUILT_AT_UTC, and EJAM_API_ENV as Docker build args, Docker env vars, or Cloud Run env vars.
• Keep EJAM_VERSION already present in the Dockerfile.
• Read the installed EJAM package version from R.
• Read ejamdata_version.txt from the installed EJAM package data folder if present.
• Make /status cheap and dependency-light; it should not run an EJAM analysis.
• Return Cache-Control: no-store from /status.

Dev no-cache requirement:

• When EJAM_API_ENV=dev, the API should return Cache-Control: no-store for dev responses, including successful GET /report responses.
• Production may keep its current successful GET /report caching behavior.
• This can be implemented with a small helper such as is_dev_api <- identical(Sys.getenv("EJAM_API_ENV"), "dev") and by passing cache_header = NULL for dev report responses.

This matters because apidev.ejanalysis.com is for testing current behavior. Stale cached report output would make staging harder to trust.

6. Access Needed

Whoever performs the setup needs permission to:

• Enable Google Cloud APIs.
• Create Artifact Registry repositories.
• Create service accounts.
• Create Workload Identity Federation pools/providers.
• Grant IAM roles.
• Create and update Cloud Run services.
• Add GitHub repo variables/secrets or have admin access to the GitHub repository.

Use Google Cloud Shell for the commands below. That avoids installing gcloud locally.

Cloudflare access is not required for the Google Cloud Run / EJAM-API owner work. @ejanalysis will configure apidev.ejanalysis.com separately after the Cloud Run dev URL is available.

7. One-Time Google Cloud Setup

Open Google Cloud Shell in the Google Cloud project that owns the production API.

Set variables:

export GCP_PROJECT_ID="<actual-google-cloud-project-id>"
export GAR_LOCATION="us-central1"
export GAR_REPOSITORY="ejamapi"
export CLOUD_RUN_DEV_SERVICE="ejamapi-dev"
export CLOUD_RUN_REGION="us-central1"
export DEV_RUNTIME_SERVICE_ACCOUNT="ejamapi-dev-runner"
export DEPLOYER_SERVICE_ACCOUNT="github-ejam-api-deployer"
export WIF_POOL_ID="github-actions"
export WIF_PROVIDER_ID="github"
export GITHUB_REPO="Public-Environmental-Data-Partners/EJAM-API"

gcloud config set project "$GCP_PROJECT_ID"
export GCP_PROJECT_NUMBER="$(gcloud projects describe "$GCP_PROJECT_ID" --format='value(projectNumber)')"

Confirm the project number matches the production URL:

echo "$GCP_PROJECT_NUMBER"

Expected:

84652557241

If it does not match, stop and confirm the correct Google Cloud project before creating anything.

Enable required APIs:

gcloud services enable \
  run.googleapis.com \
  artifactregistry.googleapis.com \
  iamcredentials.googleapis.com \
  cloudresourcemanager.googleapis.com \
  sts.googleapis.com

Create the Artifact Registry Docker repository:

gcloud artifacts repositories create "$GAR_REPOSITORY" \
  --repository-format=docker \
  --location="$GAR_LOCATION" \
  --description="EJAM-API container images"

If the repository already exists, use it and continue:

gcloud artifacts repositories describe "$GAR_REPOSITORY" \
  --location="$GAR_LOCATION"

Create the Cloud Run runtime service account:

gcloud iam service-accounts create "$DEV_RUNTIME_SERVICE_ACCOUNT" \
  --display-name="EJAM-API dev Cloud Run runtime"

This runtime service account should start with no extra roles. EJAM-API currently uses public network resources and does not need Google Cloud write access for normal operation.

Create the GitHub Actions deployer service account:

gcloud iam service-accounts create "$DEPLOYER_SERVICE_ACCOUNT" \
  --display-name="GitHub Actions deployer for EJAM-API"

Set email variables:

export DEV_RUNTIME_SA_EMAIL="${DEV_RUNTIME_SERVICE_ACCOUNT}@${GCP_PROJECT_ID}.iam.gserviceaccount.com"
export DEPLOYER_SA_EMAIL="${DEPLOYER_SERVICE_ACCOUNT}@${GCP_PROJECT_ID}.iam.gserviceaccount.com"

Grant deployer access to push images:

gcloud artifacts repositories add-iam-policy-binding "$GAR_REPOSITORY" \
  --location="$GAR_LOCATION" \
  --member="serviceAccount:${DEPLOYER_SA_EMAIL}" \
  --role="roles/artifactregistry.writer"

Grant deployer access to manage Cloud Run:

gcloud projects add-iam-policy-binding "$GCP_PROJECT_ID" \
  --member="serviceAccount:${DEPLOYER_SA_EMAIL}" \
  --role="roles/run.admin"

Allow deployer to deploy services as the runtime service account:

gcloud iam service-accounts add-iam-policy-binding "$DEV_RUNTIME_SA_EMAIL" \
  --member="serviceAccount:${DEPLOYER_SA_EMAIL}" \
  --role="roles/iam.serviceAccountUser"

8. Set Up GitHub Actions Authentication Without Static Keys

Preferred method: Google Workload Identity Federation. This avoids storing a long-lived Google Cloud JSON key in GitHub.

Create a Workload Identity Pool:

gcloud iam workload-identity-pools create "$WIF_POOL_ID" \
  --project="$GCP_PROJECT_ID" \
  --location="global" \
  --display-name="GitHub Actions"

Create the GitHub OIDC provider:

gcloud iam workload-identity-pools providers create-oidc "$WIF_PROVIDER_ID" \
  --project="$GCP_PROJECT_ID" \
  --location="global" \
  --workload-identity-pool="$WIF_POOL_ID" \
  --display-name="GitHub Actions provider" \
  --attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository,attribute.ref=assertion.ref" \
  --issuer-uri="https://token.actions.githubusercontent.com"

Allow only the EJAM-API repo to impersonate the deployer service account:

gcloud iam service-accounts add-iam-policy-binding "$DEPLOYER_SA_EMAIL" \
  --project="$GCP_PROJECT_ID" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${WIF_POOL_ID}/attribute.repository/${GITHUB_REPO}"

Get the provider resource name for GitHub:

gcloud iam workload-identity-pools providers describe "$WIF_PROVIDER_ID" \
  --project="$GCP_PROJECT_ID" \
  --location="global" \
  --workload-identity-pool="$WIF_POOL_ID" \
  --format="value(name)"

It will look like:

projects/84652557241/locations/global/workloadIdentityPools/github-actions/providers/github

Save that value. It goes into a GitHub Actions variable named GCP_WORKLOAD_IDENTITY_PROVIDER.

9. GitHub Repository Variables

In GitHub:

Public-Environmental-Data-Partners/EJAM-API -> Settings -> Secrets and variables -> Actions -> Variables

Create these repository variables:

GCP_PROJECT_ID=<actual-google-cloud-project-id>
GCP_PROJECT_NUMBER=84652557241
GCP_WORKLOAD_IDENTITY_PROVIDER=projects/84652557241/locations/global/workloadIdentityPools/github-actions/providers/github
GCP_DEPLOYER_SERVICE_ACCOUNT=github-ejam-api-deployer@<actual-google-cloud-project-id>.iam.gserviceaccount.com
GAR_LOCATION=us-central1
GAR_REPOSITORY=ejamapi
CLOUD_RUN_DEV_SERVICE=ejamapi-dev
CLOUD_RUN_REGION=us-central1
DEV_RUNTIME_SERVICE_ACCOUNT=ejamapi-dev-runner@<actual-google-cloud-project-id>.iam.gserviceaccount.com
EJAM_VERSION=v3.2022.1

No Google Cloud JSON key secret is needed when Workload Identity Federation is used.

Optional GitHub secret:

EJAM_DATA_GITHUB_PAT=<fine-grained GitHub token with read access to public repos, only if rate limits become a problem>

The Dockerfile can use a BuildKit secret named github_pat while baking EJAM Arrow data into the image. The normal GitHub Actions token may be enough for public downloads, but a dedicated secret is useful if builds hit GitHub rate limits.

10. Add a Manual Dev Deploy Workflow

Create this file in the EJAM-API repo:

.github/workflows/deploy-dev-cloud-run.yaml

Proposed workflow:

name: Deploy dev EJAM-API to Cloud Run

on:
  workflow_dispatch:
    inputs:
      ejam_version:
        description: "EJAM git ref to install into the API image, such as v3.2022.1 or development"
        required: true
        default: "v3.2022.1"
      no_cache:
        description: "Force a fresh Docker build. Use true when building from a moving branch such as development."
        required: true
        default: "false"
        type: choice
        options:
          - "false"
          - "true"

permissions:
  contents: read
  id-token: write

env:
  IMAGE_NAME: ejamapi-dev

jobs:
  build-and-deploy-dev:
    name: Build and deploy dev service
    runs-on: ubuntu-latest

    steps:
      - name: Check out EJAM-API
        uses: actions/checkout@v4

      - name: Authenticate to Google Cloud
        uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: ${{ vars.GCP_WORKLOAD_IDENTITY_PROVIDER }}
          service_account: ${{ vars.GCP_DEPLOYER_SERVICE_ACCOUNT }}

      - name: Set up gcloud
        uses: google-github-actions/setup-gcloud@v2

      - name: Configure Docker for Artifact Registry
        run: gcloud auth configure-docker "${{ vars.GAR_LOCATION }}-docker.pkg.dev" --quiet

      - name: Set image tags
        id: image
        shell: bash
        run: |
          IMAGE_BASE="${{ vars.GAR_LOCATION }}-docker.pkg.dev/${{ vars.GCP_PROJECT_ID }}/${{ vars.GAR_REPOSITORY }}/${IMAGE_NAME}"
          echo "sha_tag=${IMAGE_BASE}:${GITHUB_SHA}" >> "$GITHUB_OUTPUT"
          echo "staging_tag=${IMAGE_BASE}:staging" >> "$GITHUB_OUTPUT"
          echo "built_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build and push dev image
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          no-cache: ${{ inputs.no_cache == 'true' }}
          tags: |
            ${{ steps.image.outputs.sha_tag }}
            ${{ steps.image.outputs.staging_tag }}
          build-args: |
            EJAM_VERSION=${{ inputs.ejam_version }}
          secrets: |
            github_pat=${{ secrets.EJAM_DATA_GITHUB_PAT || github.token }}

      - name: Deploy to Cloud Run dev service
        uses: google-github-actions/deploy-cloudrun@v2
        with:
          service: ${{ vars.CLOUD_RUN_DEV_SERVICE }}
          region: ${{ vars.CLOUD_RUN_REGION }}
          image: ${{ steps.image.outputs.sha_tag }}
          flags: >-
            --allow-unauthenticated
            --port=8080
            --memory=4Gi
            --cpu=2
            --timeout=3600
            --concurrency=1
            --min-instances=0
            --max-instances=1
            --service-account=${{ vars.DEV_RUNTIME_SERVICE_ACCOUNT }}
            --set-env-vars=EJAM_VERSION=${{ inputs.ejam_version }},EJAM_API_ENV=dev,API_GIT_SHA=${{ github.sha }},IMAGE_TAG=${{ steps.image.outputs.sha_tag }},BUILT_AT_UTC=${{ steps.image.outputs.built_at }}

      - name: Show dev service URL
        run: |
          gcloud run services describe "${{ vars.CLOUD_RUN_DEV_SERVICE }}" \
            --region="${{ vars.CLOUD_RUN_REGION }}" \
            --format='value(status.url)'

Why these Cloud Run flags:

• --allow-unauthenticated: makes dev behave like the public production API.
• --port=8080: matches main.r.
• --memory=4Gi: conservative starting point for EJAM plus baked Arrow data and PDF generation.
• --cpu=2: conservative starting point for report generation.
• --timeout=3600: avoids killing longer report/PDF runs while testing.
• --concurrency=1: avoids multiple heavy report renders competing inside one R process.
• --max-instances=1: keeps /handoff token behavior reliable while tokens are stored in process memory.
• --min-instances=0: keeps cost low. Change to 1 during active testing if cold starts are too slow.
• --set-env-vars: gives /status enough metadata to identify what is running, and sets EJAM_API_ENV=dev so dev can disable caching.

11. First Dev Deployment

After the workflow file is merged or available on a branch:

1. Go to GitHub Actions.
2. Select Deploy dev EJAM-API to Cloud Run.
3. Click Run workflow.
4. Use:

ejam_version: v3.2022.1
no_cache: false

For a moving EJAM branch build:

ejam_version: development
no_cache: true

After the workflow finishes, copy the service URL from the final step. It should look like:

https://ejamapi-dev-84652557241.us-central1.run.app

The exact hostname may differ. Use the workflow output as the source of truth.

12. Manual Validation Before Cloudflare

Run these from a terminal or Cloud Shell after replacing DEV_URL:

export DEV_URL="https://ejamapi-dev-84652557241.us-central1.run.app"

Docs endpoint:

curl -sSIL "${DEV_URL}/__docs__/"

Expected:

HTTP/2 200
content-type: text/html
access-control-allow-origin: *

Root redirect:

curl -sSIL "${DEV_URL}/"

Expected:

HTTP/2 302
location: /__docs__/

Status endpoint:

curl -sS "${DEV_URL}/status"

Expected:

JSON response with environment=dev, API git SHA, image tag, EJAM version, and build time.

Fast HTML report:

curl -sS -D /tmp/ejamapi-dev-report.headers \
  -o /tmp/ejamapi-dev-report.html \
  "${DEV_URL}/report?fips=10001&fileextension=html"

head -20 /tmp/ejamapi-dev-report.headers
grep -i "EJSCREEN" /tmp/ejamapi-dev-report.html | head

Expected:

HTTP status 200
content-type: text/html
cache-control: no-store

Data endpoint:

curl -sS -X POST "${DEV_URL}/data" \
  -H "Content-Type: application/json" \
  -d '{"fips":"10","scale":"county","buffer":0}' \
  | head -c 500

Expected:

JSON response body, not an HTML error page

Handoff token:

TOKEN="$(curl -sS -X POST "${DEV_URL}/handoff" \
  -H "Content-Type: application/json" \
  -d '{"fips":["10001"],"radius":0}' \
  | sed -n 's/.*"token"[ ]*:[ ]*"\([^"]*\)".*/\1/p')"

echo "$TOKEN"
curl -sS "${DEV_URL}/handoff/${TOKEN}"

Expected:

The second call returns the stored fips/radius payload.

13. Separate @ejanalysis Cloudflare Setup for apidev.ejanalysis.com

Goal:

https://apidev.ejanalysis.com -> DEV_URL

This is a separate @ejanalysis workstream. The EJAM-API / Cloud Run owner should provide the working Cloud Run dev URL; @ejanalysis should then create the public apidev.ejanalysis.com alias in Cloudflare.

Decision:

apidev.ejanalysis.com is public.

Important caution: a plain Cloudflare DNS CNAME to a run.app hostname may not be sufficient if Cloud Run rejects the incoming Host: apidev.ejanalysis.com header. Production appears to be proxied through Cloudflare rather than exposed as a visible redirect, so copy the production pattern if possible.

Use one of these patterns.

Preferred Cloudflare Pattern: Worker Proxy

Create a Cloudflare Worker that forwards requests to the Cloud Run dev URL.

Worker route:

apidev.ejanalysis.com/*

Worker code:

const ORIGIN = "https://ejamapi-dev-84652557241.us-central1.run.app";

export default {
  async fetch(request) {
    const incoming = new URL(request.url);
    const target = new URL(ORIGIN);
    target.pathname = incoming.pathname;
    target.search = incoming.search;

    const response = await fetch(target.toString(), {
      method: request.method,
      headers: request.headers,
      body: request.method === "GET" || request.method === "HEAD" ? undefined : request.body,
      redirect: "manual"
    });

    const headers = new Headers(response.headers);
    headers.set("Cache-Control", "no-store");

    return new Response(response.body, {
      status: response.status,
      statusText: response.statusText,
      headers
    });
  }
};

Replace ORIGIN with the actual Cloud Run dev URL from the GitHub Actions deploy output. The Worker forces Cache-Control: no-store so the dev alias is not cached at the Cloudflare edge.

Alternative Cloudflare Pattern: Origin Rule

If Cloudflare Origin Rules are available:

• DNS name: apidev
• Proxy status: proxied
• Origin host: the Cloud Run dev hostname, such as ejamapi-dev-84652557241.us-central1.run.app
• Host header override: same Cloud Run dev hostname

This keeps the browser URL at apidev.ejanalysis.com while sending the request to the Cloud Run origin host Cloud Run expects.

Alternative Google Pattern: Cloud Run Domain Mapping

Cloud Run has domain mapping support, but Google describes Cloud Run domain mappings as preview/limited availability and not the recommended production option. For dev it may be acceptable, but since production already uses Cloudflare, using the same Cloudflare approach is cleaner.

14. Dev Cache Policy

Do not use caching for the dev API server.

The dev no-cache policy has two layers:

• EJAM-API should return Cache-Control: no-store for dev responses when EJAM_API_ENV=dev.
• Cloudflare should bypass cache for apidev.ejanalysis.com.

Cloudflare should bypass cache for:

apidev.ejanalysis.com/*
apidev.ejanalysis.com/handoff*
apidev.ejanalysis.com/data
apidev.ejanalysis.com/query
Any POST request
Any response with Cache-Control: no-store

Do not add a dev cache rule for GET /report. Production can continue caching successful report responses, but dev should prefer freshness and deploy clarity over speed.

15. Separate @ejanalysis Validation After Cloudflare

Set:

export DEV_ALIAS="https://apidev.ejanalysis.com"

Run the same tests:

curl -sSIL "${DEV_ALIAS}/__docs__/"
curl -sSIL "${DEV_ALIAS}/"
curl -sS "${DEV_ALIAS}/status"
curl -sS -D /tmp/apidev-report.headers -o /tmp/apidev-report.html "${DEV_ALIAS}/report?fips=10001&fileextension=html"
curl -sS -X POST "${DEV_ALIAS}/data" -H "Content-Type: application/json" -d '{"fips":"10","scale":"county","buffer":0}' | head -c 500

Compare headers to production:

curl -sSIL "https://api.ejanalysis.com/__docs__/"
curl -sSIL "https://apidev.ejanalysis.com/__docs__/"

Expected:

• Both return 200 for /__docs__/.
• Both include access-control-allow-origin: *.
• Dev clearly reaches the new Cloud Run dev service, not production.
• Dev report responses include cache-control: no-store.
• Dev /status reports environment=dev.

16. Confirm /status in Production and Dev

The /status endpoint is an immediate requirement, not a later hardening task.

Confirm production after the /status production deploy:

curl -sS "https://api.ejanalysis.com/status"

Expected:

JSON response with environment=prod and production image/code metadata.

Confirm development after the /status dev deploy:

curl -sS "https://apidev.ejanalysis.com/status"

Expected:

JSON response with environment=dev and dev image/code metadata.

If environment, image tag, or git SHA is missing, treat the endpoint as incomplete. The endpoint is only useful if it can distinguish production from development and identify the deployed build.

17. Next Phase: Smoke Tests and Image-Digest Promotion

Do not automate production deployment yet. First prove the manual dev deploy flow and the public apidev.ejanalysis.com alias.

After the flow works, add automated smoke tests and image-digest promotion to production. Production should eventually use image promotion, not rebuild-from-scratch.

Recommended flow:

1. Build a Git SHA image once.
2. Deploy that exact image digest to ejamapi-dev.
3. Run automated smoke tests against https://apidev.ejanalysis.com.
4. If tests pass, manually approve deploying the same image digest to production ejamapi.
5. Deploy the approved digest to production with a manual production workflow.
6. Purge relevant production Cloudflare report cache if the API behavior or data changed.

This prevents a common failure mode where dev tests one build but production receives a slightly different rebuild.

Future production workflow can use:

manual workflow input: image_digest
target service: ejamapi
target URL: https://api.ejanalysis.com

Do not automate production deploy from every push. Keep production manual until the staging process is proven.

18. Rollback Plan

Cloud Run keeps previous revisions.

For dev:

gcloud run revisions list \
  --service=ejamapi-dev \
  --region=us-central1

Route dev traffic back to a previous revision:

gcloud run services update-traffic ejamapi-dev \
  --region=us-central1 \
  --to-revisions=<REVISION_NAME>=100

Return to latest revision later:

gcloud run services update-traffic ejamapi-dev \
  --region=us-central1 \
  --to-latest

For production, use the same commands against service ejamapi, but only after confirming the exact revision and current production state.

19. Cost Controls

Initial dev settings keep costs modest:

min-instances=0
max-instances=1
concurrency=1

Tradeoff:

• min-instances=0 costs less but cold starts can be slow.
• min-instances=1 costs more but makes dev testing feel much better.

Use min-instances=1 during active testing windows, then set it back to 0:

gcloud run services update ejamapi-dev \
  --region=us-central1 \
  --min-instances=1

Return to lower cost:

gcloud run services update ejamapi-dev \
  --region=us-central1 \
  --min-instances=0

20. Implementation Checklist

ASAP API code changes

• Add GET /status to EJAM-API.
• Deploy /status to production so https://api.ejanalysis.com/status identifies the current production build.
• Deploy /status to dev so https://apidev.ejanalysis.com/status identifies the current dev build.
• Make dev responses return Cache-Control: no-store when EJAM_API_ENV=dev.

Cloud setup

• Confirm the Google Cloud project ID for production project number 84652557241.
• Enable required Google Cloud APIs.
• Create Artifact Registry repository ejamapi in us-central1.
• Create runtime service account ejamapi-dev-runner.
• Create deployer service account github-ejam-api-deployer.
• Grant Artifact Registry writer to deployer.
• Grant Cloud Run admin to deployer.
• Grant deployer iam.serviceAccountUser on runtime service account.
• Create GitHub Actions Workload Identity Federation pool/provider.
• Bind WIF access to only Public-Environmental-Data-Partners/EJAM-API.

GitHub setup

• Add GitHub Actions repository variables.
• Add optional EJAM_DATA_GITHUB_PAT secret if builds hit rate limits.
• Add .github/workflows/deploy-dev-cloud-run.yaml.
• Keep the dev deploy workflow manual-only.
• Run workflow manually with ejam_version=v3.2022.1.
• Save the Cloud Run dev URL from the workflow output.

Cloud Run validation

• Confirm /__docs__/ returns 200.
• Confirm / redirects to /__docs__/.
• Confirm /status reports environment=dev.
• Confirm /report?fips=10001&fileextension=html returns HTML.
• Confirm dev report responses use cache-control: no-store.
• Confirm POST /data returns JSON.
• Confirm /handoff token create/read works.
• Confirm logs show requests reaching ejamapi-dev.

Separate @ejanalysis Cloudflare setup

• Create public apidev.ejanalysis.com route using a Worker or Origin Rule.
• Point it at the actual Cloud Run dev URL.
• Bypass Cloudflare cache for all apidev.ejanalysis.com/* requests.
• Manually validate https://apidev.ejanalysis.com.
• Compare headers and behavior with https://api.ejanalysis.com.

Follow-up after the manual dev flow works

• Add automated dev-vs-prod smoke tests.
• Add a production promotion workflow that deploys an already-tested image digest.
• Export current production Cloud Run settings and mirror the important resource settings in dev.

21. Files to Add to EJAM-API

Minimum:

rest_controller.r
.github/workflows/deploy-dev-cloud-run.yaml

Recommended:

docs/deployment/dev-cloud-run.md

Later:

scripts/smoke-test-api.sh

for automated smoke tests after the manual dev flow works.

22. Remaining Decisions

1. Confirm the actual Google Cloud project ID for project number 84652557241.
2. Decide whether the manual dev workflow may run from feature branches, or only from main.
3. Confirm the production Cloud Run service's current memory, CPU, timeout, concurrency, and min/max instance settings so dev can mirror the important parts.

Recommendation:

• Start public, min-instances=0, uncached for dev, manual workflow only.
• Add /status immediately.
• After the manual dev flow works, add automated smoke tests and image-digest promotion to production.

23. References

• EJAM-API repo: https://github.com/Public-Environmental-Data-Partners/EJAM-API
• Production API: https://api.ejanalysis.com
• Production Cloud Run URL documented in repo: https://ejamapi-84652557241.us-central1.run.app
• Google Cloud Run deploy container docs: https://cloud.google.com/run/docs/deploying
• Google Cloud Run custom domain docs: https://cloud.google.com/run/docs/mapping-custom-domains
• Google Cloud Run traffic rollback docs: https://cloud.google.com/run/docs/rollouts-rollbacks-traffic-migration
• Docker Hub push docs, useful only for comparison/fallback: https://docs.docker.com/docker-hub/repos/manage/hub-images/push/