oppia / oppia/analytics-data-platform

Use Workload Identity Federation for GCP auth

Open
#2 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

This comment came up as part of a Copilot review in https://github.com/oppia/analytics-data-platform/pull/1#discussion_r3790752232:

This workflow relies on a long-lived service account JSON key and assumes gcloud is available on the runner. Prefer GitHub’s recommended GCP auth approach (Workload Identity Federation via google-github-actions/auth) and, if gcloud is needed, install/configure it explicitly (e.g., google-github-actions/setup-gcloud). This reduces key-leak risk and avoids failures on runners without gcloud.

If we go this route, this would need to be done for all our github workflows (e.g. pr_validation and weekly_run). It requires setting up a Workload Identity Pool/Provider in GCP and adding two new repo secrets, GCP_WORKLOAD_IDENTITY_PROVIDER and GCP_SERVICE_ACCOUNT_EMAIL, then removing the old GCP_SERVICE_ACCOUNT_KEY secret once the integration is confirmed working.

Sample patch file with expected changes:

--- a/.github/workflows/weekly_run.yml
+++ b/.github/workflows/weekly_run.yml
@@ -9,7 +9,7 @@ name: Weekly Production Analytics Run
 #
 # Environment Target: Production (oppia-analytics-prod)
 # Key Operations:
-#   1. Authenticates to GCP using an automated service account.
+#   1. Authenticates to GCP via Workload Identity Federation (no long-lived keys).
 #   2. Dynamically isolates database credentials to prevent local Git leaks.
 #   3. Runs data quality assertions and transforms fresh production analytics logs.
 # ==============================================================================
@@ -25,6 +25,9 @@ jobs:
   dbt_run:
     name: Execute dbt Production Pipeline
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write # required to mint the OIDC token for Workload Identity Federation
 
     steps:
       # Clone the repository codebase onto the temporary GitHub virtual runner
@@ -43,21 +46,14 @@ jobs:
           pip install --upgrade pip
           pip install dbt-bigquery
 
-      # Exchange GitHub secret tokens for active, authenticated Google Cloud sessions
-      - name: Authenticate to GCP Natively
-        run: |
-          # Write the secret JSON token safely as a raw literal string
-          cat << 'EOF' > ${HOME}/gcp_key.json
-          ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
-          EOF
-
-          # Activate the service account using the pre-installed gcloud CLI
-          gcloud auth activate-service-account --key-file=${HOME}/gcp_key.json
-
-          # Set the application default credentials environment variable for dbt
-          echo "GOOGLE_APPLICATION_CREDENTIALS=${HOME}/gcp_key.json" >> $GITHUB_ENV
+      # Keyless auth: exchanges a GitHub OIDC token for short-lived GCP credentials
+      - name: Authenticate to GCP via Workload Identity Federation
+        uses: google-github-actions/auth@v2
+        with:
+          workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
+          service_account: ${{ secrets.GCP_SERVICE_ACCOUNT_EMAIL }}
 
-      # Generate a production connection profile using service-account credentials
+      # Generate a production connection profile using the ADC credentials from the auth step
       - name: Create dynamic profiles.yml for dbt
         run: |
           cat << EOF > profiles.yml
@@ -66,8 +62,7 @@ jobs:
             outputs:
               prod:
                 type: bigquery
-                method: service-account
-                keyfile: ${{ env.GOOGLE_APPLICATION_CREDENTIALS }}
+                method: oauth
                 project: oppia-analytics-prod
                 dataset: analytics_production
                 threads: 4

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review .github/workflows/weekly_run.yml and the pr_validation workflow first, then confirm the GCP Workload Identity Pool/Provider and the two listed repository secrets. Update all GitHub workflows to use short-lived federation credentials, configure gcloud explicitly only where needed, and remove GCP_SERVICE_ACCOUNT_KEY after the integration is confirmed working.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, github-actions
Domain
ci-cd, cloud, security
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.