PostHog / PostHog/posthog

GitHub warehouse source: deployment_statuses can never sync (PostHog GitHub App lacks deployments:read)

Open
#76,104 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Summary

The deployment_statuses schema on the GitHub data warehouse source fails on every run with 403 Resource not accessible by integration, permanently pauses itself, and generates a recurring "Data warehouse syncs paused" alert email that the user has no way to action.

The cause is not user configuration. The PostHog GitHub App does not declare the deployments permission, so no installer can grant it.

Evidence

Raw sync log (ERROR level), project 388173:

Github API error: status=403,
body={"message":"Resource not accessible by integration",
      "documentation_url":"https://docs.github.com/rest/deployments/deployments#list-deployments",
      "status":"403"},
url=https://api.github.com/repos/<owner>/<repo>/deployments?per_page=100

GET /apps/posthog returns these declared permissions:

actions, checks, contents, emails, issues, members,
metadata, pull_requests, repository_hooks, statuses, workflows

There is no deployments entry. The installed permission set on our org matches the declared set exactly, so there is no pending permission-approval request for an admin to accept.

GET /repos/{owner}/{repo}/deployments requires the Deployments repository permission (read) for a GitHub App, so this call returns 403 for every installation.

Why the webhook doesn't save it

From the DEBUG logs, each run does two things in sequence:

  1. Drains the webhook parquet buffer. This succeeds: Webhook source reading 9 files then write_to_deltalake: mode = overwrite then Updating rows_synced with +9.
  2. Runs the reconciliation poll, which 403s and retries 4 times before giving up.

Step 2 appears unavoidable by design. deployment_statuses has no repo-wide list endpoint, so reconciling fans out over /deployments first, and per test_webhook_enabled_deployment_statuses_reconciles_inactive_statuses the poll exists because GitHub fires no deployment_status webhook for the inactive transition.

Because the job fails at step 2, the table is never registered: the schema ends with table: null and last_synced_at: null despite rows_synced: 9. The webhook rows are written and then effectively discarded.

By contrast the sibling deployments schema is genuinely webhook-only, makes zero HTTP calls to the GitHub API, and succeeds. That is what makes the failure look inconsistent from the UI: two deployment tables, same source, one green and one permanently red.

The state is unrecoverable from the user side

Once auto-paused, status: Failed and latest_error persist. Neither PATCH should_sync=false nor DELETE .../delete-data clears them, and only a successful sync would, which cannot happen. The alert email therefore repeats indefinitely with no available user action.

Suggested fixes (any one)

  1. Add deployments: read to the GitHub App's requested permissions. Existing installations would surface a normal permission-approval request.
  2. Make deployment_statuses truly webhook-only and drop the reconcile, accepting that the inactive transition is not captured.
  3. Detect the missing permission at schema-discovery time and ship deployment_statuses deselected with an explanatory note, rather than letting it fail and alert.

Separately, the user-facing error string "Access forbidden. Your token may lack required permissions or have hit rate limits." conflates a permanent permission failure with a transient rate limit. That sent us looking at rate limits and at re-granting repo access, neither of which was relevant. Surfacing GitHub's actual message would have made this self-diagnosing.

Secondary bug

PATCH /api/projects/:id/external_data_schemas/:schema_id/ with {"sync_frequency": "never"} on a sync_type: webhook schema returns HTTP 500:

{"type":"server_error","code":"error","detail":"A server error occurred.","attr":null}

{"should_sync": false} on the same schema succeeds. never is an advertised enum value in the API schema, so this should either work or return a 400.

Environment

  • PostHog Cloud US, project 388173
  • Source type: Github, api_version 2026-03-10
  • Failing schema id: 019fac79-b1f9-0000-686f-24a13669ff08
  • Observed 2026-07-29 and 2026-07-31

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the GitHub warehouse source's deployment_statuses reconciliation and the test_webhook_enabled_deployment_statuses_reconciles_inactive_statuses test. Check the GitHub App permission configuration and the PATCH sync_frequency handling described in the issue. Done means selecting and implementing a defined fix, covering the affected behavior with tests, and preventing the failing schema from repeatedly pausing and alerting.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, python
Domain
api, backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.