PostHog / PostHog/posthog

Max authors SQLV2 notebook cells for users without revamped-py-notebooks, so Run always 404s

Open
#87,654 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug feature/notebooks feature/posthog-ai team/self-driving
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Problem

Max's create_notebook tool unconditionally authors <SQLV2 /> / <PythonV2 /> cells (ee/hogai/tools/create_notebook/tool.py, same guidance in ee/hogai/context/context.py), with no is_sql_v2_enabled check. SQLV2 execution is gated behind the internal-only revamped-py-notebooks flag (0% rollout + posthog.com dogfooding), so for any external user the result is:

  1. Max builds them a notebook full of SQLV2 cells.
  2. The frontend renders those cells for everyone by design — only insertion is flag-gated, rendering of existing nodes is not (frontend/src/scenes/notebooks/Notebook/markdownNotebookRegistry.tsx). The user sees normal-looking SQL cells with Run buttons.
  3. Every Run click hits sql_v2_run, which raises a bare Http404() when the flag is off (products/notebooks/backend/presentation/views/notebook.py:1143). The UI surfaces an unexplained "not found".
  4. The same SQL works in the standalone SQL editor, which has no such gate — making the failure look like notebook corruption rather than gating.

Evidence

A paid customer hit exactly this and filed a support ticket ("why do all these queries in this notebook return not found — it works in the standalone SQL editor"). Their session replay shows Max creating the notebook via create_notebook, a $feature_flag_called event evaluating revamped-py-notebooks to false, and then 22× POST /api/projects/:id/notebooks/:short_id/sql_v2/run/ → 404 over ~45 minutes before they gave up and reproduced the query in the SQL editor.

Internal links (PostHog staff): support ticket 68438 · inbox report

Proposed fixes (in priority order)

  • (a) Root cause — Max should not author SQLV2/PythonV2 cells for users without the flag. Fall back to legacy <Query /> HogQL cells (which run ungated), or condition the notebook-authoring prompt/context on is_sql_v2_enabled.
  • (b) Frontend mitigation — when revamped-py-notebooks is off, disable Run on rendered SQLV2 nodes (or offer one-click conversion to a standard SQL cell) instead of a Run button guaranteed to 404.
  • (c) Backend — distinguish the flag-gate 404 from a genuine missing-notebook 404 (e.g. a distinct error code), so the frontend can explain correctly.

Relation to #86890

#86890 improves not-found handling for notebook cells, but after its review fix it renders every SQLV2 run-dispatch 404 as "This notebook could not be found. It may have been deleted." For flag-gated users — likely the most common source of that 404, per the above — the notebook visibly exists, so that message is confidently wrong. (c) should land with or before that message change.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ee/hogai/tools/create_notebook/tool.py and ee/hogai/context/context.py, then trace the flag gate in products/notebooks/backend/presentation/views/notebook.py and rendering in frontend/src/scenes/notebooks/Notebook/markdownNotebookRegistry.tsx. Compare flag-on and flag-off behavior and determine which proposed mitigation is appropriate; done means external users no longer receive runnable cells that predictably return 404, without breaking enabled users.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
api, backend, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.