PostHog / PostHog/posthog

HogQL rejects a WITH alias that references a column of the queried table

Open
#99,992 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
7h 27m
Merged PRs (30d)
222

Description

Bug description

ClickHouse accepts a scalar WITH alias whose expression references columns of the table in
FROM:

WITH toStartOfDay(timestamp) AS day
SELECT day, count()
FROM events
GROUP BY day

HogQL raises No scope or CTE available instead, because the WITH clause is resolved before
the query scope exists.

Mechanism

In posthog/hogql/resolver.py, visit_select_query begins at line 881:

  • line 897 — "First step: resolve all the WITH CTEs onto self.ctes"
  • line 923 — node.select_from is first visited

So a WITH expression is resolved 26 lines before anything knows what FROM contains. On a
top-level query there is neither an enclosing scope nor a CTE to fall back to, and the lookup
reaches posthog/hogql/resolver.py:2697:

raise QueryError("No scope or CTE available")
Suggested direction

Split the two kinds of WITH entry rather than resolving them in one pass:

  • subquery CTEs must resolve first, because FROM may reference them
  • expression CTEs can resolve after select_from, where a scope exists

The Rust parser should be checked for the same ordering, so the two do not diverge.

Notes

Traced in the code on master today. Not executed — a resolver-level repro needs a database
context, which needs a running Postgres.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in posthog/hogql/resolver.py at visit_select_query around lines 881-923 and inspect the lookup at line 2697. Reproduce the scalar WITH alias against a table-backed query with a running Postgres database, then verify subquery CTEs still resolve before FROM while expression CTEs resolve after select_from. Check the Rust parser for matching ordering and confirm both implementations behave consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.