PostHog / PostHog/posthog

feat(signals): backfill report embeddings for existing reports

Open
#73,706 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Problem

#73704 added a report-level document to document_embeddings (product='signals', document_type='report'), written by a post_save receiver whenever a report's title or summary changes.

That path is forward-only. Reports already in Postgres never pass through it, and many READY or RESOLVED reports will never have their text change again, so they stay permanently absent from the report feature set.

This matters for the inbox ranking model specifically. The label stream (signal_report_status_changed, emitted from products/signals/backend/receivers.py) already covers historical reports, so without a backfill the features and the labels cover different populations, biasing any training set toward reports created after deploy.

Scope

Emit a report document for existing non-deleted reports that have renderable text, reusing render_report_document and emit_report_embedding from products/signals/backend/report_embeddings.py.

Requirements worth getting right:

  • Pin timestamp to the report's created_at, as the receiver does. The table partitions by toMonday(timestamp) and orders by toDate(timestamp), so a backfill stamped "now" would create a second row per report rather than the intended single one.
  • Respect the safety gate. Skip reports whose latest safety_judgment artefact is choice: false; _is_safety_suppressed in receivers.py is the existing check.
  • Skip deleted reports, and consider whether to emit tombstones for them instead.
  • Batch and rate-limit. Each document costs one embedding per configured model, currently two, so a full-fleet backfill is a real load on the embedding worker.
  • Note the table's 3 month TTL runs from timestamp, i.e. from report creation, so reports older than that will expire immediately and are not worth emitting.

A management command under products/signals/backend/management/commands/ fits the existing pattern.

Context

Raised during review of #73704 and named there as intended follow-up scope.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with products/signals/backend/report_embeddings.py and the report signal receiver in products/signals/backend/receivers.py, then inspect existing management commands under products/signals/backend/management/commands/. Implement a batched, rate-limited backfill using the named rendering and emission entry points, covering the safety, deletion, timestamp, and TTL requirements; verify the command handles existing reports without duplicating current receiver behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
backend, databases, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.