PostHog / PostHog/posthog

`$identify` property is lost when a later event in the same batch re-reads the person

Open
#95,302 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug feature/ingestion
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Summary

A property sent with $identify never reaches the person profile if a second event for the same person lands in the same ingestion batch through a distinct ID that is not cached yet. The usual shape is $identify with a $set, then $create_alias for a distinct ID that already resolves to the same person.

The pipeline accepts the event and raises no error. The profile keeps its old value. Clients resend the same value on every $identify, so the property never converges, and the customer sees the old value for as long as they keep sending.

Where

BatchWritingPersonsStore in nodejs/src/ingestion/common/persons/batch-writing-person-store.ts.

The store caches person state per batch, keyed by person ID, with a distinct-ID to person-ID map in front. A distinct ID missing from that map sends the store back to Postgres, and the store folds that row on top of the writes the batch has already queued. The database value overwrites the queued one. The rest of the batch then reads the stale value, and the flush writes it back.

Impact

  • Person properties revert for any customer whose traffic produces a second read of the same person inside one batch. $identify followed by $create_alias is ordinary browser SDK traffic.
  • The flush writes the person's whole property blob, so a stale cached value can also revert a key this batch never touched, written by another pod.
  • The stale value goes into the person Kafka message as well as the Postgres row.
  • Nothing signals it. No warning, no retry, and no current metric separates a write that carried the queued value from one that carried a stale one.

Reproduction

Unit level, in nodejs/src/ingestion/common/persons/batch-writing-person-store.test.ts, with a repository mock that returns the same person for several distinct IDs:

  1. Read the person for update through distinct ID A. Stored properties include plan: 'free'.
  2. Queue plan: 'pro' on that person through A.
  3. Read the person for update through distinct ID B, which resolves to the same person but is not cached, so the store re-reads Postgres.
  4. The returned person has plan: 'free'.
  5. Flush. The store hands the repository a payload carrying 'free'.

End to end, matching the ticket:

  1. The profile has plan: 'free'. Send $identify for distinct_id = X with $set: {plan: 'pro'}.
  2. Send $create_alias for distinct_id = X, alias = Y, where Y already resolves to the same person, so the merge is a no-op.
  3. Both events have to land in the same batch. Sending them back to back from one client is enough.
  4. Read the profile. Still plan: 'free', and repeating the pair never changes it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with nodejs/src/ingestion/common/persons/batch-writing-person-store.ts and its batch-writing-person-store.test.ts. Run the existing unit tests, then reproduce the sequence using distinct IDs A and B to trace how the second repository read interacts with queued writes. Done means the regression test shows that flushing preserves the queued property and the resulting person and Kafka message retain the updated value.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, node.js, postgresql, typescript
Domain
backend, data-engineering, databases, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.