`$identify` property is lost when a later event in the same batch re-reads the person
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Summary
A property sent with $identify never reaches the person profile if a second event for the same person lands in the same ingestion batch through a distinct ID that is not cached yet. The usual shape is $identify with a $set, then $create_alias for a distinct ID that already resolves to the same person.
The pipeline accepts the event and raises no error. The profile keeps its old value. Clients resend the same value on every $identify, so the property never converges, and the customer sees the old value for as long as they keep sending.
Where
BatchWritingPersonsStore in nodejs/src/ingestion/common/persons/batch-writing-person-store.ts.
The store caches person state per batch, keyed by person ID, with a distinct-ID to person-ID map in front. A distinct ID missing from that map sends the store back to Postgres, and the store folds that row on top of the writes the batch has already queued. The database value overwrites the queued one. The rest of the batch then reads the stale value, and the flush writes it back.
Impact
- Person properties revert for any customer whose traffic produces a second read of the same person inside one batch.
$identifyfollowed by$create_aliasis ordinary browser SDK traffic. - The flush writes the person's whole property blob, so a stale cached value can also revert a key this batch never touched, written by another pod.
- The stale value goes into the person Kafka message as well as the Postgres row.
- Nothing signals it. No warning, no retry, and no current metric separates a write that carried the queued value from one that carried a stale one.
Reproduction
Unit level, in nodejs/src/ingestion/common/persons/batch-writing-person-store.test.ts, with a repository mock that returns the same person for several distinct IDs:
- Read the person for update through distinct ID
A. Stored properties includeplan: 'free'. - Queue
plan: 'pro'on that person throughA. - Read the person for update through distinct ID
B, which resolves to the same person but is not cached, so the store re-reads Postgres. - The returned person has
plan: 'free'. - Flush. The store hands the repository a payload carrying
'free'.
End to end, matching the ticket:
- The profile has
plan: 'free'. Send$identifyfordistinct_id = Xwith$set: {plan: 'pro'}. - Send
$create_aliasfordistinct_id = X,alias = Y, whereYalready resolves to the same person, so the merge is a no-op. - Both events have to land in the same batch. Sending them back to back from one client is enough.
- Read the profile. Still
plan: 'free', and repeating the pair never changes it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with nodejs/src/ingestion/common/persons/batch-writing-person-store.ts and its batch-writing-person-store.test.ts. Run the existing unit tests, then reproduce the sequence using distinct IDs A and B to trace how the second repository read interacts with queued writes. Done means the regression test shows that flushing preserves the queued property and the resulting person and Kafka message retain the updated value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, node.js, postgresql, typescript
- Domain
- backend, data-engineering, databases, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100