Event ingestion filtering: keep only the first event per person per day, gated by a feature flag

Feature request

Problem

A customer runs a distributed architecture. Many nodes send one keepalive event for the same person, dozens of times per person per day. That event is the majority of their event volume. They use the first event per person per day for analysis.

• Their nodes share no state. No node knows if another node already sent the event today.
• No customer-facing tool at ingestion time holds state.

What already exists

nodejs/src/ingestion/common/feature-flag-called-dedup/ dedupes $feature_flag_called per person against Redis. Phil Haack shipped it in #62793.

It has disabled / shadow / drop modes, a team allowlist, an excludedTeams escape hatch, a TTL guard, a JSON-hashed claim key, a local-first check, and Redis metrics.

What is missing

Gap	Today	Needed
Key identity	(team_id, distinct_id, flag_key, response, groups, has_experiment)	(team_id, event_name, distinct_id, day)
Day scope	TTL only	The day inside the key
Configuration	Environment variable	The event filter tree
Rollout control	None	A customer-owned feature flag

Why the other tools do not work

Tool	Reason
Event ingestion filtering	event_name and distinct_id only, equals and contains only, no state
Drop events transformation	No state. Runs after filters.
Downsampling (#93939)	Fixed percentage. It keeps the wrong events.
Capture overflow limiter	Reroutes above a threshold. It does not drop. The window is 1 minute.

A workaround works today

Four code paths dedupe on the same identity, (day, event, distinct_id, uuid):

Path	Expression
get_teams_with_billable_event_count_in_period	distinct toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid)
get_teams_with_billable_enhanced_persons_event_count_in_period	The same expression
analyticsRecordId in nodejs/src/ingestion/common/steps/usage-records-steps.ts	day plus sha256([event, distinctId, eventUuid])
The events ReplacingMergeTree	ORDER BY (team_id, toDate(timestamp), event, cityHash64(distinct_id), cityHash64(uuid))

So the customer derives the UUID from the person and the UTC day. Every node computes the same UUID, and no node needs shared state. posthog-node accepts uuid on the capture message. prepare-event-step.ts passes it through unchanged.

posthog.capture({ distinctId, event: 'keepalive', uuid: uuidv5(`${distinctId}:${utcDate}`, NS) })

Questions

1. Do we support that identity, or must a customer not rely on it? If we support it, then one doc page closes this issue.
2. get_teams_with_event_count_with_groups_in_period uses count(1), not the distinct expression. The group analytics add-on therefore bills every duplicate copy. Is that intended?
3. The events engine version column is _timestamp. The surviving row is the last copy of the day, not the first.

Proposal, if the answer to question 1 is no

Add a seen_today node to the filter tree in nodejs/src/ingestion/common/event-filters/. Call the existing dedup service with a second key builder. Gate the rule on a customer-owned feature flag, which gives the customer a rollout ramp and a kill switch.

• Key: hash a JSON-encoded (event_name, distinct_id, day) tuple. Do not join the values with a colon. Both fields accept any character, so a:b with c and a with b:c collide and drop a first event.
• Day: the UTC date of the event timestamp, which matches toDate(timestamp).
• Cost: unique persons × filtered events keys per day. One million persons on one event is about 100 MB.
• Failure: a Redis error makes the node evaluate false, and PostHog ingests the event. Fail open.
• Verify: total event count falls, and unique persons per day stays flat.
• Dropped events do not count toward billing.

nodejs/src/ingestion/** has no CODEOWNERS entry, so a PR there requests no reviewer. #91119 and #93845 touch the same filter step, model and API.

Debug info

• PostHog Cloud