dotCMS / dotCMS/core

Introduce is_human_interaction Dimension Across MVs/RMVs and Dashboards (Phased Rollout)

Open
#34,698 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : Analytics Priority : 3 Average Team : Falcon
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Description

Recent testing confirmed that crawlers and bots are generating events (pageviews and content impressions), which are currently included in our dashboards. This inflates metrics and impacts data accuracy.

We need to introduce a consistent mechanism to identify and filter human interaction across our analytics pipeline.

Scope of this task:

  • Add a new is_human_interaction column to the relevant MV or RMV tables.
  • Update MV/RMV logic to populate this column using a unified detection approach.
  • Update Cubes to expose is_human_interaction as a dimension.
  • Ensure UI dashboards only display metrics where is_human_interaction = true.
  • Define and execute a safe refresh/backfill plan for existing data.
  • Implement changes incrementally:
    • Phase 1: Pageviews
    • Phase 2: Conversions
    • Phase 3: Engagement
  • Reuse shared logic (functions, views, or reusable SQL patterns) so that bot detection rules are defined in a single place and not duplicated across MVs/RMVs.

The solution must avoid regressions and maintain compatibility with current dashboards and replicated/non-replicated environments.

Acceptance Criteria
  • A new is_human_interaction column is added to the relevant MV/RMV tables.
  • A single reusable logic definition (e.g., SQL function, view, or shared expression) is created for bot/human classification. If possible
  • All updated MVs/RMVs reference the shared classification logic (no duplicated logic across tables). If possible
  • Cubes are updated to include is_human_interaction as a dimension.
  • A documented migration plan exists to refresh/backfill MV/RMV data safely.
  • MV/RMV refresh is executed without data loss.
  • No regression is introduced in existing dashboards.
  • Changes are implemented incrementally (Pageviews → Conversions → Engagement).
  • Performance impact is validated and remains stable or improved.
Priority

High

Additional Context

https://gist.github.com/erickgonzalez/0dd76b1c0c37112834978857d0c45db2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked gist and tracing the Pageviews MV/RMV tables, Cubes, and dashboard paths before extending the work to Conversions and Engagement. Done means the phased data changes, shared classification logic, safe refresh/backfill plan, performance validation, and dashboard compatibility are documented and verified without data loss.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
analytics, backend, data, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.