Codeinwp / Codeinwp/otter-blocks

Telemetry: Custom CSS is over-tracked (per-keystroke, ~78% of all events) — replace with bucketed depth + error signal

Open
#2,867 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
198
Forks
36
Avg merge
3d 4h
Merged PRs (30d)
30

Description

Summary

Otter over-tracks Custom CSS: the editor fires a telemetry event on every keystroke, and the event carries no payload. In the latest 3-month usage window this single event was 5,194 of 6,658 total events (78%) — it dominates the dataset, drowns out every high-intent feature (Conditions, Forms, AI) in the raw feed, and yet tells us nothing actionable.

This proposes replacing the per-keystroke counter with two low-cardinality, non-PII signal events emitted on the validation settle that already runs — cutting ~78% of event volume and finally answering whether Custom CSS is a one-off tweak or a power-user layer, and how often it's invalid.

Current behavior

src/css/editor.js (~L108–116):

const onChange = () => {
    window?.oTrk?.add({ feature: 'custom-css', featureComponent: 'used' });
    clearTimeout( inputTimeout );
    inputTimeout = setTimeout( () => { checkInput( editorRef.current ); }, 500 );
};
editorRef.current.on( 'change', onChange );
  • oTrk.add(...) fires on every CodeMirror change (i.e. per keystroke), outside the existing 500ms debounce.
  • The payload is just { feature: 'custom-css', featureComponent: 'used' } — no featureValue, so it answers only "someone typed in the box," at extreme frequency.

Why this is a problem

  • Noise + skew: 78% of all tracked events. The usage dashboard needs a log-scale toggle purely to see past it; week-over-week movement is mostly CSS keystroke jitter, not product signal.
  • Pipeline waste: dozens of batched events per editing session where one would suffice.
  • Zero insight: the event can't distinguish a 2-line tweak from a 200-line stylesheet, can't say which block, and can't say whether the CSS is valid.

Proposed change

  1. Delete the per-keystroke oTrk.add({ feature: 'custom-css', featureComponent: 'used' }) at L109.

  2. Inside the already-debounced checkInput() (runs 500ms after typing settles; it already computes editorErrors and has the CSS value via editor.getValue()), emit two keyed oTrk.set events per block (set, keyed by block clientId, so it dedups to the latest state per block per settle rather than accumulating):

    // size: coarse bucket by CSS volume (e.g. rule/declaration or non-empty-line count)
    window?.oTrk?.set( `custom-css-size-${clientId}`, {
        feature: 'custom-css', featureComponent: 'size-bucket', featureValue: sizeBucket   // '0' | '1-3' | '4-10' | '10+'
    });
    
    // validity: coarse bucket of lint errors (always emit, including '0')
    window?.oTrk?.set( `custom-css-errors-${clientId}`, {
        feature: 'custom-css', featureComponent: 'validation', featureValue: errorBucket   // '0' | '1-2' | '3-5' | '6+'
    });
    

    (Exact bucket edges are open to tuning; the point is bounded, low-cardinality enums.)

Compliance

  • Non-PII, no payload: never send the CSS text itself or any lint-error message — only the bucketed counts above. featureValue stays a closed enum string.
  • No consent change: reuses the existing window.oTrk (tiTrk.with('otter')) accumulator and the existing otter_blocks_logger_flag consent gate. No { consent: true } bypass.

Expected impact

  • ~78% reduction in tracked event volume; the dashboard's other features become legible without log-scale gymnastics.
  • New, decision-relevant signal: is Custom CSS a quick one-off tweak (→ candidate for a native control) or a power-user escape hatch (→ protect it), plus a CSS error rate that feeds the broader durability/quality picture.

References

  • Usage analysis: INSIGHTS.md §1 (the 78% / 5,194-event figure) and the telemetry-expansion roadmap item "A1 — Custom CSS depth + error bucket."
  • Pattern precedent: PR #2862 (block add/remove tracking) and the sibling telemetry PRs #2864 / #2865 / #2866.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/css/editor.js, focusing on the onChange handler and the debounced checkInput flow, then read INSIGHTS.md §1 and the referenced telemetry PRs for precedent. Done means removing the per-keystroke event and emitting keyed size and validation buckets per block while preserving the existing consent gate.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
frontend, observability
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.