dotCMS / dotCMS/core

Publish aggregate request-cost (token) telemetry to external endpoint

Open
#35,750 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement stale
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Parent Issue

NA — internal observability enhancement.

Background

com.dotcms.cost.RequestCostApiImpl already aggregates per-request token counts (window + lifetime) and a ScheduledExecutorService ticks every REQUEST_COST_TIME_WINDOW_SECONDS (default 60s) to emit a local log line. These aggregates aren't queryable across a cluster — they only live in the local log.

Proposal

Wire the same aggregates into an external time-series sink so per-cluster, per-environment token consumption can be tracked centrally.

  • Reuse the existing monitor tick — no new scheduler.
  • Dispatch the HTTP POST through DotConcurrentFactory.getInstance().getSubmitter() so a slow or dead collector never blocks the monitor thread.
  • Opt-in. Activate implicitly when both REQUEST_COST_PUSH_URL and REQUEST_COST_PUSH_TOKEN are configured.
  • Drop-on-failure (observational telemetry, not durable accounting). Rate-limited warn logs.

Payload

10 fields, exact:

{
  "clusterId": "...",
  "environmentId": "...",
  "timestamp": "2026-05-19T18:48:00Z",
  "windowSeconds": 60,
  "windowRequests": 1234,
  "windowTokens": 5678.5,
  "windowAvgTokensPerRequest": 4.6,
  "lifetimeRequests": 999999,
  "lifetimeTokens": 12345678.25,
  "lifetimeAvgTokensPerRequest": 12.35
}
  • clusterIdClusterFactory.getClusterId()
  • environmentIdConfigUtils.getServerId()
  • Token values pre-divided by REQUEST_COST_DENOMINATOR.

Config

Key Default Notes
REQUEST_COST_PUSH_URL URL + token activates the publisher
REQUEST_COST_PUSH_TOKEN Bearer token
REQUEST_COST_PUSH_TIMEOUT_MS 5000 Per-request timeout

Acceptance criteria

  • Publisher activates only when both URL and token are set.
  • HTTP POST never blocks the monitor scheduler.
  • Transport failures are rate-limited warns, snapshot dropped.
  • Unit tests lock the JSON shape (exactly 10 fields) and the enable gate.
  • Manual smoke against a request bin shows a snapshot every window tick.

Implementation

See PR #35749.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with com.dotcms.cost.RequestCostApiImpl and PR #35749, then inspect the existing monitor tick and the mentioned DotConcurrentFactory, ClusterFactory, and ConfigUtils entry points. Verify the completed acceptance criteria, especially the exact 10-field JSON shape and enable gate; the remaining stated work is a manual smoke test showing one snapshot per window tick.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.