refactor(telemetry): Use {feature}_{name} composite key as internal metric identifier

Summary

Introduce getQualifiedName() on MetricType returning {feature}_{name} (e.g., CONTENTLETS_COUNT) as the internal unique identifier for metrics. Keep the external wire format unchanged (separate name and feature fields in JSON). Revert the metric name change that broke downstream reporting.

This fixes the contentlets_count reporting gap (#35002) and properly resolves the metric naming ambiguity (#34042).

Root Cause — Confirmed from Production Database

The reporting layer derives keys as {feature}_{name}

The telemetry database (dotcms-metrics RDS) stores raw JSON snapshots. Two PostgreSQL views (metrics_table_view and time_series_all_metrics_view) pivot this JSON into columns using:

concat('"', lower(ms.metric ->> 'feature'), '_', lower(ms.metric ->> 'name'), '":', ms.metric ->> 'value')

This is mapped into a hardcoded PostgreSQL composite type metric_stat via json_populate_record(NULL::metric_stat, ...). The type has 58 fixed columns including contentlets_count.

What broke

PR #34479 renamed TotalContentsDatabaseMetricType.getName() from "COUNT" to "COUNT_CONTENT". The derived key changed from contentlets_count → contentlets_count_content. Since contentlets_count_content has no matching column in metric_stat, json_populate_record silently drops the value. The contentlets_count column returns NULL for all 145 clients running the new code.

Timeline — two separate causes, confirmed from production data

Date	Event	Cause
~Jan 7-8	contentlets_count goes NULL in reporting views for all clients	Name change deployed — COUNT → COUNT_CONTENT means derived key becomes contentlets_count_content, which has no matching column in metric_stat. Silently dropped by json_populate_record.
~Feb 5	All telemetry stops — zero rows from all affected clients	Connection leak (#34926) — FEATURE_FLAG_TELEMETRY_CORE_ENABLED=false deployed as workaround for ThreadLocal connection poisoning
~Feb 19	Telemetry resumes, contentlets_count still NULL	Connection leak fixed, telemetry re-enabled — but renamed metric still doesn't match metric_stat

Net result: contentlets_count has been NULL since ~Jan 7 for 145 of 153 clients (~10 weeks of missing data). 8 clients on older versions still report correctly.

Fix — Revert metric name, introduce getQualifiedName() internally

1. Revert the wire format (immediate fix for #35002)

Revert TotalContentsDatabaseMetricType.getName() from "COUNT_CONTENT" back to "COUNT". This restores the derived key contentlets_count that the downstream metric_stat type expects.

2. Add getQualifiedName() for internal uniqueness

Add a default method to MetricType:

default String getQualifiedName() {
    return getFeature().name() + "_" + getName();
}

Use getQualifiedName() as the unique key in all internal consumers (cache, map keys, name filters, i18n). This eliminates the naming collision between TotalContentsDatabaseMetricType and TotalLanguagesDatabaseMetricType (both return "COUNT") without changing the wire format.

3. Remove the UsageResource hack

Remove the getMappedMetricName() workaround in UsageResource.java (lines 310-326) and the duplicate disambiguation logic (lines 189-215). The getQualifiedName() method replaces this entirely.

Changes Required

File	Change
MetricType.java	Add default getQualifiedName() method
TotalContentsDatabaseMetricType.java	Revert getName() from "COUNT_CONTENT" back to "COUNT"
MetricStatsCollector.java	Use getQualifiedName() for name filter matching and cache key
MetricCacheManager.java	Document that cache keys are qualified names
UsageResource.java	Remove getMappedMetricName() hack; use getQualifiedName() for map keys
DashboardMetricsProvider.java	getDashboardMetricByName() should match on qualified name
MetricTiming.java	Use qualified name for timing records
Language.properties (all locales)	Update i18n keys to use qualified names (e.g., usage.metric.CONTENTLETS_COUNT.label)

What does NOT change

• The 106 concrete MetricType classes (except reverting one)
• The external telemetry wire format — Metric serializes separate name/feature/category/description fields
• The MetricValue @JsonUnwrapped serialization
• The metric_stat composite type or reporting views (they already use the {feature}_{name} convention)

---

Reporting Layer Audit: metric_stat Column Mapping

The metric_stat PostgreSQL composite type defines which metrics are available to downstream reporting. Any metric not listed here is silently dropped by json_populate_record. This type is a static contract that must be updated manually when metrics are added or removed in core.

Currently mapped correctly (48 metrics — receiving data)

metric_stat column	Core class	Status
contentlets_contents_recently_edited	RecentlyEditedContentDatabaseMetricType	Working
contentlets_count	TotalContentsDatabaseMetricType	BROKEN — sends COUNT_CONTENT since ~Jan 7. Fix: revert to COUNT
contentlets_live_not_default_language_count	LiveNotDefaultLanguageContentsDatabaseMetricType	Working
contentlets_working_not_default_language_count	WorkingNotDefaultLanguageContentsDatabaseMetricType	Working
languages_count	TotalLanguagesDatabaseMetricType	Working
languages_is_default_language_not_english	HasChangeDefaultLanguagesDatabaseMetricType	Working
languages_live_language_variable_count	TotalLiveLanguagesVariablesDatabaseMetricType	Working
languages_old_style_languages_variable_count	OldStyleLanguagesVarialeMetricType	Working
languages_unique_count	TotalUniqueLanguagesDatabaseMetricType	Working
languages_working_language_variable_count	TotalWorkingLanguagesVariablesDatabaseMetricType	Working
layout_count_file_containers_used_in_live_pages	TotalFileContainersInLivePageDatabaseMetricType	Working
layout_count_file_containers_used_in_live_templates	TotalFileContainersInLiveTemplatesDatabaseMetricType	Working
layout_count_file_containers_used_in_working_pages	TotalFileContainersInWorkingPageDatabaseMetricType	Working
layout_count_of_advanced_templates	TotalAdvancedTemplatesDatabaseMetricType	Working
layout_count_of_live_containers	TotalLiveContainerDatabaseMetricType	Working
layout_count_of_template_builder_templates	TotalBuilderTemplatesDatabaseMetricType	Working
layout_count_of_templates	TotalTemplatesDatabaseMetricType	Working
layout_count_of_templates_used_in_live_pages	TotalTemplatesInLivePagesDatabaseMetricType	Working
layout_count_of_templates_used_in_working_pages	TotalTemplatesInWorkingPagesDatabaseMetricType	Working
layout_count_of_working_containers	TotalWorkingContainerDatabaseMetricType	Working
layout_count_standard_containers_used_in_live_pages	TotalStandardContainersInLivePageDatabaseMetricType	Working
layout_count_standard_containers_used_in_live_templates	TotalStandardContainersInLiveTemplatesDatabaseMetricType	Working
layout_count_standard_containers_used_in_working_pages	TotalStandardContainersInWorkingPageDatabaseMetricType	Working
layout_total_files_in_themes	TotalFilesInThemeMetricType	Working
layout_total_live_files_in_themes	TotalLiveFilesInThemeMetricType	Working
layout_total_themes	TotalThemeMetricType	Working
layout_total_used_themes_in_live_templates	TotalThemeUsedInLiveTemplatesMetricType	Working
layout_total_used_themes_in_working_templates	TotalThemeUsedInWorkingTemplatesMetricType	Working
sites_aliases_active_sites_count	TotalAliasesActiveSitesDatabaseMetricType	Working
sites_aliases_sites_count	TotalAliasesAllSitesDatabaseMetricType	Working
sites_count_of_active_sites	TotalActiveSitesDatabaseMetricType	Working
sites_count_of_sites	TotalSitesDatabaseMetricType	Working
sites_non_system_fields_on_content_type_count	SitesWithNoSystemFieldsDatabaseMetricType	Working
sites_sites_non_default_tag_storage_count	SitesWithNoDefaultTagStorageDatabaseMetricType	Working
sites_sites_run_dashboard_true_count	SitesWithRunDashboardDatabaseMetricType	Working
sites_sites_with_individual_permissions	CountOfSitesWithIndividualPermissionsMetricType	Working
sites_sites_with_thumbnail_count	CountOfSitesWithThumbnailsMetricType	Working
site_search_indices_count	CountSiteSearchIndicesMetricType	Working
site_search_indices_document_count	CountSiteSearchDocumentMetricType	Working
site_search_total_indices_size	TotalSizeSiteSearchIndicesMetricType	Working
url_maps_content_types_with_url_map	ContentTypesWithUrlMapDatabaseMetricType	Working
url_maps_count_url_map_patter_with_more_that_one_variable	UrlMapPatterWithTwoVariablesDatabaseMetricType	Working
url_maps_live_contentlets_in_content_types_with_url_map	LiveContentInUrlMapDatabaseMetricType	Working
url_maps_working_contentlets_in_content_types_with_url_map	WorkingContentInUrlMapDatabaseMetricType	Working
users_active_users_count	ActiveUsersDatabaseMetricType	Working
workflow_actions_count	ActionsDatabaseMetricType	Working
workflow_content_types_assigned	ContentTypesDatabaseMetricType	Working
workflow_schemes_count	SchemesDatabaseMetricType	Working
workflow_steps_count	StepsDatabaseMetricType	Working
workflow_subactions_count	SubActionsDatabaseMetricType	Working
workflow_unique_subactions_count	UniqueSubActionsDatabaseMetricType	Working

Dead columns in metric_stat (7 columns — no data source)

metric_stat column	Status	Notes
image_api_count_of_be_contentasset_calls	Permanently dead	API request counting was never a working solution for these types of metrics. The system relied on a web interceptor approach that was architecturally flawed — it attempted to count API calls via request interception but never produced reliable data in production. Removed in PR #34111.
image_api_count_of_be_da_calls	Permanently dead	Same — never functional, permanently removed
image_api_count_of_fe_contentasset_calls	Permanently dead	Same — never functional, permanently removed
image_api_count_of_fe_da_calls	Permanently dead	Same — never functional, permanently removed
layout_reuse_count_of_advanced_templates	Dead	No matching metric in core — reporting-side addition never backed by a collector
layout_reuse_count_of_templates	Dead	Same — no matching metric in core
site_search_rate_of_search	Dead	No matching metric in core

Metrics collected by core but SILENTLY DROPPED by metric_stat (48 metrics — data lost)

These metrics are collected, sent to the telemetry endpoint, and stored in the snapshot JSON — but the reporting views discard them because metric_stat has no matching column.

AI (3 metrics)

• ai_total_embeddings_indexes — TotalEmbeddingsIndexesMetricType
• ai_total_sites_using_dotai — TotalSitesUsingDotaiMetricType
• ai_total_sites_with_auto_index_content_config — TotalSitesWithAutoIndexContentConfigMetricType

Content Types (1 metric)

• content_types_count_of_content_types — CountOfContentTypesMetricType

Content Type Fields (30 metrics)

• All content_type_fields_count_* metrics (binary, block editor, category, checkbox, column, constant, date, date_time, file, hidden, image, json, key_value, line_divider, multi_select, permissions, radio, relationship, row, select, site_or_folder, tab_divider, tag, text_area, text, time, wysiwyg fields)

Contentlets (2 metrics)

• contentlets_import_contentlets_job_triggered — ImportContentletsJobTriggeredMetricType
• contentlets_last_content_edited — LastContentEditedDatabaseMetricType

Experiments (15 metrics)

• All experiments_count_* metrics (edited in past 30 days, bounce/exit/reach/url parameter goals, pages with archived/draft/ended/running/scheduled experiments, variants in archived/scheduled/draft/ended/running experiments)

Sites (2 metrics)

• sites_with_live_site_variables — CountOfLiveSitesWithSiteVariablesMetricType
• sites_with_working_site_variables — CountOfWorkingSitesWithSiteVariablesMetricType

Layout (1 metric)

• layout_total_size_of_files_per_theme — TotalSizeOfFilesPerThemeMetricType

Users (3 metrics)

• users_count_of_users — TotalUsersDatabaseMetricType
• users_last_login — LastLoginDatabaseMetricType
• users_last_login_user — LastLoginUserDatabaseMetric

---

Additional Infrastructure Issues Found

Missing database indexes

The metric table (319K rows) is missing 3 of 4 indexes defined in the Flyway migrations:

Index	Status
idx_metric_client_name	Missing — was on old table (now renamed to metric_random)
idx_metric_client_env	Missing — was on old table
idx_metric_insert_date	Missing — was on old table
idx_metric_client_category	Present

The old metric table was renamed to metric_random (1.8M rows, has all indexes) and a new metric table was created without reapplying the indexes. Every query against client_name or insert_date does a full table scan.

Fragile metric_stat composite type

The metric_stat PostgreSQL type is a static contract that must be manually updated via DDL whenever metrics are added or removed in core. There is no automation or validation connecting the two. This is the root architectural fragility — core developers can add metrics freely, but they silently go nowhere unless someone also updates metric_stat, the views, and any downstream dashboards.

Acceptance Criteria

• TotalContentsDatabaseMetricType.getName() returns "COUNT" (reverted)
• External telemetry payload sends "name": "COUNT" with "feature": "CONTENTLETS" (restoring contentlets_count in reporting views)
• getQualifiedName() returns {FEATURE}_{NAME} for all metrics
• All internal consumers (cache, maps, filters, i18n) use getQualifiedName() as the unique key
• UsageResource disambiguation hack removed
• Usage Dashboard displays metrics correctly
• No duplicate qualified names exist (verified by test)

Closes

• Closes #34042
• Fixes #35002 (restores external contentlets_count key)

Related

• #34926 — ThreadLocal connection poisoning (caused the ~2 week telemetry blackout when feature flag was disabled as workaround)
• #34837 — Parent epic: Database Connection and Transaction Management
• #34479 — PR that introduced the breaking rename

Start with MetricType.java and compare the listed consumers: MetricStatsCollector.java, MetricCacheManager.java, UsageResource.java, DashboardMetricsProvider.java, and MetricTiming.java. Check the existing metric-name handling and all locale Language.properties files, then verify that internal keys use qualified names while the external name and feature fields remain unchanged and TotalContentsDatabaseMetricType reports COUNT.