cockroachdb / cockroachdb/cockroach

roachtest: datadog instrumentation across all TeamCity Builds & release branches

Open
#170,750 1 comment 0 reactions 0 assignees View on GitHub
A-testeng-infra C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

### Background
During an investigation (https://cockroachlabs.slack.com/archives/C03SG8QKYRJ/p1778270425439519), it was observed that expected test runner logs in datadog were not present. This issue is to investigate missing logs, and make the roachtest instrumentation consistent across branch versions and cloud provider CI, and to establish next steps.

### Goal
Roachtest operational logs (non test artifacts) are currently instrumented with Datadog for master (Test logs have been uploaded since either 25.4 or 26.1). As the capability for splitting roachtest operations and test execution logs already exists (https://github.com/cockroachdb/cockroach/pull/166175), all operational logs i.e. the test runner & cluster creation logs, should be uploaded to datadog.
* Note: Test logs across all branches and cloud provider CI is too expensive https://github.com/cockroachdb/cockroach/pull/163239#issuecomment-3930422144
Various backports need to be added across release branches, missing backports may be why the experience has been inconsistent. Also the Datadog API key needs to be across all TC Builds.
Then verify instrumentation is behaving as expected.

Related: https://github.com/cockroachdb/cockroach/issues/155854

Jira issue: CRDB-64157

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.