apache / apache/hudi

[SUPPORT] DataDog Metrics Reporter using outdated DataDog API endpoint

Open
#8,785 1 comment 0 reactions 0 assignees View on GitHub
area:metrics priority:medium
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**

I'm trying to configure `DataDogMetricsReporter` for many `DeltaStreamer` jobs and getting the following issue when using the `0.13.0` documentation for metrics (https://hudi.apache.org/docs/metrics/#datadogmetricsreporter):
```
Caused by: javax.net.ssl.SSLPeerUnverifiedException: Host name 'app.us.datadoghq.com'
does not match the certificate subject provided by
the peer (CN=*.datadoghq.com, O="Datadog, Inc.", L=New York, ST=New York, C=US)
```

**To Reproduce**

Steps to reproduce the behavior:

1. Follow Metrics documentation for DataDogMetricsReporter
2. Add API Key and turn metrics on
3. Deploy delta streamer job with the appropriate properties

**Expected behavior**

I expected to see the `hoodie.` prefixed metrics in DataDog.

**Environment Description**

* Hudi version : 0.13.0

* Spark version : 3.1

* Hive version : 2.3.1

* Hadoop version : 2.10.1

* Storage (HDFS/S3/GCS..) : S3

* Running on Docker? (yes/no) : No

**Additional context**

I've reached out to DataDog Support and they seem to say something similar regarding the outdated endpoint being the possible cause of this failure to write metrics to DataDog.

The Metrics Writer uses ["app.us.datadoghq"](https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/datadog/DatadogHttpClient.java#L51)
but the official documentation says the API needs to be [api.datadoghq.com](https://docs.datadoghq.com/api/latest/metrics/#submit-metrics).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/datadog/DatadogHttpClient.java at the endpoint referenced on line 51, then compare it with DataDog's current metrics API documentation. Reproduce the DeltaStreamer configuration from the issue and verify that hoodie.-prefixed metrics reach DataDog successfully using the supported endpoint.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability-sre
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.