apache / apache/hudi

[SUPPORT] Datadog Metrics reporter fails with null pointer exception using hudi 0.14.0

Open
#11,278 0 comments 0 reactions 0 assignees View on GitHub
area:metrics
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**
I am trying to setup the datadog metrics reporter with an api key for some hudi spark jobs and getting the following issue when using `0.14.0` and spark `3.1.3` on dataproc gke. I am following the documentation outlined here, https://hudi.apache.org/docs/metrics/#datadogmetricsreporter.

I am getting a NullPointerException where it is returning null for the name of the Datadog api site.

**To Reproduce**

Steps to reproduce the behavior:

1. Use hudi 0.14.0
2. Use Spark 3.1.3
3. Setup hoodies metrics config like so
```
# metrics
"hoodie.metrics.on": "true",
"hoodie.metrics.reporter.type": "DATADOG",
"hoodie.metrics.datadog.api.site": "US",
"hoodie.metrics.datadog.api.key": dd_api_key, # from vault
"hoodie.metrics.datadog.metric.prefix": f"app_name_{self.env}",
"hoodie.metrics.datadog.metric.host": "api.datadoghq",
```

**Expected behavior**

Custom metrics should be reported to datadog with the spark job not running into any issues.

**Environment Description**

* Hudi version : 0.14.0

* Spark version : 3.1.3

* Storage (HDFS/S3/GCS..) : S3

* Running on Docker? (yes/no) : no

**Stacktrace**

```
py4j.protocol.Py4JJavaError: An error occurred while calling o255.save.
: org.apache.hudi.exception.HoodieRollbackException: Failed to rollback s3a://thescore-datalake-enriched-uat/enriched/tracker/tracker_validated/sports/button commits 20240522021059764
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:827)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:731)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:715)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:710)
at org.apache.hudi.client.BaseHoodieWriteClient.lambda$startCommitWithTime$97cdbdca$1(BaseHoodieWriteClient.java:839)
at org.apache.hudi.common.util.CleanerUtils.rollbackFailedWrites(CleanerUtils.java:156)
at org.apache.hudi.client.BaseHoodieWriteClient.startCommitWithTime(BaseHoodieWriteClient.java:838)
.
.
.
.
Caused by: java.lang.NullPointerException: Name is null
at java.lang.Enum.valueOf(Enum.java:236)
at org.apache.hudi.metrics.datadog.DatadogHttpClient$ApiSite.valueOf(DatadogHttpClient.java:120)
at org.apache.hudi.config.HoodieWriteConfig.getDatadogApiSite(HoodieWriteConfig.java:2090)
at org.apache.hudi.metrics.datadog.DatadogMetricsReporter.(DatadogMetricsReporter.java:48)
at org.apache.hudi.metrics.MetricsReporterFactory.createReporter(MetricsReporterFactory.java:78)
at org.apache.hudi.metrics.Metrics.(Metrics.java:58)
at org.apache.hudi.metrics.Metrics.getInstance(Metrics.java:82)
at org.apache.hudi.metrics.HoodieMetrics.(HoodieMetrics.java:72)
at org.apache.hudi.client.BaseHoodieClient.(BaseHoodieClient.java:91)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with DatadogMetricsReporter.java, then trace the Datadog API-site value through HoodieWriteConfig.java and DatadogHttpClient.java. Reproduce the configuration using Hudi 0.14.0 and Spark 3.1.3, and verify that the Datadog reporter no longer fails with a NullPointerException while custom metrics are reported.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.