Registry.getRegistryOfClass returns a registry of the wrong class instead of failing
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Found while reviewing #19892 (part of #19524).
`Registry.REGISTRY_MAP` (hudi-io, `org.apache.hudi.common.metrics.Registry`) is a JVM-wide static that is never cleared. `SparkHoodieBackedTableMetadataWriter.initRegistry()` interns the metadata registry under the table-name-less key `""::HoodieMetadata`. When metrics are on and executor metrics are off it stores a `LocalRegistry`; when executor metrics are on with a non-INMEMORY reporter it calls `Registry.getRegistryOfClass(..., DistributedRegistry.class)`. On a class mismatch `getRegistryOfClass` only logs "already exists with a different class" and returns the existing `LocalRegistry`, which the caller then hard-casts.
Reproduces at one surefire fork just by ordering two existing tests in one JVM:
```
mvn test -am -pl hudi-spark-datasource/hudi-spark -Dspark3.5 -Dscala-2.12 -DskipUTs=true \
-Dhudi.surefire.forkCount=1 -Dsurefire.runOrder=reversealphabetical \
-Dtest='TestMetricsReporter,TestMetadataTableWithSparkSQL' ...
# ClassCastException: LocalRegistry cannot be cast to DistributedRegistry
```
The alphabetical order passes, and each class passes alone. Any change to how classes are grouped into JVMs (fork count, new tests, renamed classes) can flip it.
Proposed fix: key the metadata registry per table (or per base path), and have `getRegistryOfClass` replace or throw on a class mismatch rather than return the wrong type. A test-side reset of `REGISTRY_MAP` between classes would hide the symptom but not the production bug.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with Registry.REGISTRY_MAP and Registry.getRegistryOfClass, then trace SparkHoodieBackedTableMetadataWriter.initRegistry(). Reproduce the collision with TestMetricsReporter and TestMetadataTableWithSparkSQL using one surefire fork and reverse order. Done means the metadata registry is isolated by table or base path and a class mismatch cannot return a registry that causes a ClassCastException, with a regression test covering the ordering.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100