[Bug] Reject ambiguous external metadata local-name mappings
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
## Search before asking
I searched existing Apache Doris issues for duplicate external metadata local names, `meta_names_mapping` conflicts, and ambiguous remote-to-local name resolution. I did not find an issue covering this exact gap.
## Version
Apache Doris `master` at `2215dc476a1f9a1f6edba8c7a056e450165f7001`.
This problem was found while reviewing #65126, but the ambiguous mapping state is already possible on the master baseline and was not introduced by that PR.
## What is wrong?
External metadata name loading does not unconditionally enforce that one exact local database/table name identifies only one remote object.
For example, a custom naming hook or mapping can produce:
```text
RemoteA -> LocalX
RemoteB -> LocalX
```
This state is not resolvable: SQL can reference only `LocalX`, while the object cache key and generated Doris metadata ID are also derived from `LocalX`.
The generic conflict checks currently run only under selected case-insensitive/lower-case modes. In ordinary mode 0, a custom connector naming hook or an uncovered mapping collision can still produce the same exact local name for multiple remote objects.
## Master baseline behavior
The legacy `MetaCache` keeps the complete remote/local pairs in a mutable list.
- `MetaCache.listNames()` returns both local entries, so database enumeration can expose duplicate `LocalX` values.
- `MetaCache.getRemoteName(LocalX)` uses `findFirst()`, so only `RemoteA` is selected.
- The metadata object cache is keyed only by `LocalX`, so `RemoteB` cannot be addressed independently.
- Incremental `updateCache()` appends another pair without replacing the existing local identity, making the result event-order dependent.
Relevant code:
- https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java#L83-L92
- https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java#L128-L138
- https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java#L240-L267
- https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java#L593-L618
## Impact
The trigger is narrow because normal remote metadata normally has unique names and standard mapping implementations reject many direct conflicts. The remaining gap mainly affects custom connector naming hooks, general `ExternalCatalog` subclasses, or uncovered mode-0 mapping collisions.
Once triggered, the impact is a correctness problem:
- one remote database/table becomes inaccessible or is resolved through the wrong remote identity
- cached and cache-bypass paths may select different remote objects
- object-cache keys and deterministic IDs collide
- database enumeration may contain duplicate local names
- incremental events may change the selected remote object according to event order
## Expected behavior
Within one catalog scope for databases, and within one database scope for tables:
- an exact local name must identify at most one remote object
- two different remote names mapping to the same exact local name must fail with a clear conflict error
- case-sensitive mode 0 must continue to allow distinct local names such as `Foo` and `foo`
- mode-specific case-insensitive conflict checks must remain in effect
- duplicate/replayed incremental events must remain idempotent
## Suggested fix
1. Validate the exact local-name uniqueness invariant when building the complete external metadata names snapshot.
2. Report both conflicting remote names and the shared local name.
3. Add a defensive invariant in the shared names value abstraction so custom loaders cannot publish an ambiguous snapshot.
4. Keep mode-aware case-insensitive collision validation in the catalog/database loader.
5. Add database and table tests, including a custom naming hook in mode 0.
PR #65126 plans to fix this by validating the invariant in its immutable `NameCacheValue` snapshot before publication.
Contributor guide
Research direction
Start with MetaCache.java, ExternalDatabase.java, and ExternalCatalog.java to trace snapshot loading, lookup, and incremental updates. Review PR #65126 and its immutable NameCacheValue approach, then examine the database and table test coverage needed for mode 0 custom naming hooks. Done means exact local-name collisions are rejected with both remote names while valid case-sensitive names and replayed events remain supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100