apache / apache/doris

[Tracking] Follow-up issues and baseline findings related to PR #65126

Open
#66,026 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

## Current status after connector SPI merge (#64304)

Updated after #64304 was merged into master and #65126 was rebased on top of it.

| Tracking item | Current status |
|---|---|
| #65750 HMS incremental-event consistency | Partially fixed: item 1 is fixed by #65126; items 2 and 4 were structurally fixed/removed by #64304; items 3, 5, and 6 remain open. A new item 7 now tracks missing connector-cache invalidation for database/table CREATE, DROP, and RENAME events. |
| #65779 names snapshot rebuild cost | Open; immutable NameCacheValue single-item updates still rebuild O(N) state. |
| #65792 unified cache statistics | Open and expanded: direct FE catalog/database entries and connector-owned entries are both absent from the unified statistics path. |
| #66001 global invalidation ordering | Open; #64304 does not provide one ordering barrier across FE names/object/ID state. |
| #66025 delegated-session SHOW TABLES N+1 | Open; the current getTables path still repeats remote name enumeration. |
| #66024 MetaInfoAction privilege filtering | Open; unrelated to #64304 and #65126 cache behavior. |
| #66233 ambiguous local-name mappings | Fixed by the current #65126 NameCacheValue implementation, pending PR merge; not fixed by #64304. |

### Current PR-only review items not closed by #64304

- Legacy ID-only refresh replay can still lose invalidation when the database object entry is cold.
- An unrelated key in the same MetaCacheEntry stripe can suppress required ID-map publication; the active-key side-effect fence is still pending.
- Targeted DROP/rename scans the retained ID map under the cache publication window; this remains a performance follow-up.

### Scope decision for connector event invalidation

The missing connector invalidation for HMS database/table events is a correctness gap introduced by #64304's cache-ownership split, but it is already present on the new master baseline. It is tracked in #65750 and does not have to block #65126 as long as the PR claims only FE metadata-cache convergence and the follow-up risk is explicit.

## Summary

This issue aggregates the follow-up issues, review findings, and baseline problems tracked outside PR #65126:

- PR: #65126
- PR URL: https://github.com/apache/doris/pull/65126

The goal is to provide one place where reviewers and later contributors can quickly find the problems that were discovered during the review of PR #65126, including both:

1. issues introduced or clarified during the refactor follow-up process, and
2. pre-existing master/baseline problems that were surfaced by the same review thread but intentionally tracked out of scope.

## Tracked issues

### Correctness / consistency

- #65750 `[Bug] HMS incremental events have multiple metadata cache consistency gaps`
- umbrella issue for HMS incremental-event consistency problems
- includes the mode-1 database-event canonicalization gap identified during PR review

- #66001 `[Bug][Meta Cache] Complete global invalidation ordering after PR #65126`
- tracks the remaining global invalidation ordering/fence gap after the per-key publication ordering fixes

### Performance / behavior follow-ups

- #65779 `[Performance] Reduce full names snapshot rebuild cost on HMS events`
- tracks snapshot rebuild overhead on HMS event paths

- #65792 `[Enhancement] Include catalog and database MetaCacheEntry instances in unified cache statistics`
- tracks follow-up observability work for the refactored cache entries

- #66025 `[Performance] Avoid N+1 remote table enumeration in external session bypass SHOW TABLES path`
- baseline performance issue surfaced during the PR review
- not introduced by PR #65126, but related to the same external metadata/session-bypass area

### Other related findings found during review

- #66024 `[bug](http) MetaInfoAction.getAllDatabases() leaks unfiltered db list, bypassing SHOW privilege`
- separate pre-existing master bug surfaced while reviewing the full PR scope

## Notes

- This issue is intentionally a tracking index only.
- Individual bug discussions, fixes, priority, and closure should happen in the linked issues.
- New issues discovered while reviewing or following up PR #65126 can be appended here later.

Contributor guide

Open the contributing guide

Research direction

Start with the linked follow-up issues #65750, #65779, #65792, #66001, #66024, and #66025, then review PR #65126 and the baseline merge #64304 for context. This issue is an index rather than a standalone task; done means resolving or updating the individual findings and keeping their status accurately reflected here.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.