Problematic separation between IndexerMetadataStorageCoordinator and MetadataSegmentManager abstractions
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
Several methods from `IndexerMetadataStorageCoordinator`, namely
- `getUsedSegmentsForInterval`
- `getAllUsedSegments`
- `getUsedSegmentsAndCreatedDates`
- `getUsedSegmentsForIntervals`
- `getveUnusedSegmentsForInterval`
Duplicate the `MetadataSegmentManager`'s functionality and look like much better suited for the latter class.
The conceptual difference is that `MetadataSegmentManager` caches the segments' data in memory, while `IndexerMetadataStorageCoordinator` always accesses the metadata store. This difference should be retained, but there should be comments to those methods, as well as parallel methods returning cached metadata with warnings and guidance regarding which method to use. For example, `MaterializedViewSupervisor` uses the methods from `IndexerMetadataStorageCoordinator` which seems unnecessary to me (could use the methods from `MetadataSegmentManager` instead).
If we decide that "cached metadata - direct access to metadata store" is the separation principle between `IndexerMetadataStorageCoordinator` and `MetadataSegmentManager`, then this fact should be reflected in these interfaces' names, Javadocs, and `retrieveAllDataSourceNames()` method should be moved from `MetadataSegmentManager` to `IndexerMetadataStorageCoordinator`.
Contributor guide
Research direction
Start by comparing the IndexerMetadataStorageCoordinator and MetadataSegmentManager abstractions and their listed segment methods, then inspect MaterializedViewSupervisor's usage. Clarify whether the intended boundary is cached metadata versus direct metadata-store access. Done means the chosen separation is consistently reflected in method placement, interface names, Javadocs, and guidance for callers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100