MetadataSegmentView should correctly handle the situation of Coordinator being down or changed leadership
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
Regardless of whether any of the designs proposed for `MetadataSegmentView` here: https://github.com/apache/incubator-druid/issues/6834#issuecomment-478569365 is applied, `MetadataSegmentView` should document its behavior in the face of the Coordinator being down, or just started, or leadership changed.
Currently, there is a single comment ["Note that coordinator must be up to get segments"](https://github.com/apache/incubator-druid/pull/6901/files#diff-5707cd19ee7a43a258f29e61fd385d4cR181) which, IMO, only adds FUD and doesn't explain anything.
Ideally, `MetadataSegmentView` should correctly recognize the situations of changed Coordinator's leadership and not to flush all cached published segments.
Explanation may look like the following:
```
/**
* When Coordinator is just started or leadership changed,
* its SegmentsMetadata is unpopulated. Then the endpoint /metadata/abc/foo/bar
* which we are querying returns code XXX (see {@link DataSourcesResource#theRelevantMethod}).
* We process this code here and don't flush {@link #cachedPublishedSegments} in this case.
*/
```
FYI @surekhasaharan
Contributor guide
Research direction
Start by locating MetadataSegmentView and DataSourcesResource#theRelevantMethod, then read the linked discussion in issue 6834 and the referenced change in pull request 6901. Trace the response used when the Coordinator is starting or has changed leadership, and verify that the documented and implemented behavior preserves cachedPublishedSegments rather than flushing them unnecessarily.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100