[PROPOSAL] Prevent historicals whose segment caches contain unknown segments from corrupting cluster

# Motivation

If you are running multiple independent Druid clusters with the same data source names (eg, a production and a staging cluster) and accidentally start up a historical on a machine with a segment cache from the wrong cluster, the results can be disastrous. The historical will announce the segments it found on disk and immediately begin serving them segments to users. Worse, the coordinator can mark the segments that actually are part of your cluster as unused if they're overshadowed, and even delete it from deep storage if kill tasks are turned on.

This is not a theoretical problem: my company had a major outage because we accidentally did this. Fortunately we had disabled kill tasks literally that morning, so we had no major data loss, and I understood the innards of the druid_segments SQL table well enough to figure out how to recover, but it was still challenging.

While users should be careful not to reuse segment caches, it would be nice to have some protection against this failure mode.

# Proposed changes

When historicals run `SegmentLoadDropHandler.loadLocalCache` on startup, they can check each segment that they find against the druid_segments table, and ignore any segments that aren't there. (They should ignore them and send an alert rather than crash, like it currently does for segment files whose info_dir filename doesn't match the actual segment id.)

I don't know the most appropriate way to do this check is. Do historicals have access to MetadataSegmentManager or a similar class? I think not and they probably shouldn't just for this one. Or do they have access to something like CoordinatorClient to talk to coordinators? (Do historicals ever talk to anything other than ZooKeeper and deep storage?)

(I see that `SegmentLoadDropHandler` is also part of realtime nodes, but I know nothing about realtime nodes. I don't know if this suggestion should be applied to them too or if they should be left alone.)

# Rationale

Other considered approaches:
- Allow you to give a "name" to a Druid cluster. Store the cluster name in a market file in the segment cache. Ignore segment caches that have the wrong name in them. (This also lets you use the name for other purposes — protecting server-to-server communication against misconfiguration that sets up nodes to talk to nodes in the wrong cluster, displaying the cluster name in the web console, etc.)
- Instead of storing the cluster name in a marker file, add it to the DataSegment class directly. This accomplishes a similar goal with a lot more changes, and necessitates adding another configuration to allow clusters to load segments that were created before the cluster got a name.
- Instead of a single startup-time check in historicals, brokers and coordinators could ignore all segments *from historicals* that aren't in the `druid_segments` table. This potentially solves other hypothetical failure modes (running a historical on a machine with an existing segment cache which had been turned off for a long time, and some segments on it had been deleted already?) but requires more machines to talk to the metadata db more often, and you have to be careful to only apply this logic to segments announced from historicals, not segments announced by indexing tasks.

# Operational impact

Minimal unless you run a historical with a segment cache with unknown segments, in which case it rescues you from the problems described above.

This does mean that historicals won't be able to start up if the coordinator is down (unless they talk directly to the metadata database). The implementation could assume segments are valid if the coordinator can't be reached, to make historicals more independent.

If you rely on the ability for historicals to serve segments even if the `druid_segments` SQL table is corrupted, then this could be a problem.

# Test plan

Tests can validate that unknown segments are skipped.

Start with SegmentLoadDropHandler.loadLocalCache and trace how historicals discover and announce cached segments. Determine how the node can check the druid_segments table or coordinator, including the behavior when that service is unavailable and whether realtime nodes are affected. Done means unknown segments are safely skipped with an alert and tests validate that behavior.

Written by the indexing model from the issue text.