Add rack/leader affinity to Redpanda

Who is this for and what problem do they have today?

This ticket is to publicly track adding rack/leader affinity to Redpanda. From the internal ticket:

To facilitate the most cost-efficient use of cross-AZ/region clusters, some customers need to control placement of partition leadership for a given topic within an AZ/region (as represented by the rack ID), because all/most clients for a given topic may be AZ/region-local, and transfer cost/latency is minimized with this capability, leading to substantial cost savings.

For clusters where rack-awareness has been configured, topics should be able to declare which rack should host all partition leaders for the topic (follower replicas should be stored in other racks per the normal rack aware replica placement for resiliency).

What are the success criteria?

A topic's leader can be set by the user on topic creation.

Why is solving this problem impactful?

Some future features that would be either enabled or unblocked by this implementation are:

• allow producers to write to a topic's partition which is within the same AZ
• allow consumers to read from the closest replica partition

Additional notes

Implementing rack/leader affinity is related to these other issues:

• https://github.com/redpanda-data/redpanda/issues/4103

JIRA Link: CORE-1071

Start by reading the linked issue #4103 and the referenced internal ticket CORE-1071 to understand the related work and requirements. The stated outcome is that users can select a rack for topic leaders at topic creation; the issue does not name implementation files or tests, so the remaining scope needs investigation.