[RFC]: Add optional Store transport connection warmup

### Changes proposed

## Motivation

In a multi-node Mooncake Store deployment, data may remain cold for a period of time before a burst of GET or write traffic arrives.

When transport connections to remote segments have not yet been established, many requests may concurrently enter the connection setup path and repeatedly poll the connection state. A sudden wave of these requests can create a connection storm, consume a large amount of client and transport resources, and in severe cases make the affected processes unstable or unresponsive.

The latency and resource pressure are especially visible when:

- multiple clients become active at approximately the same time;
- remote data has not been accessed recently;
- many requests target endpoints without reusable connections;
- first-access latency is important.

## Proposed change

Add an optional, best-effort Store transport connection warmup during client setup.

The proposed flow is:

1. The Store client asks the Master for a bounded list of eligible remote segment endpoints.
2. The Master filters unavailable segments, the requester's own segments, incompatible protocols, empty endpoints, and duplicate endpoints.
3. The client submits a small READ-only probe to each selected target through the existing Transfer Engine.
4. The probe triggers the normal metadata lookup, handshake, endpoint creation, and connection caching paths before business traffic arrives.
5. Failures remain best-effort and do not fail normal client setup.

This is transport connection warmup rather than KV data or cache preloading. It does not write remote memory.

## Safety and scalability

The feature should have the following safeguards:

- disabled by default;
- finite default target count;
- a defensive Master-side target limit;
- bounded client-side concurrency;
- READ-only probes;
- exact protocol-token matching;
- endpoint deduplication;
- safe ownership of probe buffers while transfers are still pending;
- batch-ID cleanup retries;
- warmup shutdown before probe memory is unregistered.

If a probe times out, its destination buffer must remain owned until the Transfer Engine reports that the transfer can no longer write to it.

## Scope

This proposal only adds Store-level connection warmup.

It does not:

- change the general Transfer Engine handshake concurrency model;
- add receiver-side handshake worker pools;
- guarantee successful warmup for every target;
- guarantee connection reuse for transports that do not cache connections;
- add periodic warmup;
- add a new scheduling framework.

## Configuration

The initial implementation provides configuration for:

- enabling Store warmup;
- READ probe size;
- per-target timeout;
- client-side concurrency;
- maximum requested targets.

The default requested target count is finite, and the Master applies an additional hard cap to each request.

## Expected benefit

For warmup targets that are later accessed by business traffic, the first request can avoid part of the metadata lookup, handshake, and endpoint initialization latency.

The expected benefit is primarily:

- lower first-access latency;
- reduced concurrent connection-state polling during traffic bursts;
- reduced risk of a connection-establishment storm.

This proposal is not expected to increase steady-state transfer bandwidth.

## Validation plan

The implementation should include:

- Master target-selection unit tests;
- protocol filtering and endpoint deduplication tests;
- maximum-target and Master hard-cap tests;
- pending-transfer buffer-lifetime tests;
- batch cleanup fairness and retry tests;
- TCP loopback integration tests covering real READ probes;
- repeated integration-test runs for basic stability.

Real cross-machine, RDMA/UB hardware, large-scale stress, and explicit connection-reuse measurements can be validated separately.

### Before submitting a new issue...

- [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

Start by locating Store client setup, Master target-selection handling, and the existing Transfer Engine READ, metadata lookup, handshake, and endpoint-caching paths; the issue provides no file names. Use the listed validation plan to define done: selection and filtering tests, buffer-lifetime and cleanup tests, and TCP loopback integration tests for real READ probes, with warmup disabled by default and normal setup unaffected by failures.

Written by the indexing model from the issue text.