[Enhancement] Maintain key ordering during re-partitioning

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### Version

Linux, java 21.0.4, from `apachepulsar/pulsar@sha256:51e92bc45ba495b9753585b12625579ddaeea8e02d08aecd336ceff26e1099f1`
environment:zookeeper.version=3.9.2-e454e8c7283100c7caec6dcae2bc82aaecb63023, built on 2024-02-12 20:59 UTC
Pulsar Broker service; version: '4.0.0' Git Revision 92448d5f3113d783d41de6bae718d34cd73ca934
Client is also at `4.0.0`

### Minimal reproduce step

Create a partitioned topic with 5 partitions.
Add a subscription and 5 `Key_Shared` consumers, running in parallel.
Start a producer producing messages with 20 different keys.
Increase the number of partitions to 10.
Restart the producer.

### What did you expect to see?

Each consumer should be in charge of a portion of the key space, once attributed the consumer-key should be stable.

### What did you see instead?

The following sequence:
```
[Consumer 4] In charge of key 7 # first time Consumer 4 sees key 7
[Consumer 4] received MSG 7.1 (key 7)
[Consumer 4] received MSG 7.2 (key 7)
[Consumer 4] received MSG 7.3 (key 7)
...
<--- REPARTITION --->
<--- PRODUCER RESTART --->
[Consumer 4] received MSG 7.10 (key 7)

[Consumer 3] In charge of key 7 # oops... consumer 4 is still ongoing
[Consumer 3] received MSG 7.11 (key 7)
[Consumer 3] received MSG 7.12 (key 7)
[Consumer 3] received MSG 7.13 (key 7)

[Consumer 4] received MSG 7.14 (key 7)
[Consumer 4] received MSG 7.15 (key 7)
...
```

### Anything else?

As @lhotari [pointed out](https://github.com/apache/pulsar/issues/23603#issuecomment-2478709054), this could be solved by offering a more advanced automated procedure for re-partitioning. While shutting down all producers and waiting for all messages to be consumed would solve the problem, I wonder how applicable this would be in real life...

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Start with the reported five-to-ten partition reproduction using a Key_Shared subscription, five consumers, and a producer restart. The payload names no source files or tests; done means a key remains assigned to one consumer without overlapping delivery during and after re-partitioning.

Written by the indexing model from the issue text.