BookKeeper rack-aware placement policies

We are attempting to set the rack-aware policies on our bookies, but are seeing some odd behaviors that ultimately end in the rack-awareness not working.

### Pulsar Version
2.2.0

### Our Setup
- 6 Bookies
- 3 on rack-1, 3 on rack-2
- Ensemble: 3
- Write Quorum: 2
- Read Quorum: 2

### Issue 1

1. We check the cluster's default rack policy before making any changes and it shows to be empty. That's what we'd expect.
```sh
# List Bookie Racks
$ $PULSAR_HOME/bin/pulsar-admin bookies racks-placement
{ }
```
2. Next we add the desired rack configurations so that we have the 2 racks of 3 as follows
```sh
# Add Bookies to Racks
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie1:3181 --group group-1 --hostname bookie1 --rack rack-1
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie2:3181 --group group-1 --hostname bookie2 --rack rack-1
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie3:3181 --group group-1 --hostname bookie3 --rack rack-1
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie4:3181 --group group-2 --hostname bookie4 --rack rack-2
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie5:3181 --group group-2 --hostname bookie5 --rack rack-2
$PULSAR_HOME/bin/pulsar-admin bookies set-bookie-rack --bookie bookie6:3181 --group group-2 --hostname bookie6 --rack rack-2
```
3. We list the rack policies again to be sure they are in zookeeper, also looks good
```sh
# List Bookie Racks
$ $PULSAR_HOME/bin/pulsar-admin bookies racks-placement

{
"group-1" : {
"bookie1:3181" : {
"rack" : "rack-1",
"hostname" : "bookie1"
},
"bookie2:3181" : {
"rack" : "rack-1",
"hostname" : "bookie2"
},
"bookie3:3181" : {
"rack" : "rack-1",
"hostname" : "bookie3"
}
},
"group-2" : {
"bookie4:3181" : {
"rack" : "rack-2",
"hostname" : "bookie4"
},
"bookie5:3181" : {
"rack" : "rack-2",
"hostname" : "bookie5"
},
"bookie6:3181" : {
"rack" : "rack-2",
"hostname" : "bookie6"
}
}
}
```
4. We check the broker logs to see if the new rack settings show up. As you can see it still pegs the bookies to /default-rack, not expected
```sh
14:01:24.828 [pulsar-ordered-OrderedExecutor-5-0-EventThread] INFO org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Reloading the bookie rack affinity mapping cache.
14:01:24.860 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping - Bookie rack info updated to {group-1={bookie1:3181=BookieInfo(rack=rack-1, hostname=bookie1), bookie2:3181=BookieInfo(rack=rack-1, hostname=bookie2), bookie3:3181=BookieInfo(rack=rack-1, hostname=bookie3)}, group-2={bookie4:3181=BookieInfo(rack=rack-2, hostname=bookie4), bookie5:3181=BookieInfo(rack=rack-2, hostname=bookie5), bookie6:3181=BookieInfo(rack=rack-2, hostname=bookie6)}, group-1={bookie1:3181=BookieInfo(rack=rack-1, hostname=bookie1), bookie2:3181=BookieInfo(rack=rack-1, hostname=bookie2), bookie3:3181=BookieInfo(rack=rack-1, hostname=bookie3)}, group-2={bookie4:3181=BookieInfo(rack=rack-2, hostname=bookie4), bookie5:3181=BookieInfo(rack=rack-2, hostname=bookie5), bookie6:3181=BookieInfo(rack=rack-2, hostname=bookie6)}}. Notifying rackaware policy.
14:01:24.861 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie1:3181
14:01:24.864 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie1:3181
14:01:24.864 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie2:3181
14:01:24.867 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie2:3181
14:01:24.867 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie3:3181
14:01:24.874 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie3:3181
14:01:24.874 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie4:3181
14:01:24.877 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie4:3181
14:01:24.877 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie5:3181
14:01:24.879 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie5:3181
14:01:24.879 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Removing a node: /default-rack/bookie6:3181
14:01:24.882 [zk-cache-callback-executor-OrderedExecutor-15-0] INFO org.apache.bookkeeper.net.NetworkTopologyImpl - Adding a new node: /default-rack/bookie6:3181

```
Taking down bookie nodes always shows /default-rack and not our currently set policies.

This behavior holds true no matter if you use:
- Admin CLI
- curl REST
- Postman REST

### Issue 2
Attempting to delete a rack policy shows to work without error but the data is actually unaffected in zookeeper.

1. Try a delete
```sh
$PULSAR_HOME/bin/pulsar-admin bookies delete-bookie-rack --bookie bookie1:3181
```

2. List the rack policies in zookeeper to make sure they've changed. As you can see they did not.
```sh
# List Bookie Racks
$ $PULSAR_HOME/bin/pulsar-admin bookies racks-placement

{
"group-1" : {
"bookie1:3181" : {
"rack" : "rack-1",
"hostname" : "bookie1"
},
"bookie2:3181" : {
"rack" : "rack-1",
"hostname" : "bookie2"
},
"bookie3:3181" : {
"rack" : "rack-1",
"hostname" : "bookie3"
}
},
"group-2" : {
"bookie4:3181" : {
"rack" : "rack-2",
"hostname" : "bookie4"
},
"bookie5:3181" : {
"rack" : "rack-2",
"hostname" : "bookie5"
},
"bookie6:3181" : {
"rack" : "rack-2",
"hostname" : "bookie6"
}
}
}
```
This behavior holds true no matter if you use:
- Admin CLI
- curl REST
- Postman REST

### Other Questions
1. What is the impact of not being able to have separate rack policies? We are currently stuck with /default-rack of 6 bookie nodes.

2. How many nodes can we loose without data loss? We are worried about being resilient and need to be able to loose 3 nodes without data loss.

Thanks!