Multiple tabletpools of same type not supported

Use case

Creating distinct tablet pools of the same type, to create distinct configurations for tablets in the same Keyspace - ie. migrating from one volumeClaimTemplate to another.

Issue

I have a keyspace with 2 tablet pools - default (replica) and reporting (rdonly). This works fine.

I added a third tablet pool local, of type replica again (different volumeClaimTemplate and some labels distinct to the tablet pool) with a single replica (default x3, local x1) but when the operator applies the change, it instead scaled down a tablet from the default pool, leaving me with default x2, local x1. This happened again when I tried to scale up to local x2 resulting in default x2, local x2.

Root cause

It looks like the pool's name is only factored into the generation of the tablet UID when ExternalDatastore = true. This means that the 2x tablet pools' replicas are getting the same UIDs across the pools (based on their replica index), and as such are overwriting one-another.

https://github.com/planetscale/vitess-operator/blob/2d43a0ded4841ce5ea598c4f550532ed4f808121/pkg/controller/vitessshard/reconcile_tablets.go#L287-L290

However, the docs say There must be at most one pool in this list for each (cell,type,name), which leads me to think that the if ExternalDatastore != nil condition can be removed?

Version

Operator: v2.14.0
Vitess: v21.0.3

Looking at the source, this is still the same in main

I didn't see an issue template so I tried to include what I though what was relevant - Happy to provide more info here or on Slack.

Start in pkg/controller/vitessshard/reconcile_tablets.go around lines 287-290 and compare the UID generation with the documented (cell,type,name) tablet-pool constraint. Reproduce reconciliation with two same-type pools and verify that each pool retains its requested replica count and distinct tablet UIDs.