Clarification on the Inner Workings of Auto Rehoming

Hello,

I have recently read the CockroachDB paper and was fascinated by the concept of auto rehoming and how the `crdb_region` column represents this. However, I had a few points of confusion that I was hoping could be clarified.

1. **Process of Rehoming**: When a rehome operation is triggered, what exactly happens within CockroachDB? My understanding is that it starts a transaction to update the `crdb_region` field of the row. Then, once this update is committed, the Raft protocol ensures this update gets replicated across all copies. Is this understanding correct?

2. **Conflict Resolution**: In the event of simultaneous rehome operations being triggered by different regional clients, how does CockroachDB handle this conflict? As far as I understand, every transaction in CockroachDB carries a timestamp and must be committed in the order of their timestamps. Therefore, in a case where two transactions attempt to modify the same row, only one transaction will succeed and the other will be blocked until the first one commits or aborts. Can you confirm if this is the case?

3. **Impact on Ongoing Transactions**: How does a rehome operation affect an ongoing transaction that is currently accessing the key? I assume that in CockroachDB, transactions are isolated so that a transaction would not see the modifications made by another ongoing transaction. Therefore, if a transaction is accessing a row that is being rehomed, it would continue its operations without seeing the rehome operation. Only when this transaction tries to commit, and if the rehome operation has been committed, this transaction might fail due to concurrency conflict. Is this correct?

5. **Rehoming on Non-Leader Nodes**: If a client triggers a rehome operation, and the node it is interacting with is not the leader, how does the rehome operation occur? As I understand, the non-leader node forwards the request to the leader, which then logs and replicates the operation. Can you confirm or elaborate on this?

I would appreciate your help to clarify these points and deepen my understanding of the workings of CockroachDB.

Thank you.

Jira issue: CRDB-29439

Epic CRDB-45396