couchbaselabs / couchbaselabs/sdk-rfcs

GetReplica: wrap and the out-of-bounds bound are implemented two ways

Closed
#164 0 comments 0 reactions 1 assignee Claimed by @programmatix View on GitHub
Dominant language
No language data
Stars
42
Forks
20
PR merge metrics
No merged PRs in 30d

Description

The Go and C++ SDKs have each implemented `GetReplica`, and they resolve two cases differently. Each
behaviour is pinned by a unit test in its own repository, so neither is accidental. Both readings are
defensible from the current text, which is why this is an RFC question rather than a bug report
against one SDK.

All quotes below are from one section: [`0053-sdk3-crud.md` → `GetReplicaStrategy.fromIndex`](https://github.com/couchbaselabs/sdk-rfcs/blob/f690b9c46bf562465c10fc5ff5e02d9ec29876ce/rfc/0053-sdk3-crud.md?plain=1#L686).

| | implementation | change |
|---|---|---|
| Go | [`IndexReplicaSelector.selectReplica`](https://github.com/couchbase/gocbcore/blob/c0571fc4c1548a2e627de21f956dbdb83a30adc3/crud_replica.go#L19-L48) | [gocbcore 253207](https://review.couchbase.org/c/gocbcore/+/253207) (merged), consumed by [gocb 253297](https://review.couchbase.org/c/gocb/+/253297) (open) |
| C++ | [`resolve_replica_index`](https://github.com/couchbase/couchbase-cxx-client/blob/4bcea91846a565687c95fd1687c8fcb7a7d62621/core/impl/replica_utils.cxx#L47-L107) | [couchbase-cxx-client#1078](https://github.com/couchbase/couchbase-cxx-client/pull/1078) (open) |

**Indexing.** [Line 701](https://github.com/couchbaselabs/sdk-rfcs/blob/f690b9c46bf562465c10fc5ff5e02d9ec29876ce/rfc/0053-sdk3-crud.md?plain=1#L701) says `ReplicaIndex.FIRST` is the 0-th element of the replica chain,
and the chain excludes the active copy. Go's `ReplicaIdx` is 1-based against the raw vbucket-map row;
C++'s index is 0-based against the replica array. Go N is C++ N-1. The tables below use the RFC's
names (`FIRST`, `SECOND`, `THIRD`) and show vbucket-map rows with the active at position 0.

---

## 1. Which length bounds the index

[Line 706](https://github.com/couchbaselabs/sdk-rfcs/blob/f690b9c46bf562465c10fc5ff5e02d9ec29876ce/rfc/0053-sdk3-crud.md?plain=1#L706):

> if the replica index is higher than the current replica chain for that vbucket, the SDK will
> fast-fail with a `ReplicaIndexOutOfBoundsException` exception (new for this feature) without
> hitting the network.

- **Go** bounds on `numReplicas`, the bucket's configured replica count. An index within that but past
the row falls through to the unavailable branch.
- **C++** bounds on the row: `min(row length - 1, numReplicas)`.

The two differ when the row is shorter than `numReplicas + 1` — the state between a replica-count
*increase* being published and the rebalance that widens the rows.

| row | `numReplicas` | requested | `wrap` | Go | C++ |
|---|---|---|---|---|---|
| `[0, 1]` | 2 | `SECOND` | false | `ReplicaIndexCurrentlyUnavailable` | `ReplicaIndexOutOfBounds` |

Pinned by [`ReplicasInVBucketMapFewerThanNumReplicas`](https://github.com/couchbase/gocbcore/blob/c0571fc4c1548a2e627de21f956dbdb83a30adc3/crud_replica_test.go#L104-L111) and
[`the_chain_length_caps_a_num_replicas_larger_than_the_chain`](https://github.com/couchbase/couchbase-cxx-client/blob/4bcea91846a565687c95fd1687c8fcb7a7d62621/test/unit/core/get_replica.cxx#L390-L398).

**To decide:** is the bound the vbucket's replica chain, the bucket's configured replica count, or the
smaller of the two?

---

## 2. What `wrap` does

Two passages point in different directions.

**Procedure** — [lines 707-709](https://github.com/couchbaselabs/sdk-rfcs/blob/f690b9c46bf562465c10fc5ff5e02d9ec29876ce/rfc/0053-sdk3-crud.md?plain=1#L707-L709):

> Unless the `wrap` option is true, in which case the replica index should be used modulo the length
> of the replica array.

> If the given replica has a -1 entry in the vbucket map indicating the node is currently
> unavailable, raise a `ReplicaIndexCurrentlyUnavailableException` (new for this feature). Unless
> `wrap` is set, in which case, automatically move to the next replica (wrapping around if necessary).

> `wrap` edge-case: if the SDK somehow wraps all the way around to the starting point in the same
> loop (e.g. if the replica chain somehow contains all -1 entries), then raise a
> `ReplicaIndexCurrentlyUnavailableException`.

**Option description** — [line 698](https://github.com/couchbaselabs/sdk-rfcs/blob/f690b9c46bf562465c10fc5ff5e02d9ec29876ce/rfc/0053-sdk3-crud.md?plain=1#L698):

> `Boolean wrap` - whether to wrap around the available replicas, rather than throwing
> `ReplicaIndexOutOfBoundsException`. Defaults to false.

Go implements the option description — drop the unavailable entries, then index into what remains:

```go
return availableReplicaIdxs[(s.ReplicaIdx-1)%len(availableReplicaIdxs)], nil
```

C++ implements the procedure — modulo the replica-array length, then step forward one position per
iteration (`offset` is the step counter), at most one lap:

```cpp
const auto position = ((requested_replica + offset) % number_of_replicas) + 1;
```

Dropping entries renumbers every index above the gap, so the two answer differently when the replica
named is reachable and a `-1` sits below it. They always agree when no `-1` precedes the named
replica. They also agree when the named replica is itself `-1` and another `-1` sits below it:
a bucket has at most three replicas, so two gaps leave one reachable replica and both sides return
it.

| row | `numReplicas` | requested | `wrap` | Go | C++ |
|---|---|---|---|---|---|
| `[0, 1, -1, 2]` | 3 | `THIRD` | true | replica 1 | replica 3 |
| `[0, -1, 1, 2]` | 3 | `THIRD` | true | replica 2 | replica 3 |
| `[0, -1, 1]` | 2 | `SECOND` | true | replica 2 | replica 2 |
| `[0, -1, 1]` | 2 | `FIRST` | true | replica 2 | replica 2 |
| `[0, 1, -1, 2]` | 3 | past the end (Go 7, C++ 6) | true | replica 1 | replica 1 |

The C++ reading is pinned by
[`wrap_keeps_a_reachable_replica_an_earlier_gap_precedes`](https://github.com/couchbase/couchbase-cxx-client/blob/4bcea91846a565687c95fd1687c8fcb7a7d62621/test/unit/core/get_replica.cxx#L328-L340),
added after this issue was opened; before it, the C++ suite did not distinguish the two readings.

Rows 1 and 2 are the divergence: `THIRD` is present and reachable, and Go answers from a
lower-numbered replica. Read the other way, a caller who set `wrap` has said a different replica is
acceptable, in which case C++ is the implementation that would need to change.

**To decide:** does `wrap` renumber the replica array by dropping unavailable entries, or keep the
numbering and walk forward from the named position?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.