todo: segment identity needs a working fabric rerun and the migration step

What is left undone

Two things, after a second cluster run closed the TCP control.

1. The packaged image has been exercised only halfway. A host-network backend has now run under
   gpustack/mirrored-mooncake:0.3.13.post1-cpu, so the tag gap on the identity and attribution
   reading is filled. What that run does NOT support is the transport half: EFA itself did not come
   up on it (the member could not reach the fabric), so "the EFA path works on the packaged image"
   is still unread.

   What blocked that re-run has since changed, and this paragraph used to point at the wrong
   thing. It cited #347 and #348 as two standing blockers. #347 has merged: it adds the EFA
   device-resource request, so an EFA member is no longer denied open() on the device by the
   device cgroup. #348 is still open but it covers the RDMA branch of that same function only —
   applyMemberFabric returns before the resource request whenever the protocol is not efa
   (pkg/worker/kvcache/mooncake/member_workload.go, read 2026-09-12) — so it does not block an EFA
   re-run.

   ⚠️ What #347 has NOT got is a reading on real hardware. No EFA member has been observed
   reaching the fabric since it merged, because the cluster it would run on was destroyed. So the
   honest state of this item is "waiting on an environment", not "waiting on a fix" — and the
   re-run is also the first evidence that #347 works.

2. The upgrade/migration step was not run.

What has already been measured

On a cluster with two host-network EFA nodes:

• Step 1. Both host-network member Pods came up Running, co-located on one node.

• Steps 3 and 4, in part. The leader reported two protocol: efa segments sharing the
  host-network address, with segment_id and client_id both non-empty and distinct. The controller
  could not uniquely attribute either row to a Pod from a shared address and reported
  MembersMounted=False with reason AmbiguousMemberIdentity.

  ⚠️ Step 4's criterion has been restated and the reading above predates it. The original step
  asked that both ambiguous rows leave nodeName and medium empty. That bound two separate things
  into one sentence, and only the second is required: what must not happen is a Pod being guessed.
  Since #345 an ambiguous row reports the node and the medium every candidate agrees on, which is
  well defined because the candidates are on one node by construction. The criterion is now that
  MembersMounted stays False with reason AmbiguousMemberIdentity and that no segmentID or
  clientID is attributed to a Pod that was merely guessed. An empty nodeName is no longer
  evidence of anything, and a populated one is no longer a defect.

• Step 5. With the groups moved to separate nodes, the two rows gained distinct identities,
  MembersMounted became True/Mounted, and the backend Phase became Ready.

• The TCP control (formerly step 6), now run. Two TCP groups pinned to one node took their own
  Pod IPs, MembersMounted became True/Mounted, Phase became Ready, and both rows carried a
  nodeName and a medium. This is the positive baseline the ambiguity reading needed: it shows
  the mechanism reports mounted when identity is unambiguous, rather than reporting ambiguity
  unconditionally.

⚠️ The protocol: efa in that reading does not mean the members used EFA. That field reports what
the client declared when it registered the segment, not the transport it negotiated - see #341. It is
recorded here as the identity reading it is, not as transport evidence.

The "portless segment name" criterion was wrong twice, and both readings are now in

This issue was filed asking for rows whose segment_name is portless. It is not, on either
transport: the leader's actual output carries a dynamic port in both the host-network case and
the TCP case. The portless shape came from the fixture in PR #296, not from a live Mooncake v0.3.13,
and it was carried into this issue as though it had been observed. Anyone following the original
wording gets a mismatch on the first read and has every reason to file it as a segment-naming
defect. It is not one.

⛔ A third reading, which the fixture work has to account for: within a single row, the port in
segment_name and the port in te_endpoint are DIFFERENT. They are two independently assigned
values, not one value rendered twice. Correcting the fixture by copying te_endpoint's port onto
segment_name would produce a shape the leader never emits.

That fixture is a fidelity gap and not a masking one: the attribution and ambiguity decisions read
te_endpoint, never segment_name (hostOf(segment.TEEndpoint)), and the fixtures' te_endpoint
values already carry ports. segment_name bears weight only in the encoded-size budget, where the
boundary case uses a name four orders of magnitude longer than a real one.

How to verify what remains

1. Re-run the co-located and separated host-network shapes under
   gpustack/mirrored-mooncake:0.3.13.post1-cpu on members that actually reach the fabric,
   asserting the same identity properties as steps 3-5 above, and recording the leader's
   /get_segments_detail response verbatim - both segment_name and te_endpoint, since their
   ports differ and each is its own reading.
2. On a cluster at Kubernetes 1.23 or above - the lowest version this project supports, not that
   exact version - install the pre-change CRD and controller, create a stored status.members row
   without segmentID or clientID, then upgrade the CRD and controller while the leader's
   /get_segments_detail request fails. Trigger both the backend listing path and the pool
   controller's independent usedBy status write. Both writes must succeed, the entire incompatible
   member listing must be omitted, and the backend must be Degraded with MembersMounted=False and
   reason LegacyMemberStatus. Restore the leader response and verify that real IDs repopulate the
   rows, the migration reason clears, and normal phase/condition reporting resumes.

⚠️ Step 2 must run last on any shared cluster: it downgrades the CRD and controller and upgrades
them again, which invalidates the premises of every other assertion on that cluster. And a green
result on a server that has CRD validation ratcheting should be recorded as holding on that version -
it does not establish the pre-ratcheting path, though a red result there would establish its failure.

What does NOT fill these

• Re-running the host-network path on the same upstream image. The gap is the tag, not the path.
• The identity reading that has already run under the packaged tag. It settles attribution on
  that image and says nothing about whether the transport works there, which is the half still open.
• Reading the migration logic from unit tests. Those pass, and what they do not prove is that
  both API-server status transitions succeed against a real installed CRD.

/area kv-cache

Start with pkg/worker/kvcache/mooncake/member_workload.go and the leader's /get_segments_detail response, then prepare a Kubernetes 1.23+ environment using the packaged image. Run the co-located and separated EFA cases on members that reach the fabric, followed last by the pre-change CRD/controller migration. Done means transport and identity evidence is recorded, incompatible legacy status is safely omitted with LegacyMemberStatus, and normal IDs and conditions return after recovery.