Location-transparent placement: "spawn on" should not care whether the device is in this box

The vision, stated plainly: when a program says spawn on(T), T may be a GPU in this box, a GPU in the next rack, or a GPU in another datacentre, and the program text is the same. The programmer names where, not how to get there.

This reframes M4 in docs/discussions/implementation_plans/gpu_disaggregated_inference.md, which specifies the KV handoff as TCP written in Vx source with std::net — "the handoff is Vx source, that is the claim". Under this vision that is backwards. Sockets belong below the plugin ABI, not in the program. A program that opens a socket has stopped being location-transparent and started being a network application that happens to compile with Vx.

The abstraction hides the mechanism, not the price

This is the distinction that keeps it from being a leaky abstraction, and it is the thing Vx can do that a runtime-only system cannot.

A local peer copy is microseconds. The same copy across InfiniBand is milliseconds. Hiding that would be dishonest — it is the difference between a placement that works and one that does not. But hiding it is not what is proposed. The fleet models already declare the edges:

transfer Memory::HBM      -> Memory::NIC_RAM    : 200,
transfer Memory::NIC_RAM  -> Memory::Remote_HBM : 2400

and src/arch.rs already routes across them with Dijkstra, picking the cheaper multi-hop path over a direct one where that is what the hardware offers. So the compiler can price a remote placement before the machine is rented — which is hiraditya/Vx.1#319's whole thesis, applied one level out.

"The device may be anywhere, and the compiler tells you what anywhere costs." That is a stronger sentence than either half alone, and it is the one this issue is for.

The seam is already the right shape

vx_plugin_transfer_peer(src_device_ptr, src_topology_id, dst_topology_id, bytes) (fcfd81cf) names a movement by its two endpoints and hides how it happens. The CUDA plugin answers with cudaMemcpyPeer; a fleet plugin would read the ids, notice the destination is not local, and do RDMA. The program text does not change. Same for vx_plugin_dispatch_async, which already carries topo=N in its payload.

So nothing about the ABI needs redesigning. Three specific things are missing.

1. No topology can name a remote device

Topology::GPU[1] means cudaSetDevice(1) on this box. There is no spelling for "GPU 1 on node B". The memory vocabulary for remote exists; the placement vocabulary does not.

Open question, and a naming decision rather than an implementation one: whether this is a second index (Topology::GPU[node, dev]), a qualified topology, or a fleet-level name declared in the machine model the way Memory and Topology already are. The last is probably right — it keeps the program naming a role and the machine file saying where that role lives, which is what makes one program text run against different fleets.

2. memory_space_dispatch_id collapses remote onto AMX

// Network memory has no dedicated topology; kept at 300 (overlaps AMX) for now.
MemorySpace::NicRam | MemorySpace::RemoteHbm => 300,

Topology::AMX is also 300. A transfer targeting remote memory therefore emits target_topology = 300 and a plugin reads it as Apple AMX. Two remote nodes are also indistinguishable from each other. The comment already calls it out as provisional; this is the issue it was waiting for.

3. Dispatch arguments are pointers into this process

device_args[i] points at a value in this address space, and libffi reconstructs the call locally. A remote dispatch has to marshal instead. The groundwork is there — arg_tags describe every argument, and vx_memref_aligned / vx_memref_offset / vx_memref_write_desc in include/vx_hardware_runtime.h already decode a memref descriptor well enough to serialise one — but nothing walks them for the wire.

This is runtime work, not language work, and it is the largest of the three.

Relationship to hiraditya/Vx#347

hiraditya/Vx#347 Step 1 (landed) is the first implementation of the abstraction with both devices in one box. It is worth having proved there first: it is the only configuration where the four defects in hiraditya/Vx.1#346 were observable, and it establishes that the seam carries a real device-to-device movement before that movement has to cross a network too.

hiraditya/Vx#347 Step 2 should become this, rather than the socket-in-Vx form: the same handoff_kv source, a topology that names a device on another machine, and a plugin that notices. Nothing in tests/backend/pass/llama2.vx would change.

Explicitly not claimed

That remote and local perform alike. They do not, by three orders of magnitude, and the admission verdict is where that is supposed to show up rather than being smoothed over. A placement whose handoff does not fit its declared link budget should be refused at compile time for the same reason one that does not fit HBM already is.

Start with docs/discussions/implementation_plans/gpu_disaggregated_inference.md, src/arch.rs, and the runtime ABI references in include/vx_hardware_runtime.h; compare them with hiraditya/Vx#347 and tests/backend/pass/llama2.vx. Trace topology and memory-space dispatch IDs, then inspect argument tags and memref descriptors. Done means remote placements can be named, remain distinguishable through dispatch, marshal arguments, and enforce declared link budgets.