virtio: no way for a device to persist its own state across save/restore

While implementing VIRTIO_NET_F_CTRL_GUEST_OFFLOADS I ran into what looks like a structural gap rather than a missing field, so I wanted to ask before building anything.

What I see

VirtioDevice (vm/devices/virtio/virtio/src/device.rs:22) exposes traits, queue_size, read_registers_u32, write_registers_u32, set_shared_memory_region, start_queue and stop_queue. There is no save/restore method.

The transport saves CommonSavedState (device status, driver feature banks, the selects, config generation) plus the per-queue ring state stop_queue returns. All of that is transport-level. As far as I can tell a device has nowhere to put state of its own, and no virtio device implements SaveRestore (the one impl in vhost_user_frontend is a test mock).

Why the control virtqueue needs it

The guest changes device state at runtime through the control queue, and that state cannot be re-derived from the negotiated features afterwards:

• VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET - the current offload mask. Linux sends it from virtnet_set_features, and the kernel also sends it unprompted when the interface joins a bridge (dev_disable_lro).
• VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET - the active queue pair count.
• RX mode, MAC table and VLAN filters, for a device offering those classes.

For offloads the failure is not just a lost setting. On restore the device re-derives the mask from the negotiated features, so coalescing comes back on while the guest's own view (ethtool -k) still says off, and the guest is handed coalesced frames it believes it disabled. Persisting the number is also not sufficient on its own: the backend has to be re-programmed from it on restore, or the device and the tap disagree.

Why the feature matters beyond that

This may be worth knowing independently of save/restore. virtnet_xdp_set refuses to attach an XDP program when the device offers any guest offload but not VIRTIO_NET_F_CTRL_GUEST_OFFLOADS:

"Can't set XDP while host is implementing GRO_HW/CSUM, disable GRO_HW/CSUM first"
-> -EOPNOTSUPP

The check includes VIRTIO_NET_F_GUEST_CSUM, which traits() offers today. So as far as I can tell XDP cannot currently be attached to a virtio-net interface on any openvmm guest. NETIF_F_GRO_HW is likewise only exposed to the guest when the control feature is offered, so the kernel also cannot disable lossy receive coalescing when an interface joins a bridge.

Where I got to

The backend half works: the tap now applies a changed offload mask to live queues rather than only at the next get_queues, since the guest changes this mid-traffic and a mask that waits for a queue restart leaves the tap coalescing frames the guest has stopped accepting. That needed no new API.

The device half is where I stopped, because persisting the mask needs somewhere to put it.

Questions

1. Is a device-level save/restore hook planned, or is device state deliberately out of scope for now?
2. If a PR would be welcome, what shape would you want? The obvious one is a method pair on VirtioDevice returning a mesh Protobuf payload that the transport stores alongside CommonSavedState, with devices free to return nothing.

We need this on our side regardless, so I am going to implement something. I would rather build it in the shape you would accept, so it can come back as a PR instead of turning into a permanent divergence - which is why I am asking before writing it rather than after.

Start by reviewing VirtioDevice in vm/devices/virtio/virtio/src/device.rs and the transport's CommonSavedState and queue save/restore flow. Determine the accepted shape for device-owned state, with completion requiring runtime control-queue state to survive restore and the backend to be re-programmed consistently.