microsoft / microsoft/openvmm

virtio-net: mergeable receive buffers are never offered, so a Linux guest posts 19 buffers per received packet

Open
#4,035 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
1.9k
Forks
238
Avg merge
1d 15h
Merged PRs (30d)
100

Description

virtio-net advertises GUEST_TSO4/GUEST_TSO6 and GUEST_USO4/GUEST_USO6 but never
VIRTIO_NET_F_MRG_RXBUF. The feature name exists as an unused field in
NetworkFeaturesBank0 (vm/devices/virtio/virtio_net/src/lib.rs) and appears nowhere else
in the virtio tree.

A Linux guest that negotiates the guest-side segmentation features without mergeable receive
buffers falls back to its big-packets receive layout, which posts MAX_SKB_FRAGS + 2 buffers
for every received packet. I measured 19 on a 6.x guest by logging the payload length of each
receive chain.

That multiplies every per-descriptor cost by 19. It also costs one GuestMemory::subrange per
packet, which allocates an Arc, because a chain that long arrives as an indirect table.

For a sense of scale: removing an unrelated per-packet allocation and a redundant descriptor
read from the same path (#4034) recovered roughly 18 to 23 percent of the drained packet rate
on a saturated receive queue. The 19x amplification is the larger remaining factor there.

Offering the feature is not just setting the bit. The device would have to spread one received
packet across several buffers and report the count in num_buffers, and the BufferAccess
contract in net_backend pre-assigns exactly one buffer per RxId: the backend asks
capacity(id), then writes the whole frame into that buffer, with no way to acquire more part
way through the write. The spec expects the device to take buffers as it needs them while
filling.

One shape that would leave net_backend untouched: hold queued chains in a free list inside
virtio-net's own pool, have capacity() report the aggregate, and let the write consume as
many chains as the frame needs, setting num_buffers to the count. Three constraints come
with it. The device advertises VIRTIO_F_IN_ORDER, so chains would have to be consumed and
completed strictly in available order. fill_ready must not report more RxIds than it can
back. And push_guest_addresses, which the MANA path uses for zero copy, only knows its
segments once the chains are chosen.

I plan to implement this, roughly along the lines above. Raising it first in case you would
rather it took a different shape, or in case there is a reason the feature was left out that I
am missing.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in vm/devices/virtio/virtio_net/src/lib.rs, then read the BufferAccess contract in net_backend and the fill_ready and push_guest_addresses paths. Confirm how queued chains, available-order consumption, and MANA zero-copy addressing interact before choosing the design. Done means negotiating VIRTIO_NET_F_MRG_RXBUF, distributing a frame across queued buffers, reporting num_buffers, and respecting the stated ordering and capacity constraints.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.