Reducing allocations for transfer-heavy protocols by using borrowed data in protobuf files

Description

This is inspired by PRs from @joshuef (#4751, #4753, #4754).

The idea is to use Bytes to avoid potentially costly clones of large chunks of memory as we are processing the messages. For gossipsub, a single message might be sent to multiple clients. For kademlia, we might serve the same record multiple times from the record store. In both cases, there shouldn't be a need for any additional allocations.

The main problem though is that the generated protobuf structs use Vec<u8> and thus force an allocation onto us every time we want to send one of these. https://github.com/tafia/quick-protobuf does have an option to generate Cow instead of Vec<u8> and thus allow encoding of borrowed data. In the past, we weren't able to use this because asynchronous-codec couldn't encode borrowed data. But it can now! See https://github.com/mxinden/asynchronous-codec/pull/9.

As a result, I think it is worth experimenting whether or not we can now use the generated protobuf structs.

To actually make use of this, we need to:

• Re-run pb-rs with the correct options to use Cow instead of Vec:

  pb-rs **/generated/*.proto`
  

• Work out whether or not we can adapt quick-protobuf-codec to work with the new protobuf files
• Update the CI check to not fail on the now differing files
• Actually make use of this in (all?) protocols to avoid allocations where possible

Motivation

Less memory allocations.

Current Implementation

We allocate each message again just to serialize it to the wire.

Are you planning to do it yourself in a pull request ?

No

Start with the generated protobuf files under **/generated/*.proto and the pb-rs command described in the issue, then inspect quick-protobuf-codec and the CI check for generated-file differences. Trace the gossipsub and kademlia protocols to assess where borrowed data can be used. Done means the protobufs and codec work together, CI passes, and relevant protocols avoid unnecessary allocations.