beyond-all-reason / beyond-all-reason/RecoilEngine
Make network/replay protocol forward-compatible
- Dominant language
- C++
- Stars
- 679
- Forks
- 290
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 40
Description
Replays are just a log of network traffic so any change to the network protocol breaks replay parsers. Design some way to let us make changes to the protocol without ruining parsers.
I am unfamiliar with replay parsers so I don't even know the main hurdles (contacting some existing replay parser
maintainers and figuring that out would be part of the task).
The obvious problem is that there isn't any sort of versioning or anything. In theory you can look at the engine version but there is nothing that tells you what packet types exist in any given version so you'd have to hardcode it. A new packet type with a version and perhaps some extra metadata (see below) sounds good to have.
I assume that the main practical hurdle would be that an unknown packet type has an unknown length, so you don't know how much to skip.
Here's an **example** solution: add a new packet type, say NETMSG_INIT, with the following current signature: `u16 length, u8 packet_length_pairs, vector > packet_type_to_packet_length`. Any packet length that is specified as 0 must start with u16 length.
Say we later add three new packet types with IDs 101, 102, and 103 where the first two are constant lengths 2 and 5 bytes and the third is dynamically sized. And we added some extra metadata to INIT (in the example below that's the 3 bytes with value 123). So the INIT packet would look like this:
```
12 3 101 2 102 5 103 0 123 123 123
│ │ │ │ │
│ │ │ │ └ some unknown future metadata
│ │ │ └ packet 103 is dynamically sized and must have u16 length at the start
│ │ └ packet 101 is 4 bytes, and packet 102 is 5 bytes
│ └ there are 3 packets since the introduction of NETMSG_INIT, what follows are their sizes
└ the init packet packet is 12 bytes (we can read 9 of those, there's 3 unknown bytes which we can skip)
```
This way a parser always knows the lengths of each packet (const-size from the listing in init, and dynamic from the mandatory length field) so anything unknown can be skipped, and we also leave space in init for more metadata in the future (like [this](https://github.com/beyond-all-reason/spring/issues/2042)).
Contributor guide
Research direction
Start by locating the network/replay protocol implementation and contacting existing replay-parser maintainers, since no file or parser entry point is named. Compare current packet framing with the proposed NETMSG_INIT metadata and define how unknown packets can be skipped; done means an agreed forward-compatible design covering packet lengths and future metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100