0xMiden / 0xMiden/node

Response size estimates are off

Offen
#2,310 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
rpc store
Vorherrschende Sprache
Rust
Sterne
104
Forks
138
Ø Merge
1 T. 13 Std.
Gemergte PRs (30 T.)
56

Beschreibung

`SyncTransactions` can return responses larger than the default tonic decode limit of 4 MB. One observed response from a pioneer was about 6 MB:

```rust
RpcError(RequestError {
endpoint: SyncTransactions,
error_kind: OutOfRange,
source: Status { code: OutOfRange,
message: "Error, decoded message length too large: found 6146702 bytes, the limit is:
4194304 bytes" }
})
```

Account ID was `0x6d37b2d4aedd697140338bb31c67e3`.

For pagination, the response uses the store's `size_in_bytes` estimate, which is obviously different than the actual protobuf-encoded response size. That estimate also undercounts the real payload: `fee` is not counted at all, unauthenticated input notes are estimated at 64 bytes but it ignores the note headers, etc. All this without counting the protobuf framing overhead.

I think @Dominik1999 also hit a similar issue on the `GetNotesById` endpoint, as part of the client's sync state process.

Overall, there are a couple of things to address:

- While the estimate is calculated based on the serialized raw structs, the payload may be filled too close to the 4 MB limit, which will cause the protobuf-encoded message to go over that due to the framing overhead
- Even so, it's clear some limits need to be reviewed because they may not be correctly counted (i.e., 4 MB vs 6 MB is a big enough difference)
- From the client, we do two things: we honor the node-communicated limits for parameters, and we keep the default 4 MB as the maximum-allowed payload size. We should probably bump this limit by 10-15% to allow for overhead if we can't enforce the limit correctly on the node.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.