0xMiden / 0xMiden/node

Response size estimates are off

Aperta
#2,310 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
rpc store
Lingua principale
Rust
Stelle
104
Fork
138
Merge medio
1g 13h
PR unite (30g)
56

Descrizione

`SyncTransactions` can return responses larger than the default tonic decode limit of 4 MB. One observed response from a pioneer was about 6 MB:

```rust
RpcError(RequestError {
endpoint: SyncTransactions,
error_kind: OutOfRange,
source: Status { code: OutOfRange,
message: "Error, decoded message length too large: found 6146702 bytes, the limit is:
4194304 bytes" }
})
```

Account ID was `0x6d37b2d4aedd697140338bb31c67e3`.

For pagination, the response uses the store's `size_in_bytes` estimate, which is obviously different than the actual protobuf-encoded response size. That estimate also undercounts the real payload: `fee` is not counted at all, unauthenticated input notes are estimated at 64 bytes but it ignores the note headers, etc. All this without counting the protobuf framing overhead.

I think @Dominik1999 also hit a similar issue on the `GetNotesById` endpoint, as part of the client's sync state process.

Overall, there are a couple of things to address:

- While the estimate is calculated based on the serialized raw structs, the payload may be filled too close to the 4 MB limit, which will cause the protobuf-encoded message to go over that due to the framing overhead
- Even so, it's clear some limits need to be reviewed because they may not be correctly counted (i.e., 4 MB vs 6 MB is a big enough difference)
- From the client, we do two things: we honor the node-communicated limits for parameters, and we keep the default 4 MB as the maximum-allowed payload size. We should probably bump this limit by 10-15% to allow for overhead if we can't enforce the limit correctly on the node.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.