0xMiden / 0xMiden/node

Use response payload budget instead of per-field entry limits for gRPC responses

未关闭
#2,182 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
rpc store
主要语言
Rust
星标
104
派生
138
平均合并
1 天 14 小时
30 天内合并 PR
54

描述

## Background

In #2153, we added support for `GetAccount` to return all storage maps. During review, we noted that the current API limit model is still too rigid: some response paths cap individual fields by number of elements, even though tonic enforces an overall 4 MiB gRPC message size limit.

For example, storage map details currently cap individual maps at 1000 entries. That means a response with 10,000 entries from one map may be rejected/truncated, while a response with 1000 entries from each of 10 maps may be allowed, even though both have similar payload-size implications.

The review suggestion was to move away from individual field limits where the real constraint is response size.

## Current behavior

- `GetAccount` has special response budgeting for `AllStorageMaps`, but individual storage maps still use `AccountStorageMapDetails::MAX_RETURN_ENTRIES = 1000`.
- `AccountVaultDetails` also uses `MAX_RETURN_ENTRIES = 1000`.
- Shared RPC limits in `crates/utils/src/limiter.rs` use element-count limits for several endpoints, with comments tying those limits to the 4 MiB payload budget.
- Some endpoints already use payload-size-aware pagination, e.g. `SyncNotes` / `SyncTransactions`, so behavior is inconsistent across APIs.

## Problem

Per-field element caps are not a good proxy for the actual gRPC response size. They can reject valid responses that fit under 4 MiB and allow combinations that only happen to stay safe because each individual field is capped.

This makes the API less flexible than necessary and spreads response-size reasoning across endpoint-specific constants.

## Proposed direction

Refactor response-size enforcement so that endpoints whose main constraint is tonic's 4 MiB response limit apply an overall encoded-size budget instead of arbitrary per-field entry limits.

For `GetAccount`, this likely means:

- apply a single response payload budget across account code, vault details, storage header, and storage map details;
- allow any individual storage map to exceed 1000 entries if the final encoded response remains within the budget;
- return existing `too_many_entries` / `too_many_assets` markers only when including that data would exceed the response budget;
- keep stricter request/work limits where they protect query fan-out, proof generation, or database load rather than response size.

## Acceptance criteria

- `GetAccount` enforces the 4 MiB response payload budget at the response level, not via a hard 1000-entry cap per storage map.
- Existing request fan-out limits, such as explicit storage map key proof requests, remain in place where needed.
- Other RPC/store endpoints with element-count limits are audited and categorized as either:
- true request/work limits, or
- response-size limits that should become payload-budget based.
- `GetLimits` documentation/API output is updated if any exposed limit semantics change.
- Tests cover cases where:
- a single large field fits under the response budget and is returned;
- a combination of fields exceeds the response budget and is truncated/paginated/marked consistently;
- generated responses stay below tonic's 4 MiB limit.

贡献指南

打开贡献指南

调研方向

该 issue 指向用于共享 RPC 限制的 crates/utils/src/limiter.rs,并提到了 GetAccount endpoint。首先检查当前按字段设置的 entry 限制(例如 AccountStorageMapDetails::MAX_RETURN_ENTRIES)以及 tonic 中 4 MiB 的 gRPC 约束。查看 SyncNotes/SyncTransactions 如何实现感知 payload 大小的分页。目标是用整体的编码后大小预算替代按字段设置的上限,确保 response 保持在 4 MiB 以下。测试应验证预算强制执行,并验证现有的 request fan-out 限制仍然保留。

由索引模型根据 Issue 内容生成。

评估

技术栈
grpc, rust
领域
api, backend
Issue 类型
重构
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
描述清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。