Feature Request: Lazy contract state queries via RPC

Feature Request: Lazy contract state queries via RPC

Describe the proposed feature:

A new RPC endpoint midnight_queryContractState that allows clients to read specific fields and collection entries from a contract's state without downloading the full serialized blob.

Each query is a path of serialized AlignedValue keys, mirroring the VM's idx instruction. The key interpretation depends on the current node type:

• Array: key converted to u8 index
• Map/Set: key used directly for HashMap::get
• MerkleTree: key converted to u64 leaf position

{
  "method": "midnight_queryContractState",
  "params": [
    "<contract_address>",
    [
      { "path": ["0108"] },                // read Cell at Array[1] (u8(1) = "0108")
      { "path": ["4001", "412a41"] },       // look up Map entry: Array[0] → Map[Fr(42)]
      { "path": ["4001", "412a41", "0108"] } // nested: Array[0] → Map[Fr(42)] → Array[1]
    ],
    null  // optional block hash
  ]
}

Each path element is a hex-encoded serialized AlignedValue. The node navigates the state tree step by step, interpreting each key based on the StateValue variant at that level. If the final value is a collection (Map, Set, MerkleTree), the query returns an error, the client must provide a deeper path.

Response:

{
  "result": [
    { "query": { "path": ["0108"] }, "value": "6d6964..." },
    { "query": { "path": ["4001", "412a41"] }, "value": "6d6964..." },
    { "query": { "path": ["4001", "412a41"] } }  // not found (no value, no error)
  ]
}

• value present: found, hex-encoded tagged_serialize'd StateValue
• value absent, no error: key not found
• error present: query failed (bad path, invalid key encoding, etc.)
• value and error fields are omitted from JSON when None

User Need:

Currently, reading any part of a contract's state requires downloading the entire state blob via midnight_contractState. This works for small contracts, but as collection fields grow, the full-blob approach becomes impractical:

(Measured on a devnet with node v0.22.1 and a Map<Field, Bytes<32>> contract)

Clients that only need a single field or map entry (indexers, dApp frontends, monitoring tools) are forced to download and deserialize the full blob. The midnight-js SDK also downloads the full blob from the indexer for every transaction it prepares, and as contract state grows, the SDK spends increasingly more time downloading and deserializing it.

Expected Benefit:

• Clients can read individual fields or look up specific collection entries in O(log n) instead of downloading the full O(n) blob
• Reduces bandwidth and CPU for both the node and consumers
• Enables efficient state monitoring for contracts with large collections
• Supports batch queries (multiple paths in a single call)
• Handles arbitrary nesting (Map inside Array, struct fields inside Map values)
• Backward compatible, midnight_contractState remains unchanged

Details:

The navigation model mirrors the existing VM idx instruction (onchain-vm/src/vm.rs), which already navigates StateValue trees using AlignedValue keys for Array, Map, and MerkleTree access. The RPC endpoint applies the same logic without the VM's cache tracking and gas metering.

The core navigation function (idx) can be implemented in midnight-onchain-state alongside StateValue, making it reusable by any consumer. The RPC endpoint deserializes each path step from bytes to AlignedValue, calls idx in a loop, and tagged_serializes the final leaf value.

Everything can be implemented in midnight-node. The query logic could be simplified if midnight-ledger exposed a generic idx/idx_path helper in midnight-onchain-state, since that's where StateValue and the collection types are defined.

Start with the VM idx implementation in onchain-vm/src/vm.rs and the StateValue and collection types in midnight-onchain-state. Trace how midnight-node exposes RPC endpoints and how midnight-ledger provides the state data. Done means a batch midnight_queryContractState endpoint navigates the requested paths, returns leaf values or not-found results, and reports invalid queries without changing midnight_contractState.