iotaledger / iotaledger/iota

feat(graphql): expose available range of data on each query

Open
#12,840 0 comments 0 reactions 1 assignee View on GitHub

@tomxey is already working on this.

Since Sep 4, 2026.

infrastructure
Dominant language
Rust
Stars
110
Forks
71
Avg merge
3d 4h
Merged PRs (30d)
129

Description

Part of #12409.

Add the available range to the GraphQL connections that can return an incomplete result because data is missing from the database.

Which connections

Three types. Each is a single type in the schema, so one field on each covers every field that returns it.

Connection Fields that return it Tables that bound the result
TransactionBlockConnection 12 fields, including Query.transactionBlocks, Address.transactionBlocks, Epoch.transactionBlocks, Object.receivedTransactionBlocks — all go through TransactionBlock::paginate tx_global_order plus the lookup table for the filter: tx_senders (sentAddress), tx_recipients (recvAddress), tx_senders and tx_recipients (affectedAddress), tx_input_objects, tx_changed_objects, tx_wrapped_or_deleted_objects, tx_calls_pkg/_mod/_fun (function), tx_kinds (kind); transactions when no filter is given; and checkpoints when afterCheckpoint or atCheckpoint is given, because the checkpoint bounds are turned into transaction sequence numbers by reading that table
EventConnection Query.events, TransactionBlockEffects.events, GenesisTransaction.events events plus the lookup table for the filter: event_senders (sender), event_struct_package/_module/_name/_instantiation (eventType), event_emit_package/_module (emittingModule)
CheckpointConnection Query.checkpoints, Epoch.checkpoints checkpoints

TransactionBlockConnection comes from our own ScanConnection (src/connection.rs), which is only ever used for transaction blocks. EventConnection and CheckpointConnection come from async_graphql::connection::Connection, which carries extra fields through Connection::with_additional_fields.

TransactionBlockEffects.events and GenesisTransaction.events list the events of a single transaction that we already hold, so there is no range to report there. The field has to be nullable so those two can leave it out.

Also in scope

Two things that make the new field usable:

  • A transactions or events query whose whole range has been pruned returns an empty connection today, which looks the same as no matches. TxBounds::query returns None when the checkpoint row for the lower bound is gone, and Event::paginate has no lower bound check at all. Both should report that the data is pruned, like Query.checkpoints already does.
  • Query.availableRange is described as the range of checkpoints the RPC has data for, but it returns the range for consistent object queries only: the objects_backward_history bound, further limited by max_available_range (9000 checkpoints by default). That is much shorter than the range for transactions, events and checkpoints. Fix the wording so the two are not mixed up.
Not in scope
  • Consistent object queries: ObjectConnection, MoveObjectConnection, BalanceConnection, CoinConnection, StakedIotaConnection, NameRegistrationConnection, DynamicFieldConnection. They already fail with "Requested data is outside the available range" when the checkpoint is outside the window, and Query.availableRange already reports that window.
  • Single object, transaction and checkpoint lookups. They already return an error saying the data was pruned when the historical fallback is not configured.
  • Connections built from data already in hand (transaction inputs, object changes, balance changes, Move modules, validators and so on), and MovePackageConnection, which reads packages and is always complete.
Where the value comes from

The watermarks table, which the indexer keeps in memory. GraphQL already reads it through db.inner.watermark_cache() (see types/checkpoint.rs), and get_lowest_available_cp_for_tables returns the bound for a set of tables.

Take the bound from the watermark rows of the tables a query reads, rather than from a fixed list of the tables the pruner deletes from. Two things write those rows:

  • the pruner, for the tables it prunes
  • a restore from snapshot, which sets a lower bound for every prunable table and also for objects_version, because the restore keeps only the object versions that were live at the end of the snapshot epoch

The upper bound is the checkpoint the query is viewed at, so the existing AvailableRange type fits. Its description needs rewording to cover both uses.

Work
  • add the field to ScanConnection, and to the event and checkpoint connections
  • map the GraphQL transaction and event filters to the tables they read. IndexerReader::tx_tables_for_filter does this for the JSON-RPC filter types, so the function cannot be reused, but the table sets can be shared
  • report pruned data instead of an empty connection
  • update schema.graphql and its snapshot test
  • tests against a pruned database
Open questions
  • whether to report a wider range when the historical fallback is configured
  • what to report while the watermarks are still zero after a cold start (#11828)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.