paritytech / paritytech/subxt

Duplicate storage iterator items using lightclient

Open
#1,743 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
489
Forks
293
Avg merge
18h 35m
Merged PRs (30d)
3

Description

Hi there,

This issue is a follow up of #1722

Fetching storage items through an iterator returns duplicates when using lightclient, hanging the loop forever with the current version of subxt. The behaviour is improved using the latest smoldot dependencies but there are still duplicates.

This isue can be confirmed when running the example below:

#![allow(missing_docs)]
use subxt::{client::OnlineClient, lightclient::LightClient, PolkadotConfig};
use std::collections::BTreeMap;

// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")]
pub mod polkadot {}

const POLKADOT_SPEC: &str = include_str!("../../artifacts/demo_chain_specs/polkadot.json");

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // The lightclient logs are informative:
    tracing_subscriber::fmt::init();

    let (_, polkadot_rpc) = LightClient::relay_chain(POLKADOT_SPEC)?;
    
    // Create Subxt clients from these Smoldot backed RPC clients.
    let api = OnlineClient::<PolkadotConfig>::from_rpc_client(polkadot_rpc).await?;

    let address = polkadot::storage()
        .bounties()
        .bounties_iter();

    let mut iter = api.storage().at_latest().await?.iter(address).await?;

    let mut data = BTreeMap::new();
    while let Some(Ok(storage)) = iter.next().await {
        let id = get_bounty_id_from_storage_key(storage.key_bytes);
        data.entry(id).and_modify(|n| *n += 1).or_insert(1);
        assert_eq!(data.len() as u32, data.iter().map(|(_, v)| v).sum::<u32>());
    }
    
    Ok(())
}

pub fn get_bounty_id_from_storage_key(key: Vec<u8>) -> u32 {
    let s = &key[key.len() - 4..];
    let v: [u8; 4] = s.try_into().expect("slice with incorrect length");
    u32::from_le_bytes(v)
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the Rust example in the report, focusing on the lightclient-backed storage iterator and its interaction with the smoldot dependencies. Trace the iterator behavior and compare results with the updated dependencies; done means storage iteration no longer yields duplicate items and the loop terminates normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.