Speed up reverts

未关闭
#1,705 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
30/100
Issue 类型
功能
描述清晰度
需要澄清
活跃度
停滞
技术栈
rust, sql

调研方向

首先对 graph-node 当前的区块回滚查询进行性能分析,并将其与按受影响实体版本 ID (vid) 缩小范围的查询进行比较。使用这些测量结果确定按区块划分的内存中数据形态和保留上限;完成的标准是测得回滚速度有所提升,并且在没有数据时仍可使用现有行为。

由索引模型根据 Issue 内容生成。

描述

area/indexing ops performance

For some subgraphs, reverting blocks is still slow. The best way to speed this up might be to restrict the queries we run to revert the block by the vid of the entity versions that are actually affected by the revert. To facilitate that, graph-node should keep a list of the vid's of entities by block in memory as it moves forward and processes blocks. That list can then be used to speed up reverts.

Even if we only keep this data for a small number of blocks (say 5), it should help in speeding up reverts already. It's ok if we do not have that data for a revert (e.g., after a cold start), we can just fall back to the current behavior. The amount of data to keep should be relatively small, as mappings typically only alter a small number of entities for each block, but we might want to limit this by only keeping the data if there are fewer than N vid's to keep for a block.

Before implementing this, we should analyze the performance of the current queries and compare it to the performance of queries including the vid of the entities affected by the rollback. That should also inform the shape of the data we keep in memory, but will likely look like this for the different operations in a specific block:

  • create: remember the vid of the new entity
  • update: remember the vid of the old and the new version of the entity
  • delete: remember the vid of the deleted entity version

During a revert, we'd then use this information to narrow down which rows in a table to change, for example the query to delete entity versions that are now in the future would become

delete from things where vid in ($vids)

where vids contains what we recorded as new versions for a create or update

主要语言
Rust
星标
3.2k
派生
1.1k
平均合并
4 天 1 小时
30 天内合并 PR
1

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

graphprotocol/graph-node 的其他 Issue

查看 graphprotocol/graph-node 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。