graphprotocol / graphprotocol/graph-node
Rules-based system to control index creation and similar attributes
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.2k
- Forks
- 1.1k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
There are a few things that can be set manually through graphman or directly by manipulating the database that affect how subgraph data is stored, and can help query performance. Those items include:
- custom indexes (multicolumn indexes or partial indexes)
- creation of default attribute indexes (a lot of them are never used)
- account-like optimization
- statistics target
- amount of history
An example of what that could like like in YAML form is
# Sketch of what an index definition file could look like. The file will
# contain many entries like this one
- rule: Messari analytics subgraphs
match:
# All subgraphs whose name matches this regex will be treated according
# to these actions
name: messari/.*
actions:
# Keep 10,000 blocks of history
- history: 10000
- entity:
name: LiquidityPoolSnapshot
index:
- on: [pool, timestamp]
- entity:
name: LiquidityPool
# Turn off attribute indexes
default_indexes: false
# Set the table to account-like
account-like: true
# Increase the statistics target
stats-target: 300
# Create some custom indexes
index:
# Use a GiST index since we use block_range
- on: [block_range, timestamp, id]
method: gist
# A partial index that only covers blocks after a certain point
# TODO: since the block number means different things on
# different networks, we should either use a timespan or let users
# match on network
- on: [timestamp, id]
after: 15664000
- on: [pool_address, block_range, timestamp]
method: gist
We need to think more about the way in which operators interact with these rules. Some requirements:
- changing rules should not require a restart of
graph-node - it should be possible to list rules, delete them etc.
- rules should be applied when a subgraph is deployed, but there should also be tooling to apply a new rule to an existing deployment.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Use graphman and the existing database-manipulation paths as entry points; first map how current index, history, and attribute settings are represented and applied. Define how rules are listed, deleted, reloaded without restarting graph-node, applied at deployment, and applied to existing deployments; done means these workflows are specified and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100