crypto-com / crypto-com/chain-indexing
Problem: Current projection design is not scalable
- Dominant language
- Go
- Stars
- 34
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
Currently, as Crypto.org Chain has more than 3 million blocks, it takes a long time for a newly indexing server to catch up the latest block height.
This will force us to spend a long time in:
- re-indexing existing projections
- deploying and testing newly introduced projections
One possible solution is to make use of `EVENT_STORE` mode. I think a more efficient way to catch up would be:
1. `EVENT_STORE` mode indexing server starts to fetch block data and write to DB.
2. Each projection creates multiple goroutines, to parallel handle these blocks.
3. After catching up to the latest height, each projection may only need one goroutine to continue working.
The difficulty here I think is mainly on Step 2, as for some projections, we have the assumption that the blocks come in an ordered manner. So to fulfill the parallel handling purpose, these projections might need refactoring.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.