filecoin-project / filecoin-project/lily
Verify the accuracy of visor data in the analysis schema
- Dominant language
- Go
- Stars
- 52
- Forks
- 47
- PR merge metrics
- No merged PRs in 30d
Description
### Description
How would we verify that a complete extraction of chain data via visor would allow us to run aggregation queries that gave the same answer as the filecoin state tree at a given epoch? Is that even a sensible goal for the analysis db? Should users be able to run financial reports on the analysis db and expect to get the same answer as if they had queried a lotus node?
Is our goal reasonably high-precision (for graphing trends) or perfect convergence (for financial reports)? We should figure out what is possible and make a decision on it to help inform where to put our efforts.
Either way it would be useful for us to have a verification process that allows us to put a figure on how accurate the collected data is. We could show some aggregations distance from the chain state or similar. Other ideas are here needed, and this issues is intended to drive discussion at this stage.
### Acceptance criteria
- We can put a metric on how accurate the collected data is over a given window.
- this would be useful for automatically catching issues like #354
- Some tables can only be meaningfully aggregated over if they have full extractions from gensis, and we should identify which those are.
- We publish a position on what sorts of analysis are appropriate for sentinel data. (trends vs transaction reports)
### Where to begin
Discuss it!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.