filecoin-project / filecoin-project/boost
Add opt in metrics
- Dominant language
- Go
- Stars
- 121
- Forks
- 79
- Avg merge
- 10d 14h
- Merged PRs (30d)
- 1
Description
It would be helpful for debugging and improvements if we were able to collect metrics from Boost, such as deal success rates, errors, system information, etc.
This should be opt in only, and we could add this to the settings page with clear information about what we're collecting and where it's being collected.
---
In order to increase visibility into Boost and understand how Boost is used and what its performance is on SPs systems, we'd like to add functionality to Boost so that SPs can opt-in and periodically send statistics and metrics to a centralized system maintained by the development team.
We would then be able to track a measure a lot of data that is not available on-chain, such as:
- deal failure rates (after acceptance) prior to PublishStorageDeals message
- CommP benchmarks and performance
- various retrieval errors
- etc.
Given that Boost is exposed on the internet, the opt-in metrics could be implemented as either `push-` or `pull-` based.
## Opt-in push metrics
As an example of `push-` based system we could implement something similar to `go-ethereum`: https://github.com/ethereum/go-ethereum/tree/master/metrics
Each SP would then periodically emit metrics to our centralized time-series db backend.
## Opt-in pull metrics
As an example of `pull-` based system we could implement a `metrics` package similar to Lotus: https://github.com/filecoin-project/lotus/blob/master/metrics/metrics.go
Additionally to the `metrics` package, we'd need a way to `announce` and `discover` every endpoint for an SP who has enabled opt-in metrics, so that we know to `pull` them periodically.
## Issue Tracking
- [ ] https://github.com/filecoin-project/boost/issues/721
- [ ] https://github.com/filecoin-project/boost/issues/722
- [ ] Monitor retrieval reliability
## Refs
* https://github.com/ipfs/go-bitswap/pull/562
Contributor guide
Research direction
No implementation files or tests are named. Start by reviewing the linked merged pull request 809 and comparing the push-based go-ethereum and pull-based Lotus examples; done means agreeing on an approach, documenting the collected data and destination, and providing an opt-in setting with clear user-facing disclosure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100