filecoin-project / filecoin-project/boost

OpenTelemetry trace exporter does not reconnect to backend if connection is lost

Open
#812 2 comments 0 reactions 0 assignees View on GitHub
area/metrics kind/bug
Dominant language
Go
Stars
121
Forks
79
Avg merge
10d 14h
Merged PRs (30d)
1

Description

Currently if we restart the tracing setup (tempo+grafana+prometheus), we then also have to restart all services that export traces to it.

Ideally the exporters on individual services (such as boostd, booster-http, booster-bitswap) should reconnect to the tracing backend, when it becomes available.

Contributor guide

Open the contributing guide

Research direction

Start by locating the OpenTelemetry exporter setup used by boostd, booster-http, and booster-bitswap, then reproduce the issue by restarting the tempo, Grafana, and Prometheus tracing setup. Done means each service reconnects and resumes exporting traces when the tracing backend becomes available without restarting the service.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.