raystack / raystack/frontier

Add OpenTelemetry tracing

Open
#1,875 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
344
Forks
47
Avg merge
4d 4h
Merged PRs (30d)
26

Description

Frontier has logs and Prometheus metrics, but no request tracing. Metrics show counts and averages; they cannot show where one slow request spent its time.

What exists today
  • The otelconnect interceptor already collects per-RPC metrics through the OTel SDK and exports them to Prometheus (pkg/server/server.go). Tracing is explicitly disabled there with WithoutTracing().
  • Every Postgres query goes through Client.WithTimeout in pkg/db/db.go, which records Prometheus latency histograms.
  • Every SpiceDB call goes through internal/store/spicedb/relation_repository.go.
  • otelgrpc and otelhttp are already indirect dependencies.
Proposal
  • Add an OTel TracerProvider with an OTLP exporter, configured via a new config block: enabled flag, endpoint, sampling ratio.
  • Remove WithoutTracing() from the otelconnect interceptor so each RPC gets a span.
  • Add child spans for Postgres queries in WithTimeout, or adopt otelsqlx (an existing TODO in pkg/db/db.go asks for this).
  • Add spans for SpiceDB calls in the relation repository, or add the otelgrpc client interceptor to the SpiceDB connection.

With this, one slow request shows its timing breakdown — interceptors, each SQL query, each SpiceDB check — and traces can go to any OTLP backend.

Follow-up to #1783, where moving from New Relic to OpenTelemetry was suggested.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/server/server.go, where the otelconnect interceptor disables tracing, then read pkg/db/db.go and internal/store/spicedb/relation_repository.go for the database and SpiceDB call paths. Review the existing OTel dependencies and the TODO in pkg/db/db.go before choosing the exporter and span approach. Done means configurable OTLP tracing covers RPCs, Postgres queries, and SpiceDB calls with the requested timing breakdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql, prometheus
Domain
backend, databases, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.