api7 / api7/aisix

P1-8: Helicone / Langsmith / Datadog logs / Slack alerts observability sinks

Open
#57 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cross-repo gap-with-litellm P1
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Problem

`crates/aisix-obs/` ships Langfuse + 4 Prometheus series + a marker OTLP. LiteLLM has 30+ named integrations. Four that customers ask for most:

  1. Helicone — log every request via `POST https://api.helicone.ai/v1/log/request\` with a Bearer key.
  2. Langsmith — `POST https://api.smith.langchain.com/runs/batch\`.
  3. Datadog logs — HTTP intake `POST https://http-intake.logs.datadoghq.com/api/v2/logs\` with DD-API-KEY header.
  4. Slack alerts — webhook on threshold events (budget at 80%, error spike, etc.).

Scope

Each sink is a separate exporter behind the same async-batched-mpsc shape Langfuse uses (`crates/aisix-obs/src/langfuse.rs` is the model).

DP (moonming/ai-gateway)
  • 4 new sub-modules under `aisix-obs`: helicone.rs, langsmith.rs, datadog.rs, slack_alert.rs.
  • Each: capacity-bounded mpsc → background batcher → HTTP POST → structured retry with backoff.
  • Config (etcd-loaded): `observability: { sinks: [{kind, ...}] }`. Multiple sinks active simultaneously.
  • Slack alerts wire into the budget tracker (P0-6) and rate-limit-rejection counter for threshold logic.
CP (api7/api7ee-3-control-plane)
  • N/A initially (config in DP file). Optional later: `observability_sink` resource.
Dashboard UI (api7/AISIX-Cloud)
  • Settings page (env-scoped or org-scoped — likely org-scoped for billing-related sinks like Datadog/Slack).
  • Add/remove sinks form.
e2e
  • Mock each external endpoint, send a request, assert payload shape on the wire (especially the LLM-specific ones — Helicone payload schema is finicky).

Estimate

DP 1d/each = 4d, CP 0d, UI 1.5d, e2e 1d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/aisix-obs/src/langfuse.rs to understand the existing async-batched-mpsc exporter shape, then review the budget tracker and rate-limit-rejection counter referenced for Slack thresholds. Done means the four sinks, etcd-loaded configuration, Dashboard UI settings, and e2e mocks for each external endpoint are covered as specified.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, frontend, observability, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.