api7 / api7/aisix

docs: bring observability-exporters.md current with all three kinds (otlp_http + aliyun_sls + object_store)

Open
#540 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Problem

docs/configuration/observability-exporters.md still says "Current scope is kind: \"otlp_http\" only." (line 9) and documents only the OTLP/HTTP exporter. The feature has since grown to three kinds, all with real DP delivery + e2e coverage, and a generalized dashboard form — none of which is documented:

  • otlp_http — shipped; OTLP/HTTP-JSON traces. Backends: Tempo, Loki, Jaeger, Honeycomb, Grafana Cloud, Langfuse-via-OTLP. Dashboard one-click presets exist for Langfuse / Honeycomb / Grafana Cloud Tempo.
  • aliyun_sls — Aliyun SLS native PutLogs, with opt-in content_mode: full prompt/response capture (DP: api7/ai-gateway#528 + #532; CP/dashboard: api7/AISIX-Cloud#709).
  • object_store — batched NDJSON to S3 / GCS / Azure Blob (+ S3-compatible MinIO / Cloudflare R2) (DP: api7/ai-gateway#531; CP/dashboard: api7/AISIX-Cloud#706 + #714).

Scope

Update docs/configuration/observability-exporters.md (keep the existing structure — Current Fields / Endpoint Restriction / Runtime Model / Operator Guidance / Troubleshooting / Related Pages — and the Docusaurus sentence-case conventions). Anchor every field/value to current code, don't invent.

Add / revise:

  1. Kinds overview — replace the "otlp_http only" line with the three kinds + a one-line purpose each.
  2. Per-kind config + supported backends:
    • otlp_http: endpoint (full /v1/traces URL) + headers; the backend list above; note the dashboard presets (Langfuse / Honeycomb / Grafana Cloud Tempo) and that Tempo/Loki/Jaeger are self-hosted (enter the endpoint manually).
    • aliyun_sls: endpoint (bare region host, e.g. ap-southeast-3.log.aliyuncs.com), project, logstore, credential_ref, content_mode (metadata_only default | full), content_max_bytes (default 128 KiB, ≤ 1 MiB).
    • object_store: provider (s3/gcs/azure_blob), bucket, prefix, region, credential_ref, compression (gzip/none), optional endpoint for S3-compatible (MinIO / R2).
  3. credential_ref (the secret model) — for aliyun_sls + object_store, the cloud AccessKey is never in the config / cp-api / kine; only an opaque credential_ref the customer-deployed DP resolves locally (env / mounted secret). Document the DP-side env convention (SLS_CRED_<REF>_AK_ID / _AK_SECRET for SLS) and that the control plane stores only the reference.
  4. content_mode privacy notefull writes end-user prompt + response text into the customer's SLS; opt-in, per-field-capped, with a content_truncated marker. Make the privacy implication explicit (and note the dashboard surfaces a warning when enabling Full).
  5. Dashboard configuration flow — the multi-kind exporter form (Kind selector → per-kind field group), one-click presets (OTLP vendors + S3/GCS/Azure/MinIO/R2), and the credential-reference field (not a plaintext key). cp-api validates per-kind before persisting.
  6. Cross-links / Related Pages — link the design issues for depth.

References (anchor to these)

  • Code: crates/aisix-obs/src/otlp_http_sink.rs (otlp delivery + fan-out), crates/aisix-obs/src/sink/sls.rs (SLS PutLogs), crates/aisix-obs/src/sink/object_store.rs (object storage); crates/aisix-core/src/models/observability_exporter.rs (the config shapes).
  • Design: api7/AISIX-Cloud#687 (SLS), #689 (object storage), #688 (Datadog — future), #690 (Snowflake — future), epic #572.
  • e2e (the live contracts): e2e/cases/observability_exporter_test.go in api7/AISIX-Cloud (otlp → real OpenTelemetry Collector; SLS → real-logstore readback).

Not in this doc (future kinds — mention as roadmap only)

datadog_logs (#688), Snowflake (#690), Databricks Delta Lake (#380) are designed but not shipped — list under a short "Roadmap" note, don't document config that doesn't exist yet.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/configuration/observability-exporters.md and compare its fields with crates/aisix-core/src/models/observability_exporter.rs and the three sink files: otlp_http_sink.rs, sink/sls.rs, and sink/object_store.rs. Use e2e/cases/observability_exporter_test.go and the listed design issues to verify shipped behavior and distinguish roadmap items. Done means the existing sections cover all three kinds, credential and privacy behavior, dashboard flow, supported backends, and related links without documenting future config.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
documentation, observability-sre
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.