api7 / api7/aisix

BigQuery exporter for usage and audit logs

Open
#764 0 comments 0 reactions 1 assignee View on GitHub

@moonming is already working on this.

Since Jul 15, 2026.

enhancement
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Description

There is no BigQuery sink: no BigQuery client and no use of the Storage Write API anywhere in the tree. The GCP-adjacent paths today are the object-store NDJSON usage export and the scheduled CSV audit export, so reaching BigQuery means landing NDJSON in GCS and defining external tables over it.

Requested: a first-class BigQuery exporter over the Storage Write API —

  • dataset and table configuration, service-account or Workload Identity auth;
  • batched append with retry and backpressure, behind the same async-batched exporter pipeline as the other sinks;
  • a documented, stable schema for usage records (model, caller, timestamps, token breakdown, cost, latency, status) and for audit records.

Why

For organisations on GCP, BigQuery is where token-spend chargeback, per-team cost attribution and audit queries already live — every other data source they have lands there directly. Routing through object storage plus external tables adds a hop, hands the operator a pipeline to own, and lets schema drift break things quietly.

This is a sibling of the other named sinks (#57, #62) and would slot into the pluggable-sink work tracked in api7/AISIX-Cloud#692.

Priority

High.


Prior art

Product Has it Reference
LiteLLM Partial gcs_pubsub → BigQuery, user wires the subscription
Cloudflare AI Gateway Partial BigQuery Logpush destination; AI Gateway → BigQuery undocumented
Bifrost Partial Storage Write API writer (enterprise; its own docs contradict this)
Portkey No logs export is a JSONL pull
Kong AI Gateway No no BigQuery in the logging plugins
Envoy AI Gateway No usage via access logs only
TrueFoundry AI Gateway No export is an email request

No peer clearly ships a first-class BigQuery sink; the two closest are unconfirmed for LLM traffic.

Surveyed 2026-07-15; every claim rests on a fetched docs/source page.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.