BigQuery exporter for usage and audit logs
@moonming is already working on this.
Since Jul 15, 2026.
- Dominant language
- Rust
- Stars
- 157
- Forks
- 32
- Avg merge
- 1h 25m
- Merged PRs (30d)
- 145
Description
Description
There is no BigQuery sink: no BigQuery client and no use of the Storage Write API anywhere in the tree. The GCP-adjacent paths today are the object-store NDJSON usage export and the scheduled CSV audit export, so reaching BigQuery means landing NDJSON in GCS and defining external tables over it.
Requested: a first-class BigQuery exporter over the Storage Write API —
- dataset and table configuration, service-account or Workload Identity auth;
- batched append with retry and backpressure, behind the same async-batched exporter pipeline as the other sinks;
- a documented, stable schema for usage records (model, caller, timestamps, token breakdown, cost, latency, status) and for audit records.
Why
For organisations on GCP, BigQuery is where token-spend chargeback, per-team cost attribution and audit queries already live — every other data source they have lands there directly. Routing through object storage plus external tables adds a hop, hands the operator a pipeline to own, and lets schema drift break things quietly.
This is a sibling of the other named sinks (#57, #62) and would slot into the pluggable-sink work tracked in api7/AISIX-Cloud#692.
Priority
High.
Prior art
| Product | Has it | Reference |
|---|---|---|
| LiteLLM | Partial | gcs_pubsub → BigQuery, user wires the subscription |
| Cloudflare AI Gateway | Partial | BigQuery Logpush destination; AI Gateway → BigQuery undocumented |
| Bifrost | Partial | Storage Write API writer (enterprise; its own docs contradict this) |
| Portkey | No | logs export is a JSONL pull |
| Kong AI Gateway | No | no BigQuery in the logging plugins |
| Envoy AI Gateway | No | usage via access logs only |
| TrueFoundry AI Gateway | No | export is an email request |
No peer clearly ships a first-class BigQuery sink; the two closest are unconfirmed for LLM traffic.
Surveyed 2026-07-15; every claim rests on a fetched docs/source page.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.