api7 / api7/aisix

aisix-obs: object_store sink ignores `endpoint` for GCS — wire `with_base_url` or drop the schema allowance

Open
#585 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug cross-repo P1
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Summary

In crates/aisix-obs/src/sink/object_store.rs, build_object_store's GCS arm never consumes the endpoint parameter — only bucket + the service-account key. The S3 arm (with_endpoint, path-style, allow-http for loopback) and the Azure arm (MicrosoftAzureBuilder::with_endpoint, Azurite) both honor it.

The comment there says GCS "has no HTTP-endpoint override on the builder" — that's stale: object_store 0.13 (the pinned version) exposes GoogleCloudStorageBuilder::with_base_url and GoogleConfigKey::BaseUrl (google_base_url), which take precedence over the service-account JSON's gcs_base_url.

Meanwhile the watch-path config schema still allows endpoint for any provider and its loopback pattern explicitly allowlists fake-gcs-server / fake-gcs hostnames — an allowance nothing can use while the GCS arm ignores the field. A configured endpoint on a gcs exporter is silently dead today.

Options

(a) Wire itendpointwith_base_url in the GCS arm, mirroring the S3/Azure arms (incl. allow_http for the loopback emulator hosts). Gives provider symmetry and makes the fake-gcs-server emulator path work through the normal config field instead of riding gcs_base_url inside the credential JSON.

(b) Make the ignore official — remove fake-gcs-server / fake-gcs from the schema's endpoint pattern, document that GCS emulator/private base URLs ride the service-account JSON's gcs_base_url, and reject (or keep ignoring, but document) endpoint for gcs at config validation.

Either way, the stale "no host override" comment in the GCS arm should be corrected.

Notes

  • gcs_base_url in the service-account JSON is real and honored (object_store ServiceAccountCredentials.gcs_base_url → client base URL), so the current workaround stands regardless.
  • If (a) lands, control-plane-side validation that assumes endpoint-is-ignored-for-gcs can be relaxed in step.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/aisix-obs/src/sink/object_store.rs at build_object_store and compare the GCS, S3, and Azure arms, then inspect the watch-path schema and its endpoint validation. Review the object_store 0.13 APIs and choose whether GCS endpoints should be wired or rejected and documented. Done means the stale comment and related validation consistently reflect the chosen behavior, including the fake-gcs emulator path.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, rust
Domain
cloud, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.