apache / apache/opendal

feat: opendal should support gcs `authorized_user` credential

Open
#6,287 12 comments 5 reactions 0 assignees View on GitHub
enhancement services/gcs
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

### Describe the bug

GCS not work, and `Operator::check` returns an `IncompleteMessage` error.

### Steps to Reproduce

```rust
pub fn create_operator(bucket: impl AsRef) -> Result {
let builder = services::Gcs::default().bucket(bucket.as_ref());
Ok(Operator::new(builder)?.finish())
}

#[tokio::test]
async fn basic_probe() -> Result<()> {
let op = create_operator("xxx-datasets").unwrap();
dbg!(op.check().await);
Ok(())
}
```
Just block for 10 seconds or so,
```sh
[test/probe.rs:18:5] op.check().await = Err(
Error {
kind: Unexpected,
message: "loading credential to sign http request",
status: Temporary,
operation: "list",
context: [
(
"called",
"reqsign::LoadCredential",
),
(
"service",
"gcs",
),
(
"path",
"/",
),
(
"listed",
"0",
),
],
source: Some(
reqwest::Error {
kind: Request,
source: hyper_util::client::legacy::Error(
SendRequest,
hyper::Error(
IncompleteMessage,
),
),
},
),
},
)
```

### Expected Behavior

The operator of GCS can work correctly.

### Additional Context

Add `trace_subscriber` and got:
```log
running 1 test
2025-06-12T05:56:50.245403Z DEBUG ThreadId(02) opendal::services::gcs::backend: 241: backend build started: GcsBuilder { config: GcsConfig { root: None, bucket: "xvu-datasets", endpoint: None, scope: None, .. }, .. }
2025-06-12T05:56:50.245487Z DEBUG ThreadId(02) opendal::services::gcs::backend: 244: backend use root /
2025-06-12T05:56:50.245499Z DEBUG ThreadId(02) opendal::services::gcs::backend: 263: backend use endpoint: https://storage.googleapis.com
2025-06-12T05:56:50.264007Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/
2025-06-12T05:56:52.004969Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/
2025-06-12T05:56:55.422559Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/
2025-06-12T05:57:00.092319Z DEBUG ThreadId(02) reqwest::connect: 788: starting new connection: http://metadata.google.internal/
[test/probe.rs:18:5] op.check().await = Err(
Error {
kind: Unexpected,
message: "loading credential to sign http request",
status: Temporary,
...
```

It looks like failing to connect a strange address `http://metadata.google.internal/ `

### Are you willing to submit a PR to fix this bug?

- [ ] Yes, I would like to submit a PR.

Contributor guide

Open the contributing guide

Research direction

Start with the reproduction at test/probe.rs:18 and trace GCS credential loading through reqsign::LoadCredential, focusing on authorized_user credentials and the metadata endpoint fallback. Reproduce the failing Operator::check call, then verify that a GCS operator using authorized_user credentials completes the check without the IncompleteMessage error.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, rust
Domain
authentication, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.