out_stackdriver: service account authentication broken in v5.0.0
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
## Bug Report
**Describe the bug**
The stackdriver output plugin's service account authentication is completely broken in v5.0.0. Both `google_service_credentials` config option and `GOOGLE_APPLICATION_CREDENTIALS` env var are ignored — the plugin only tries the GCE metadata server, which fails on non-GCP environments (AWS, Azure, on-prem, etc.).
This is a regression from v4.2.3, where the same config works correctly.
**To Reproduce**
Run fluent-bit v5.0.0 on any non-GCE machine with a valid GCP service account JSON key:
```ini
[INPUT]
name dummy
[OUTPUT]
name stackdriver
match *
google_service_credentials /path/to/service-account-key.json
```
Or equivalently in YAML format:
```yaml
pipeline:
inputs:
- name: dummy
outputs:
- name: stackdriver
match: "*"
google_service_credentials: /path/to/service-account-key.json
```
**Expected behavior**
fluent-bit reads the service account key from the specified file and authenticates via OAuth2 JWT, as documented in https://docs.fluentbit.io/manual/data-pipeline/outputs/stackdriver
**Actual behavior**
fluent-bit ignores the credentials file and only tries the GCE metadata server:
```
[output:stackdriver:stackdriver.0] metadata_server set to http://metadata.google.internal
[error] [output:stackdriver:stackdriver.0] error retrieving oauth2 access token
[error] [output:stackdriver:stackdriver.0] cannot retrieve oauth2 token
[warn] [engine] failed to flush chunk '...' cannot be retried
```
All log chunks are dropped after retry exhaustion.
**Comparison with v4.2.3**
Same config, same credentials file, same machine:
- **v5.0.0**: `error retrieving oauth2 access token` ❌
- **v4.2.3**: `[oauth2] access token from 'oauth2.googleapis.com:443' retrieved` ✅
**Environment**
- fluent-bit version: 5.0.0 (from `packages.fluentbit.io` apt repo, Ubuntu 24.04)
- OS: Ubuntu 24.04 (AWS EC2)
- Credential type: GCP service account key JSON (`"type": "service_account"`)
**Possible cause**
The OAuth2 core library (`flb_oauth2.h`) was significantly refactored between v4.2.3 and v5.0.0 (new `flb_oauth2_config` struct, JWT support for OpenTelemetry). The stackdriver plugin code itself was not changed, suggesting the regression is in the shared OAuth2 layer's handling of service account JWT auth.
**Workaround**
Pin fluent-bit to v4.x: `sudo apt-get install -y "fluent-bit=4.*"`
Contributor guide
Assessment
This issue has not been assessed yet.