airbytehq / airbytehq/airbyte

[builder] OAuth Authenticator Misconfiguration for client_credentials Grant Type in DeclarativeSource

Đang mở
#46,903 12 bình luận 1 reaction 0 người được giao Xem trên GitHub
area/connector-builder community team/marketplace team/platform-move type/bug
Ngôn ngữ chính
Python
Star
22.1k
Fork
5.4k
Merge trung bình
5 giờ
Pull request đã merge (30 ngày)
671

Mô tả

### Helm Chart Version

I am unsure of the Helm Chart version, as this information is not explicitly mentioned in the Airbyte UI or the setup guide I followed. Could you please provide instructions on how to retrieve this information?

### What step the error happened?

On deploy

### Relevant information

### **Bug Report: OAuth Authenticator Misconfiguration for `client_credentials` Grant Type in DeclarativeSource**

#### **Helm Chart Version**:
I am unsure of the Helm Chart version, as this information is not explicitly mentioned in the Airbyte UI or the setup guide I followed. Could you please provide instructions on how to retrieve this information?

#### **What is the Helm Chart App version you're using?**
As noted above, I am not sure how to determine the Helm Chart App version for Airbyte. Any guidance on how to find this would be helpful.

#### **What step did the error happen?**
The error occurred during the initial connection test and subsequent data sync attempt while using the `OAuthAuthenticator` with the `client_credentials` grant type in a custom Airbyte source connector. I am using the declarative YAML configuration for a REST API integration.

The connection was supposed to authenticate using `client_credentials`, but Airbyte is erroneously expecting a `refresh_token`, which is not applicable for the `client_credentials` flow.

#### **Relevant Information**
1. **Context**:
I am setting up a custom Airbyte connector using `DeclarativeSource` and `OAuthAuthenticator` with the `client_credentials` grant type. The API I am connecting to does not issue `refresh_token`s, as it follows the `client_credentials` flow. I expect the connector to request a new `access_token` using `client_id` and `client_secret` directly whenever needed without involving a `refresh_token`.

2. **The problem**:
Despite configuring the OAuth flow with the `client_credentials` grant type, Airbyte throws an error during stream initialization, stating that the `OAuthAuthenticator` needs a `refresh_token` parameter, which is not required for this grant type. This is a critical misconfiguration for the `client_credentials` flow, which never involves a `refresh_token`.

3. **Expected Behavior**:
The `OAuthAuthenticator` should function correctly with the `client_credentials` grant type without requiring a `refresh_token`. The authenticator should fetch a new `access_token` using the `client_id` and `client_secret` when the current one expires.

4. **Steps to Reproduce**:
- Setup a custom Airbyte declarative YAML configuration using `DeclarativeSource` and `OAuthAuthenticator`.
- Configure the `OAuthAuthenticator` with the following setup:
- Grant type: `client_credentials`
- No `refresh_token` should be provided as it is not part of the `client_credentials` flow.
- Attempt a connection test or data sync.
- Observe that Airbyte throws an error, expecting a `refresh_token` even though it shouldn't.

Here is the configuration I used:
```yaml
version: 5.12.0

type: DeclarativeSource

check:
type: CheckStream
stream_names:
- demographics

definitions:
streams:
demographics:
type: DeclarativeStream
name: demographics
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_size_option:
type: RequestOption
field_name: resultsPerPage
inject_into: body_json
page_token_option:
type: RequestOption
field_name: page
inject_into: body_json
pagination_strategy:
type: PageIncrement
page_size: 100
start_from_page: 1
requester:
$ref: "#/definitions/base_requester"
path: ***********/query
http_method: GET
request_body_json:
top: 1
name: Demographics
page: 1
sort:
- ProviderID: ASC
fields:
- ProviderID
- FirstName
- LastName
- Degree
- Department1
- Specialty
- Office1Address
- Office1AddressCity
- Office1AddressZip
- Office1AddressTelephone
- NPI
- Gender
- Content
filter:
OnTheWeb:
- type: in
values:
- "Y"
Specialty:
- type: not in
values:
- null
source: AdHocAPI
isArray: true
resultsPerPage: 1
dynamicFilterParameters:
ReportID: 4210c249-f397-4902-ab59-e9fcc3a2fa75
FieldNames: >-
Uid,ProviderID,FirstName,LastName,Degree,Department1,Specialty,Office1Address,Office1AddressCity,Office1AddressZip,Office1AddressTelephone,OnTheWeb,NPI,Gender,Content,LastName,FirstName
request_options_provider:
request_headers:
Content-Type: application/json
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/demographics"

base_requester:
type: HttpRequester
url_base: https://***********.api.asm-cloud.com/api/
authenticator:
type: OAuthAuthenticator
scopes:
- ***********/c3c26c9e-b3c2-4903-93bc-fca7cd50c88b
client_id: "{{ config['client_id'] }}"
client_secret: "{{ config['client_secret'] }}"
expires_in_name: expires_in
access_token_name: access_token
refresh_request_body:
grant_type: client_credentials
token_refresh_endpoint: https://***********.api.asm-cloud.com/api/tokens
# Do not specify a refresh_token, as client_credentials doesn't need it
```

5. **Workarounds Tried**:
I attempted to exclude the `refresh_token` parameter and explicitly set the `grant_type` to `client_credentials`, but the error persists. It seems the connector still expects a `refresh_token` for this OAuth grant type, which is incorrect for the `client_credentials` flow.

---

#### **Relevant Log Output**
```
Traceback (most recent call last):
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/connector_builder_handler.py", line 68, in read_stream
stream_read = handler.get_message_groups(source, config, configured_catalog, state, limits.max_records)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/message_grouper.py", line 84, in get_message_groups
stream = source.streams(config)[0] # The connector builder currently only supports reading from a single stream at a time
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 95, in streams
source_streams = [
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 96, in
self._constructor.create_component(
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 299, in _create_component_from_model
return component_constructor(model=model, config=config, **kwargs)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 1065, in create_oauth_authenticator
return DeclarativeOauth2Authenticator( # type: ignore
File "", line 17, in __init__
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/auth/oauth.py", line 79, in __post_init__
raise ValueError("OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`")
ValueError: OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`
```

I strongly believe that the connector misinterprets the `client_credentials` flow and incorrectly enforces the requirement of a `refresh_token`.

#### **Additional Notes**
This bug makes it impossible to use APIs that exclusively follow the `client_credentials` grant type, limiting the effectiveness of Airbyte in such integrations.

### Relevant log output

```shell
Traceback (most recent call last):
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/connector_builder_handler.py", line 68, in read_stream
stream_read = handler.get_message_groups(source, config, configured_catalog, state, limits.max_records)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/message_grouper.py", line 84, in get_message_groups
stream = source.streams(config)[0] # The connector builder currently only supports reading from a single stream at a time
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 95, in streams
source_streams = [
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 96, in
self._constructor.create_component(
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 299, in _create_component_from_model
return component_constructor(model=model, config=config, **kwargs)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 1065, in create_oauth_authenticator
return DeclarativeOauth2Authenticator( # type: ignore
File "", line 17, in __init__
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/auth/oauth.py", line 79, in __post_init__
raise ValueError("OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`")
ValueError: OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.