[builder] OAuth Authenticator Misconfiguration for client_credentials Grant Type in DeclarativeSource
- Ngôn ngữ chính
- Python
- Star
- 22.1k
- Fork
- 5.4k
- Merge trung bình
- 5 giờ
- Pull request đã merge (30 ngày)
- 671
Mô tả
### Helm Chart Version
I am unsure of the Helm Chart version, as this information is not explicitly mentioned in the Airbyte UI or the setup guide I followed. Could you please provide instructions on how to retrieve this information?
### What step the error happened?
On deploy
### Relevant information
### **Bug Report: OAuth Authenticator Misconfiguration for `client_credentials` Grant Type in DeclarativeSource**
#### **Helm Chart Version**:
I am unsure of the Helm Chart version, as this information is not explicitly mentioned in the Airbyte UI or the setup guide I followed. Could you please provide instructions on how to retrieve this information?
#### **What is the Helm Chart App version you're using?**
As noted above, I am not sure how to determine the Helm Chart App version for Airbyte. Any guidance on how to find this would be helpful.
#### **What step did the error happen?**
The error occurred during the initial connection test and subsequent data sync attempt while using the `OAuthAuthenticator` with the `client_credentials` grant type in a custom Airbyte source connector. I am using the declarative YAML configuration for a REST API integration.
The connection was supposed to authenticate using `client_credentials`, but Airbyte is erroneously expecting a `refresh_token`, which is not applicable for the `client_credentials` flow.
#### **Relevant Information**
1. **Context**:
I am setting up a custom Airbyte connector using `DeclarativeSource` and `OAuthAuthenticator` with the `client_credentials` grant type. The API I am connecting to does not issue `refresh_token`s, as it follows the `client_credentials` flow. I expect the connector to request a new `access_token` using `client_id` and `client_secret` directly whenever needed without involving a `refresh_token`.
2. **The problem**:
Despite configuring the OAuth flow with the `client_credentials` grant type, Airbyte throws an error during stream initialization, stating that the `OAuthAuthenticator` needs a `refresh_token` parameter, which is not required for this grant type. This is a critical misconfiguration for the `client_credentials` flow, which never involves a `refresh_token`.
3. **Expected Behavior**:
The `OAuthAuthenticator` should function correctly with the `client_credentials` grant type without requiring a `refresh_token`. The authenticator should fetch a new `access_token` using the `client_id` and `client_secret` when the current one expires.
4. **Steps to Reproduce**:
- Setup a custom Airbyte declarative YAML configuration using `DeclarativeSource` and `OAuthAuthenticator`.
- Configure the `OAuthAuthenticator` with the following setup:
- Grant type: `client_credentials`
- No `refresh_token` should be provided as it is not part of the `client_credentials` flow.
- Attempt a connection test or data sync.
- Observe that Airbyte throws an error, expecting a `refresh_token` even though it shouldn't.
Here is the configuration I used:
```yaml
version: 5.12.0
type: DeclarativeSource
check:
type: CheckStream
stream_names:
- demographics
definitions:
streams:
demographics:
type: DeclarativeStream
name: demographics
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_size_option:
type: RequestOption
field_name: resultsPerPage
inject_into: body_json
page_token_option:
type: RequestOption
field_name: page
inject_into: body_json
pagination_strategy:
type: PageIncrement
page_size: 100
start_from_page: 1
requester:
$ref: "#/definitions/base_requester"
path: ***********/query
http_method: GET
request_body_json:
top: 1
name: Demographics
page: 1
sort:
- ProviderID: ASC
fields:
- ProviderID
- FirstName
- LastName
- Degree
- Department1
- Specialty
- Office1Address
- Office1AddressCity
- Office1AddressZip
- Office1AddressTelephone
- NPI
- Gender
- Content
filter:
OnTheWeb:
- type: in
values:
- "Y"
Specialty:
- type: not in
values:
- null
source: AdHocAPI
isArray: true
resultsPerPage: 1
dynamicFilterParameters:
ReportID: 4210c249-f397-4902-ab59-e9fcc3a2fa75
FieldNames: >-
Uid,ProviderID,FirstName,LastName,Degree,Department1,Specialty,Office1Address,Office1AddressCity,Office1AddressZip,Office1AddressTelephone,OnTheWeb,NPI,Gender,Content,LastName,FirstName
request_options_provider:
request_headers:
Content-Type: application/json
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/demographics"
base_requester:
type: HttpRequester
url_base: https://***********.api.asm-cloud.com/api/
authenticator:
type: OAuthAuthenticator
scopes:
- ***********/c3c26c9e-b3c2-4903-93bc-fca7cd50c88b
client_id: "{{ config['client_id'] }}"
client_secret: "{{ config['client_secret'] }}"
expires_in_name: expires_in
access_token_name: access_token
refresh_request_body:
grant_type: client_credentials
token_refresh_endpoint: https://***********.api.asm-cloud.com/api/tokens
# Do not specify a refresh_token, as client_credentials doesn't need it
```
5. **Workarounds Tried**:
I attempted to exclude the `refresh_token` parameter and explicitly set the `grant_type` to `client_credentials`, but the error persists. It seems the connector still expects a `refresh_token` for this OAuth grant type, which is incorrect for the `client_credentials` flow.
---
#### **Relevant Log Output**
```
Traceback (most recent call last):
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/connector_builder_handler.py", line 68, in read_stream
stream_read = handler.get_message_groups(source, config, configured_catalog, state, limits.max_records)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/message_grouper.py", line 84, in get_message_groups
stream = source.streams(config)[0] # The connector builder currently only supports reading from a single stream at a time
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 95, in streams
source_streams = [
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 96, in
self._constructor.create_component(
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 299, in _create_component_from_model
return component_constructor(model=model, config=config, **kwargs)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 1065, in create_oauth_authenticator
return DeclarativeOauth2Authenticator( # type: ignore
File "", line 17, in __init__
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/auth/oauth.py", line 79, in __post_init__
raise ValueError("OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`")
ValueError: OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`
```
I strongly believe that the connector misinterprets the `client_credentials` flow and incorrectly enforces the requirement of a `refresh_token`.
#### **Additional Notes**
This bug makes it impossible to use APIs that exclusively follow the `client_credentials` grant type, limiting the effectiveness of Airbyte in such integrations.
### Relevant log output
```shell
Traceback (most recent call last):
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/connector_builder_handler.py", line 68, in read_stream
stream_read = handler.get_message_groups(source, config, configured_catalog, state, limits.max_records)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/connector_builder/message_grouper.py", line 84, in get_message_groups
stream = source.streams(config)[0] # The connector builder currently only supports reading from a single stream at a time
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 95, in streams
source_streams = [
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 96, in
self._constructor.create_component(
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 299, in _create_component_from_model
return component_constructor(model=model, config=config, **kwargs)
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py", line 1065, in create_oauth_authenticator
return DeclarativeOauth2Authenticator( # type: ignore
File "", line 17, in __init__
File "/home/airbyte/.pyenv/versions/3.10.14/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/auth/oauth.py", line 79, in __post_init__
raise ValueError("OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`")
ValueError: OAuthAuthenticator needs a refresh_token parameter if grant_type is set to `refresh_token`
```
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.