airbytehq / airbytehq/airbyte-python-cdk
Custom components use case analysis: HTTP Requests for Configuration Determination
- Dominant language
- Python
- Stars
- 26
- Forks
- 53
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 10
Description
This issue documents a use case That requires custom Python. The purpose of this issue is to document the requirement and make a space for discussing if it can be supported in a YAML only format.
## Use Case
Some manifest-only connectors need to make HTTP requests during configuration setup to determine correct API endpoints or extract configuration values before stream processing begins.
## Example Implementation
**source-mailchimp** uses `ExtractAndSetDataCenterConfigValue` to make OAuth metadata requests and extract the data center from authentication credentials:
- Makes HTTP request to OAuth token info endpoint
- Extracts data center from response
- Modifies connector configuration with correct API base URL
- Handles both OAuth and API key authentication methods
**Code Reference:**
https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-mailchimp/components.py#L44-L93
## Current Gap
The YAML dialect cannot perform runtime HTTP requests to modify configuration during connector initialization, requiring custom Python components for this functionality.
Contributor guide
Assessment
This issue has not been assessed yet.