airbytehq / airbytehq/airbyte-python-cdk

Custom components use case analysis: HTTP Requests for Configuration Determination

Open
#713 0 comments 0 reactions 1 assignee Claimed by @aaronsteers View on GitHub
Dominant language
Python
Stars
26
Forks
53
Avg merge
2d 6h
Merged PRs (30d)
10

Description

This issue documents a use case That requires custom Python. The purpose of this issue is to document the requirement and make a space for discussing if it can be supported in a YAML only format.

## Use Case
Some manifest-only connectors need to make HTTP requests during configuration setup to determine correct API endpoints or extract configuration values before stream processing begins.

## Example Implementation
**source-mailchimp** uses `ExtractAndSetDataCenterConfigValue` to make OAuth metadata requests and extract the data center from authentication credentials:

- Makes HTTP request to OAuth token info endpoint
- Extracts data center from response
- Modifies connector configuration with correct API base URL
- Handles both OAuth and API key authentication methods

**Code Reference:**
https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-mailchimp/components.py#L44-L93

## Current Gap
The YAML dialect cannot perform runtime HTTP requests to modify configuration during connector initialization, requiring custom Python components for this functionality.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.