airbytehq / airbytehq/airbyte

source-google-ads: Check Connection fails with 422 Unprocessable Entity on platform updateSource (abctl/K8s deployment)

Open
#60,900 3 comments 0 reactions 0 assignees View on GitHub
area/connectors area/platform community connectors/source/google-ads team/extensibility type/bug
Dominant language
Python
Stars
22.1k
Forks
5.3k
PR merge metrics
PR metrics pending

Description

### Connector Name

source-google-ads

### Connector Version

3.7.10

### What step the error happened?

Configuring a new connector

### Relevant information

## Summary
After successfully creating the Google Ads source via the Airbyte API, the "Check Connection" test in the UI consistently fails. The connector's Python code appears to run successfully and exits with code 0. However, the Airbyte platform then fails with a `422 Unprocessable Entity` error when attempting an internal `updateSource` API call, likely triggered by a control message from the connector (e.g., to save an updated token). This issue occurs in an Airbyte OSS instance deployed via `abctl v0.24.0` (using `kind` for Kubernetes). The same Google Ads account and credentials work correctly on Airbyte Cloud.

## Airbyte Environment
* **Airbyte Deployment:** OSS, managed by `abctl v0.24.0`
* **Kubernetes Provider (from `abctl local status`):** `kind`
* **`abctl` Kubeconfig Path:** `/home/pedro/.airbyte/abctl/abctl.kubeconfig`
* **Airbyte Platform Version:** *(If you know it, add it here. If not, state "Unknown, deployed by abctl v0.24.0")*

## Steps to Reproduce
1. Deployed Airbyte OSS using `abctl v0.24.0`.
2. Ensured Google Cloud OAuth credentials (`Client ID`, `Client Secret`) were correctly configured in the environment and `Redirect URIs` (`/auth_code`, `/auth_flow`) were added to the Google Cloud Project.
3. Obtained a valid Google Ads `Developer Token` (Basic access) and a `Refresh Token`.
4. Successfully created the `source-google-ads` (v3.7.10) source using the Airbyte API (`/api/v1/sources/create`) with the following (redacted) configuration:
```json
{
"name": "Google Ads API Test - Plural",
"sourceDefinitionId": "253487c0-2246-43ba-a21f-5116b20a2c50",
"workspaceId": "4812a890-98f8-4b78-a8ac-abc782c456a5",
"connectionConfiguration": {
"credentials": {
"client_id": "REDACTED_YOUR_CLIENT_ID",
"client_secret": "REDACTED_YOUR_CLIENT_SECRET",
"refresh_token": "REDACTED_YOUR_REFRESH_TOKEN",
"developer_token": "REDACTED_YOUR_DEV_TOKEN"
},
"customer_ids": ["4783655164"],
"start_date": "2024-01-01"
// conversion_window_days was also included
}
}
```
*(Note: API call also included a Bearer token for Airbyte API authentication).*
5. Navigated to the newly created source ("Google Ads API Test - Plural") in the Airbyte UI.
6. Clicked "Check Connection."

## Current Behavior
The "Check Connection" test in the UI fails. The `jobInfo` from the UI shows:
* `succeeded: false`
* `failureOrigin: "source"`
* `internalMessage: "Unexpected error performing CHECK. The exit of the connector was: 0"`
* A Java stack trace with `Caused by: org.openapitools.client.infrastructure.ClientException: Client error : 422 Unprocessable Entity` originating from `io.airbyte.api.client.generated.SourceApi.updateSource`.

The logs from the Kubernetes pod (`e-google-ads-check-...`) for this check operation show:
* The Python connector code starts (`python /airbyte/integration_code/main.py`).
* The connector `exited with exit code 0`.
* The Airbyte sidecar logs `Optional control message present. Updating...`.
* This is immediately followed by the Java stack trace showing the `422 Unprocessable Entity` error.

## Expected behavior
The "Check Connection" test in the UI should pass successfully, similar to how it functions on Airbyte Cloud with the same Google Ads account and credentials.

## Previous Troubleshooting Steps
* Resolved initial `redirect_uri_mismatch` errors for Google OAuth.
* Resolved `None is not of type 'string'` config validation errors by moving to API creation and ensuring all credentials and `customer_id`/`customer_ids` were present.
* Confirmed via API logs that all credential strings (client_id, client_secret, refresh_token, developer_token) are non-empty when passed to the API.
* Tested creating the source via API with both `customer_id` (singular string) and `customer_ids` (plural list) in the `connectionConfiguration`. Both creation methods succeed, but the subsequent UI "Check Connection" fails with the 422 error in the same way.

### Relevant log output

```shell
pedro@vm-closedata-airbyte:~$ kubectl --kubeconfig /home/pedro/.airbyte/abctl/abctl.kubeconfig logs -f e-google-ads-check-4be4d24c-f068-43a6-8ad2-eec7d6244327-0-ijcir -n airbyte-abctl --all-containers
Unsetting empty environment variable 'AWS_SECRET_MANAGER_REGION'
Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_CREDENTIALS_PATH'
Using AIRBYTE_ENTRYPOINT: python /airbyte/integration_code/main.py
Unsetting empty environment variable 'FEATURE_FLAG_CLIENT'
Unsetting empty environment variable 'DD_AGENT_HOST'
Unsetting empty environment variable 'KEYCLOAK_INTERNAL_REALM_ISSUER'
Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_EMAIL'
Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_CREDENTIALS_PATH'
Unsetting empty environment variable 'KEYCLOAK_INTERNAL_REALM_ISSUER'
Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_EMAIL'
Unsetting empty environment variable 'KEYCLOAK_CLIENT_ID'
Unsetting empty environment variable 'AIRBYTE_ROLE'
Unsetting empty environment variable 'VAULT_PREFIX'
Unsetting empty environment variable 'AB_AZURE_KEY_VAULT_TENANT_ID'
Unsetting empty environment variable 'SECRET_STORE_GCP_PROJECT_ID'
Unsetting empty environment variable 'AWS_SECRET_MANAGER_SECRET_TAGS'
Unsetting empty environment variable 'AB_AZURE_KEY_VAULT_VAULT_URL'
Unsetting empty environment variable 'FEATURE_FLAG_BASEURL'
Unsetting empty environment variable 'VAULT_ADDRESS'
Unsetting empty environment variable 'AB_AZURE_KEY_VAULT_TAGS'
Unsetting empty environment variable 'KEYCLOAK_CLIENT_ID'
Unsetting empty environment variable 'CONTROL_PLANE_AUTH_ENDPOINT'
2025-05-24 12:07:00,296 [main] INFO i.a.c.ApplicationKt(main):13 - Sidecar start

___ _ __ __
/ | (_)____/ /_ __ __/ /____
/ /| | / / ___/ __ \/ / / / __/ _ \
/ ___ |/ / / / /_/ / /_/ / /_/ __/
/_/ |_/_/_/ /_.___/\__, /\__/\___/
/____/
: airbyte-connector-sidecar :

2025-05-24 12:07:02,230 [main] INFO i.m.c.e.DefaultEnvironment():168 - Established active environments: [worker-v2, control-plane, oss, local-secrets]
Unsetting empty environment variable 'AIRBYTE_ROLE'
Unsetting empty environment variable 'LAUNCHDARKLY_KEY'
Unsetting empty environment variable 'CONTROL_PLANE_AUTH_ENDPOINT'
Unsetting empty environment variable 'AWS_KMS_KEY_ARN'
Unsetting empty environment variable 'DD_DOGSTATSD_PORT'
2025-05-24 12:06:51,038 [main] INFO i.a.i.ApplicationKt(main):13 - Init start

___ _ __ __
/ | (_)____/ /_ __ __/ /____
/ /| | / / ___/ __ \/ / / / __/ _ \
/ ___ |/ / / / /_/ / /_/ / /_/ __/
/_/ |_/_/_/ /_.___/\__, /\__/\___/
/____/
: airbyte-workload-init-container :

2025-05-24 12:06:52,563 [main] INFO i.m.c.e.DefaultEnvironment():168 - Established active environments: [worker-v2, control-plane, oss, local-secrets]
2025-05-24 12:06:54,035 [main] INFO c.z.h.HikariDataSource():79 - config-pool - Starting...
2025-05-24 12:06:54,067 [main] INFO c.z.h.HikariDataSource():81 - config-pool - Start completed.
2025-05-24 12:06:54,703 [main] INFO i.a.i.InputFetcher(fetch):31 - Fetching workload...
2025-05-24 12:07:06,473 [pool-3-thread-1] INFO i.a.c.i.LineGobbler(voidCall):177 -
2025-05-24 12:07:06,507 [pool-3-thread-1] INFO i.a.c.i.LineGobbler(voidCall):177 - ----- START CHECK -----
2025-05-24 12:07:06,508 [pool-3-thread-1] INFO i.a.c.i.LineGobbler(voidCall):177 -
2025-05-24 12:06:56,477 [main] INFO i.a.i.InputFetcher(fetch):41 - Workload 253487c0-2246-43ba-a21f-5116b20a2c50_4be4d24c-f068-43a6-8ad2-eec7d6244327_0_check fetched.
2025-05-24 12:07:07,580 [main] INFO i.a.c.ConnectorWatcher(processConnectorOutput):122 - Connector exited, processing output
2025-05-24 12:07:07,582 [main] INFO i.a.c.ConnectorWatcher(getConnectorOutputStream):159 - Output file jobOutput.json found
2025-05-24 12:07:07,586 [main] INFO i.a.c.ConnectorWatcher(processConnectorOutput):125 - Connector exited with exit code 0
2025-05-24 12:07:07,592 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(create):198 - Reading messages from protocol version 0.2.0
2025-05-24 12:07:07,699 [main] INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):232 - Checking for optional control message...
2025-05-24 12:07:07,711 [main] INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):236 - Optional control message present. Updating...
2025-05-24 12:07:07,975 [main] ERROR i.a.c.ConnectorWatcher(handleException):183 - Error performing operation: io.airbyte.workers.exception.WorkerException
io.airbyte.workers.exception.WorkerException: Unexpected error performing CHECK. The exit of the connector was: 0
at io.airbyte.connectorSidecar.ConnectorMessageProcessor.run(ConnectorMessageProcessor.kt:95)
at io.airbyte.connectorSidecar.ConnectorWatcher.processConnectorOutput(ConnectorWatcher.kt:130)
at io.airbyte.connectorSidecar.ConnectorWatcher.run(ConnectorWatcher.kt:79)
at io.airbyte.connectorSidecar.$ConnectorWatcher$Definition.initialize(Unknown Source)
at io.airbyte.connectorSidecar.$ConnectorWatcher$Definition.instantiate(Unknown Source)
at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2334)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2304)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2316)
at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3127)
at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
2025-05-24 12:06:56,478 [main] INFO i.a.i.InputFetcher(fetch):43 - Processing workload...
at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:3029)
at io.micronaut.context.DefaultBeanContext.initializeEagerBean(DefaultBeanContext.java:2702)
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1994)
at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:314)
at io.micronaut.context.DefaultBeanContext.configureAndStartContext(DefaultBeanContext.java:3318)
at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:345)
2025-05-24 12:06:57,193 [main] WARN o.j.t.JooqLogger(lambda$warn$7):461 - Version mismatch : Database version is older than what dialect POSTGRES supports: 13.20. Consider https://www.jooq.org/download/support-matrix to see what jOOQ version and edition supports which RDBMS versions.
2025-05-24 12:06:57,511 [main] INFO i.a.i.InputFetcher(fetch):49 - Workload processed.
2025-05-24 12:06:57,663 [main] INFO i.m.r.Micronaut(start):170 - No embedded container found. Running as CLI application
2025-05-24 12:06:57,691 [main] INFO c.z.h.HikariDataSource(close):349 - config-pool - Shutdown initiated...
2025-05-24 12:06:57,698 [main] INFO c.z.h.HikariDataSource(close):351 - config-pool - Shutdown completed.
2025-05-24 12:06:57,702 [main] INFO i.a.i.ApplicationKt(main):27 - Init end
at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:216)
at io.micronaut.runtime.Micronaut.start(Micronaut.java:75)
at io.airbyte.connectorSidecar.ApplicationKt.main(Application.kt:18)
at io.airbyte.connectorSidecar.ApplicationKt.main(Application.kt)
Caused by: org.openapitools.client.infrastructure.ClientException: Client error : 422 Unprocessable Entity
at io.airbyte.api.client.generated.SourceApi.updateSource(SourceApi.kt:948)
at io.airbyte.commons.converters.ConnectorConfigUpdater.updateSource(ConnectorConfigUpdater.kt:45)
at io.airbyte.connectorSidecar.ConnectorMessageProcessor.updateConfigFromControlMessage(ConnectorMessageProcessor.kt:239)
at io.airbyte.connectorSidecar.ConnectorMessageProcessor.updateConfigFromControlMessagePerMessageType(ConnectorMessageProcessor.kt:119)
at io.airbyte.connectorSidecar.ConnectorMessageProcessor.run(ConnectorMessageProcessor.kt:85)
... 19 common frames omitted

2025-05-24 12:07:08,214 [main] INFO i.a.c.ConnectorWatcher(failWorkload):314 - Failing workload 253487c0-2246-43ba-a21f-5116b20a2c50_4be4d24c-f068-43a6-8ad2-eec7d6244327_0_check.
2025-05-24 12:07:08,261 [main] INFO i.a.c.ConnectorWatcher(exitInternalError):226 - Deliberately exiting process with code 1.
pedro@vm-closedata-airbyte:~$
```

### Contribute

- [ ] Yes, I want to contribute

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.