airbytehq / airbytehq/PyAirbyte

Failure to run `spec` on `source-airtable` ("Error: Validation against json schema defined in declarative_component_schema.yaml schema failed")

Abierto
#671 2 comentarios 8 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

Reported in slack:

- https://airbytehq-team.slack.com/archives/C06FZ238P8W/p1747412543018439

The failure occurs when invoking `source-airtable`:

```
Error: Validation against json schema defined in declarative_component_schema.yaml schema failed.
```

Easy to repro:

```
uv tool install airbyte-source-airtable
source-airtable spec
```

Error:

Show/Hide

> `$ source-airtable spec`
> `{"type":"TRACE","trace":{"type":"ERROR","emitted_at":1747413222276,"error":{"message":"Error starting the sync. This could be due to an invalid configuration or catalog. Please contact Support for assistance. Error: Validation against json schema defined in declarative_component_schema.yaml schema failed","stack_trace":"Traceback (most recent call last):\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 394, in _validate_source\n validate(self._source_config, self._declarative_component_schema)\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/jsonschema/validators.py\", line 1121, in validate\n raise error\njsonschema.exceptions.ValidationError: {'type': 'HttpComponentsResolver', 'retriever': {'type': 'SimpleRetriever', 'requester': {'type': 'HttpRequester', 'url_base': 'https://api.airtable.com/v0/', 'authenticator': {'type': 'SelectiveAuthenticator', 'authenticator_selection_path': ['credentials', 'auth_method'], 'authenticators': {'oauth2.0': {'type': 'OAuthAuthenticator', 'client_id': \"{{ config['credentials']['client_id'] }}\", 'client_secret': \"{{ config['credentials']['client_secret'] }}\", 'refresh_token': \"{{ config['credentials']['refresh_token'] }}\", 'grant_type': 'refresh_token', 'refresh_request_headers': {'Authorization': \"Basic {{ [config['credentials']['client_id'], config['credentials']['client_secret']] | join(':') | base64encode }}\", 'Content-Type': 'application/x-www-form-urlencoded'}, 'refresh_token_updater': {'refresh_token_error_status_codes': [400, 401], 'refresh_token_error_key': 'error', 'refresh_token_error_values': ['invalid_grant', 'invalid_client'], 'refresh_token_name': 'refresh_token', 'access_token_config_path': ['credentials', 'access_token'], 'token_expiry_date_config_path': ['credentials', 'token_expiry_date'], 'refresh_token_config_path': ['credentials', 'refresh_token']}, 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token'}, 'api_key': {'type': 'BearerAuthenticator', 'api_token': \"{{ config['credentials']['api_key'] }}\"}}}, 'error_handler': {'type': 'DefaultErrorHandler', 'max_retries': 10, 'backoff_strategies': [{'type': 'ConstantBackoffStrategy', 'backoff_time_in_seconds': 30}], 'response_filters': [{'predicate': \"{{ response.status_code == 403 and response.get('error', {}).get('type') == 'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' }}\", 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': \"{{ 'Personal Access Token does not have required permissions, please add all required permissions to existed one or create new PAT, see docs for more info: https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' if config.get('credentials', {}). get('auth_method', '') == 'api_key' else 'Access Token does not have required permissions, please reauthenticate.' }}\", 'type': 'HttpResponseFilter'}, {'http_codes': [403, 422], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Permission denied or entity is unprocessable.', 'type': 'HttpResponseFilter'}, {'http_codes': [401], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Invalid credentials were provided. Please re-authenticate to restore access to Airtable.', 'type': 'HttpResponseFilter'}]}, 'path': 'meta/bases/{{ stream_partition.base_id }}/tables', 'http_method': 'GET'}, 'record_selector': {'type': 'RecordSelector', 'extractor': {'type': 'DpathExtractor', 'field_path': ['tables']}, 'pagination_strategy': {'type': 'CursorPagination', 'cursor_value': '{{ response.get(\"offset\", {}) }}', 'stop_condition': '{{ not response.get(\"offset\", {}) }}'}}, 'partition_router': {'type': 'SubstreamPartitionRouter', 'parent_stream_configs': [{'type': 'ParentStreamConfig', 'parent_key': 'id', 'partition_field': 'base_id', 'extra_fields': [['name']], 'stream': {'type': 'DeclarativeStream', 'name': 'bases', 'retriever': {'type': 'SimpleRetriever', 'requester': {'type': 'HttpRequester', 'url_base': 'https://api.airtable.com/v0/', 'authenticator': {'type': 'SelectiveAuthenticator', 'authenticator_selection_path': ['credentials', 'auth_method'], 'authenticators': {'oauth2.0': {'type': 'OAuthAuthenticator', 'client_id': \"{{ config['credentials']['client_id'] }}\", 'client_secret': \"{{ config['credentials']['client_secret'] }}\", 'refresh_token': \"{{ config['credentials']['refresh_token'] }}\", 'grant_type': 'refresh_token', 'refresh_request_headers': {'Authorization': \"Basic {{ [config['credentials']['client_id'], config['credentials']['client_secret']] | join(':') | base64encode }}\", 'Content-Type': 'application/x-www-form-urlencoded'}, 'refresh_token_updater': {'refresh_token_error_status_codes': [400, 401], 'refresh_token_error_key': 'error', 'refresh_token_error_values': ['invalid_grant', 'invalid_client'], 'refresh_token_name': 'refresh_token', 'access_token_config_path': ['credentials', 'access_token'], 'token_expiry_date_config_path': ['credentials', 'token_expiry_date'], 'refresh_token_config_path': ['credentials', 'refresh_token']}, 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token'}, 'api_key': {'type': 'BearerAuthenticator', 'api_token': \"{{ config['credentials']['api_key'] }}\"}}}, 'error_handler': {'type': 'DefaultErrorHandler', 'max_retries': 10, 'backoff_strategies': [{'type': 'ConstantBackoffStrategy', 'backoff_time_in_seconds': 30}], 'response_filters': [{'predicate': \"{{ response.status_code == 403 and response.get('error', {}).get('type') == 'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' }}\", 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': \"{{ 'Personal Access Token does not have required permissions, please add all required permissions to existed one or create new PAT, see docs for more info: https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' if config.get('credentials', {}). get('auth_method', '') == 'api_key' else 'Access Token does not have required permissions, please reauthenticate.' }}\", 'type': 'HttpResponseFilter'}, {'http_codes': [403, 422], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Permission denied or entity is unprocessable.', 'type': 'HttpResponseFilter'}, {'http_codes': [401], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Invalid credentials were provided. Please re-authenticate to restore access to Airtable.', 'type': 'HttpResponseFilter'}]}, 'path': 'meta/bases', 'http_method': 'GET'}, 'record_selector': {'type': 'RecordSelector', 'extractor': {'type': 'DpathExtractor', 'field_path': ['bases']}, 'record_filter': {'type': 'RecordFilter', 'condition': '{{ record.get(\"permissionLevel\") }}'}}, 'paginator': {'type': 'DefaultPaginator', 'page_token_option': {'type': 'RequestOption', 'inject_into': 'request_parameter', 'field_name': 'offset'}, 'pagination_strategy': {'type': 'CursorPagination', 'cursor_value': '{{ response.get(\"offset\", {}) }}', 'stop_condition': '{{ not response.get(\"offset\", {}) }}'}}}, 'schema_loader': {'type': 'InlineSchemaLoader', 'schema': {'type': 'object', '$schema': 'http://json-schema.org/schema#', 'additionalProperties': True, 'properties': {'id': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'permissionLevel': {'type': ['string', 'null']}}}}}}]}}, 'components_mapping': [{'type': 'ComponentMappingDefinition', 'field_path': ['name'], 'value': \"{{ stream_slice.extra_fields.name.replace(' ', '_').lower().strip() }}/{{ components_values.name.replace(' ', '_').lower().strip() }}/{{ components_values.id }}\"}, {'type': 'ComponentMappingDefinition', 'field_path': ['retriever', 'requester', 'path'], 'value': '{{ stream_slice.base_id }}/{{ components_values.id }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'requester', 'path'], 'value': '{{ stream_slice.base_id }}/{{ components_values.id }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['transformations', 0, 'fields', 0, 'value'], 'value': '{{ components_values.name }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'requester', 'path'], 'value': 'meta/bases/{{ stream_slice.base_id }}/tables'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'record_selector', 'record_filter', '$parameters', 'table_id'], 'value': '{{ components_values.id }}'}]} is not valid under any of the given schemas\n\nFailed validating 'anyOf' in schema['properties']['dynamic_streams']['items']['properties']['components_resolver']:\n {'anyOf': [{'$ref': '#/definitions/HttpComponentsResolver'},\n {'$ref': '#/definitions/ConfigComponentsResolver'}],\n 'description': 'Component resolve and populates stream templates with '\n 'components values.',\n 'title': 'Components Resolver'}\n\nOn instance['dynamic_streams'][0]['components_resolver']:\n {'components_mapping': [{'field_path': ['name'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ '\n \"stream_slice.extra_fields.name.replace(' \"\n \"', '_').lower().strip() }}/{{ \"\n \"components_values.name.replace(' ', \"\n \"'_').lower().strip() }}/{{ \"\n 'components_values.id }}'},\n {'field_path': ['retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ stream_slice.base_id }}/{{ '\n 'components_values.id }}'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ stream_slice.base_id }}/{{ '\n 'components_values.id }}'},\n {'field_path': ['transformations',\n 0,\n 'fields',\n 0,\n 'value'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ components_values.name }}'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': 'meta/bases/{{ stream_slice.base_id '\n '}}/tables'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'record_selector',\n 'record_filter',\n '$parameters',\n 'table_id'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ components_values.id }}'}],\n 'retriever': {'partition_router': {'parent_stream_configs': [{'extra_fields': [['name']],\n 'parent_key': 'id',\n 'partition_field': 'base_id',\n 'stream': {'name': 'bases',\n 'retriever': {'paginator': {'page_token_option': {'field_name': 'offset',\n 'inject_into': 'request_parameter',\n 'type': 'RequestOption'},\n 'pagination_strategy': {'cursor_value': '{{ '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'stop_condition': '{{ '\n 'not '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'type': 'CursorPagination'},\n 'type': 'DefaultPaginator'},\n 'record_selector': {'extractor': {'field_path': ['bases'],\n 'type': 'DpathExtractor'},\n 'record_filter': {'condition': '{{ '\n 'record.get(\"permissionLevel\") '\n '}}',\n 'type': 'RecordFilter'},\n 'type': 'RecordSelector'},\n 'requester': {'authenticator': {'authenticator_selection_path': ['credentials',\n 'auth_method'],\n 'authenticators': {'api_key': {'api_token': '{{ '\n \"config['credentials']['api_key'] \"\n '}}',\n 'type': 'BearerAuthenticator'},\n 'oauth2.0': {'client_id': '{{ '\n \"config['credentials']['client_id'] \"\n '}}',\n 'client_secret': '{{ '\n \"config['credentials']['client_secret'] \"\n '}}',\n 'grant_type': 'refresh_token',\n 'refresh_request_headers': {'Authorization': 'Basic '\n '{{ '\n \"[config['credentials']['client_id'], \"\n \"config['credentials']['client_secret']] \"\n '| '\n \"join(':') \"\n '| '\n 'base64encode '\n '}}',\n 'Content-Type': 'application/x-www-form-urlencoded'},\n 'refresh_token': '{{ '\n \"config['credentials']['refresh_token'] \"\n '}}',\n 'refresh_token_updater': {'access_token_config_path': ['credentials',\n 'access_token'],\n 'refresh_token_config_path': ['credentials',\n 'refresh_token'],\n 'refresh_token_error_key': 'error',\n 'refresh_token_error_status_codes': [400,\n 401],\n 'refresh_token_error_values': ['invalid_grant',\n 'invalid_client'],\n 'refresh_token_name': 'refresh_token',\n 'token_expiry_date_config_path': ['credentials',\n 'token_expiry_date']},\n 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token',\n 'type': 'OAuthAuthenticator'}},\n 'type': 'SelectiveAuthenticator'},\n 'error_handler': {'backoff_strategies': [{'backoff_time_in_seconds': 30,\n 'type': 'ConstantBackoffStrategy'}],\n 'max_retries': 10,\n 'response_filters': [{'action': 'FAIL',\n 'error_message': '{{ '\n \"'Personal \"\n 'Access '\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n 'add '\n 'all '\n 'required '\n 'permissions '\n 'to '\n 'existed '\n 'one '\n 'or '\n 'create '\n 'new '\n 'PAT, '\n 'see '\n 'docs '\n 'for '\n 'more '\n 'info: '\n \"https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' \"\n 'if '\n \"config.get('credentials', \"\n '{}). '\n \"get('auth_method', \"\n \"'') \"\n '== '\n \"'api_key' \"\n 'else '\n \"'Access \"\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n \"reauthenticate.' \"\n '}}',\n 'failure_type': 'config_error',\n 'predicate': '{{ '\n 'response.status_code '\n '== '\n '403 '\n 'and '\n \"response.get('error', \"\n \"{}).get('type') \"\n '== '\n \"'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' \"\n '}}',\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Permission '\n 'denied '\n 'or '\n 'entity '\n 'is '\n 'unprocessable.',\n 'failure_type': 'config_error',\n 'http_codes': [403,\n 422],\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Invalid '\n 'credentials '\n 'were '\n 'provided. '\n 'Please '\n 're-authenticate '\n 'to '\n 'restore '\n 'access '\n 'to '\n 'Airtable.',\n 'failure_type': 'config_error',\n 'http_codes': [401],\n 'type': 'HttpResponseFilter'}],\n 'type': 'DefaultErrorHandler'},\n 'http_method': 'GET',\n 'path': 'meta/bases',\n 'type': 'HttpRequester',\n 'url_base': 'https://api.airtable.com/v0/'},\n 'type': 'SimpleRetriever'},\n 'schema_loader': {'schema': {'$schema': 'http://json-schema.org/schema#',\n 'additionalProperties': True,\n 'properties': {'id': {'type': ['string',\n 'null']},\n 'name': {'type': ['string',\n 'null']},\n 'permissionLevel': {'type': ['string',\n 'null']}},\n 'type': 'object'},\n 'type': 'InlineSchemaLoader'},\n 'type': 'DeclarativeStream'},\n 'type': 'ParentStreamConfig'}],\n 'type': 'SubstreamPartitionRouter'},\n 'record_selector': {'extractor': {'field_path': ['tables'],\n 'type': 'DpathExtractor'},\n 'pagination_strategy': {'cursor_value': '{{ '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'stop_condition': '{{ '\n 'not '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'type': 'CursorPagination'},\n 'type': 'RecordSelector'},\n 'requester': {'authenticator': {'authenticator_selection_path': ['credentials',\n 'auth_method'],\n 'authenticators': {'api_key': {'api_token': '{{ '\n \"config['credentials']['api_key'] \"\n '}}',\n 'type': 'BearerAuthenticator'},\n 'oauth2.0': {'client_id': '{{ '\n \"config['credentials']['client_id'] \"\n '}}',\n 'client_secret': '{{ '\n \"config['credentials']['client_secret'] \"\n '}}',\n 'grant_type': 'refresh_token',\n 'refresh_request_headers': {'Authorization': 'Basic '\n '{{ '\n \"[config['credentials']['client_id'], \"\n \"config['credentials']['client_secret']] \"\n '| '\n \"join(':') \"\n '| '\n 'base64encode '\n '}}',\n 'Content-Type': 'application/x-www-form-urlencoded'},\n 'refresh_token': '{{ '\n \"config['credentials']['refresh_token'] \"\n '}}',\n 'refresh_token_updater': {'access_token_config_path': ['credentials',\n 'access_token'],\n 'refresh_token_config_path': ['credentials',\n 'refresh_token'],\n 'refresh_token_error_key': 'error',\n 'refresh_token_error_status_codes': [400,\n 401],\n 'refresh_token_error_values': ['invalid_grant',\n 'invalid_client'],\n 'refresh_token_name': 'refresh_token',\n 'token_expiry_date_config_path': ['credentials',\n 'token_expiry_date']},\n 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token',\n 'type': 'OAuthAuthenticator'}},\n 'type': 'SelectiveAuthenticator'},\n 'error_handler': {'backoff_strategies': [{'backoff_time_in_seconds': 30,\n 'type': 'ConstantBackoffStrategy'}],\n 'max_retries': 10,\n 'response_filters': [{'action': 'FAIL',\n 'error_message': '{{ '\n \"'Personal \"\n 'Access '\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n 'add '\n 'all '\n 'required '\n 'permissions '\n 'to '\n 'existed '\n 'one '\n 'or '\n 'create '\n 'new '\n 'PAT, '\n 'see '\n 'docs '\n 'for '\n 'more '\n 'info: '\n \"https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' \"\n 'if '\n \"config.get('credentials', \"\n '{}). '\n \"get('auth_method', \"\n \"'') \"\n '== '\n \"'api_key' \"\n 'else '\n \"'Access \"\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n \"reauthenticate.' \"\n '}}',\n 'failure_type': 'config_error',\n 'predicate': '{{ '\n 'response.status_code '\n '== '\n '403 '\n 'and '\n \"response.get('error', \"\n \"{}).get('type') \"\n '== '\n \"'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' \"\n '}}',\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Permission '\n 'denied '\n 'or '\n 'entity '\n 'is '\n 'unprocessable.',\n 'failure_type': 'config_error',\n 'http_codes': [403,\n 422],\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Invalid '\n 'credentials '\n 'were '\n 'provided. '\n 'Please '\n 're-authenticate '\n 'to '\n 'restore '\n 'access '\n 'to '\n 'Airtable.',\n 'failure_type': 'config_error',\n 'http_codes': [401],\n 'type': 'HttpResponseFilter'}],\n 'type': 'DefaultErrorHandler'},\n 'http_method': 'GET',\n 'path': 'meta/bases/{{ '\n 'stream_partition.base_id '\n '}}/tables',\n 'type': 'HttpRequester',\n 'url_base': 'https://api.airtable.com/v0/'},\n 'type': 'SimpleRetriever'},\n 'type': 'HttpComponentsResolver'}\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/source_airtable/run.py\", line 25, in _get_source\n return SourceAirtable(\n ^^^^^^^^^^^^^^^\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/source_airtable/source.py\", line 13, in __init__\n super().__init__(catalog=catalog, config=config, state=state, **{\"path_to_yaml\": \"manifest.yaml\"})\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/yaml_declarative_source.py\", line 34, in __init__\n super().__init__(\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/concurrent_declarative_source.py\", line 93, in __init__\n super().__init__(\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 138, in __init__\n self._validate_source()\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 396, in _validate_source\n raise ValidationError(\njsonschema.exceptions.ValidationError: Validation against json schema defined in declarative_component_schema.yaml schema failed\n"}}}`
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.