airbytehq / airbytehq/PyAirbyte

Failure to run `spec` on `source-airtable` ("Error: Validation against json schema defined in declarative_component_schema.yaml schema failed")

Đang mở
#671 2 bình luận 8 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
344
Fork
77
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
35

Mô tả

Reported in slack:

- https://airbytehq-team.slack.com/archives/C06FZ238P8W/p1747412543018439

The failure occurs when invoking `source-airtable`:

```
Error: Validation against json schema defined in declarative_component_schema.yaml schema failed.
```

Easy to repro:

```
uv tool install airbyte-source-airtable
source-airtable spec
```

Error:

Show/Hide

> `$ source-airtable spec`
> `{"type":"TRACE","trace":{"type":"ERROR","emitted_at":1747413222276,"error":{"message":"Error starting the sync. This could be due to an invalid configuration or catalog. Please contact Support for assistance. Error: Validation against json schema defined in declarative_component_schema.yaml schema failed","stack_trace":"Traceback (most recent call last):\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 394, in _validate_source\n validate(self._source_config, self._declarative_component_schema)\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/jsonschema/validators.py\", line 1121, in validate\n raise error\njsonschema.exceptions.ValidationError: {'type': 'HttpComponentsResolver', 'retriever': {'type': 'SimpleRetriever', 'requester': {'type': 'HttpRequester', 'url_base': 'https://api.airtable.com/v0/', 'authenticator': {'type': 'SelectiveAuthenticator', 'authenticator_selection_path': ['credentials', 'auth_method'], 'authenticators': {'oauth2.0': {'type': 'OAuthAuthenticator', 'client_id': \"{{ config['credentials']['client_id'] }}\", 'client_secret': \"{{ config['credentials']['client_secret'] }}\", 'refresh_token': \"{{ config['credentials']['refresh_token'] }}\", 'grant_type': 'refresh_token', 'refresh_request_headers': {'Authorization': \"Basic {{ [config['credentials']['client_id'], config['credentials']['client_secret']] | join(':') | base64encode }}\", 'Content-Type': 'application/x-www-form-urlencoded'}, 'refresh_token_updater': {'refresh_token_error_status_codes': [400, 401], 'refresh_token_error_key': 'error', 'refresh_token_error_values': ['invalid_grant', 'invalid_client'], 'refresh_token_name': 'refresh_token', 'access_token_config_path': ['credentials', 'access_token'], 'token_expiry_date_config_path': ['credentials', 'token_expiry_date'], 'refresh_token_config_path': ['credentials', 'refresh_token']}, 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token'}, 'api_key': {'type': 'BearerAuthenticator', 'api_token': \"{{ config['credentials']['api_key'] }}\"}}}, 'error_handler': {'type': 'DefaultErrorHandler', 'max_retries': 10, 'backoff_strategies': [{'type': 'ConstantBackoffStrategy', 'backoff_time_in_seconds': 30}], 'response_filters': [{'predicate': \"{{ response.status_code == 403 and response.get('error', {}).get('type') == 'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' }}\", 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': \"{{ 'Personal Access Token does not have required permissions, please add all required permissions to existed one or create new PAT, see docs for more info: https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' if config.get('credentials', {}). get('auth_method', '') == 'api_key' else 'Access Token does not have required permissions, please reauthenticate.' }}\", 'type': 'HttpResponseFilter'}, {'http_codes': [403, 422], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Permission denied or entity is unprocessable.', 'type': 'HttpResponseFilter'}, {'http_codes': [401], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Invalid credentials were provided. Please re-authenticate to restore access to Airtable.', 'type': 'HttpResponseFilter'}]}, 'path': 'meta/bases/{{ stream_partition.base_id }}/tables', 'http_method': 'GET'}, 'record_selector': {'type': 'RecordSelector', 'extractor': {'type': 'DpathExtractor', 'field_path': ['tables']}, 'pagination_strategy': {'type': 'CursorPagination', 'cursor_value': '{{ response.get(\"offset\", {}) }}', 'stop_condition': '{{ not response.get(\"offset\", {}) }}'}}, 'partition_router': {'type': 'SubstreamPartitionRouter', 'parent_stream_configs': [{'type': 'ParentStreamConfig', 'parent_key': 'id', 'partition_field': 'base_id', 'extra_fields': [['name']], 'stream': {'type': 'DeclarativeStream', 'name': 'bases', 'retriever': {'type': 'SimpleRetriever', 'requester': {'type': 'HttpRequester', 'url_base': 'https://api.airtable.com/v0/', 'authenticator': {'type': 'SelectiveAuthenticator', 'authenticator_selection_path': ['credentials', 'auth_method'], 'authenticators': {'oauth2.0': {'type': 'OAuthAuthenticator', 'client_id': \"{{ config['credentials']['client_id'] }}\", 'client_secret': \"{{ config['credentials']['client_secret'] }}\", 'refresh_token': \"{{ config['credentials']['refresh_token'] }}\", 'grant_type': 'refresh_token', 'refresh_request_headers': {'Authorization': \"Basic {{ [config['credentials']['client_id'], config['credentials']['client_secret']] | join(':') | base64encode }}\", 'Content-Type': 'application/x-www-form-urlencoded'}, 'refresh_token_updater': {'refresh_token_error_status_codes': [400, 401], 'refresh_token_error_key': 'error', 'refresh_token_error_values': ['invalid_grant', 'invalid_client'], 'refresh_token_name': 'refresh_token', 'access_token_config_path': ['credentials', 'access_token'], 'token_expiry_date_config_path': ['credentials', 'token_expiry_date'], 'refresh_token_config_path': ['credentials', 'refresh_token']}, 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token'}, 'api_key': {'type': 'BearerAuthenticator', 'api_token': \"{{ config['credentials']['api_key'] }}\"}}}, 'error_handler': {'type': 'DefaultErrorHandler', 'max_retries': 10, 'backoff_strategies': [{'type': 'ConstantBackoffStrategy', 'backoff_time_in_seconds': 30}], 'response_filters': [{'predicate': \"{{ response.status_code == 403 and response.get('error', {}).get('type') == 'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' }}\", 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': \"{{ 'Personal Access Token does not have required permissions, please add all required permissions to existed one or create new PAT, see docs for more info: https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' if config.get('credentials', {}). get('auth_method', '') == 'api_key' else 'Access Token does not have required permissions, please reauthenticate.' }}\", 'type': 'HttpResponseFilter'}, {'http_codes': [403, 422], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Permission denied or entity is unprocessable.', 'type': 'HttpResponseFilter'}, {'http_codes': [401], 'action': 'FAIL', 'failure_type': 'config_error', 'error_message': 'Invalid credentials were provided. Please re-authenticate to restore access to Airtable.', 'type': 'HttpResponseFilter'}]}, 'path': 'meta/bases', 'http_method': 'GET'}, 'record_selector': {'type': 'RecordSelector', 'extractor': {'type': 'DpathExtractor', 'field_path': ['bases']}, 'record_filter': {'type': 'RecordFilter', 'condition': '{{ record.get(\"permissionLevel\") }}'}}, 'paginator': {'type': 'DefaultPaginator', 'page_token_option': {'type': 'RequestOption', 'inject_into': 'request_parameter', 'field_name': 'offset'}, 'pagination_strategy': {'type': 'CursorPagination', 'cursor_value': '{{ response.get(\"offset\", {}) }}', 'stop_condition': '{{ not response.get(\"offset\", {}) }}'}}}, 'schema_loader': {'type': 'InlineSchemaLoader', 'schema': {'type': 'object', '$schema': 'http://json-schema.org/schema#', 'additionalProperties': True, 'properties': {'id': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'permissionLevel': {'type': ['string', 'null']}}}}}}]}}, 'components_mapping': [{'type': 'ComponentMappingDefinition', 'field_path': ['name'], 'value': \"{{ stream_slice.extra_fields.name.replace(' ', '_').lower().strip() }}/{{ components_values.name.replace(' ', '_').lower().strip() }}/{{ components_values.id }}\"}, {'type': 'ComponentMappingDefinition', 'field_path': ['retriever', 'requester', 'path'], 'value': '{{ stream_slice.base_id }}/{{ components_values.id }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'requester', 'path'], 'value': '{{ stream_slice.base_id }}/{{ components_values.id }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['transformations', 0, 'fields', 0, 'value'], 'value': '{{ components_values.name }}'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'requester', 'path'], 'value': 'meta/bases/{{ stream_slice.base_id }}/tables'}, {'type': 'ComponentMappingDefinition', 'field_path': ['schema_loader', 'retriever', 'record_selector', 'record_filter', '$parameters', 'table_id'], 'value': '{{ components_values.id }}'}]} is not valid under any of the given schemas\n\nFailed validating 'anyOf' in schema['properties']['dynamic_streams']['items']['properties']['components_resolver']:\n {'anyOf': [{'$ref': '#/definitions/HttpComponentsResolver'},\n {'$ref': '#/definitions/ConfigComponentsResolver'}],\n 'description': 'Component resolve and populates stream templates with '\n 'components values.',\n 'title': 'Components Resolver'}\n\nOn instance['dynamic_streams'][0]['components_resolver']:\n {'components_mapping': [{'field_path': ['name'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ '\n \"stream_slice.extra_fields.name.replace(' \"\n \"', '_').lower().strip() }}/{{ \"\n \"components_values.name.replace(' ', \"\n \"'_').lower().strip() }}/{{ \"\n 'components_values.id }}'},\n {'field_path': ['retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ stream_slice.base_id }}/{{ '\n 'components_values.id }}'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ stream_slice.base_id }}/{{ '\n 'components_values.id }}'},\n {'field_path': ['transformations',\n 0,\n 'fields',\n 0,\n 'value'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ components_values.name }}'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'requester',\n 'path'],\n 'type': 'ComponentMappingDefinition',\n 'value': 'meta/bases/{{ stream_slice.base_id '\n '}}/tables'},\n {'field_path': ['schema_loader',\n 'retriever',\n 'record_selector',\n 'record_filter',\n '$parameters',\n 'table_id'],\n 'type': 'ComponentMappingDefinition',\n 'value': '{{ components_values.id }}'}],\n 'retriever': {'partition_router': {'parent_stream_configs': [{'extra_fields': [['name']],\n 'parent_key': 'id',\n 'partition_field': 'base_id',\n 'stream': {'name': 'bases',\n 'retriever': {'paginator': {'page_token_option': {'field_name': 'offset',\n 'inject_into': 'request_parameter',\n 'type': 'RequestOption'},\n 'pagination_strategy': {'cursor_value': '{{ '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'stop_condition': '{{ '\n 'not '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'type': 'CursorPagination'},\n 'type': 'DefaultPaginator'},\n 'record_selector': {'extractor': {'field_path': ['bases'],\n 'type': 'DpathExtractor'},\n 'record_filter': {'condition': '{{ '\n 'record.get(\"permissionLevel\") '\n '}}',\n 'type': 'RecordFilter'},\n 'type': 'RecordSelector'},\n 'requester': {'authenticator': {'authenticator_selection_path': ['credentials',\n 'auth_method'],\n 'authenticators': {'api_key': {'api_token': '{{ '\n \"config['credentials']['api_key'] \"\n '}}',\n 'type': 'BearerAuthenticator'},\n 'oauth2.0': {'client_id': '{{ '\n \"config['credentials']['client_id'] \"\n '}}',\n 'client_secret': '{{ '\n \"config['credentials']['client_secret'] \"\n '}}',\n 'grant_type': 'refresh_token',\n 'refresh_request_headers': {'Authorization': 'Basic '\n '{{ '\n \"[config['credentials']['client_id'], \"\n \"config['credentials']['client_secret']] \"\n '| '\n \"join(':') \"\n '| '\n 'base64encode '\n '}}',\n 'Content-Type': 'application/x-www-form-urlencoded'},\n 'refresh_token': '{{ '\n \"config['credentials']['refresh_token'] \"\n '}}',\n 'refresh_token_updater': {'access_token_config_path': ['credentials',\n 'access_token'],\n 'refresh_token_config_path': ['credentials',\n 'refresh_token'],\n 'refresh_token_error_key': 'error',\n 'refresh_token_error_status_codes': [400,\n 401],\n 'refresh_token_error_values': ['invalid_grant',\n 'invalid_client'],\n 'refresh_token_name': 'refresh_token',\n 'token_expiry_date_config_path': ['credentials',\n 'token_expiry_date']},\n 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token',\n 'type': 'OAuthAuthenticator'}},\n 'type': 'SelectiveAuthenticator'},\n 'error_handler': {'backoff_strategies': [{'backoff_time_in_seconds': 30,\n 'type': 'ConstantBackoffStrategy'}],\n 'max_retries': 10,\n 'response_filters': [{'action': 'FAIL',\n 'error_message': '{{ '\n \"'Personal \"\n 'Access '\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n 'add '\n 'all '\n 'required '\n 'permissions '\n 'to '\n 'existed '\n 'one '\n 'or '\n 'create '\n 'new '\n 'PAT, '\n 'see '\n 'docs '\n 'for '\n 'more '\n 'info: '\n \"https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' \"\n 'if '\n \"config.get('credentials', \"\n '{}). '\n \"get('auth_method', \"\n \"'') \"\n '== '\n \"'api_key' \"\n 'else '\n \"'Access \"\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n \"reauthenticate.' \"\n '}}',\n 'failure_type': 'config_error',\n 'predicate': '{{ '\n 'response.status_code '\n '== '\n '403 '\n 'and '\n \"response.get('error', \"\n \"{}).get('type') \"\n '== '\n \"'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' \"\n '}}',\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Permission '\n 'denied '\n 'or '\n 'entity '\n 'is '\n 'unprocessable.',\n 'failure_type': 'config_error',\n 'http_codes': [403,\n 422],\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Invalid '\n 'credentials '\n 'were '\n 'provided. '\n 'Please '\n 're-authenticate '\n 'to '\n 'restore '\n 'access '\n 'to '\n 'Airtable.',\n 'failure_type': 'config_error',\n 'http_codes': [401],\n 'type': 'HttpResponseFilter'}],\n 'type': 'DefaultErrorHandler'},\n 'http_method': 'GET',\n 'path': 'meta/bases',\n 'type': 'HttpRequester',\n 'url_base': 'https://api.airtable.com/v0/'},\n 'type': 'SimpleRetriever'},\n 'schema_loader': {'schema': {'$schema': 'http://json-schema.org/schema#',\n 'additionalProperties': True,\n 'properties': {'id': {'type': ['string',\n 'null']},\n 'name': {'type': ['string',\n 'null']},\n 'permissionLevel': {'type': ['string',\n 'null']}},\n 'type': 'object'},\n 'type': 'InlineSchemaLoader'},\n 'type': 'DeclarativeStream'},\n 'type': 'ParentStreamConfig'}],\n 'type': 'SubstreamPartitionRouter'},\n 'record_selector': {'extractor': {'field_path': ['tables'],\n 'type': 'DpathExtractor'},\n 'pagination_strategy': {'cursor_value': '{{ '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'stop_condition': '{{ '\n 'not '\n 'response.get(\"offset\", '\n '{}) '\n '}}',\n 'type': 'CursorPagination'},\n 'type': 'RecordSelector'},\n 'requester': {'authenticator': {'authenticator_selection_path': ['credentials',\n 'auth_method'],\n 'authenticators': {'api_key': {'api_token': '{{ '\n \"config['credentials']['api_key'] \"\n '}}',\n 'type': 'BearerAuthenticator'},\n 'oauth2.0': {'client_id': '{{ '\n \"config['credentials']['client_id'] \"\n '}}',\n 'client_secret': '{{ '\n \"config['credentials']['client_secret'] \"\n '}}',\n 'grant_type': 'refresh_token',\n 'refresh_request_headers': {'Authorization': 'Basic '\n '{{ '\n \"[config['credentials']['client_id'], \"\n \"config['credentials']['client_secret']] \"\n '| '\n \"join(':') \"\n '| '\n 'base64encode '\n '}}',\n 'Content-Type': 'application/x-www-form-urlencoded'},\n 'refresh_token': '{{ '\n \"config['credentials']['refresh_token'] \"\n '}}',\n 'refresh_token_updater': {'access_token_config_path': ['credentials',\n 'access_token'],\n 'refresh_token_config_path': ['credentials',\n 'refresh_token'],\n 'refresh_token_error_key': 'error',\n 'refresh_token_error_status_codes': [400,\n 401],\n 'refresh_token_error_values': ['invalid_grant',\n 'invalid_client'],\n 'refresh_token_name': 'refresh_token',\n 'token_expiry_date_config_path': ['credentials',\n 'token_expiry_date']},\n 'token_refresh_endpoint': 'https://airtable.com/oauth2/v1/token',\n 'type': 'OAuthAuthenticator'}},\n 'type': 'SelectiveAuthenticator'},\n 'error_handler': {'backoff_strategies': [{'backoff_time_in_seconds': 30,\n 'type': 'ConstantBackoffStrategy'}],\n 'max_retries': 10,\n 'response_filters': [{'action': 'FAIL',\n 'error_message': '{{ '\n \"'Personal \"\n 'Access '\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n 'add '\n 'all '\n 'required '\n 'permissions '\n 'to '\n 'existed '\n 'one '\n 'or '\n 'create '\n 'new '\n 'PAT, '\n 'see '\n 'docs '\n 'for '\n 'more '\n 'info: '\n \"https://docs.airbyte.com/integrations/sources/airtable#step-1-set-up-airtable' \"\n 'if '\n \"config.get('credentials', \"\n '{}). '\n \"get('auth_method', \"\n \"'') \"\n '== '\n \"'api_key' \"\n 'else '\n \"'Access \"\n 'Token '\n 'does '\n 'not '\n 'have '\n 'required '\n 'permissions, '\n 'please '\n \"reauthenticate.' \"\n '}}',\n 'failure_type': 'config_error',\n 'predicate': '{{ '\n 'response.status_code '\n '== '\n '403 '\n 'and '\n \"response.get('error', \"\n \"{}).get('type') \"\n '== '\n \"'INVALID_PERMISSIONS_OR_MODEL_NOT_FOUND' \"\n '}}',\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Permission '\n 'denied '\n 'or '\n 'entity '\n 'is '\n 'unprocessable.',\n 'failure_type': 'config_error',\n 'http_codes': [403,\n 422],\n 'type': 'HttpResponseFilter'},\n {'action': 'FAIL',\n 'error_message': 'Invalid '\n 'credentials '\n 'were '\n 'provided. '\n 'Please '\n 're-authenticate '\n 'to '\n 'restore '\n 'access '\n 'to '\n 'Airtable.',\n 'failure_type': 'config_error',\n 'http_codes': [401],\n 'type': 'HttpResponseFilter'}],\n 'type': 'DefaultErrorHandler'},\n 'http_method': 'GET',\n 'path': 'meta/bases/{{ '\n 'stream_partition.base_id '\n '}}/tables',\n 'type': 'HttpRequester',\n 'url_base': 'https://api.airtable.com/v0/'},\n 'type': 'SimpleRetriever'},\n 'type': 'HttpComponentsResolver'}\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/source_airtable/run.py\", line 25, in _get_source\n return SourceAirtable(\n ^^^^^^^^^^^^^^^\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/source_airtable/source.py\", line 13, in __init__\n super().__init__(catalog=catalog, config=config, state=state, **{\"path_to_yaml\": \"manifest.yaml\"})\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/yaml_declarative_source.py\", line 34, in __init__\n super().__init__(\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/concurrent_declarative_source.py\", line 93, in __init__\n super().__init__(\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 138, in __init__\n self._validate_source()\n File \"/Users/ajsteers/.local/share/uv/tools/airbyte-source-airtable/lib/python3.12/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py\", line 396, in _validate_source\n raise ValidationError(\njsonschema.exceptions.ValidationError: Validation against json schema defined in declarative_component_schema.yaml schema failed\n"}}}`
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.