airbytehq / airbytehq/airbyte-python-cdk

Bug: Declarative component schema should validate inline stream schemas against JSON Schema spec

未關閉
#832 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
26
分支
53
平均合併
2 天 6 小時
30 天內合併 PR
10

描述

Workaround in MCP:

- https://github.com/airbytehq/connector-builder-mcp/pull/175

## Current Behavior

The `InlineSchemaLoader.schema` field in `declarative_component_schema.yaml` is currently defined as:

```yaml
schema:
type: object
additionalProperties: true
```

This allows any object, including invalid JSON schemas. For example, these invalid schemas currently pass validation:

Example 1: Invalid type value

```
schema:
type: object
properties:
id:
type: not_a_valid_type # Should fail but doesn't
```

Example 2: Malformed structure
schema: "string instead of object" # Causes Python error later

Example 3: Invalid property definitions

```
schema:
type: object
properties:
field: "should be object with type" # Invalid structure
```

Problem

Invalid schemas pass manifest validation but cause issues later:

1. Cryptic Python errors during stream reading - e.g., 'str' object has no attribute 'get' when schema is a string
2. Runtime failures instead of validation-time failures - errors only appear when trying to read records
3. Poor developer experience - no clear feedback about what's wrong with the schema

Expected Behavior

The declarative component schema should validate that InlineSchemaLoader.schema is a valid JSON Schema conforming to the JSON Schema Draft 7
specification.

Invalid schemas should be caught during manifest validation with clear error messages like:
- "Invalid JSON Schema type: 'not_a_valid_type' is not a valid type"
- "Schema must be an object, got string"
- "Properties must be objects with type definitions"

Proposed Solution

Update the InlineSchemaLoader definition in declarative_component_schema.yaml to validate against the JSON Schema meta-schema.

One approach would be to add a reference to the JSON Schema Draft 7 meta-schema:

```
InlineSchemaLoader:
properties:
schema:
title: "Schema"
description: "Describes a streams' schema..."
type: object
# Validate that this is a valid JSON Schema
"$ref": "http://json-schema.org/draft-07/schema#"
```

Alternatively, add custom validation logic in the CDK to ensure the schema structure is valid before attempting to use it.

Impact

This affects all connector builders using declarative YAML manifests with inline schemas. The fix would:
- Catch schema errors earlier in the development process
- Provide clearer error messages
- Improve developer experience
- Prevent runtime errors from malformed schemas

Workaround

We've added validation in our MCP server as a temporary workaround, but this should be fixed upstream in the CDK for all users.

Environment

- CDK version: 6.61.6+ (latest)
- Component: airbyte_cdk.sources.declarative.declarative_component_schema.yaml

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。