airbytehq / airbytehq/airbyte-python-cdk

ManifestComponentTransformer._is_json_schema_object is order-sensitive: type: [object, null] schemas get $parameters propagated into every property

Đang mở Phù hợp với người mới
#1,137 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
community
Ngôn ngữ chính
Python
Star
26
Fork
53
Merge trung bình
2 ngày 6 giờ
Pull request đã merge (30 ngày)
10

Mô tả

## Symptom

When a manifest carries a `$parameters` block above an inline record schema (e.g. a stream definition used as a dynamic-stream template), and that schema's root declares `type: ["object", "null"]` (reversed order), the discovered `json_schema` gets polluted: `name` and `$parameters` keys are written onto the schema root, into every property dict, and appear as new property names under `properties` - so typed destinations create junk columns.

## Root cause

`airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py` (~lines 195-200 at 7.x):

```python
@staticmethod
def _is_json_schema_object(propagated_component: Mapping[str, Any]) -> bool:
return propagated_component.get("type") == "object" or propagated_component.get("type") == [
"null",
"object",
]
```

The guard that stops `$parameters` propagation at a JSON-schema boundary compares the `type` array as an exact ordered list literal. JSON Schema `type` arrays are unordered sets: `["object", "null"]` and `["null", "object"]` are semantically identical, but only the latter is recognised. With the reversed order the schema root is treated as a component, `propagate_types_and_parameters` recurses into `properties.*` (every property dict has a `type` key, also not recognised), and `$parameters`/`name` get attached at every level.

## Reproduction

Manifest with:

```yaml
definitions:
streams:
my_stream:
$parameters:
name: my_stream
schema_loader:
type: InlineSchemaLoader
schema:
$schema: https://json-schema.org/draft-07/schema#
type:
- object
- "null"
properties:
id: {type: ["null", "string"]}
```

Run discover >> the emitted schema root and `properties.id` carry `name: my_stream` and `$parameters: {name: my_stream}`, and `properties` gains bogus `name`/`$parameters` entries. Flip the root to `["null", "object"]` >> clean.

## Suggested fix

Compare as a set, e.g. `set(t) == {"null", "object"}` for list values (and arguably accept any `type` list that contains "object"), instead of the exact list literal.

## Impact / precedent

Found by a connector regression run during review of https://github.com/airbytehq/airbyte/pull/77625 (source-stripe): `schemas.payment_methods` had `type: [object, "null"]` and was the single stream whose discovered schema regressed once it moved under a dynamic-stream template with `$parameters`. Fixed connector-side by reordering the two array entries - a latent landmine for any manifest with a non-canonical `type` order under a `$parameters` scope.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start in airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py around _is_json_schema_object and trace propagate_types_and_parameters. Reproduce the inline schema shown, then compare discovery for both type-array orders. Done means the reversed ["object", "null"] root is recognized as a JSON Schema boundary and emitted schemas do not gain name, $parameters, or bogus properties.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
tooling
Loại issue
Lỗi
Độ khó
2/5
Thời gian dự kiến
1-3 giờ
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
84/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.