airbytehq / airbytehq/airbyte-python-cdk

ManifestComponentTransformer._is_json_schema_object is order-sensitive: type: [object, null] schemas get $parameters propagated into every property

オープン 初心者向け
#1,137 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
community
主要言語
Python
スター
26
フォーク
53
平均マージ
2日 6時間
マージ済み PR(30日)
10

説明

## Symptom

When a manifest carries a `$parameters` block above an inline record schema (e.g. a stream definition used as a dynamic-stream template), and that schema's root declares `type: ["object", "null"]` (reversed order), the discovered `json_schema` gets polluted: `name` and `$parameters` keys are written onto the schema root, into every property dict, and appear as new property names under `properties` - so typed destinations create junk columns.

## Root cause

`airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py` (~lines 195-200 at 7.x):

```python
@staticmethod
def _is_json_schema_object(propagated_component: Mapping[str, Any]) -> bool:
return propagated_component.get("type") == "object" or propagated_component.get("type") == [
"null",
"object",
]
```

The guard that stops `$parameters` propagation at a JSON-schema boundary compares the `type` array as an exact ordered list literal. JSON Schema `type` arrays are unordered sets: `["object", "null"]` and `["null", "object"]` are semantically identical, but only the latter is recognised. With the reversed order the schema root is treated as a component, `propagate_types_and_parameters` recurses into `properties.*` (every property dict has a `type` key, also not recognised), and `$parameters`/`name` get attached at every level.

## Reproduction

Manifest with:

```yaml
definitions:
streams:
my_stream:
$parameters:
name: my_stream
schema_loader:
type: InlineSchemaLoader
schema:
$schema: https://json-schema.org/draft-07/schema#
type:
- object
- "null"
properties:
id: {type: ["null", "string"]}
```

Run discover >> the emitted schema root and `properties.id` carry `name: my_stream` and `$parameters: {name: my_stream}`, and `properties` gains bogus `name`/`$parameters` entries. Flip the root to `["null", "object"]` >> clean.

## Suggested fix

Compare as a set, e.g. `set(t) == {"null", "object"}` for list values (and arguably accept any `type` list that contains "object"), instead of the exact list literal.

## Impact / precedent

Found by a connector regression run during review of https://github.com/airbytehq/airbyte/pull/77625 (source-stripe): `schemas.payment_methods` had `type: [object, "null"]` and was the single stream whose discovered schema regressed once it moved under a dynamic-stream template with `$parameters`. Fixed connector-side by reordering the two array entries - a latent landmine for any manifest with a non-canonical `type` order under a `$parameters` scope.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start in airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py around _is_json_schema_object and trace propagate_types_and_parameters. Reproduce the inline schema shown, then compare discovery for both type-array orders. Done means the reversed ["object", "null"] root is recognized as a JSON Schema boundary and emitted schemas do not gain name, $parameters, or bogus properties.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
tooling
issue の種類
バグ
難易度
2/5
見積もり時間
1〜3時間
活発さ
活発
明瞭さ
明確に書かれている
初心者へのやさしさ
84/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。