airbytehq / airbytehq/airbyte-python-cdk
ManifestComponentTransformer._is_json_schema_object is order-sensitive: type: [object, null] schemas get $parameters propagated into every property
- Dominant language
- Python
- Stars
- 26
- Forks
- 53
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 10
Description
## Symptom
When a manifest carries a `$parameters` block above an inline record schema (e.g. a stream definition used as a dynamic-stream template), and that schema's root declares `type: ["object", "null"]` (reversed order), the discovered `json_schema` gets polluted: `name` and `$parameters` keys are written onto the schema root, into every property dict, and appear as new property names under `properties` - so typed destinations create junk columns.
## Root cause
`airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py` (~lines 195-200 at 7.x):
```python
@staticmethod
def _is_json_schema_object(propagated_component: Mapping[str, Any]) -> bool:
return propagated_component.get("type") == "object" or propagated_component.get("type") == [
"null",
"object",
]
```
The guard that stops `$parameters` propagation at a JSON-schema boundary compares the `type` array as an exact ordered list literal. JSON Schema `type` arrays are unordered sets: `["object", "null"]` and `["null", "object"]` are semantically identical, but only the latter is recognised. With the reversed order the schema root is treated as a component, `propagate_types_and_parameters` recurses into `properties.*` (every property dict has a `type` key, also not recognised), and `$parameters`/`name` get attached at every level.
## Reproduction
Manifest with:
```yaml
definitions:
streams:
my_stream:
$parameters:
name: my_stream
schema_loader:
type: InlineSchemaLoader
schema:
$schema: https://json-schema.org/draft-07/schema#
type:
- object
- "null"
properties:
id: {type: ["null", "string"]}
```
Run discover >> the emitted schema root and `properties.id` carry `name: my_stream` and `$parameters: {name: my_stream}`, and `properties` gains bogus `name`/`$parameters` entries. Flip the root to `["null", "object"]` >> clean.
## Suggested fix
Compare as a set, e.g. `set(t) == {"null", "object"}` for list values (and arguably accept any `type` list that contains "object"), instead of the exact list literal.
## Impact / precedent
Found by a connector regression run during review of https://github.com/airbytehq/airbyte/pull/77625 (source-stripe): `schemas.payment_methods` had `type: [object, "null"]` and was the single stream whose discovered schema regressed once it moved under a dynamic-stream template with `$parameters`. Fixed connector-side by reordering the two array entries - a latent landmine for any manifest with a non-canonical `type` order under a `$parameters` scope.
Contributor guide
Research direction
Start in airbyte_cdk/sources/declarative/parsers/manifest_component_transformer.py around _is_json_schema_object and trace propagate_types_and_parameters. Reproduce the inline schema shown, then compare discovery for both type-array orders. Done means the reversed ["object", "null"] root is recognized as a JSON Schema boundary and emitted schemas do not gain name, $parameters, or bogus properties.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100