airbytehq / airbytehq/airbyte-python-cdk

Improve error messages when json schema validation fails (Pydantic discriminators)

未關閉
#615 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
26
分支
53
平均合併
2 天 6 小時
30 天內合併 PR
10

描述

We get really noising and difficult to parse error messages when manifests don't successfully validate against the json schema.

JSON Schema (and Pydantic) have a feature called "discriminator" which can help with this:

https://docs.pydantic.dev/latest/concepts/unions/#discriminated-unions

> When Union validation fails, **error messages can be quite verbose**, as they will produce validation errors for each case in the union. This is especially noticeable when dealing with recursive models, where reasons may be generated at each level of recursion. **Discriminated unions help to simplify error messages in this case, as validation errors are only produced for the case with a matching discriminator value.**

Today: we get a huge error of the input not matching one of n "anyOf" conditions, which could be tens or hundreds of lines of error, without explicitly naming the field that was extra or missing.

Afterwards: we would get an error like `field 'foo' is missing` or `unexpected field 'food' field`

Sample code from the docs:

```py
class Model(BaseModel):
pet: Cat | Dog | Lizard = Field(discriminator='pet_type')
n: int
```

Taking our [`SimpleRetriever.selector`](https://github.com/airbytehq/airbyte/blob/57acc8b84c0b43cc836ab7b79ec02c60e18d2aab/airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/migrate_to_manifest_only/declarative_component_schema.py#L1545-L1551) field, for example:

```py
class SimpleRetriever(BaseModel):
...
requester: Union[CustomRequester, HttpRequester] = Field(
...,
description="Requester component that describes how to prepare HTTP requests to send to the source API.",
)
```

We'd update as follows to declare `discriminator`:

```py
class SimpleRetriever(BaseModel):
...
requester: Union[CustomRequester, HttpRequester] = Field(
...,
description="Requester component that describes how to prepare HTTP requests to send to the source API.",
discriminator="type",
)
```

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。