airbytehq / airbytehq/PyAirbyte

Support for Prefect 3

Offen
#365 7 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
344
Forks
77
Ø Merge
1 T. 11 Std.
Gemergte PRs (30 T.)
35

Beschreibung

### Topic

Make Airbyte compatible with Prefect 3

### Relevant information

Since we are blocked by [parallelism support](https://github.com/airbytehq/airbyte/issues/7750) we are running Prefect + Pyairbyte and want to upgrade to Prefect 3 but are blocked on the dependency of airbyte-cdk (which is a dependency of pyairbyte) on jsonschema = "~3.2.0" and pendulum = "<3.0.0".
[Prefect 3](https://docs.prefect.io/3.0/resources/whats-new-prefect-3) brings a lot of performance improvements to utilise pyairbyte to the fullest.
Moreover,
1. jsonschema 4.0.0 was released on Sep 30, 2021 and not upgrading does not make sense.
2. pendulum 3.0.0 brings lots of performance improvement with extensions rewritten in Rust

Upgrading these core dependencies ensures compatibility across various other integrations with airbyte.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.