airbytehq / airbytehq/PyAirbyte
Support for Prefect 3
- Lenguaje dominante
- Python
- Estrellas
- 344
- Forks
- 77
- Merge medio
- 1 d 11 h
- PR fusionados (30 d)
- 35
Descripción
### Topic
Make Airbyte compatible with Prefect 3
### Relevant information
Since we are blocked by [parallelism support](https://github.com/airbytehq/airbyte/issues/7750) we are running Prefect + Pyairbyte and want to upgrade to Prefect 3 but are blocked on the dependency of airbyte-cdk (which is a dependency of pyairbyte) on jsonschema = "~3.2.0" and pendulum = "<3.0.0".
[Prefect 3](https://docs.prefect.io/3.0/resources/whats-new-prefect-3) brings a lot of performance improvements to utilise pyairbyte to the fullest.
Moreover,
1. jsonschema 4.0.0 was released on Sep 30, 2021 and not upgrading does not make sense.
2. pendulum 3.0.0 brings lots of performance improvement with extensions rewritten in Rust
Upgrading these core dependencies ensures compatibility across various other integrations with airbyte.
Guía de contribución
Línea de trabajo
No file or test is named. Start by locating the airbyte-cdk dependency declarations for jsonschema ~3.2.0 and pendulum <3.0.0, then inspect how those constraints affect PyAirbyte and Prefect 3 integration. Done means the dependency constraints are upgraded and compatibility with Prefect 3 is verified.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- python
- Área
- data-engineering
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100