Is PyDeequ 2.0 still being actively developed? State of PyDeequ in general

We’re currently running PyDeequ on Spark 3.5 and planning a future migration to Spark 4.1 / Delta Lake 4.1 / Python 3.13. Before we commit further to PyDeequ, we’d like to understand the project’s current state.

The 2.0.0b1 beta was announced in January 2025 with Spark Connect support — great direction, since _jvm bridge calls aren’t supported under Spark Connect which is the default in Spark 4.x. However, the last PyPI release was v1.5 in April 2025 and there’s been no follow-up since.

Two questions:

1. Is the project in active development or maintenance mode?

2. Is there a plan to support Spark 4.1 and get 2.0 published to PyPI?

This will directly inform whether we continue building on PyDeequ or evaluate alternatives ahead of our migration. Thanks for any visibility.

Leggi prima l’issue e la relativa discussione di 12 commenti; non sono identificati alcun file sorgente, test o punto di ingresso. La discussione richiede una decisione da parte di un maintainer sullo stato dello sviluppo, sul supporto a Spark 4.1 e sulla pubblicazione o meno di PyDeequ 2.0 prima che si possa definire qualsiasi contributo concreto.