awslabs / awslabs/python-deequ
spark-submit job is not exiting until I hit ctrl+C if I use Check class
Abierto
enhancement
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 826
- Forks
- 158
- Merge medio
- 9 d 22 h
- PR fusionados (30 d)
- 3
Descripción
**Is your feature request related to a problem? Please describe.**
spark-submit job is not exiting until I hit ctrl+C if I use Check function
**Describe the solution you'd like**
Check function would start a self._spark_session.sparkContext._gateway server. If my code is completed, the server is still hang on that causes the spark job can't stop normally.
Adding a close function to class Check
def close(self):
self._spark_session.sparkContext._gateway.close()
return
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.