awslabs / awslabs/python-deequ

spark-submit job is not exiting until I hit ctrl+C if I use Check class

Aperta
#7 7 commenti 2 reazioni 1 assegnatario Rivendicata da @jaoanan1126 Vedi su GitHub
enhancement
Lingua principale
Jupyter Notebook
Stelle
826
Fork
158
Merge medio
9g 22h
PR unite (30g)
3

Descrizione

**Is your feature request related to a problem? Please describe.**

spark-submit job is not exiting until I hit ctrl+C if I use Check function

**Describe the solution you'd like**

Check function would start a self._spark_session.sparkContext._gateway server. If my code is completed, the server is still hang on that causes the spark job can't stop normally.

Adding a close function to class Check

def close(self):
self._spark_session.sparkContext._gateway.close()
return

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.