awslabs / awslabs/python-deequ
spark-submit job is not exiting until I hit ctrl+C if I use Check class
Aperta
enhancement
- Lingua principale
- Jupyter Notebook
- Stelle
- 826
- Fork
- 158
- Merge medio
- 9g 22h
- PR unite (30g)
- 3
Descrizione
**Is your feature request related to a problem? Please describe.**
spark-submit job is not exiting until I hit ctrl+C if I use Check function
**Describe the solution you'd like**
Check function would start a self._spark_session.sparkContext._gateway server. If my code is completed, the server is still hang on that causes the spark job can't stop normally.
Adding a close function to class Check
def close(self):
self._spark_session.sparkContext._gateway.close()
return
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.