awslabs / awslabs/python-deequ
spark-submit job is not exiting until I hit ctrl+C if I use Check class
Offen
enhancement
- Vorherrschende Sprache
- Jupyter Notebook
- Sterne
- 826
- Forks
- 158
- Ø Merge
- 9 T. 22 Std.
- Gemergte PRs (30 T.)
- 3
Beschreibung
**Is your feature request related to a problem? Please describe.**
spark-submit job is not exiting until I hit ctrl+C if I use Check function
**Describe the solution you'd like**
Check function would start a self._spark_session.sparkContext._gateway server. If my code is completed, the server is still hang on that causes the spark job can't stop normally.
Adding a close function to class Check
def close(self):
self._spark_session.sparkContext._gateway.close()
return
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.