awslabs / awslabs/python-deequ
spark-submit job is not exiting until I hit ctrl+C if I use Check class
Đang mở
enhancement
- Ngôn ngữ chính
- Jupyter Notebook
- Star
- 826
- Fork
- 158
- Merge trung bình
- 9 ngày 22 giờ
- Pull request đã merge (30 ngày)
- 3
Mô tả
**Is your feature request related to a problem? Please describe.**
spark-submit job is not exiting until I hit ctrl+C if I use Check function
**Describe the solution you'd like**
Check function would start a self._spark_session.sparkContext._gateway server. If my code is completed, the server is still hang on that causes the spark job can't stop normally.
Adding a close function to class Check
def close(self):
self._spark_session.sparkContext._gateway.close()
return
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.