log streaming does not terminate if a job is terminated while streaming logs
Open
@pavithrasv is already working on this.
Since Jul 17, 2020.
bug
- Dominant language
- Python
- Stars
- 383
- Forks
- 93
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 1
Description
I started a job with log streaming enabled. While the job was running I manually terminated the job in AI Platform training by clicking on stop job in the UI. The log streaming continued to run after 5 minutes until manual termination with the following:
File ".../cloud/tensorflow_cloud/deploy.py", line 211, in _stream_logs
output = process.stdout.readline()
I expected when the job is terminated on AI Platform training the log streaming should also terminate / stop.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.