tensorflow / tensorflow/tensorboard

Implement liveness check for notebook extensions

Open
#2,483 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

core:backend os:windows type:bug
Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

Currently, each TensorBoard process writes its meta-information to a
file in the shared .tensorboard-info temp directory, and tries to
clean up the file on graceful exit. This has two problems on Windows:

  • The base temporary directory %TMP% is never automatically cleaned,
    even after logout or reboot.
  • It is not possible to gracefully shut down an arbitrary process
    given its PID.

The result is that most any TensorBoards started by %tensorboard will
leave their info files around forever, unless manually cleaned up, and
the instructions suggested to the user (“use !kill …”) are not
adequate to effect this cleanup. See #2481.

We can ameliorate this by implementing the liveness check mentioned in a
TODO in manager.py, cleaning up the dead info file on failure:

https://github.com/tensorflow/tensorboard/blob/c1c2771a5ed5ce1b9ad605dd1834616905c2f190/tensorboard/manager.py#L453-L458

We can also provide a notebook.kill(pid) function with implementation
something vaguely like

def kill(pid):
  if os.name == "nt":
    subprocess.check_output(["taskkill", "/pid", str(int(pid)), "/f"])
    manager.remove_info_file(pid)
  else:
    os.kill(pid, signal.SIGTERM)

and then replace the “reusing TensorBoard” user-facing messaging on
Windows with something like

template = (
    "Reusing TensorBoard on port {port} (pid {pid}), started {delta} ago. "
    "To kill it, run `from tensorboard import notebook; notebook.kill({pid})`."
)

though that incantation still is a bit of a mouthful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the liveness-check TODO in manager.py around the referenced lines, then inspect the notebook entry point and existing info-file handling. Trace how dead processes are detected and how Windows cleanup differs from other platforms. Done means dead info files are removed, notebook.kill(pid) handles the platform-specific shutdown, and the Windows reuse message points users to it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.