[RFC] Extending the Tensorboard Notebook Display API

Three possible enhancements are proposed below to improve the Tensorboard experience on various notebooks environments. Please add your comments or concerns.

Custom Display API

A recent pull request made it possible for users to run Tensorboard in environments where the Jupyter host machine is not directly exposed to the notebook user. We are proposing to make the notebook display API more flexible.

The proposed enhancement is an API that lets users to directly set an inline display function that works in their environment, providing more flexibility for Tensorboard to be used on various notebook environments.

Here is an example use-case of this API.

# In the notebook Python shell init script
def my_custom_display(port, height, display_handle)
    del display_handel
    html_template = …
    display_function(html_template) # this function is specific to the cloud environment

# import hook
Import wrapt
@wrapt.when_import(“tensorboard.notebook”)
def hook_tensorboard(module):
    module.register_custom_display(my_custom_display)

# In notebook cells
%load_ext tensorboard

%tensorboard --logdir logs
# uses the custom display function, which successfully renders the Tensorboard UI inline 

Flag to prevent process reuse

Process reuse is a convenient feature when a single user runs Tensorboard on their personal machine. However, in multi-tenant environments where users share the same machine or container, reusing processes can lead to mutual interference. One possible scenario: user A starts a process on a log directory and user B starts tensorboard on the same directory, hence reusing user A's process. User B kills the Tensorboard process and user A's Tensorboard UI becomes stale (to user A’s surprise).

By adding a flag to Tensorboard that prevents process re-use, e.g.--new_process, users can set that flag by default to prevent interfering with each other on a multi-tenant environment

Increasing the limit for the number of concurrent processes

Currently, there is a default and hard-coded limit of 10 concurrent Tensorboard processes in the port selection logic. For some environments, e.g. multi-tenant environments with powerful computing resources this is too restrictive. We propose increasing this default limit to 100.

Start by reviewing the linked pull request 3674 and the tensorboard.notebook entry point, then inspect the port selection logic mentioned for concurrent processes. The RFC covers custom display registration, a --new_process flag, and increasing the process limit; done would require a decided scope and implementation plan for these proposals.