Tensorboard Callback profile_batch causes Segmentation Fault

Environment information

Suggestion: Fix conflicting installations

Namely:
pip uninstall tensorboard tensorflow tensorflow-estimator tensorflow-gpu
pip install tensorflow # or tensorflow-gpu, or tf-nightly, ...

Issue description

System:

• Ubuntu 18.04
• TF2.0

Error:

I am using tf.keras and the model.fit() function to train my model. I have added the tf.keras.callbacks.TensorBoard callback to my .fit() call.

I am encountering an issue where if:

• profile_batch=2 --> Segmentation Fault (core dumped) when a call is made to .fit()
• profile_batch=0 --> No segfault (disabled profiling)

Note: This error only occurs sometimes; Run the model.fit() a couple of times to reproduce the error; This error happens independent of the directory being logged to

Background Info:

Currently, the input to my model is in the form of a tf.data.Dataset. The documentation for the callback says:

profile_batch: Profile the batch to sample compute characteristics. By default, it will profile the second batch. Set profile_batch=0 to disable profiling. Must run in TensorFlow eager mode.

I use the dataset.map() function in my input pipeline to transform my input data. However, since .map() does not execute eagerly, I wrap it around a tf.py_function, which should make it execute eagerly (I've verified that the map function does in fact run eagerly after using py_function).

However, sometimes, I still get a Segmentation Fault error, as described above. On the occasion where the Segmentation Fault does not occur, the profile logged to the Tensorboard is a .profile-empty file.

https://github.com/tensorflow/tensorboard/issues/2084 may be related.