tensorflow / tensorflow/tensorboard

XLA not working

Open
#2,767 0 comments 2 reactions 1 assignee View on GitHub

@davidsoergel is already working on this.

Since Oct 14, 2019.

core:frontend plugin:graph stat:awaiting tensorflower type:support
Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

hello, everyone
As the left sidebar shows, the tensorboard 1.14 .0 has supported display a graph with XLA enabled, but in my XLA test, the graph in tensorboard was not converted as i wish: the nodes added to the same XLA cluster would be replaced by a XlaCompileOp + XlaRunOp. At present, i can only figure out which node is compiled by its attribute, which way i still doubted because it seemed that the function related to XLA does not work well,
the source code i tested:

mnist_with_summaries.py
to enable XLA, i turn JIT by modifying the function:

  def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):
    """Reusable code for making a simple neural net layer.
    It does a matrix multiply, bias add, and then uses ReLU to nonlinearize.
    It also sets up name scoping so that the resultant graph is easy to read,
    and adds a number of summary ops.
    """
    jit_scope = tf.contrib.compiler.jit.experimental_jit_scope(compile_ops=True)
    with jit_scope:
      # Adding a name scope ensures logical grouping of the layers in the graph.
      with tf.name_scope(layer_name):
        # This Variable will hold the state of the weights for the layer
        with tf.name_scope('weights'):
          weights = weight_variable([input_dim, output_dim])
          variable_summaries(weights)
        with tf.name_scope('biases'):
          biases = bias_variable([output_dim])
          variable_summaries(biases)
        with tf.name_scope('Wx_plus_b'):
          preactivate = tf.matmul(input_tensor, weights) + biases
          tf.summary.histogram('pre_activations', preactivate)
      activations = act(preactivate, name='activation')
      tf.summary.histogram('activations', activations)
      return activations

the following is the original graph when the "Default" tag is selected:
Screen Shot 2019-10-13 at 20 00 15

but when i chose the after several steps, the graph made me confused.
Screen Shot 2019-10-13 at 19 50 12
Screen Shot 2019-10-13 at 19 50 33

Question 1: the "XLA Cluster" was still unselectable even though XLA was enabled
Question 2: Intuitively, all the converted to brown(unused structure) were added to a XLA Cluster, but i could not know that according to the attribute of the selected node. So, why this node changed to unused?
Question 3: just as question 2,the node in the middle graph, as i selected, clearly belonged to a XLA Cluster

In a word, i am confused by the current XLA function in Tensorboard. if these questions were caused by my unfamiliarity of tensorboard , thank you very much for giving me some help.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.