tensorflow / tensorflow/tensorboard

Feature request for visualising groups of runs

Open
#1,450 7 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type:feature
Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

Currently, when having multiple event files in some subdirectory of the log directory, it is assumed that these event files are to be appended to get one single graph. This is rather often not the case, however, which leads to a bunch of users ending up with a bunch of useless lines connecting the different runs, e.g. as illustrated here.

Now, I wanted to conduct several experiments and obviously repeat the experiments (with the same parameters) to assure these results are reproducible. This means that I have different runs (with different parameters) that are repeated for the sake of reproducibility. As a good tensorboard user, I put each run in its own subdirectory, but in the end, I have a multi-coloured chaos of learning curves from which I can barely learn anything. It is mostly doable to compare the different runs, but the repetitions have completely different colours and make it almost impossible to see things.

What I would like to see now, is some option that allows to use a single directory to store repeated runs of a single experiment that does away with the connections between different runs in the plots. Ideally, this option would also allow to plot the mean curve with error bars (over all runs) rather than the separate events.

I am not sure whether my explanation makes a lot of sense or how feasible my suggestions are, but it would be really helpful to plot multiple events in some subdirectory in a single colour.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points; begin by tracing how TensorBoard discovers event files in subdirectories and renders their learning curves. Done should mean repeated runs can be grouped without connecting lines, with the desired scope clarified for shared colors, mean curves, and error bars.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data-visualization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.