tensorflow / tensorflow/tensorboard

Adding new hyperparameters should be supported

Open
#2,014 5 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin:hparams type:feature
Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

Adding new hyperparameters should be supported

c/o @omoindrot, who says:

3. Adding new hyperparameters is not fully supported. If I add a new parameter new and re-create the event file for the HParams tab, the previous runs that don't contain new will all disappear if I choose to display new.
Ideally I would be able to add values to my old summaries, but I couldn't find a way to do that.
For instance if I add a batch_size parameter, I would need add the default value 32 to all previous runs.
In general it would be great to be able to manipulate old summaries easily.

One approach would be to modify the HParamInfo proto to
include an optional default_value attribute, which would be set
whenever a new hyperparameter is created and all previous runs had a
fixed value for the parameter (which should be the common case). Users
certainly shouldn’t have to modify the old event files for data from
previous runs to still be useful.

Ideally, the visualizations should be able to show partial data for runs
where one or more hyperparameters is missing: the table view could show
an empty cell to denote a missing value, the parallel coordinates view
could omit the line segments through an axis with missing data, and the
scatter plot matrix could simply omit some points.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the HParamInfo definition in tensorboard/plugins/hparams/api.proto and review the HParams tab visualizations described in the issue. Determine how missing hyperparameters should appear in the table, parallel coordinates, and scatter plot matrix, and define completion as preserving older runs while displaying partial data without requiring event-file edits.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data-visualization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.