tensorflow / tensorflow/tensorboard

hparams with separate train and validation summary writers

Open
#3,675 5 comments 0 reactions 1 assignee View on GitHub

@bmd3k is already working on this.

Since May 29, 2020.

core:frontend plugin:hparams stat:awaiting tensorflower type:support
Dominant language
TypeScript
Stars
7.2k
Forks
1.7k
Avg merge
4d 22h
Merged PRs (30d)
1

Description

I have a model that I train with hp.KerasCallback and TensorBoard callback as explained in the example:

model.fit(
    ...,
    callbacks=[
        tf.keras.callbacks.TensorBoard(logdir),  # log metrics
        hp.KerasCallback(logdir_hp, hparams),  # log hparams
    ],
)

This successfully prints all my input (hyper)parameters (in hparams dict) and

  • train metrics, if logdir_hp is /train (or a summary writer to train dir)
  • validation metrics, if logdir_hp is /validation (or a summary writer to validation dir)

but never both because of the ambiguous summary tensor name!

I expect to log the same metric but in both runs (train and validation) using prefixes (like acc and val_acc) for better analysis

image

The above result was obtained using estimator api which automatically appends the run name based on the folder (train on current folder, so no prefix, for validation it's eval and I had an additional evaluation every x steps saved in global_eval), it would be great to achieve the same in keras model.fit()

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.