tensorflow / tensorflow/tensorboard
Discrete histograms/counters across steps/epochs
@wchargin is already working on this.
Since Mar 5, 2021.
- Dominant language
- TypeScript
- Stars
- 7.2k
- Forks
- 1.7k
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 1
Description
I'm currently tracking several discrete values, e.g. the number of iterations a loop has gone through, over several training steps and dumping them to a tf.summary.histogram at the end of each epoch.
My current approach amounts to storing counts of the values seen in a counts = tf.Variable(shape=[max_counter_value], initializer='zeros', dtype=tf.int32, trainable=False) and then constructing a "fake" tf.summary_histogram, e.g.
data = tf.repeat(
tf.range(max_counter_value, dtype=tf.float32),
repeats=counts)
tf.summary.histogram(name, data, step=step, buckets=2 * max_counter_value - 1)
This is a bit clunky for several reasons:
- I need to inflate the
countsinto thedataTensor, which can contain several thousands/millions of values, even thoughtf.summary.histogramprobably re-computescountsinternally, - Plotting the data in TensorBoard smears the discrete values over several buckets, so I never actually see the total count,
- (Somewhat tangentially) I have to handle accumulating and flushing the counters myself.
What would be really super cool would be a function named something like tf.summary.counter(value: tf.Tensor, name: str, flush_every_n_epochs:int = 1) where I can just dump in Tensors of integer types and get the discrete (unsmoothed) histograms every 'n' epochs.
I'm guessing the third part (accumulating values across steps) is probably a bit iffy since it would require maintaining some kind of state, but I'm hoping something like calling tf.summary.histogram with a set of pre-bucketed counts should be possible?
Cheers, Pedro
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.