bigscience-workshop / bigscience-workshop/Megatron-DeepSpeed

[Tensorboard] Log text prediction in evaluation

Open
#163 14 comments 1 reaction 0 assignees View on GitHub
Good First Issue
Dominant language
Python
Stars
1.4k
Forks
226
PR merge metrics
No merged PRs in 30d

Description

A very useful tool in order to understand model performance beyond obtaining loss: Actually show what are the predictions.

It'd be very useful to be able to "see" the output of the model during evaluation in text format. These should be logged in tensorboard. Tensorboard likely supports markdown style where you can put prediction in bold.

Maybe we can only print out the first batch as we should get a good amount of example from it.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.