apple / apple/pfl-research

Visualize Histogram metrics in Wandb

Open
#92 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
358
Forks
43
Avg merge
9h 2m
Merged PRs (30d)
2

Description

pfl supports reporting metrics to Weights & biases with https://apple.github.io/pfl-research/reference/callback.html#pfl.callback.WandbCallback.
Currently we only report the line metrics printed out in console.
Wandb has support for visualizing histograms.
It can be very useful if we can make use of this feature to visualize histogram of a metric over the cohort of users.
This depends on having https://github.com/apple/pfl-research/issues/91 implemented such that we can use that data for wandb visualization.

Contributor guide

Open the contributing guide

Research direction

Start with the documented pfl.callback.WandbCallback entry point and review issue #91, which this work depends on. Determine how the cohort metric data becomes available, then verify that the resulting metric histogram is reported for visualization in Weights & Biases.

Written by the indexing model from the issue text.

Assessment

Domain
data-visualization, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.