aws / aws/sagemaker-experiments

Add method to log custom visualizations in `Tracker`

Open
#149 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
129
Forks
36
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
My feature is not related to a specific problem. However, it would be useful to have the ability to store custom plots that evaluate the results of each experiment. For example, we could store Shapley Feature Explanation plots or simple feature importance plots for each experiment. It would be helpful to have the ability to upload these images/plots as part of each trial.

**Describe the solution you'd like**
```python
with Tracker.create(display_name="Preprocessing", sagemaker_boto_client=sm) as tracker:
feat_importances = pd.DataFrame(model.feature_importances_, index=iris.feature_names, columns=["Importance"])
feat_importances.sort_values(by='Importance', ascending=False, inplace=True)
ft_importances_plot = feat_importances.plot(kind='bar', figsize=(8,6))
tracker.log_figure(name="feature-importances", ft_importances_plot)
```
Ideally, the `Tracker` class would have a method called `log_figure` that would allow us to log any figure that we would consider useful.

**Describe alternatives you've considered**
I have considered using the `log_artifact` method of the `Tracker` class. This method is not implemented specifically for visualizations.

**Additional context**
I believe Comet has this feature implemented [`Experiment.log_figure` method](https://www.comet.ml/docs/python-sdk/Experiment/#experimentlog_figure). It would be extremely useful to add this feature to sagemaker-experiments.

Thank you!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.