iterative / iterative/cml

Bug: cml comment --publishNative on gitlab ignores large amount of images in stacked markdown spans

Open
#1,446 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
JavaScript
Stars
4.2k
Forks
345
PR merge metrics
No merged PRs in 30d

Description

## Description

We currently use `cml-publish` to let CML add plots of different metrics to our merge requests so we can do a visual check of the training results. Since `cml-publish` is deprecated, we want to use `cml comment create --publishNative` instead. At the moment, there are >300 plots. They are arranged in stacked markdown spans so we can keep the irrelevant ones hidden.

`cml comment create --publishNative` only uploads 49 of the images in the markdown file and its own watermark. The "metric plots" span contains five more spans for different customers that in turn contain multiple plots each. `cml comment create --publishNative` only properly uploads the plots for the first customer. The other customers just get one plot each with the exception of testcustomerB, who gets 6.

I have attached an anonymized example:
[report_before_cml_comment.md](https://github.com/iterative/cml/files/15024524/report_before_cml_comment.md)
[report_after_cml_comment.md](https://github.com/iterative/cml/files/15024525/report_after_cml_comment.md)

## Reproduction

Run report_before_cml_comment.md through a gitlab pipeline using `cml comment --publishNative` (CML 0.20.0) and have any images in the paths it uses.
Observe it creating a comment with similar content to report_after_cml_comment.md

## Expected behaviour

`cml comment create --publishNative` should upload all images in the markdown file.
If anything prevents it from doing that, it should throw an error instead of failing silently.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.