iterative / iterative/cml

cml pr: is racy, generates a PR for different submissions

Open
#1,126 5 comments 0 reactions 1 assignee Claimed by @dacbd View on GitHub
cml-pr documentation
Dominant language
JavaScript
Stars
4.2k
Forks
345
PR merge metrics
No merged PRs in 30d

Description

Sorry for the title. I don't have a good way to express what went wrong here.

This happened twice in `ldb-hackathon` for me where it generated a PR, where the contents were quite different from the result that I was expecting. In fact, the same contents were already generated before my workflow even ran. I am not quite sure here why cml is creating a PR with content that is already in the repo (even if the `git pull` is done before `cml pr` to try to keep the repo in sync).

1. `13:53 GMT`: This PR [#94](https://github.com/iterative/ldb-hackathon/pull/94/files) was published.
2. [`13:55 GMT`](https://github.com/iterative/ldb-hackathon/actions/runs/2847051140): Triggered workflow.
3. [`13:57 GMT`](https://github.com/iterative/ldb-hackathon/runs/7807546288?check_suite_focus=true#step:1:1): Started "evaluate" job/`train` step.
4. [`16:47:50 GMT`](https://github.com/iterative/ldb-hackathon/runs/7807546288?check_suite_focus=true#step:13:28733): Finished training
5. [`16:47:58 GMT`](https://github.com/iterative/ldb-hackathon/runs/7807546288?check_suite_focus=true#step:15:13): Created a PR [#106](https://github.com/iterative/ldb-hackathon/pull/106/files) with identical contents to [#94](https://github.com/iterative/ldb-hackathon/pull/94/files).

```console
Run cml pr --rebase 'submissions/*.json'
cml pr --rebase 'submissions/*.json'
shell: sh -e {0}
env:
AWS_DEFAULT_REGION: us-east-1
AWS_REGION: us-east-1
AWS_ACCESS_KEY_ID: ***
AWS_SECRET_ACCESS_KEY: ***
AWS_SESSION_TOKEN: ***
REPO_TOKEN: ***
{"level":"warn","message":"Failed to enable auto-merge: Enable the feature in your repository settings: https://github.com/iterative/ldb-hackathon/settings#merge_types_auto_merge. Trying to merge immediately..."}
https://github.com/iterative/ldb-hackathon/pull/106(https://github.com/iterative/ldb-hackathon/runs/7807546288?check_suite_focus=true#step:15:6)
```

Also I am a bit surprised to see `cml pr` using same branch name for submissions, which may likely be the issue here.
The commits are same and is from earlier time.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.