OpenML Python runs may have swapped truth and prediction labels (at least for classification, regression)

Description

For a small set of flows, the predictions.arff files of some runs contain faulty entries. In these entries, the prediction does not correspond to the class with the highest confidence.

As far as I was able to find out, all affected flows are sklearn pipelines and published/uploaded using openml-python.
Moreover, the confidences of these pipelines should be, to the best of my knowledge, representative for the prediction (unlike, for example, the confidences of SVM).
Furthermore, the confidences are off by a large margin. This is not a result of two or more classes having almost equal confidences or a precision problem.

Example

Flow 19039 with Run 10581112 and the associated predictions file.

Expected Results

The predictions in the predictions.arff should correspond to the class with the highest confidence in the predictions.arff.

Actual Results

The predictions in the predictions.arff correspond to the class with the second highest confidence. In other cases, the prediction does not correspond to a high-confidence class at all but seems to be chosen at random.

Affected Flows

In my research, I have found the following list of flows to run into this problem at least once: [19030, 19037, 19039, 19035, 18818, 17839, 17761].
These include sklearn pipelines using decision trees (19030, 18818), Gradient Boosting (19307,19039), KNN (19035), SGD (17839), and LDA (17761).

Versions

I assume that the flows [19030, 19037, 19039, 1903] used the newest version of openml-python based on their upload date and feedback gather by the original uploader. For the other flows, I am not certain which version was used.

Start by reproducing the listed OpenML flows and inspecting their predictions.arff files, focusing on whether the predicted class matches the highest confidence. Compare the affected sklearn pipelines and openml-python upload behavior; done means identifying and correcting the cause so generated predictions consistently match their highest confidence class.