Model builder training appears to leak data somehow into the training set

Windwos 11
ML.Net 3.0.1
.Net 4.8

When traingin a large csv my model would get consistently high results that I could not replicate in testing outside of model builder. I was letting model builder handle the trainign/validation split, though I tried all those options. Folds, 70/30, 80/20, etc. Always ended up >90% micro accuracy over training time if left, but never got even close when run in real time. After many days - I today split the SAME data file into 2 different files, telling model builder the validation data is in that separate file, and hey presto, can;t train more than 45%... This is better (for worse!). The 2 files are a 80/20 split - I just did it myself. Give model builder the whole file and tell it to do the 80/20 split, and it will train to >93% again. Something in there is broken it seems! So little visibility for me into what is going on, I don't have much more to offer in terms of what. it would appear the validation data is somehow leaked into the training set.

Seperate validation file

Combined file letting model builder do the split will train to >0.93, same data and metrics.

Model builder version is 17.18.2.2415501

Riproduci la discrepanza con lo stesso CSV diviso in una coppia 80/20, quindi confrontala con Model Builder che esegue la suddivisione internamente. Inizia esaminando i due workflow di training e validazione e le metriche riportate; il lavoro è completo quando la suddivisione del file combinato non produce più un’accuratezza fuorviante rispetto al file di validazione separato.

Scritto dal modello di indicizzazione a partire dal testo della issue.