AzureML TabularDataSet via parquet and pandas index error

Azure's TabularDataset implementation introduces an index, \_\_index\_level_0\_\_ when creating or reading parquet files that were originally written by Pandas/Python. This occurs when an index is unnamed but has been modified at some point; if an index is named we get an extra column with the same name as the index.

When making changes to datasets, this additional field causes Azure errors if not handled. Depending on what's been done to the index of the original dataset, you may or may not get that additional field.

I have an example notebook that can be run to reproduce the issue. It's here: https://github.com/vla6/Azure_notes/blob/main/tabulardataset_parquet_index_di_issue.ipynb

The notebook requires an Azure Machine Learning workspace and a storage account to run