Provide test datasets where labels are hidden

Description

My understanding is that when creating a task, the selected estimation procedure defines the train/test splits of the dataset. When creating a run, a model is evaluated on the given dataset using some estimation procedure such as cross-validation.

Is it possible to provde hidden test sets for evaluation of solutions? There is a "Note" about that on https://openml.github.io/OpenML/#circles-under-construction in the section about Tasks but I'm not sure if this version of the docs is up-to-date.

For my use case, this issue is connected to openml/openml-python#1231. I would like the user to be able to upload the predictions for a test set where the labels are hidden, so the user has no access to it. Is this possible by default? Or would I need to create a separate task with a separate dataset that only contains X, but no labels? What would be the recommende approach here?

Thanks!

Start with the Tasks documentation at the linked OpenML page and review the related openml-python#1231 issue. Determine whether hidden-label test sets and prediction uploads are supported, then document the recommended workflow or define the required feature and its completion criteria.