Column names with '\%' are renamed
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 755
- Forks
- 128
- PR merge metrics
- No merged PRs in 30d
Description
Description
Datasets with column names that contain '%' are escaped by pandas, but not in the list of feature names.
That creates errors whenever you first get the list of features and then try to look up some of them.
Not sure what the best action is. One possible solution is to always escape them, also in the feature list created by get_dataset.
I only found this to happen on one dataset so far, but there may be more. It's not hard to work around this, but it breaks automated tests.
Steps/Code to Reproduce
Example:
import openml
d = openml.datasets.get_dataset(70)
df, *_ = d.get_data(dataset_format="dataframe",include_row_id=True, include_ignore_attribute=True)
print(df.columns)
print([f.name for f in d.features.values()])
Expected Results
Identical lists of feature names
Actual Results
Some feature names are different, e.g. bw\%2Fme, blue\%2Fbright\%2Fvarn\%2Fclean
versus bw%2Fme, blue%2Fbright%2Fvarn%2Fclean
Versions
Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
Python 3.7.11 (default, Jul 3 2021, 18:01:19)
[GCC 7.5.0]
NumPy 1.19.5
SciPy 1.4.1
Scikit-Learn 0.22.2.post1
OpenML 0.12.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the get_dataset and get_data entry points shown in the reproduction, then compare the returned dataframe columns with the names in d.features. Reproduce the mismatch using dataset 70 and verify that both lists contain identical feature names, including names containing escaped percent sequences.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100