Handling of non-string DataFrame column names
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 124
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
The test suite is currently broken on `master` after the latest dev release of HoloViews. The breakage is due to https://github.com/holoviz/holoviews/pull/5354 that raises an error earlier than before if the column names of a DataFrame contain an integer.
It means that in the current state this breaks white it used to work:
```python
import numpy as np; import pandas as pd
import hvplot.pandas
df = pd.DataFrame(np.random.rand(10, 2)) # columns is [0, 1]
df.hvplot()
```
and raises:
```
DataError: pandas DataFrame column names used as dimensions must be strings not integers.
PandasInterface expects tabular data, for more information on supported datatypes see http://holoviews.org/user_guide/Tabular_Datasets.html
```
### Pandas .plot
Pandas `.plot` is actually very flexible on the column names it accepts, these all work:
```python
df.plot()
df.plot(y=1)
dft = pd.DataFrame(np.random.rand(10, 2), columns=[pd.Timestamp('2022/01/01'), pd.Timestamp('2023/01/01')])
dft.plot()
dft.plot(y=pd.Timestamp('2022/01/01'))
```
### hvPlot, before
Things worked partially, i.e. plotting all the columns at once worked while to specify a given column you'd have to find out its string representation.
```python
df.hvplot() # Works
df.hvplot(y=1) # Error!
df.hvplot(y='1') # Works!
dft.hvplot() # Works
df.hvplot(y=pd.Timestamp('2022/01/01')) # Error!
df.hvplot(y='2022-01-01 00:00:00') # Works!
```
### hvPlot, now
All the examples above fail with a `DataError`. What happens is that at line L1226 a `DataError` is raised (also at L1224 but it's caught) when trying to instantiate the `hv.Dataset`, that is given `self.source_data`. If it was given `self.data` instead no `DataError` would be raised as the columns of `self.data` are converted to strings the `_transform_columnar_data` method. I'm not sure why `hv.Dataset` is given `self.source_data`.
https://github.com/holoviz/hvplot/blob/6f6da2d2f39970bb3ea0731641c4038bc934ec27/hvplot/converter.py#L1210-L1226
### Solutions
The current state is clearly a regression, compared to a previous state that wasn't already ideal.
HoloViews should:
1. [ ] not just check for integers before raising a `DataError` at it seems that Pandas allows more than strings and integers as column names
2. maybe add support to non-string column names, if that's even technically possible
Until 2. happens, a solution should be found for hvPlot itself:
- [ ] Untangle the usage of `self.source_data` and `self.data` I referred to above, to avoid the `DataError` when calling `df.hvplot()`
- [ ] To allow users to reference the actual column name (e.g. `df.hvplot(y=1)`) it may be required to record a mapping of the original column names with their string representation.
Feedbacks on this @jlstevens @philippjfr ?
Note that it also broke the following but I expect the potential hvPlot fixes listed above to fix that too:
```
s = pd.Series(np.random.rand(10))
s.hvplot()
```
Contributor guide
Assessment
This issue has not been assessed yet.