Cannot search with regex on columns with iterables
- Dominant language
- Python
- Stars
- 164
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
### Description
Having created an `esm_datastore`, I'd like to use regular expressions (regex) to search for datasets that contain a given variable. However, the use of regex does not appear to be supported on columns with iterables. For example:
```
import intake
> datastore = intake.open_esm_datastore(
'cryo_input_datastore.json',
columns_with_iterables=['variable'])
```
This successfully loads the datastore which contains multiple records with the following unique variables:
```
> datastore.unique().variable
['bed', 'dataid', 'errbed', 'firn', 'geoid', 'mask', 'source', 'surface', 'thickness', 'x', 'y', 'bed_topography', 'bed_uncertainty', 'ice_thickness', 'surface_topography', 'thickness_survey_count', 'thickness_uncertainty', 'CNT', 'ERRX', 'ERRY', 'STDX', 'STDY', 'VX', 'VY', 'lat', 'lon']
```
To filter the datastore to show datasets that contain bed topography, I use the `.*bed.*` regex to search, but this returns no entries:
```
> ds = datastore.search(variable = '.*bed.*')
> print(ds)
```
I would expect it to return entries for all datasets that contain `bed` in the `variable` field. If I use `variable_standard_name` in the search function, it returns entries as expected. This is because `variable_standard_name` is not included in the `columns_with_iterables` when loading the datastore.
### Version information: output of `intake_esm.show_versions()`
```
INSTALLED VERSIONS
------------------
cftime: 1.6.4
dask: 2025.5.1
fastprogress: 1.0.3
fsspec: 2025.7.0
gcsfs: 2025.7.0
intake: 2.0.8
intake_esm: 2025.7.9
netCDF4: 1.7.2
pandas: 2.3.1
requests: 2.32.4
s3fs: 2025.7.0
xarray: 2025.6.1
zarr: 2.18.7
```
### Cause
This behaviour appears to be intended and is implemented [here](https://github.com/intake/intake-esm/blob/b9c14d9414a80e8b2ec834dcad29e73e9578e4a5/intake_esm/_search.py#L47-L51).
Contributor guide
Assessment
This issue has not been assessed yet.