bool category not properly parsed
- Dominant language
- Python
- Stars
- 16
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
Assume I have a metadata category like `infection` with values `TRUE` or `FALSE`. If I load these data as in your example `metadata = pd.read_table("data/metadata.tsv", sep="\t", index_col=0)` they are of type `object` and proper boolean values, i.e. `True` and `False`. If I would add a `dtype=str`, the values are still of type `object` but strings, namely `'TRUE'` and `'FALSE'`.
Only the `dtype=str` way works for me. Otherwise evident throws the error:
```
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3628 try:
-> 3629 return self._engine.get_loc(casted_key)
3630 except KeyError as err:
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
KeyError: 'infection'
The above exception was the direct cause of the following exception:
KeyError Traceback (most recent call last)
/tmp/ipykernel_1855806/1849832882.py in
1 for cat in ["birth_timestamp","cage","genotype","infection"]:
----> 2 print(adh.calculate_effect_size(column=cat))
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/evident/data_handler.py in calculate_effect_size(self, column, difference)
112 :rtype: evident.results.EffectSizeResult
113 """
--> 114 if self.metadata[column].dtype != np.dtype("object"):
115 raise exc.NonCategoricalColumnError(self.metadata[column])
116
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
3503 if self.columns.nlevels > 1:
3504 return self._getitem_multilevel(key)
-> 3505 indexer = self.columns.get_loc(key)
3506 if is_integer(indexer):
3507 indexer = [indexer]
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3629 return self._engine.get_loc(casted_key)
3630 except KeyError as err:
-> 3631 raise KeyError(key) from err
3632 except TypeError:
3633 # If we have a listlike key, _check_indexing_error will raise
KeyError: 'infection'
```
You might want to return a more explicit error message in those cases.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with evident/data_handler.py at calculate_effect_size, which the traceback identifies, and reproduce the issue using metadata columns loaded as booleans and as strings. Determine how categorical boolean columns are expected to behave and how invalid input should be reported; done means the infection category no longer fails ambiguously or produces an unexplained KeyError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100