biocore / biocore/evident

bool category not properly parsed

Open
#27 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
16
Forks
9
PR merge metrics
No merged PRs in 30d

Description

Assume I have a metadata category like `infection` with values `TRUE` or `FALSE`. If I load these data as in your example `metadata = pd.read_table("data/metadata.tsv", sep="\t", index_col=0)` they are of type `object` and proper boolean values, i.e. `True` and `False`. If I would add a `dtype=str`, the values are still of type `object` but strings, namely `'TRUE'` and `'FALSE'`.

Only the `dtype=str` way works for me. Otherwise evident throws the error:
```
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3628 try:
-> 3629 return self._engine.get_loc(casted_key)
3630 except KeyError as err:

~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'infection'

The above exception was the direct cause of the following exception:

KeyError Traceback (most recent call last)
/tmp/ipykernel_1855806/1849832882.py in
1 for cat in ["birth_timestamp","cage","genotype","infection"]:
----> 2 print(adh.calculate_effect_size(column=cat))

~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/evident/data_handler.py in calculate_effect_size(self, column, difference)
112 :rtype: evident.results.EffectSizeResult
113 """
--> 114 if self.metadata[column].dtype != np.dtype("object"):
115 raise exc.NonCategoricalColumnError(self.metadata[column])
116

~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
3503 if self.columns.nlevels > 1:
3504 return self._getitem_multilevel(key)
-> 3505 indexer = self.columns.get_loc(key)
3506 if is_integer(indexer):
3507 indexer = [indexer]

~/miniconda3/envs/qiime2-2022.8/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3629 return self._engine.get_loc(casted_key)
3630 except KeyError as err:
-> 3631 raise KeyError(key) from err
3632 except TypeError:
3633 # If we have a listlike key, _check_indexing_error will raise

KeyError: 'infection'
```

You might want to return a more explicit error message in those cases.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with evident/data_handler.py at calculate_effect_size, which the traceback identifies, and reproduce the issue using metadata columns loaded as booleans and as strings. Determine how categorical boolean columns are expected to behave and how invalid input should be reported; done means the infection category no longer fails ambiguously or produces an unexplained KeyError.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.