lincc-frameworks / lincc-frameworks/nested-pandas
Packing on an object-dtyped column may fail if items are not comparable
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 26
- Forks
- 8
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 9
Description
**Bug report**
Today @mi-dai and myself were working on packing SALT3 data into nested pandas. When we tried to run `nf.add_nested(src, on='SNID', name='lc')` it failed, because it happened that `SNID` column had object dtype and consisted of heterogeneous data: numbers and strings. A reproducible example:
```python
from nested_pandas import NestedFrame
nf = NestedFrame({"SNID": [1, 'abc']})
nf.add_nested(NestedFrame({"SNID": [1, 1, 'abc', 'abc'], "x": [1, 2, 3, 4]}), name='lc', on='SNID')
```
It failed in our code, basically because this doesn't work in pandas:
```
nd.set_index('SNID').sort_index()
```
I'm not sure if it is our problem or not. We potentially may fix this use-case with catching the `TypeError` and trying to cast `on` column values to `str`. Or we can do nothing, because we wouldn't like to change user's data and fix pandas behavior.
**Before submitting**
Please check the following:
- [x] I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
- [x] I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
- [x] If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproducible NestedFrame.add_nested example in the issue and trace the path that calls set_index('SNID').sort_index(). Add a regression test using mixed numeric and string SNID values, then verify the chosen behavior handles the example without changing unrelated user data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100