lincc-frameworks / lincc-frameworks/nested-pandas

`map_rows(..., columns=["nested"])` should pass subcolumns to the UDF, not a frame

Open
#555 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
26
Forks
8
Avg merge
2d 2h
Merged PRs (30d)
9

Description

**Bug report**

The behavior of `map_rows(columns=...)` changes when passing a nested column name, compared to both passing nothing and passing a list of the subcolumns. We should change it to be consistent with other ways of specifying columns.

```python
import pandas as pd
from nested_pandas.datasets import generate_data

nf = generate_data(3, 10)
result = nf.map_rows(lambda row: row["nested"].__class__, columns=["nested"])
# should be a dict here, not a frame
assert result.iloc[0] == pd.DataFrame
```

**Before submitting**
Please check the following:

- [x] I have described the situation in which the bug arose, including what code was executed, and any applicable data others will need to reproduce the problem.
- [ ] I have included information about my environment, including the version of this package (e.g. `nested_pandas.__version__`)
- [x] I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
- [x] If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the map_rows implementation and run the pandas reproduction from the issue with nested_pandas.datasets.generate_data. Check how columns=["nested"] is converted before the UDF runs, then compare it with no columns and an explicit list of subcolumns. Done means nested selection passes the same subcolumn representation as the equivalent explicit selection, with a regression test covering the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.