lincc-frameworks / lincc-frameworks/nested-pandas

NestedFrame constructor to pack series with hierarchical names

Open
#101 0 comments 0 reactions 0 assignees View on GitHub
enhancement interface
Dominant language
Python
Stars
26
Forks
8
Avg merge
2d 2h
Merged PRs (30d)
9

Description

**Feature request**

It would be cool to allow users provide "layer.col" list-array series as input to `NestedFrame` so we automatically pack them.

```python
nested_frame = NestedFrame(
{
{"id": [1,2,3]},
{"lc.time": [0.1, 0.2], [0.1, 0.3], [0.2, 0.3, 0.4]},
{"lc.counts": [100, 200], [55, 75], [33, 100, 155]},
}
)
```

We can use it in `reduce` to allow a simple interface of generating new nested columns from it.

```python
def apply_reference(ref_mag: float, ref_magerr: float, diff_mag: np.ndarray, diff_magerr: np.ndarray):
mag = ref_mag + diff_mag
magerr = np.hypot(ref_magerr, diff_magerr)
return {"corrected_lc.mag": mag, "corrected_lc.magerr": magerr}

corrected_mag_df = nested_frame.reduce(apply_reference, "ref_mag", "ref_magerr", "lc.mag", "lc.magerr")
nested_frame = nested_frame.join(corrected_mag_df)
# Now nested_frame has "corrected_lc" column
```

**Before submitting**
Please check the following:

- [x] I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
- [x] I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
- [ ] If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the NestedFrame constructor and the reduce and join entry points described in the issue. Read how nested columns are currently represented, then verify that hierarchical list-array inputs are packed and that columns returned by reduce can be joined as a new nested column.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.