apache / apache/hamilton

Expose feature name to data savers

Open
#312 3 comments 1 reaction 0 assignees View on GitHub
decorators io
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

Courtesy of @janhurst: https://hamilton-opensource.slack.com/archives/C03MANME6G5/p1693385993967759.

**Is your feature request related to a problem? Please describe.**
I've been paying around with the data savers and have implemented my own @save_to functionality.
I have something that looks a little like this:

```python
@save_to.feature(feature_name="my_feature_1")
def my_feature_1(foo: pd.Series, bar: pd.Series) -> pd.Series:
return foo + bar
```
my saving logic is some existing infrastructure but essentially is just using the name to figure out the right place to save to.... but really i want the function name to be accessible in my DataSaver.... i couldn't see that this available at present... any ideas on how I could achieve this?

**Describe the solution you'd like**
Let's add a `__node_name` field to the dataclass that gets auto-populated in data savers.
We could theoretically add other metadata later on that starts with `__`, but I think this is a good start.

**Describe alternatives you've considered**
The approach @janhurst is taking now is the right way, but this seems like a nice feature to have.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Research direction

Start at the @save_to.feature entry point and the DataSaver implementation, then inspect the dataclass used by data savers. Add the requested __node_name metadata and verify that it is populated with the feature function name when a saver runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.