Add resolved_kwargs to data_saver and data_loader tags
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 213
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
When I have a built dataflow I would like to be able to see which paths are entered in @load_from and @save_to.
**Describe the solution you'd like**
After executing the dataflow I can see the paths in the results, but I'd like to be able to see them without executing the dataflow.
Some metadata is already being written to tags: https://github.com/DAGWorks-Inc/hamilton/blob/main/hamilton/function_modifiers/adapters.py#L578
I tested adding the following line there:
```python
"hamilton.data_saver.kwargs": resolved_kwargs,
```
Then I tried running `examples/parallelism/star_counting/run.py` with the dr.execute statement replaced by:
```python
node = next(
node for node in dr.list_available_variables() if node.name == "save.unique_stargazers"
)
print(node.as_dict()["tags"])
```
This gives the output I was hoping for:
```python
{'hamilton.data_saver': True, 'hamilton.data_saver.sink': 'csv', 'hamilton.data_saver.classname': 'PandasCSVWriter', 'hamilton.data_saver.kwargs': {'path': 'unique_stargazers.csv'}}
```
**Describe alternatives you've considered**
Maybe a custom DataLoader and DataSaver that store the arguments they were initiated with?
Contributor guide
Assessment
This issue has not been assessed yet.