Data loaders should be able to inject tags to the graph
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 213
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
I'm a central platform team. I own some data adapters. I want to be able to inject some tags into the graph on the nodes.
**Describe the solution you'd like**
Data loaders can be used in many contexts. So I think the only way to get some tags is after instantiation, and a function is called.
Something like
```python
@dataclasses.dataclass
class JSONDataLoader(DataLoader):
path: str
@classmethod
def applicable_types(cls) -> Collection[Type]:
return [dict]
def load_data(self, type_: Type) -> Tuple[dict, Dict[str, Any]]:
with open(self.path, "r") as f:
return json.load(f), get_file_metadata(self.path)
@classmethod
def name(cls) -> str:
return "json"
def tags(self) -> dict: # <------------- add something like this
return {"some_special_tag": self.path, ...}
```
**Describe alternatives you've considered**
It's not critical, but something to think about.
**Additional context**
If one was to build more capabilities using tag metadata, then we'd want this here.
Contributor guide
Assessment
This issue has not been assessed yet.