pytask-dev / pytask-dev/pytask

ENH: add some more common Nodes

Open
#503 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
146
Forks
14
Avg merge
3d 16h
Merged PRs (30d)
34

Description

Writing pandas dataframes to disk seems really common. I am going to have to write my own DFNode class for this. Would you like me to generalize it enough and then contribute it here?

I would imagine something like

class DFNode:

    def __init__(self, path: str | ..., kind: Literal["csv", "parquet", ...], load_kwargs: dict[str, Any = {}, save_kwargs: dict[str, Any]):
        self.path = path
        self.kind = kind
        self.load_kwargs = load_kwargs
        self.save_kwargs = save_kwargs
    
    def save(self, value: Any) -> None:
        saver = getattr(value, f"to_{self.kind}")
        saver(self.path, **self.save_kwargs)

    def load(self, is_product: bool) -> Any:
        if is_product:
            return self
        loader = getattr(pd,  f"read_{self.kind}")
        return loader(self.path, **self.load_kwargs)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue provides a proposed DFNode class but names no repository files, tests, or entry point. Start by reviewing the existing Node abstractions and pandas read_* and to_* conventions; the work would be complete when the project has an agreed, tested way to read and write common dataframe formats with load and save options.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.