Simplifying driver behaviors for now

Currently, the implementation of data loading and saving (drivers) is very simple in pins-python. Despite this, tests of data loading are very complex to setup, because it requires a Meta object as its first argument.

This was done to match R pins, but looking at the r pins source (below), it seems that only really a type (e.g. "csv") argument is needed, along with a file (or files) to load.

https://github.com/rstudio/pins/blob/debc94529b8b73cda41ab50419d6276cd6c56d24/R/pin-read-write.R#L165

I think the reason for removing meta in python-pins goes as follows:

• In R pins it's passed because it holds context for finding a pre-downloaded file locally
• In python pins an abstract IOBase file object can be passed (that handle e.g. downloading files locally behind the scenes)
• convenience functions can still exist to load data from a Meta object

Alternatively, there could be a stripped down version for Meta (or an interface), that defines the bare minimum required for data loading (i.e. type, filename).

Example of how python pins can abstract file operations

The example below shows how a local filesystem object (fs) can hide the details of where the data is coming from. Whether s3 or local disk.

import joblib
import fsspec

# note that fs could be pointing at S3 or a local filesystem
# for s3 to work your path needs to also include a bucket name

fs = fsspec.filesystem("file")
# fs = fsspec.filesystem("s3")

with fs.open("test.joblib", "wb") as f:
    joblib.dump({"some_data": 1}, f)

with fs.open("test.joblib", "rb") as f:
    joblib.read(f)

Example

import tempfile
import fsspec

# set up data ---------

import pandas as pd
df = pd.DataFrame({"x": [1,2,3]})


# everything needed to reach load_data in a test ---------
# would be convenient to remove the need to create Meta
# when few details (type, files) are used.

from pins.drivers import load_data, save_data
from pins.meta import MetaFactory
from pathlib import Path

mf = MetaFactory()
fs = fsspec.filesystem("file")

# CURRENT APPROACH --------

with tempfile.NamedTemporaryFile() as tmp_file:

    df.to_csv(tmp_file.file)
    tmp_file.file.close()

    meta = mf.create(
        files=tmp_file.name,
        type="csv",
        name=Path(tmp_file.name).name,
        title=""
    )

    # meta mostly used for meta.type
    data = load_data(meta, fs, Path(tmp_file.name).parent)

# BETTER APPROACH WOULD ALLOW... --------

with tempfile.NamedTemporaryFile() as tmp_file:
    df.to_csv(tmp_file.file)
    tmp_file.file.close()
    
    # much simpler
    # could also pass a dictionary of options (e.g. from Meta.user)
    data = load_data(fs, tmp_file.name, type="csv")

The issue names pins.drivers.load_data/save_data and MetaFactory, and refers to complex data-loading tests without giving paths. Start by locating those entry points and tests, then resolve whether Meta should be removed or reduced before updating the API and its callers; done means loading and saving work with the agreed simpler interface.