ckan / ckan/ckanapi

DOC: How to read data into pandas / dask / xarray

Open
#145 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
212
Forks
81
PR merge metrics
No merged PRs in 30d

Description

Is there a good reference or a ckanapi function on how to read datasets from a CKAN instance into pandas and/or dask and/or xarray?

Pandas
- ``pandas.read_json("https://url.to/dataset.json")``
- https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#json
- https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_json.html
- https://pandas.pydata.org/pandas-docs/stable/ecosystem.html#api
- https://github.com/sinhrks/pyopendata/issues/3
- pandas-datareader, pandaSDMX, fredapi, quandl (Toronto)

Dask
- ``dask.dataframe.read_json("https://url.to/dataset.json")``
- https://docs.dask.org/en/latest/remote-data-services.html
- https://filesystem-spec.readthedocs.io/en/latest/features.html#instance-caching , arrow
- https://docs.dask.org/en/latest/dataframe-api.html#dask.dataframe.read_json
- https://docs.dask.org/en/latest/dataframe.html#common-uses-and-anti-uses
- https://docs.dask.org/en/latest/bag.html
- https://examples.dask.org/bag.html
- https://examples.dask.org/applications/json-data-on-the-web.html
- https://ml.dask.org/ (dask + {scikit-learn, TensorFlow, XGBoost})

xarray
- https://xarray.pydata.org/en/stable/generated/xarray.open_dataset.html
- https://xarray.pydata.org/en/stable/generated/xarray.Dataset.from_dataframe.html
- https://xarray.pydata.org/en/stable/generated/xarray.Dataset.from_dict.html
- https://xarray.pydata.org/en/stable/data-structures.html#dataarray

Caching
- https://github.com/reclosedev/requests-cache (sqlite)
- https://pandas-datareader.readthedocs.io/en/latest/cache.html
- https://github.com/ionrock/cachecontrol (dict, file, redis, [sqlite](https://github.com/ishitatsuyuki/cachecontrol-sqlite))

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no repository files, tests, or documentation entry point. First inspect the existing ckanapi documentation and API surface to determine whether a reading helper already exists, then compare the requested pandas, Dask, and xarray workflows. Done should be a focused reference explaining supported loading approaches and any relevant caching behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.