DOC: How to read data into pandas / dask / xarray
- Dominant language
- Python
- Stars
- 212
- Forks
- 81
- PR merge metrics
- No merged PRs in 30d
Description
Is there a good reference or a ckanapi function on how to read datasets from a CKAN instance into pandas and/or dask and/or xarray?
Pandas
- ``pandas.read_json("https://url.to/dataset.json")``
- https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#json
- https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_json.html
- https://pandas.pydata.org/pandas-docs/stable/ecosystem.html#api
- https://github.com/sinhrks/pyopendata/issues/3
- pandas-datareader, pandaSDMX, fredapi, quandl (Toronto)
Dask
- ``dask.dataframe.read_json("https://url.to/dataset.json")``
- https://docs.dask.org/en/latest/remote-data-services.html
- https://filesystem-spec.readthedocs.io/en/latest/features.html#instance-caching , arrow
- https://docs.dask.org/en/latest/dataframe-api.html#dask.dataframe.read_json
- https://docs.dask.org/en/latest/dataframe.html#common-uses-and-anti-uses
- https://docs.dask.org/en/latest/bag.html
- https://examples.dask.org/bag.html
- https://examples.dask.org/applications/json-data-on-the-web.html
- https://ml.dask.org/ (dask + {scikit-learn, TensorFlow, XGBoost})
xarray
- https://xarray.pydata.org/en/stable/generated/xarray.open_dataset.html
- https://xarray.pydata.org/en/stable/generated/xarray.Dataset.from_dataframe.html
- https://xarray.pydata.org/en/stable/generated/xarray.Dataset.from_dict.html
- https://xarray.pydata.org/en/stable/data-structures.html#dataarray
Caching
- https://github.com/reclosedev/requests-cache (sqlite)
- https://pandas-datareader.readthedocs.io/en/latest/cache.html
- https://github.com/ionrock/cachecontrol (dict, file, redis, [sqlite](https://github.com/ishitatsuyuki/cachecontrol-sqlite))
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no repository files, tests, or documentation entry point. First inspect the existing ckanapi documentation and API surface to determine whether a reading helper already exists, then compare the requested pandas, Dask, and xarray workflows. Done should be a focused reference explaining supported loading approaches and any relevant caching behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100