pydata / pydata/xarray

Loading a reduced dataset by calling an OpenDAP server-side function?

Open
#4,967 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement topic-backends
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

The OpenDAP spec provides for server-side functions to aggregate and reduce file size prior to transmission over the network see here. Pydap offers an API to write custom server-side functions (See source code here and here).

I work a lot with time-series data from instruments in my laboratory. I typically will sample at 10-1000 Hz, but when I analyze the data, it's easier to see long-term drift by resampling the data at ~1 minute. This reduces the size of the dataset by a factor of 600-60k. I'm thinking about writing some plugins for Pydap using their server-side-function API. My question is whether or not xarray supports calling OpenDAP server-side functions; if not, is there a plan to provide for this?

I'm kind of new to OpenDAP, and haven't gotten my hands too dirty yet. All I've done is install a Pydap server, and load a PyTables file. I'd love to use xarray more in my workflow.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking how xarray currently accesses OpenDAP resources and whether server-side function requests are exposed. Compare that behavior with the Pydap examples in src/pydap/wsgi/functions.py and src/pydap/wsgi/ssf.py. The issue does not name an xarray file or test; done would require a defined support plan or an agreed implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.