MIT-LCP / MIT-LCP/wfdb-python

New interface for specifying different data sources for read/write

Open
#367 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
853
Forks
322
PR merge metrics
No merged PRs in 30d

Description

Looking at the current rdrecord for example, there are two parameters used to specify the location of the record:

  1. record_name : str
  2. pn_dir : str

The current package supports reading files locally and from the global database index URL, which defaults to PhysioNet, as specified in download.py.

There are several things that we should aim to support:

  • Reading/writing from more types of data sources, such as S3, and GCS.
  • Having more than one remote source configured at a time.

One proposal might be to have a new DataSource class, and a global config dictionary with key:value pairs of ds_name(str):ds(DataSource). ie.

class DataSourceType(Enum):
    LOCAL = 1  # Not sure if this is necessary?
    HTTP = 2
    GCS = 3
    S3 = 4

class DataSource:
    ds_type : DataSourceType
    # Other type-specific params here
    
_physionet_ds = DataSource(ds_type=DataSourceType.HTTP, base_url="https://physionet.org/content/")

data_sources = { 'physionet' : _physionet_ds }

And the read/write functions could use these params:

  1. record_name: str
  2. data_source: str | DataSource - The key of the data source in the global data sources map, or a DataSource object.

This would be much more explicit. Thoughts?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the current rdrecord interface and download.py, which the issue identifies as the relevant entry points for local and PhysioNet access. Compare how those paths would need to represent multiple remote sources, then document a concrete API and acceptance criteria for local, HTTP, S3, and GCS read/write support before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.