New interface for specifying different data sources for read/write
还没有人认领这个 Issue。
- 主要语言
- Jupyter Notebook
- 星标
- 853
- 派生
- 322
- PR 合并指标
- 30 天内没有已合并 PR
描述
Looking at the current rdrecord for example, there are two parameters used to specify the location of the record:
record_name: strpn_dir: str
The current package supports reading files locally and from the global database index URL, which defaults to PhysioNet, as specified in download.py.
There are several things that we should aim to support:
- Reading/writing from more types of data sources, such as S3, and GCS.
- Having more than one remote source configured at a time.
One proposal might be to have a new DataSource class, and a global config dictionary with key:value pairs of ds_name(str):ds(DataSource). ie.
class DataSourceType(Enum):
LOCAL = 1 # Not sure if this is necessary?
HTTP = 2
GCS = 3
S3 = 4
class DataSource:
ds_type : DataSourceType
# Other type-specific params here
_physionet_ds = DataSource(ds_type=DataSourceType.HTTP, base_url="https://physionet.org/content/")
data_sources = { 'physionet' : _physionet_ds }
And the read/write functions could use these params:
record_name: strdata_source: str | DataSource - The key of the data source in the global data sources map, or aDataSourceobject.
This would be much more explicit. Thoughts?
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
首先阅读当前的 rdrecord 接口和 download.py,issue 将它们确定为本地访问和 PhysioNet 访问的相关入口点。比较这些路径需要如何表示多个远程源,然后在实现之前,为本地、HTTP、S3 和 GCS 的读写支持记录具体的 API 和验收标准。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- backend-api-design
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100