MIT-LCP / MIT-LCP/wfdb-python

Authentication handling in wfdb.io.dl_files, error 403

Abierto
#568 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Lenguaje dominante
Jupyter Notebook
Estrellas
853
Forks
322
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Context

I am trying to download specific files (subsets) from credentialed databases on PhysioNet using wfdb helper functions.

What I am trying to do is downloading selected files (not the entire dataset), e.g. :

from wfdb.io import dl_files
dl_files('mimiciv','data\\tmp',['hosp/patients.csv.gz'])

Stack trace

Created local base download directory: data\tmp
Downloading files...
Traceback (most recent call last):
  File "c:\Users\pa10198\Documents\_code\zarrow\dev.py", line 12, in <module>
    dl_files('mimiciv','data\\tmp',['hosp/patients.csv.gz'])
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\download.py", line 544, in dl_files
    pool.map(dl_pn_file, dl_inputs)
  File "C:\Users\pa10198\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 774, in get
    raise self._value
  File "C:\Users\pa10198\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\download.py", line 450, in dl_pn_file
    dl_full_file(url, local_file)
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\download.py", line 472, in dl_full_file
    content = readfile.read()
              ^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\_url.py", line 580, in read
    result = b"".join(self._read_range(start, end))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\_url.py", line 473, in _read_range
    with RangeTransfer(self._current_url, req_start, req_end) as xfer:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\_url.py", line 167, in __init__
    self._parse_headers(method, self._response)
  File "C:\Users\pa10198\Documents\_code\zarrow\.venv\Lib\site-packages\wfdb\io\_url.py", line 213, in _parse_headers
    raise cls(
wfdb.io._url.NetFilePermissionError: 403 Error: Forbidden for url: https://physionet.org/files/mimiciv/3.1/hosp/patients.csv.gz

Remarks

  • I have the credentialed physionet account and access to the databases I request.
  • The same URL works in a browser after login and the file downloads successfully.

Questions

  • Is there any supported way to pass authentification when using dl_files or related functions, e.g. through env vars, config, etc.
  • If not, are there recommended workarounds (e.g., using cookies, tokens, or another library)? Ideally not leaving python and avoiding the multiplication of libraries within projects.

Feature idea

It would be very useful if wfdb supported:

  • authentication for credentialed MIMIC-IV (and similar datasets), and
  • partial downloads (which already works on the open datasets and seem supported structurally, but fail due to auth)

This would allow to avoid downloading entire large datasets and keep workflows within a single library.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza en wfdb/io/download.py, en dl_files, dl_pn_file y dl_full_file; después inspecciona RangeTransfer y _parse_headers en wfdb/io/_url.py. Reproduce la solicitud 403 contra la URL autenticada de PhysioNet y sigue el flujo de gestión del acceso; se considera terminado cuando exista una ruta de autenticación documentada y compatible que permita a dl_files obtener archivos protegidos seleccionados.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
authentication, data
Tipo de issue
Nueva funcionalidad
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Tranquilo
Claridad
Bastante claro
Aptitud para principiantes
45/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.