`client.dandi_authenticate` blocks during programmatic use
- Dominant language
- Python
- Stars
- 28
- Forks
- 37
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 9
Description
Hello!
I'm downloading datasets programmatically, and ran into a stumbling block with auth.
## Problem
Currently, when trying to download an embargoed dataset, the client (when invoked with the cli `download` command or the underlying `download` function) will stop and wait for user authentication input. This happens in the `ParsedDandiURL.navigation` method:
https://github.com/dandi/dandi-cli/blob/10d076e7bd9625840d8a4e1cab8a49d986c415ea/dandi/dandiarchive.py#L170-L180
called by the `Downloader.download_generator` method:
https://github.com/dandi/dandi-cli/blob/10d076e7bd9625840d8a4e1cab8a49d986c415ea/dandi/download.py#L229
This makes sense for interactive use, but makes programmatic use difficult - since the exception is not thrown, there is no way to catch it. So eg. when downloading a list of datasets where we aren't certain in advance whether we have access to them, i would want to skip the ones we don't have access to, but in order to do that I had to pre-check each dataset like this:
```python
def check_auth(url:ParsedDandiURL) -> bool:
with url.get_client() as client:
try:
dandiset = url.get_dandiset(client, lazy=False)
except requests.HTTPError as e:
if e.response.status_code == 401:
return False
else:
raise
return True
```
## Options
### Detect click
click allows you to check whether you are inside of a click context, so the `dandi_authenticate` method could check if we're in an interactive session and only ask for credentials if so (eg. before getting to the interactive block here: https://github.com/dandi/dandi-cli/blob/master/dandi/dandiapi.py#L499
```python
import click
def is_interactive() -> bool:
try:
_ = click.get_current_context()
return True
except RuntimeError as e:
if 'no active click context' in str(e).lower():
return False
else:
raise
def dandi_authenticate(self) -> None:
# ...
if is_interactive():
while True:
# ...
else:
raise UnauthorizedError('need to provide credentials...') # or whatever the exception would be named
```
I see that there [already is](https://github.com/dandi/dandi-cli/blob/10d076e7bd9625840d8a4e1cab8a49d986c415ea/dandi/utils.py#L71) a `is_interactive` function, so not sure if that is the more appropriate check or whether looking for a click context is.
### Provide Auth explicitly
Currently the `authenticate` arg to `navigate` is a bool, and it's not possible to pass auth information from the top-level `download` function. If instead the `download` function had a `credentials` or `key` or whatever you want to call it arg, then absence of credentials could always raise an exception rather than prompting when not invoked via the click download command.
It is currently possible to provide credentials to the `DandiAPIClient` object, though it is not possible to provide an already-instantiated client to the `download` method, so it is possible to authenticate properly in a few ways, but it is not possible to purposely skip authentication when we intend/expect to fail auth. Propagating creds from the top-level `download` function would be one way of a) allowing explicit auth while also b) allowing explicit non-auth, depending on whether creds are provided.
---
as usual, lmk if there is something i'm missing, or if there is an intended way to skip unauthorized downloads when using the API programmatically. I'd be happy to draft PRs for either of the above options or for docs if i'm just missing some intended use.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.