Support for presigned URLs
- Langage dominant
- Python
- Étoiles
- 47
- Forks
- 22
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
**Is your feature request related to a problem? Please describe.**
I love using this library, but I have a pretty specific problem: Some of the URLs that I'd like to download are presigned R2 Cloudflare urls: https://developers.cloudflare.com/r2/api/s3/presigned-urls/
This routes to the `HTTPClient` which then throws an 403 error when making the head request.
**Describe the solution you'd like**
I'm not sure what is the ideal way to support this, but I currently patched the code to fallback to a get request. It would be nice if something official was supported however.
My current workaround:
```
from cached_path.schemes.http import (
HttpClient,
MaxRetryError,
RecoverableServerError,
session_with_backoff,
)
class HttpClient(HttpClient):
@property
def head_response(self):
"""
Presigned URLs from Cloudflare R2 throw 403 for HEAD requests: https://developers.cloudflare.com/r2/api/s3/presigned-urls/
This is a workaround to be able to avoid a fatal error.
"""
if self._head_response is None:
try:
with session_with_backoff(self.headers) as session:
response = session.head(self.resource, allow_redirects=True)
if response.status_code == 403:
response = session.get(self.resource, allow_redirects=True)
except MaxRetryError as e:
raise RecoverableServerError(e.reason)
self.validate_response(response)
self._head_response = response
return self._head_response
else:
return self._head_response
```
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.