geopython / geopython/GeoHealthCheck
Retry on http502 error during OWSlib request
- Lingua principale
- Python
- Stelle
- 92
- Fork
- 75
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
_This issue is related to issue #366._
**Is your feature request related to a problem? Please describe.**
During a stress test of GHC (lots of resources with high frequency), I am getting mysterious http502 errors during `WebFeatureService` request in the `get_metadata` function inside `wfs.py`, `wms.py`, etc. These errors can be avoided by retrying requests.
**Describe the solution you'd like**
Currently, in [probe.py](https://github.com/geopython/GeoHealthCheck/blob/master/GeoHealthCheck/probe.py#L94) there is already something in place for this using the `create_requests_retry_session()` function in [util.py](https://github.com/geopython/GeoHealthCheck/blob/master/GeoHealthCheck/util.py#L267). The same principle could be applied to the request that is done in `get_metadata` functions of each probe (wfs.py, wms.py, etc.).
[wfs.py](https://github.com/geopython/GeoHealthCheck/blob/master/GeoHealthCheck/plugins/probe/wfs.py#L123)
This request however uses the `WebFeatureService()` function from OWSlib, so the best solution in my opinion would be to use the `create_requests_retry_session()` in OWSlib. Maybe other OWSlib users have also seen this http502 error? This fix might improve OWSlib for other users as well.
**Describe alternatives you've considered**
An alternative is a simple loop with an exception clause which retries the `WebFeatureService()` request up to 3 times:
```
def get_metadata(self, resource, version='1.1.0', retries=3):
"""
Get metadata, specific per Resource type.
:param resource:
:param version:
:return: Metadata object
request_headers = self.get_request_headers()
"""
for i in range(retries):
try:
return WebFeatureService(
resource.url,
version=version,
headers=request_headers)
except:
continue
raise Exception(f'Get Metadata Request Exceeded {retries} Retries')
```
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia da GeoHealthCheck/probe.py e GeoHealthCheck/util.py, in particolare da create_requests_retry_session(), poi confronta gli entry point get_metadata in GeoHealthCheck/plugins/probe/wfs.py, wms.py e negli altri probe. Verifica come viene effettuata la richiesta WebFeatureService() di OWSlib e determina il comportamento di retry necessario per le risposte HTTP 502. Il lavoro è concluso quando le richieste di metadati interessate vengono ritentate in modo affidabile senza compromettere gli errori normali.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- api
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 40/100