geopython / geopython/GeoHealthCheck

Show resource ID in all log messages

Aperta
#377 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Python
Stelle
92
Fork
75
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

I've just performed a test by adding many resources to the GHC DB and with all resources checked every 5 minutes. This generates a lot of log message. This is not an issue by itself, but I think it is better if every log message (which is related to a single resource being tested) contains the actual resource ID. This way it is easier to identify messages which belong to the same resource. This could be helpful for debugging, especially when a lot of resources are being tested.

Example:
```
ghc_web | 2021-06-30 15:52:45,590 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.NotContainsOwsException result=True
ghc_web | 2021-06-30 15:52:45,590 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.HttpHasImageContentType result=True
ghc_web | 2021-06-30 15:52:45,591 - GeoHealthCheck.probe - INFO - Requesting: GET url=https:///nwb_hectopunten/ows
ghc_web | 2021-06-30 15:52:45,744 - urllib3.connectionpool - DEBUG - https://:443 "GET /netnummergebieden/ows?service=WFS&request=GetCapabilities&version=1.1.0 HTTP/1.1" 200 1736
ghc_web | 2021-06-30 15:52:45,749 - GeoHealthCheck.probe - INFO - Requesting: POST url=https:///netnummergebieden/ows
ghc_web | 2021-06-30 15:52:45,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): :443
ghc_web | 2021-06-30 15:52:45,807 - urllib3.connectionpool - DEBUG - https://:443 "GET /oov_pois/ows?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities HTTP/1.1" 200 2000
ghc_web | 2021-06-30 15:52:45,808 - GeoHealthCheck.probe - INFO - response: status=200
ghc_web | 2021-06-30 15:52:45,809 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.HttpStatusNoError result=True
ghc_web | 2021-06-30 15:52:45,810 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.XmlParse result=True
ghc_web | 2021-06-30 15:52:45,810 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.NotContainsOwsException result=True
ghc_web | 2021-06-30 15:52:45,810 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.ContainsStrings result=True
ghc_web | 2021-06-30 15:52:45,810 - GeoHealthCheck.probe - INFO - Result: success=True msg=OK response_time=0.333574
ghc_web | 2021-06-30 15:52:45,823 - urllib3.connectionpool - DEBUG - https://:443 "GET /nwb_hectopunten/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=hectopunten&SRS=EPSG:28992&BBOX=120000,435000,155000,470000&WIDTH=256&HEIGHT=256&FORMAT=image/png&STYLES=&EXCEPTIONS=application/vnd.ogc.se_xml HTTP/1.1" 200 95
ghc_web | 2021-06-30 15:52:45,824 - GeoHealthCheck.probe - INFO - response: status=200
ghc_web | 2021-06-30 15:52:45,824 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.HttpStatusNoError result=True
ghc_web | 2021-06-30 15:52:45,825 - chardet.charsetprober - DEBUG - SHIFT_JIS Japanese prober hit error at byte 54
ghc_web | 2021-06-30 15:52:45,825 - chardet.charsetprober - DEBUG - EUC-JP Japanese prober hit error at byte 0
ghc_web | 2021-06-30 15:52:45,826 - chardet.charsetprober - DEBUG - GB2312 Chinese prober hit error at byte 46
ghc_web | 2021-06-30 15:52:45,826 - chardet.charsetprober - DEBUG - EUC-KR Korean prober hit error at byte 0
ghc_web | 2021-06-30 15:52:45,826 - chardet.charsetprober - DEBUG - CP949 Korean prober hit error at byte 53
ghc_web | 2021-06-30 15:52:45,827 - chardet.charsetprober - DEBUG - Big5 Chinese prober hit error at byte 0
ghc_web | 2021-06-30 15:52:45,827 - chardet.charsetprober - DEBUG - EUC-TW Taiwan prober hit error at byte 0
ghc_web | 2021-06-30 15:52:45,829 - chardet.charsetprober - DEBUG - windows-1251 Russian confidence = 0.0
ghc_web | 2021-06-30 15:52:45,829 - chardet.charsetprober - DEBUG - KOI8-R Russian confidence = 0.1981857351126757
ghc_web | 2021-06-30 15:52:45,829 - chardet.charsetprober - DEBUG - ISO-8859-5 Russian confidence = 0.0
ghc_web | 2021-06-30 15:52:45,830 - chardet.charsetprober - DEBUG - MacCyrillic Russian confidence = 0.0
ghc_web | 2021-06-30 15:52:45,831 - urllib3.connectionpool - DEBUG - https://:443 "POST /oliepompinstallaties/ows HTTP/1.1" 200 382
ghc_web | 2021-06-30 15:52:45,831 - chardet.charsetprober - DEBUG - IBM866 Russian confidence = 0.0
ghc_web | 2021-06-30 15:52:45,832 - chardet.charsetprober - DEBUG - IBM855 Russian confidence = 0.13212382340845047
ghc_web | 2021-06-30 15:52:45,833 - GeoHealthCheck.probe - INFO - response: status=200
ghc_web | 2021-06-30 15:52:45,833 - chardet.charsetprober - DEBUG - ISO-8859-7 Greek confidence = 0.0
ghc_web | 2021-06-30 15:52:45,833 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.HttpStatusNoError result=True
ghc_web | 2021-06-30 15:52:45,833 - chardet.charsetprober - DEBUG - windows-1253 Greek confidence = 0.0
ghc_web | 2021-06-30 15:52:45,834 - chardet.charsetprober - DEBUG - ISO-8859-5 Bulgarian confidence = 0.01
ghc_web | 2021-06-30 15:52:45,834 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.XmlParse result=True
ghc_web | 2021-06-30 15:52:45,834 - chardet.charsetprober - DEBUG - windows-1251 Bulgarian confidence = 0.0
ghc_web | 2021-06-30 15:52:45,835 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.NotContainsOwsException result=True
ghc_web | 2021-06-30 15:52:45,835 - chardet.charsetprober - DEBUG - TIS-620 Thai confidence = 0.01
ghc_web | 2021-06-30 15:52:45,835 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.ContainsStrings result=True
ghc_web | 2021-06-30 15:52:45,836 - chardet.charsetprober - DEBUG - ISO-8859-9 Turkish confidence = 0.0
ghc_web | 2021-06-30 15:52:45,836 - scheduler - INFO - 228 run_resource OK
ghc_web | 2021-06-30 15:52:45,836 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.HttpStatusNoError result=True
ghc_web | 2021-06-30 15:52:45,836 - chardet.charsetprober - DEBUG - windows-1255 Hebrew confidence = 0.0
ghc_web | 2021-06-30 15:52:45,837 - apscheduler.executors.default - INFO - Job "run_job (trigger: interval[0:05:00], next run at: 2021-06-30 15:55:43 CEST)" executed successfully
ghc_web | 2021-06-30 15:52:45,837 - chardet.charsetprober - DEBUG - windows-1255 Hebrew confidence = 0.0
ghc_web | 2021-06-30 15:52:45,837 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.XmlParse result=True
ghc_web | 2021-06-30 15:52:45,838 - apscheduler.executors.default - INFO - Running job "run_job (trigger: interval[0:05:00], next run at: 2021-06-30 15:55:43 CEST)" (scheduled at 2021-06-30 15:50:43.671300+02:00)
ghc_web | 2021-06-30 15:52:45,838 - chardet.charsetprober - DEBUG - windows-1255 Hebrew confidence = 0.0
ghc_web | 2021-06-30 15:52:45,839 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.NotContainsOwsException result=True
ghc_web | 2021-06-30 15:52:45,841 - chardet.universaldetector - DEBUG - no probers hit minimum threshold
ghc_web | 2021-06-30 15:52:45,841 - GeoHealthCheck.probe - INFO - Check: fun=GeoHealthCheck.plugins.check.checks.ContainsStrings result=True
ghc_web | 2021-06-30 15:52:45,841 - chardet.universaldetector - DEBUG - utf-8 confidence = 0.010000000000000009
ghc_web | 2021-06-30 15:52:45,841 - GeoHealthCheck.probe - INFO - Result: success=True msg=OK response_time=0.442921
ghc_web | 2021-06-30 15:52:45,842 - chardet.universaldetector - DEBUG - SHIFT_JIS Japanese confidence = 0.01
ghc_web | 2021-06-30 15:52:45,849 - chardet.universaldetector - DEBUG - EUC-JP Japanese confidence = 0.01
ghc_web | 2021-06-30 15:52:45,849 - chardet.universaldetector - DEBUG - GB2312 Chinese confidence = 0.01
ghc_web | 2021-06-30 15:52:45,850 - chardet.universaldetector - DEBUG - EUC-KR Korean confidence = 0.01
ghc_web | 2021-06-30 15:52:45,856 - chardet.universaldetector - DEBUG - CP949 Korean confidence = 0.01
```

In this example only the following line contains the resource ID:
`ghc_web | 2021-06-30 15:52:45,836 - scheduler - INFO - 228 run_resource OK`

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia dal punto di ingresso `run_resource` dello scheduler e dai messaggi di log di `GeoHealthCheck.probe` mostrati nell’esempio, quindi segui il percorso di una risorsa attraverso i suoi controlli e le sue richieste. Il lavoro è completato quando le righe di log relative alle risorse identificano sempre l’ID della risorsa, inclusa l’output di un’esecuzione in cui vengono controllate più risorse insieme.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
observability
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.