InstIDEA / InstIDEA/controlciudadano
Contralory PDF download DAG fails when the contralory page resets a connection
- Dominant language
- TypeScript
- Stars
- 5
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
contralory_declaration_download_pdfs DAG fails with:
> [2021-02-02 03:01:58,064] {{taskinstance.py:1128}} ERROR - HTTPSConnectionPool(host='portaldjbr.contraloria.gov.py', port=443): Max retries exceeded with url: /portal-djbr/api/consulta/descargarpdf/1286712 (Caused by SSLError(SSLError("bad handshake: SysCallError(104, 'ECONNRESET')")))
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
> cnx.do_handshake()
> File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1934, in do_handshake
> self._raise_ssl_error(self._ssl, result)
> File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1663, in _raise_ssl_error
> raise SysCallError(errno, errorcode.get(errno))
> OpenSSL.SSL.SysCallError: (104, 'ECONNRESET')
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
> chunked=chunked,
> File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 376, in _make_request
> self._validate_conn(conn)
> File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
> conn.connect()
> File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 360, in connect
> ssl_context=context,
> File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
> return context.wrap_socket(sock, server_hostname=server_hostname)
> File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 491, in wrap_socket
> raise ssl.SSLError("bad handshake: %r" % e)
> ssl.SSLError: ("bad handshake: SysCallError(104, 'ECONNRESET')",)
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
> timeout=timeout
> File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
> method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
> File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 436, in increment
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
> urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='portaldjbr.contraloria.gov.py', port=443): Max retries exceeded with url: /portal-djbr/api/consulta/descargarpdf/1286712 (Caused by SSLError(SSLError("bad handshake: SysCallError(104, 'ECONNRESET')")))
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 966, in _run_raw_task
> result = task_copy.execute(context=context)
> File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 113, in execute
> return_value = self.execute_callable()
> File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 118, in execute_callable
> return self.python_callable(*self.op_args, **self.op_kwargs)
> File "/usr/local/airflow/dags/contralory_declaration_download_pdfs.py", line 185, in do_work
> record["cedula"])
> File "/usr/local/airflow/dags/contralory_declaration_download_pdfs.py", line 142, in download_pdf
> file_info = get_head(final_url, verify=False)
> File "/usr/local/airflow/dags/network_operators.py", line 146, in get_head
> head_info = requests.head(url, verify=verify)
> File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 101, in head
> return request('head', url, **kwargs)
> File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 60, in request
> return session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
> resp = self.send(prep, **send_kwargs)
> File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
> r = adapter.send(request, **kwargs)
> File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 514, in send
> raise SSLError(e, request=request)
> requests.exceptions.SSLError: HTTPSConnectionPool(host='portaldjbr.contraloria.gov.py', port=443): Max retries exceeded with url: /portal-djbr/api/consulta/descargarpdf/1286712 (Caused by SSLError(SSLError("bad handshake: SysCallError(104, 'ECONNRESET')")))
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.