ansible / ansible/awx-operator
ConnectionResetError: [Errno 104] Connection reset by peer
- Dominant language
- Jinja
- Stars
- 1.5k
- Forks
- 714
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 3
Description
### Please confirm the following
- [X] I agree to follow this project's [code of conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html).
- [X] I have checked the [current issues](https://github.com/ansible/awx-operator/issues) for duplicates.
- [X] I understand that the AWX Operator is open source software provided for free and that I might not receive a timely response.
### Bug Summary
Hello Everyone,
Suddenly getting below error while invoking API. This was working fine until 07/20, Saturday. Anyone facing same error?
Note: We are encountering this issue on all the AWX instances throughout the all locations.
AWX INFO *****************
cat kustomization.yaml
apiVersion: [kustomize.config.k8s.io/v1beta1](http://kustomize.config.k8s.io/v1beta1)
kind: Kustomization
resources:
# Find the latest tag here: https://github.com/ansible/awx-operator/releases
- [github.com/ansible/awx-operator/config/default?ref=0.29.0](http://github.com/ansible/awx-operator/config/default?ref=0.29.0)
- awx-pro.yaml
- my-ingress.yml
# Set the image tags to match the git version from above
images:
- name: [quay.io/ansible/awx-operator](http://quay.io/ansible/awx-operator)
newTag: 0.29.0
# Specify a custom namespace in which to install AWX
namespace: awx-lvn
Error Info ************************
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.9/ssl.py", line 501, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib64/python3.9/ssl.py", line 1041, in _create
self.do_handshake()
File "/usr/lib64/python3.9/ssl.py", line 1310, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
### AWX Operator version
0.29.0
### AWX version
21.6.0
### Kubernetes platform
minikube
### Kubernetes/Platform version
Minikube: 1.27
### Modifications
no
### Steps to reproduce
job_launch_url="$AWX_HOST/api/v2/job_templates/$ext_tempt_id/launch/"
job_id=$(curl -k -H "Authorization: Bearer $AWX_TOKEN" -X POST "$job_launch_url")
ext_job_id=$(extract_value 'id' "$job_id")
echo "Job '$ext_job_id' launched from template '$TEMPLATE_NAME'."
Issue occurring after job is launched through API from the client, the job launches which hits AWX server but then connection is getting rejected when AWX is trying to connect client
Error:
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.9/ssl.py", line 501, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib64/python3.9/ssl.py", line 1041, in _create
self.do_handshake()
File "/usr/lib64/python3.9/ssl.py", line 1310, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
### Expected results
AWX should launch job on the target client.
### Actual results
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.9/ssl.py", line 501, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib64/python3.9/ssl.py", line 1041, in _create
self.do_handshake()
File "/usr/lib64/python3.9/ssl.py", line 1310, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
### Additional information
_No response_
### Operator Logs
_No response_
Contributor guide
Research direction
Start with the supplied kustomization.yaml and the curl job-launch command against the AWX Operator 0.29.0 deployment on minikube. Review the reported urllib3 SSL traceback and available operator logs; no source file or test is identified. Done means reproducing the reset, identifying its cause, and documenting a verified resolution or a clear project-side diagnosis.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100