Proxy log connection reset by peer and connection refused
@cpretzer is already working on this.
Since Sep 20, 2019.
- Dominant language
- JavaScript
- Stars
- 60
- Forks
- 221
- Avg merge
- 4h 32m
- Merged PRs (30d)
- 1
Description
Bug Report
What is the issue?
I see a lot of connection reset by peer errors, from the proxy log, on all my services. I'm kind of stuck in finding the issue, so i hope someone can point me in the right direction. It dosen't seem to affect the traffic, but the success rate in Linkerd is sometimes down a few percent.
How can it be reproduced?
No i am not able to reproduce the issue, by doing a lot of requests to the service. The problem only shows in the logs.
Logs, error output, etc
ERR! [ 11082.146455s] proxy={server=in listen=0.0.0.0:4143 remote=bla.bla.bla:35314} linkerd2_proxy::app::errors unexpected error: connection error: Connection reset by peer (os error 104)
And this one:
ERR! [ 87.959649s] proxy={server=in listen=0.0.0.0:4143 remote=bla.bla.bla:50416} linkerd2_proxy::app::errors unexpected error: error trying to connect: Connection refused (os error 111) (address: 127.0.0.1:8080)
linkerd check output
kubernetes-api
√ can initialize the client
√ can query the Kubernetes API
kubernetes-version
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version
linkerd-config
√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ control plane PodSecurityPolicies exist
linkerd-existence
√ 'linkerd-config' config map exists
√ control plane replica sets are ready
√ no unschedulable pods
√ controller pod is running
√ can initialize the client
√ can query the control plane API
linkerd-api
√ control plane pods are ready
√ control plane self-check
√ [kubernetes] control plane can talk to Kubernetes
√ [prometheus] control plane can talk to Prometheus
√ no invalid service profiles
linkerd-version
√ can determine the latest version
‼ cli is up-to-date
is running version 2.4.0 but the latest stable version is 2.5.0
see https://linkerd.io/checks/#l5d-version-cli for hints
control-plane-version
‼ control plane is up-to-date
is running version 2.4.0 but the latest stable version is 2.5.0
see https://linkerd.io/checks/#l5d-version-control for hints
√ control plane and cli versions match
Status check results are √
Environment
- Kubernetes Version: 1.13.7-gke.8
- Cluster Environment: GKE
- Host OS: Ubuntu
- Linkerd version: 2.4.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.