linkerd / linkerd/website

Proxy log connection reset by peer and connection refused

Open
#617 5 comments 0 reactions 1 assignee View on GitHub

@cpretzer is already working on this.

Since Sep 20, 2019.

Dominant language
JavaScript
Stars
60
Forks
221
Avg merge
4h 32m
Merged PRs (30d)
1

Description

Bug Report

What is the issue?

I see a lot of connection reset by peer errors, from the proxy log, on all my services. I'm kind of stuck in finding the issue, so i hope someone can point me in the right direction. It dosen't seem to affect the traffic, but the success rate in Linkerd is sometimes down a few percent.

How can it be reproduced?

No i am not able to reproduce the issue, by doing a lot of requests to the service. The problem only shows in the logs.

Logs, error output, etc

ERR! [ 11082.146455s] proxy={server=in listen=0.0.0.0:4143 remote=bla.bla.bla:35314} linkerd2_proxy::app::errors unexpected error: connection error: Connection reset by peer (os error 104)

And this one:

ERR! [ 87.959649s] proxy={server=in listen=0.0.0.0:4143 remote=bla.bla.bla:50416} linkerd2_proxy::app::errors unexpected error: error trying to connect: Connection refused (os error 111) (address: 127.0.0.1:8080)

linkerd check output

kubernetes-api

√ can initialize the client
√ can query the Kubernetes API

kubernetes-version

√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

linkerd-config

√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ control plane PodSecurityPolicies exist

linkerd-existence

√ 'linkerd-config' config map exists
√ control plane replica sets are ready
√ no unschedulable pods
√ controller pod is running
√ can initialize the client
√ can query the control plane API

linkerd-api

√ control plane pods are ready
√ control plane self-check
√ [kubernetes] control plane can talk to Kubernetes
√ [prometheus] control plane can talk to Prometheus
√ no invalid service profiles

linkerd-version

√ can determine the latest version
‼ cli is up-to-date
is running version 2.4.0 but the latest stable version is 2.5.0
see https://linkerd.io/checks/#l5d-version-cli for hints

control-plane-version

‼ control plane is up-to-date
is running version 2.4.0 but the latest stable version is 2.5.0
see https://linkerd.io/checks/#l5d-version-control for hints
√ control plane and cli versions match

Status check results are √

Environment
  • Kubernetes Version: 1.13.7-gke.8
  • Cluster Environment: GKE
  • Host OS: Ubuntu
  • Linkerd version: 2.4.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.