TcpSocket probes are meaningless with meshed pods

What is the issue?

Meshed pods have all ports redirected to linkerd-proxy sidecar and as such all ports are open at the TCP level. This means probes with TCPSocketAction are meaningless, as they will succeed as long as linkerd-proxy is alive, independent if the main container is dead.

How can it be reproduced?

While this deployment will fail liveness probes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-fails
  labels:
    app: busybox-fails
spec:
  replicas: 3
  selector:
    matchLabels:
      app: busybox-fails
  template:
    metadata:
      labels:
        app: busybox-fails
    spec:
      containers:
      - name: busybox
        image: busybox:latest
        command:
        - sleep
        args:
        - "3600"
        livenessProbe:
          tcpSocket:
            port: 8080

Injecting linkerd is enough to make the probes pass:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox
  labels:
    app: busybox
spec:
  replicas: 3
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      annotations:
        linkerd.io/inject: enabled
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox:latest
        command:
        - sleep
        args:
        - "3600"
        livenessProbe:
          tcpSocket:
            port: 8080

Logs, error output, etc

Seems non-applicable.

output of linkerd check -o short

Linkerd core checks
===================


Status check results are √

Linkerd extensions checks
=========================


Status check results are √


Status check results are √

Environment

• Kubernetes v1.21.6-gke.1500 on GKE

Possible solution

As discussed in Slack, the ideal solution here is to passthrough kubelet traffic to the pod without redirecting to linkerd-proxy container, but at a minimum we should have this very clearly documented so developers are not surprised.

Additional context

No response

Would you like to work on fixing this bug?

no

Start by locating the Linkerd documentation covering Kubernetes liveness probes and sidecar traffic redirection. Reproduce the behavior with the two deployment manifests in the issue, then document the limitation and recommended probe alternatives so meshed-pod users can understand when TCP probes are unreliable.