networkservicemesh / networkservicemesh/sdk-kernel

Liveliness checker has a false positive result in case of high load

Open
#622 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
6
Forks
17
Avg merge
1h 37m
Merged PRs (30d)
5

Description

Description

The problem only occurs for UDP traffic.

Steps to reproduce:

  1. Create kind cluster
  2. Deploy Kernel2Ethernet2Kernel
  3. Install iperf3 on the NSC/NSE
  4. Start iper3 on the NSE: iperf3 -s
  5. Start iper3 on the NSC: iperf3 -c 172.16.1.100 -u -b 10G
  6. Output:
...
[  5]   5.00-6.00   sec   878 MBytes  7.37 Gbits/sec  660633  
[  5]   6.00-7.00   sec   880 MBytes  7.38 Gbits/sec  662190  
iperf3: error - unable to write to stream socket: Network unreachable
  1. NSC log:
Nov 10 08:16:16.942 [ERRO] [id:alpine-2a483148-ea71-4cc0-a3ac-22bb1e8e9a75-0] [dstIP:172.16.1.100] [srcIP:172.16.1.101] [type:networkService] (7.1)         No packets received
Nov 10 08:16:16.942 [WARN] [id:alpine-2a483148-ea71-4cc0-a3ac-22bb1e8e9a75-0] [heal:eventLoop] [type:networkService] (7.2)         Data plane is down
Nov 10 08:16:16.942 [DEBU] [id:alpine-2a483148-ea71-4cc0-a3ac-22bb1e8e9a75-0] [heal:eventLoop] [type:networkService] (7.3)         Reconnect with reselect

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the UDP case in a kind cluster with Kernel2Ethernet2Kernel and iperf3 at 10G, then correlate the NSC liveliness-checker logs with the reported socket error. The work is complete when sustained UDP traffic no longer produces a false “No packets received” result or an unnecessary data-plane reconnect.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.