nprobe packet drops during peak hours

root@ams7nprobe01:/etc/cluster# pack_drops
Mon Jul 20 05:29:16 PM UTC 2026
/proc/net/pf_ring/stats/3592845-none.1:ClusterId: 10
/proc/net/pf_ring/stats/3592845-none.1:Packets: 6083
/proc/net/pf_ring/stats/3592845-none.1:Forwarded: 12166
/proc/net/pf_ring/stats/3592898-none.3:ClusterId: 11
/proc/net/pf_ring/stats/3592898-none.3:Packets: 105218079
/proc/net/pf_ring/stats/3592898-none.3:Forwarded: 100663227
/proc/net/pf_ring/stats/3592898-none.3:IFPackets: 105218079
/proc/net/pf_ring/stats/3592898-none.3:Q0Packets: 13372435
/proc/net/pf_ring/stats/3592898-none.3:Q1Packets: 15105160
/proc/net/pf_ring/stats/3592898-none.3:Q2Packets: 13547218
/proc/net/pf_ring/stats/3592898-none.3:Q3Packets: 14256852
/proc/net/pf_ring/stats/3592898-none.3:Q4Packets: 16126656
/proc/net/pf_ring/stats/3592898-none.3:Q5Packets: 15042539
/proc/net/pf_ring/stats/3592898-none.3:Q6Packets: 13146847
/proc/net/pf_ring/stats/3592898-none.3:Q7Packets: 0
/proc/net/pf_ring/stats/3593057-none.8:Packets: 13384351/379 (rcvd/dropped)
/proc/net/pf_ring/stats/3593062-none.6:Packets: 15119997/13469 (rcvd/dropped)
/proc/net/pf_ring/stats/3593066-none.9:Packets: 13560449/419 (rcvd/dropped)
/proc/net/pf_ring/stats/3593093-none.16:Packets: 14271748/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593105-none.13:Packets: 16145339/2460 (rcvd/dropped)
/proc/net/pf_ring/stats/3593114-none.12:Packets: 15059423/1206 (rcvd/dropped)
/proc/net/pf_ring/stats/3593156-none.18:Packets: 13160389/559 (rcvd/dropped)
/proc/net/pf_ring/stats/3593325-none.24:Packets: 693/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593330-none.20:Packets: 660/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593335-none.22:Packets: 381/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593360-none.26:Packets: 1073/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593375-none.28:Packets: 175/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593390-none.30:Packets: 2219/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593413-none.32:Packets: 482/0 (rcvd/dropped)
root@ams7nprobe01:/etc/cluster# cd /proc/net/pf_ring/stats/
root@ams7nprobe01:/proc/net/pf_ring/stats# cat 3593062-none.6
Duration: 0:00:05:52:025
Bytes: 8862633493
Packets: 16455174/13469 (rcvd/dropped)
FlowCacheStats: 96004/0 (active/toBeExported)
FlowExportStats: 0/0/905097/0 (bytes/pkts/flows/flow_pkts)
FlowExportDropStats: 0/0/0 (bytes/pkts/flows)
TotalFlowStats: 0/0/905097 (bytes/pkts/flows)
ExportQueue: 0/2000000 (current/max)
root@ams7nprobe01:/proc/net/pf_ring/stats# ps -ef | grep 3593062
root 3593062 1 27 17:23 ? 00:01:40 /usr/bin/nprobe /run/nprobe-zc11_1.conf
root 3636468 3514634 0 17:29 pts/1 00:00:00 grep --color=auto 3593062
root@ams7nprobe01:/proc/net/pf_ring/stats# cd /etc/nprobe^C
root@ams7nprobe01:/proc/net/pf_ring/stats# pack_drops
Mon Jul 20 05:30:43 PM UTC 2026
/proc/net/pf_ring/stats/3592845-none.1:ClusterId: 10
/proc/net/pf_ring/stats/3592845-none.1:Packets: 7500
/proc/net/pf_ring/stats/3592845-none.1:Forwarded: 15000
/proc/net/pf_ring/stats/3592898-none.3:ClusterId: 11
/proc/net/pf_ring/stats/3592898-none.3:Packets: 133899377
/proc/net/pf_ring/stats/3592898-none.3:Forwarded: 129318183
/proc/net/pf_ring/stats/3592898-none.3:IFPackets: 133899377
/proc/net/pf_ring/stats/3592898-none.3:Q0Packets: 17059272
/proc/net/pf_ring/stats/3592898-none.3:Q1Packets: 19402744
/proc/net/pf_ring/stats/3592898-none.3:Q2Packets: 17402252
/proc/net/pf_ring/stats/3592898-none.3:Q3Packets: 18231198
/proc/net/pf_ring/stats/3592898-none.3:Q4Packets: 20890860
/proc/net/pf_ring/stats/3592898-none.3:Q5Packets: 19260966
/proc/net/pf_ring/stats/3592898-none.3:Q6Packets: 17005371
/proc/net/pf_ring/stats/3592898-none.3:Q7Packets: 0
/proc/net/pf_ring/stats/3593057-none.8:Packets: 17073165/1018 (rcvd/dropped)
/proc/net/pf_ring/stats/3593062-none.6:Packets: 19420933/19779 (rcvd/dropped)
/proc/net/pf_ring/stats/3593066-none.9:Packets: 17416324/1455 (rcvd/dropped)
/proc/net/pf_ring/stats/3593093-none.16:Packets: 18246837/3774 (rcvd/dropped)
/proc/net/pf_ring/stats/3593105-none.13:Packets: 20908664/5874 (rcvd/dropped)
/proc/net/pf_ring/stats/3593114-none.12:Packets: 19276732/9142 (rcvd/dropped)
/proc/net/pf_ring/stats/3593156-none.18:Packets: 17026010/3792 (rcvd/dropped)
/proc/net/pf_ring/stats/3593325-none.24:Packets: 897/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593330-none.20:Packets: 815/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593335-none.22:Packets: 466/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593360-none.26:Packets: 1347/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593375-none.28:Packets: 219/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593390-none.30:Packets: 2789/0 (rcvd/dropped)
/proc/net/pf_ring/stats/3593413-none.32:Packets: 568/0 (rcvd/dropped)
root@ams7nprobe01:/proc/net/pf_ring/stats#

Configuration
root@ams7nprobe01:/etc/cluster# cat cluster-11.conf

Sample configuration file for ZC cluster (zbalance_ipc)

Please also read https://www.ntop.org/guides/pf_ring/rss.html?highlight=cluster#zc-load-balancing-zbalance-ipc

Full options are available with 'zbalance_ipc -h'

Usage:

1. Copy this file to the actual configuration file including the cluster ID in the file name

cp cluster.conf.example cluster-10.conf

2. Enable the service

systemctl enable cluster@10

3. Start the configured cluster

systemctl start cluster@10

Capture Interface

Both RX and TX

-i=zc:eno5,zc:eno6

Load-Balance to 4 consumer application instances

-n=7,1
#-n=7,1

Load-Balance to 4 instances of application A, full traffic to 1 instance of application B

#-n=4,1

Distribution function

- 0 Round-Robin (default)

- 1 IP hash

- 2 Fan-out

- 3 Fan-out (1st) + Round-Robin (2nd, 3rd, ..)

- 4 GTP hash (Inner IP/Port or GTP-C Seq-Num)

- 5 GRE hash (Inner or Outer IP)

- 6 Interface X to queue X

-m=4

Cluster ID (unique identifier for the cluster)

-c=11

Capture thread CPU Core affinity

-g=49

Time thread CPU Core affinity

#-S=0
-p
root@ams7nprobe01:/etc/cluster#

root@ams7nprobe01:/etc/nprobe# cat nprobe-zc11_3.conf

Flow from ZC0 data probe.

#Connected to the specified redis server. Multiple
#--redis localhost:6380

The interface on which the streams are received

-i=zc:11@3

-L=0.0.0.0/0
-p=0/1/1/1/0/1/1/0/1

Folder for the output flows

-P=/opt/data/nProbe/all_flows

bound to specific CPU

--cpu-affinity 43

Discard IPv6 traffic

-W

Specify packet capture direction, 0=RX+TX (default), 1=RX only, 2=TX only

--capture-direction 1

size of the hash that stores the flows

-w=2000000

maximum number of active flows

-M=1000000

maximum flow lifetime

-t=60

maximum flow idle lifetime

-d=30

maximum queue timeout

#-l=1

how often the hash is walked searching expired flows

#-s=10

Dump relevant activities (e.g. nProbe start/stop or packet drop) onto the specified file.

--event-log=/opt/data/nProbe/event_log/event_log.txt

Version of the flow

--flow-version 9

Ignore UDP fragmented packets with fragment offset greater than zero, and compute the fragmented packet length on the initial fragment header.

#This flag might lead to inaccuracy in measurement but it speeds us operations with fragmented traffic.
--smart-udp-frags

Append the nProbe PID to dump files to avoid file overwrite in case multiple probes dump onto the same directory

--add-pid-to-logfile

Enable hw timestamping/stripping

--timestamp-format 2

Maximum number of lines on a dump file, or 0 = unlimited. Default: 10000

#--max-log-lines 1000000

Specify the text files separator (see -P)

--csv-separator '|'

CSV template format

-T="%FLOW_START_MILLISECONDS %FLOW_END_MILLISECONDS %FLOW_DURATION_MILLISECONDS %PROTOCOL %IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %EXPORTER_IPV4_ADDRESS %IN_PKTS %IN_BYTES %OUT_PKTS %OUT_BYTES %SRC_VLAN %DST_VLAN %BIFLOW_DIRECTION %CLIENT_TCP_FLAGS %SERVER_TCP_FLAGS %FLOW_USER_NAME %UPSTREAM_TUNNEL_ID %DOWNSTREAM_TUNNEL_ID %DNS_QUERY %DNS_QUERY_ID %DNS_QUERY_TYPE %DNS_RET_CODE %DNS_NUM_ANSWERS %DNS_TTL_ANSWER %DNS_RESPONSE %UNTUNNELED_IPV4_SRC_ADDR %UNTUNNELED_L4_SRC_PORT %UNTUNNELED_IPV4_DST_ADDR %UNTUNNELED_L4_DST_PORT %ENCAP_IPV4_SRC_ADDR %ENCAP_IPV4_DST_ADDR %ENCAP_IN_BYTES %ENCAP_OUT_BYTES"

Flat dumps, Dump files (-P) won't be saved on nested dirs.

--dont-nest-dump-dirs

No collector to send to

-n=none

--dont-drop-privileges

Unwrap GTP

--tunnel

Enable IMSI aggregation on GTPv1 signalling

#--gtpv1-track-imsi

Enable GTPv2 traffic accounting

#--gtpv2-track-imsi
--gtpv1-dump-dir /opt/data/nProbe/all_flows
--gtpv2-dump-dir /opt/data/nProbe/all_flows

Specifies (in seconds) how long the GTPv1 IMSI/TEID

--gtpv1-teid-cache-duration 2592000

Specifies (in seconds) how long the GTPv2 IMSI/TEID

--gtpv2-teid-cache-duration 2592000

If run from command line - run in daemon mode using this pid file

-G=/var/run/nprobe-zc11_3.pid
-b 1
--max-log-lines 250000
root@ams7nprobe01:/etc/nprobe#

Start with the packet counters under /proc/net/pf_ring/stats and compare them with cluster-11.conf and nprobe-zc11_3.conf, including the configured CPU affinities and queue settings. Check the referenced nProbe event log for correlated drops. The issue does not define a desired fix, so completion requires identifying the cause and agreeing on a verified remediation.