elastic / elastic/beats

Packetbeat flows enhancements

Open
#3,444 21 comments 23 reactions 0 assignees View on GitHub
discuss enhancement Packetbeat Team:Security-Service Integrations
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

Flows support in 5.0 is quite rudimentary, not really taking any network layers (arbitrary timeout) into account and only counting packets and bytes. Plus, there is a parent-child relation between flows and transactions, not yet represented in packetbeat events being published.

List of flow proposed enhancements:

- [ ] pass active flow to packet analyzer:
- [ ] add protocol specific stats to flows (e.g. number of transactions with success/failure/dropped due to packet loss/timed out)
- [ ] add flow id to protocol transaction events to establish some relationship between flows and transaction
- [ ] add support to tie flow lifetime to connection status:
- [ ] Flows for TCP/SCTP streams do not timeout while TCP connection is active
- [ ] Protocols on top of UDP managing connection state should be able to disable flow timeout
- [ ] Stop flow if connection is closed or connection attempt failed:
- [ ] for TCP stop flow on RST or normal TCP shutdown
- [ ] consider ICMP messages if TCP/UDP port is not reachable
- [ ] timeout flow in case of handshake being incomplete and no data being send in either direction
- [ ] add indicator (string?) to indicate the reason a flow has ended
- [ ] add indicator if connection has been good (established TCP connection), in case it ended forcefully
- [ ] add indicator for flow starting to capture an older TCP connection (not having seen the connection attempt itself, as packetbeat was started after)
- [ ] report TCP level stats:
- [ ] count flags usage (SYN, RST, FIN, PUSH)
- [ ] report bytes lost due to packet-loss
- [ ] report packets fully/partially resend (number of packets and bytes)

Contributor guide

Open the contributing guide

Research direction

The issue names no files or tests. Start with Packetbeat's flow handling and packet analyzer entry points, then trace how protocol transaction events are published. Scope one agreed subset of the checklist and add coverage for its flow lifetime, transaction linkage, or TCP statistics behavior; done requires that subset implemented and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.