google / google/fleetspeak

Record network usage of a Fleetspeak client (Bytes sent/received)

Open
#299 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
154
Forks
33
Avg merge
1d 13h
Merged PRs (30d)
2

Description

We want to record the network usage of a single Fleetspeak client deployed, so that we can later on plot, inspect and analyze this data for measuring performance and be alerted if there are any unexpected "spikes" in network usage.
An example for a use case is GRR, that has a dashboard for Client Load Stats (google/grr#859) which fetches its data from Fleetspeak server's datastore.

The concept is similar to other metrics gathered from Fleetspeak clients, such as [resident memory size](https://github.com/google/fleetspeak/blob/298e9ae7573cf0c2f29b0c1e68a0a11370259c53/fleetspeak/src/client/internal/monitoring/resource_usage_fetcher.go#L41).
As an example, check out #275 to learn more about how IO MiB read/write is implemented.

**Implementation details:**
Other metrics gathered for individual Fleetspeak clients are using the [`ResourceUsageFetcher`](https://github.com/google/fleetspeak/blob/298e9ae7573cf0c2f29b0c1e68a0a11370259c53/fleetspeak/src/client/internal/monitoring/resource_usage_fetcher.go#L45) struct, and specifically the method [`ResourceUsageForPID`](https://github.com/google/fleetspeak/blob/298e9ae7573cf0c2f29b0c1e68a0a11370259c53/fleetspeak/src/client/internal/monitoring/resource_usage_fetcher.go#L63). Unlike those metrics, the kernel does not have information about the network usage of a given process PID, so this cannot be fetched using the library [`gopsutil`](https://github.com/shirou/gopsutil).
Thus, an idea for implementing this: add a new struct (a singleton effectively) that will become a field in [`ResourceUsageFetcher`](https://github.com/google/fleetspeak/blob/298e9ae7573cf0c2f29b0c1e68a0a11370259c53/fleetspeak/src/client/internal/monitoring/resource_usage_fetcher.go#L45). This new singleton will also be shared with the [`StreamingCommunicator`](https://github.com/google/fleetspeak/blob/fbf9f00d8f90d29ecf545cf133228f1626aa11ab/fleetspeak/src/client/https/streaming.go#L44) struct (note that an implementation may needed for polling as well), so network usage data can be recorded when messages enter/exit the Fleetspeak client in the methods [`readLoop`](https://github.com/google/fleetspeak/blob/fbf9f00d8f90d29ecf545cf133228f1626aa11ab/fleetspeak/src/client/https/streaming.go#L473) and [`writeLoop`](https://github.com/google/fleetspeak/blob/fbf9f00d8f90d29ecf545cf133228f1626aa11ab/fleetspeak/src/client/https/streaming.go#L403), where both are called from the [`connect`](https://github.com/google/fleetspeak/blob/fbf9f00d8f90d29ecf545cf133228f1626aa11ab/fleetspeak/src/client/https/streaming.go#L202) method of `StreamingCommunicator`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.