iovisor / iovisor/bcc

[RFC] Convert trace data to CTF

Open
#1,049 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.7k
Forks
4.1k
Avg merge
10d 4h
Merged PRs (30d)
3

Description

A possible extension to BCC can be to convert trace data coming from maps and write it as [CTF](http://diamon.org/ctf/) data. The CTF format is a compact binary trace format which is used by tools such as LTTng and can allow traces to be viewed in a very nice fashion in [Trace Compass](http://tracecompass.org/). This would allow efficient trace storage for post-mortem analysis. Also, it would allow traces to be viewed graphically. Trace Compass has very interesting features such as flame-charts, and critical flow view, resource view (and [many more](https://wiki.eclipse.org/Trace_Compass/News/NewIn20)).

I have done a quick test which can write CTF traces using [babeltrace](http://diamon.org/babeltrace/) library (2.0.0-pre) Here are the results :

```
[suchakra@moria tracing]$ sudo ./open2ctf.py
trace path: /tmp/tmp26ga140x
^C[suchakra@moria tracing]$ babeltrace /tmp/tmp26ga140x
[00:24:50.796162597] (+?.?????????) 0 do_sys_open: { }, { pid = 909, comm = "iio-sensor-prox", filename = "/dev/iio:device2" }
[00:24:50.796399646] (+0.000237049) 0 do_sys_open: { }, { pid = 909, comm = "iio-sensor-prox", filename = "/dev/iio:device4" }
[00:24:50.801229686] (+0.004830040) 0 do_sys_open: { }, { pid = 1206, comm = "gnome-shell", filename = "/proc/self/stat" }
[00:24:50.801430938] (+0.000201252) 0 do_sys_open: { }, { pid = 1734, comm = "gnome-shell", filename = "/proc/self/stat" }
[00:24:50.807656245] (+0.006225307) 0 do_sys_open: { }, { pid = 18399, comm = "gnome-settings-", filename = "/proc/self/fd" }
..
..
```
Here, we collect trace data from `do_sys_open` and save it in CTF, then open the trace using `babeltrace` (which is a trace reader as well). This trace also works with Trace Compass. As we increase number of relevant trace events collected (`sched_*` etc.), Trace Compass would eventually be able to show something like [this](http://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/images/TimeAlignment_sash.png). This in PoC stage for now - the clocks and timings have to be improved. The deltas are ok, but wall clock time is not correct. We'll do a helper function in BCC itself that can help write these traces. Expect a PR soon :) If you have suggestions, leave them here!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the open2ctf.py proof of concept and its babeltrace output for do_sys_open events. Then examine how BCC collects map and sched_* trace data and how CTF timestamps are represented. Done means a defined BCC helper can write traces that babeltrace and Trace Compass read correctly, including improved clock and wall-time handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.