[RFC] Convert trace data to CTF
- Dominant language
- C
- Stars
- 22.7k
- Forks
- 4.1k
- Avg merge
- 10d 4h
- Merged PRs (30d)
- 3
Description
A possible extension to BCC can be to convert trace data coming from maps and write it as [CTF](http://diamon.org/ctf/) data. The CTF format is a compact binary trace format which is used by tools such as LTTng and can allow traces to be viewed in a very nice fashion in [Trace Compass](http://tracecompass.org/). This would allow efficient trace storage for post-mortem analysis. Also, it would allow traces to be viewed graphically. Trace Compass has very interesting features such as flame-charts, and critical flow view, resource view (and [many more](https://wiki.eclipse.org/Trace_Compass/News/NewIn20)).
I have done a quick test which can write CTF traces using [babeltrace](http://diamon.org/babeltrace/) library (2.0.0-pre) Here are the results :
```
[suchakra@moria tracing]$ sudo ./open2ctf.py
trace path: /tmp/tmp26ga140x
^C[suchakra@moria tracing]$ babeltrace /tmp/tmp26ga140x
[00:24:50.796162597] (+?.?????????) 0 do_sys_open: { }, { pid = 909, comm = "iio-sensor-prox", filename = "/dev/iio:device2" }
[00:24:50.796399646] (+0.000237049) 0 do_sys_open: { }, { pid = 909, comm = "iio-sensor-prox", filename = "/dev/iio:device4" }
[00:24:50.801229686] (+0.004830040) 0 do_sys_open: { }, { pid = 1206, comm = "gnome-shell", filename = "/proc/self/stat" }
[00:24:50.801430938] (+0.000201252) 0 do_sys_open: { }, { pid = 1734, comm = "gnome-shell", filename = "/proc/self/stat" }
[00:24:50.807656245] (+0.006225307) 0 do_sys_open: { }, { pid = 18399, comm = "gnome-settings-", filename = "/proc/self/fd" }
..
..
```
Here, we collect trace data from `do_sys_open` and save it in CTF, then open the trace using `babeltrace` (which is a trace reader as well). This trace also works with Trace Compass. As we increase number of relevant trace events collected (`sched_*` etc.), Trace Compass would eventually be able to show something like [this](http://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/images/TimeAlignment_sash.png). This in PoC stage for now - the clocks and timings have to be improved. The deltas are ok, but wall clock time is not correct. We'll do a helper function in BCC itself that can help write these traces. Expect a PR soon :) If you have suggestions, leave them here!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the open2ctf.py proof of concept and its babeltrace output for do_sys_open events. Then examine how BCC collects map and sched_* trace data and how CTF timestamps are represented. Done means a defined BCC helper can write traces that babeltrace and Trace Compass read correctly, including improved clock and wall-time handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100