cloudflare / cloudflare/workerd
workerd: macOS symbolication does not work
- Dominant language
- C++
- Stars
- 8.7k
- Forks
- 739
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 174
Description
Symbolication with `llvm-symbolizer` was enabled for the workerd github runners in `test.yml` in #1247.
Shortly after, it was suspended for macOS in commit ad20f5e039d34 in PR #1283 when additional build configurations were added because it leads to test timeouts. The symbol search process appears to be slow / misconfigured and the CPU and storage capabilities of the free-tier github runners are limited.
We've had issues with macOS, symbols for LLVM, and bazel build. [workerd explicitly sets a source-map for debugging](https://github.com/cloudflare/workerd/blob/7ec997844a83aec4980dbd1da404f8185b8e86d8/.vscode/launch.json#L61-L65). The problem in this ticket is likely related.
I've spent a day or two looking into this, but have not cracked it. This ticket is just to track that we have a problem here. The next step is to collect fs activity from `llvm-symbolizer`, macOS is very different from Linux / Windows; procmon on windows is king :-). For macOS, it looks like it needs System Integrity Protection turning off and then finding a dtrace based tool for the fs activity ([opensnoop](https://ss64.com/osx/opensnoop.html) or some-such).
It would also be good to fix this as a reasonable proportion of workerd users have macOS machines and they have no chance of reporting stack traces even if they want to.
A couple of related observations:
1) macOS has it's own `backtrace_symbols` function. This might be more expedient to use since it should give meaningful stacks without needing to run an additional tool.
2) `llvm-symbolizer` has a `--filter-markup` option that consumes [symbolizer markup](https://llvm.org/docs/SymbolizerMarkupFormat.html). This probably post-dates the code used in workerd (capnproto) and might make things marginally cleaner.
Contributor guide
Assessment
This issue has not been assessed yet.