POST: purl2sym, extracting symbols from sources for fun and for profit

# Lightweight reverse-engineering of open source binaries with purl2sym

## TL;DR

- Got binary code? Got source code?
- Need to know what open source packages it came from?
- Use **purl2sym** to extract and then index symbols, like function names, and strings.
- Extract strings from the binary and match them back to actual source packages.

## Context

As an open source developer, I further rely on open source every day. But what happens when that code is shipped as a binary, like when embedded in devices (a car, a phone, or a toaster) or in container images where the original corresponding source code is not present?

You get a blob. Maybe a stripped ELF file. How can you figure out what is inside, what is it made off, what is the source, and where is the "open" source?

This is where **purl2sym** comes in. It is one of the AboutCode https://aboutcode.org projects, funded by NLnet https://nlnet.nl/project/purl2sym and it extracts and indexes symbols from source code so you can match these back to binary symbols of open source packages, keyed by PURL https://github.com/package-url/purl-spec. It can help turn otherwise mostly obscure and opaque symbols and strings found in binaries into traceable references to specific open source code packages and specific versions.

## Problems

If you have ever faced some of these use cases, then **purl2sym** could help:

- Analyzed a binary firmware blob to find which open source packages it embeds
- Tried to comply with FOSS license obligations post-compilation, to find what the actual code that is effectively compiled and shipped (as opposed to the 10x more code used in development and testing)
- Hunted down the source code of a vulnerability found in a binary
- Built SBOMs, working from compiled codebase

## Solution

**purl2sym** extract symbols from source code and builds indexes of symbols and strings extracted from multiple tech stacks and programing languages, like C, C++, Java, Python JavaScript, packaged in Linux distros like Debian or Alpine, or application packages like PyPI, Maven and more. You can fetch the symbols by PURL (Package-URL) and cross-reference these with binary symbols to get back matches to potential packages and versions indexed in the PrulDB.

## How it works

**purl2sym** is a combo of low level libraries, scripted pipelines and high level APIs. When you call the high level REST API, you request the symbols for a PURL, and the tool:

- Downloads and extracts the source package
- Extracts symbols and strings from the code using tools like `gettext`, `treesitter`, or `pygments`.
- Stores those symbols keyed by PURL (e.g., `pkg:deb/debian/openssl@1.1.1`).
- Returns a JSON doc with the symbols and metadata for further integration, alongside hashes and fingerprints.

With this, I can analyze unknown binary blobs by extracting their symbols and finding their original sources, using symbol matching. Or determine if the symbols of a code vulnerability may be present or used in a given source or binary. (Assuming we know which code segment is the trigger for a vulnerability, but this another thing, handled in another project)/

It is designed to work with and power binary analysis tools like BANG https://github.com/armijnhemel/binaryanalysis-ng/ that can then detect reused code chunks across binaries.

## In practice

Let’s say you are working with the firmware blob of a commercial device. It includes a statically linked C executable, but it is not obvious what libraries are linked in the binary. You could pull a few function names using `strings` (or commoncode's `string` functions) , or maybe extract a specific ELF section.

You can feed that into a tool powered by purl2sym. Voila: it will suggests that OpenSSL 1.1.1d from Debian is linked in the exe, and you can further lookup in the PurlDB for complete licensing metadata and in Vulnerable for CVEs. You now have a clear path towards automating a security and license compliance.

## Get Started

- Project homepage: [https://nlnet.nl/project/purl2sym](https://nlnet.nl/project/purl2sym/)
- Docs on symbol extraction: [Symbols and Strings in PURLdb](https://aboutcode.readthedocs.io/projects/PURLdb/en/latest/how-to-guides/symbols_and_strings.html)
- Install [ScanCode.io](https://scancodeio.readthedocs.io) to run locally
- Binary analysis with BANG: [https://github.com/nexB/bang](https://github.com/nexB/bang)