Packaging issues

# Packaging issues

I just updated the nixos retdec package from version 3.0 to 3.2. That was not a pleasant experience, which is probably why it remained outdated for so long. I don't know if distribution packaging is a priority for this project, but in case it is I will outline the problems here.

## Retdec-support is huge

Only 127M compressed but 4G uncompressed. That takes up space on our mirror server but, more importantly, on every users computer. We currently remove the PE static code patterns (3G) by default and only install these when explicitly requested by the user. I've opened https://github.com/avast-tl/retdec-support/issues/3 for that.

## Dependencies are fetched at build-time

This is by far the biggest issue. NixOS and many other distros separate the build in two distinct phases: one to fetch the sources, one to build. The build phase is executed in a sandbox that does not have internet access. There are various reasons for that, including:

- reproducibility: If the build process could be influenced by the availability and content of files on the internet, there is no way to guarantee two different builds will yield the same result of even succeed at all.
- security: Some of these files could be replaced by malicious ones.
- mirroring: We are able to mirror all the source files in a content-addressable manner on our build servers.

Since the retdec build process fetches its dependencies at build time, a lot of patching is required. We have to fetch all the dependencies ourselves and then patch every CMakeLists file to use those local versions instead. Since that essentially duplicates parts of the build system, it makes a lot of manual adjustments during upgrades necessary.

I've started discussing this at https://github.com/avast-tl/retdec/issues/279.

## Dependency versions are very specific

All dependencies are fetched by commit-hash. That makes it very difficult for distros since they usually try to ship one version of every package and share those dependencies. It is generally impossible to judge the true intentions behind the build hash:
Is it "we need at least this version since we depend on feature x" or "newer versions break the api" or "we need exactly version 2.3"?
This and the lack of a central file that collects the necessary dependency versions makes it painful to adjust the dependency versions on an upgrade.

## Dependencies are forked

Capstone and yara are forked. Can the changes not be upstreamed?

## Dependencies are built from source

While a distro would ideally build all the dependencies themselves and then build retdec based on those pre-compiled dependencies, retdec instead insists on building everything itself. The exception to this is openssl, which is only built if it cannot be found on the system. That should be the case for all dependencies.

The issue names the NixOS RetDec package and the CMakeLists files that fetch and build dependencies, but no specific source file or test. Start by reviewing the dependency-fetching sections across the CMakeLists files and the NixOS packaging changes described here. Done would require an agreed approach covering offline builds, dependency versions, forks, and system dependencies.

Written by the indexing model from the issue text.