element-hq / element-hq/synapse

`ruff` is currently built from source in the nix flake, slowing down dev environment creation and requiring a moving rust version

Open
#15,939 3 comments 0 reactions 0 assignees View on GitHub
O-Uncommon T-Other Z-Dev-Wishlist Z-Upstream-Bug
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15939](https://github.com/matrix-org/synapse/issues/15939).

---

We have a [`flake.nix`](https://github.com/matrix-org/synapse/blob/20ae617d1417f8dd52e20b3a20cb01b4c2fd87c9/flake.nix) file which can be used to create a full development environment for Synapse, including python and native dependencies and things like postgres and redis.

The development environment will automatically keep its python environment up to date with `poetry` using the versions defined in `pyproject.yaml`. One of those python package is [`ruff`](https://github.com/charliermarsh/ruff), which we use as a linter.

`ruff` ships a rust binary, which is dynamically linked against either glibc or musl. As such, `ruff` expects `glibc` or `muslc` headers to exist at a certain location on your Linux box. Due to [the way NixOS works](https://nixos.wiki/wiki/Packaging/Binaries#The_Dynamic_Loader), these do not exist under the expected path, causing any `ruff` binaries we download from PyPI to fail to execute.

Thus, we have the following workaround in `flake.nix`, which forces `ruff` to be compiled on the local system:

https://github.com/matrix-org/synapse/blob/20ae617d1417f8dd52e20b3a20cb01b4c2fd87c9/flake.nix#L115-L122

This works, but is slow (as you need to compile a rust binary) and forces the development environment to maintain a rust version which is high enough to compile `ruff` (which unfortunately is almost always the latest stable. I found today that while we only require rust `1.60.0` in Synapse, the version of `ruff` we have specified in our `pyproject.toml` requires rust `1.70.0`.

The ideal solution here is for `ruff` to provide statically linked binaries to download. This has been discussed in https://github.com/astral-sh/ruff/issues/1699, and they are up for it if it does not compromise performance (or alternatively they could provide a `ruff-static` PyPI package.

Until then, we can't do much on this side.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.