Cubestore crashes with SIGILL on CPUs without AVX2 support

Open
#11,941 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
docker, rust

Research direction

Start with rust/cubestore/Dockerfile, especially the WITH_AVX2 build argument and RUSTFLAGS used by the cargo build for cubestore. Reproduce the default image startup failure on a non-AVX2 host, then compare it with a build using WITH_AVX2=0. Done means the official image no longer unconditionally requires AVX2, while retaining the documented optimization behavior where supported.

Written by the indexing model from the issue text.

Description

Describe the bug

The official cubejs/cubestore Docker image is compiled with WITH_AVX2=1, which forces RUSTFLAGS="-C target-feature=+avx2" at build time. As a result, the cubestored binary contains AVX2 instructions unconditionally and crashes immediately with SIGILL (Illegal instruction) on any CPU that does not support AVX2. In containerized deployments this causes an infinite restart loop, and a single non-AVX2 host in an otherwise modern cluster is enough to break the whole deployment.

To Reproduce

Take a host (bare metal, VM, or cloud instance) whose CPU does not expose the AVX2 feature to the guest. This can be an older Intel/AMD CPU, or a virtualized/shared instance type where the hypervisor masks AVX2.

Run the official image without overriding the build arg:

bash
docker run --rm cubejs/cubestore:latest
(or build it yourself with the default Dockerfile, which sets ARG WITH_AVX2=1)

Observe that cubestored exits immediately.

If the container is managed by an orchestrator (Docker Compose, Kubernetes, Swarm, etc.), it will be restarted and crash again in a loop.

Expected behavior

Either of the following would be acceptable:

The binary should run on any x86-64 CPU and only use AVX2 when it is actually available, or

The AVX2 requirement should be opt-in rather than the default, and clearly documented so users on older hardware can avoid the crash without debugging a SIGILL.

Ideally the image would ship a single binary that performs runtime CPU feature detection (is_x86_feature_detected!("avx2")) and dispatches to an AVX2-optimized path only when supported, so the same image works across a heterogeneous fleet.

Screenshots

N/A — the failure is a crash on startup; the container logs show something equivalent to:

text
Illegal instruction (core dumped)
or an exit code of 132 (128 + SIGILL).

Minimally reproducible Cube Schema

Not applicable — this is a build/runtime issue in cubestored, not a data modelling problem.

Version:

[1.6.7] and current master

Additional context

The relevant part of rust/cubestore/Dockerfile is:

dockerfile
ARG WITH_AVX2=1
ARG TARGETARCH
RUN [ "$WITH_AVX2" -eq "1" ] && export RUSTFLAGS="-C target-feature=+avx2";
[ "$TARGETARCH" = "arm64" ] && export JEMALLOC_SYS_WITH_LG_PAGE="16";
cargo build --release -p cubestore
Because WITH_AVX2 defaults to 1, users on older CPUs must know to rebuild with:

bash
docker build --build-arg WITH_AVX2=0 -t cubestore-compat .
which also disables the optimization on machines that do support AVX2. It would be preferable to keep the optimization while remaining portable, e.g. by using runtime feature detection. The multiversion crate (#[multiversion] + #[clone(target = "x86_64+avx2")]) is a well-established way to achieve this in Rust without restructuring the codebase.

This is especially relevant for self-hosted and on-prem deployments, where hardware is often mixed-generation and users do not control which CPU a container lands on.

Dominant language
Rust
Stars
20.9k
Forks
2.1k
Avg merge
1d 3h
Merged PRs (30d)
201

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cube-js/cube

All issues in cube-js/cube

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.