beyond-all-reason / beyond-all-reason/RecoilEngine

Implement multi-platform CI sync testing

Open
#2,906 4 comments 0 reactions 0 assignees View on GitHub
area: devops/gitops area: tools enhancement
Dominant language
C++
Stars
679
Forks
290
Avg merge
3d 2h
Merged PRs (30d)
40

Description

### Task

Create a GitHub Actions CI workflow that runs engine sync tests on both arm64 and amd64 architectures, and compares the resulting checksums against each other and against historical "golden" runs to automatically detect sync regressions.

### Context

Currently, verifying that the engine remains deterministic across different platforms requires manual testing. As we look to introduce new compiler optimizations (like enabling SSE4) or refactor core simulation code, we need automated safeguards to ensure we don't introduce desyncs between players on different hardware.

The proposed solution is to run sync tests as part of the CI pipeline on both `amd64` and `arm64` runners. A dependent job will then compare the outputs to ensure cross-platform determinism. However, comparing architectures against each other isn't enough, as a bug could theoretically alter the simulation deterministically across all platforms at once. To catch this, the CI must also compare the results against historical or "golden" checksums from previous revisions. We can draw inspiration from the BAR game repository's integration tests, which write results to disk in one job and evaluate them in another, potentially storing a rolling history of recent runs in a structured format.

### Acceptance Criteria

* GitHub Actions workflow is updated to run sync tests in separate jobs for both `amd64` and `arm64` runners.
* A dependent CI job is created to collect and compare the checksum outputs from both architectures, failing the build if they diverge.
* Test results are saved in a structured format (e.g., JSON or CSV containing PR/commit hash, contributor, test results, and timestamp) to maintain a history of recent runs.
* The comparison job evaluates current checksums against historical/golden checksums to detect cross-revision sync breakages.
* The system includes a documented mechanism to update the golden checksums when intentional changes to the simulation or sync tests are made.

------
Author: Bruno D
Discord: https://discord.com/channels/549281623154229250/724924957074915358/1484690181482221638
Model: gemini-3-pro-preview

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing GitHub Actions workflows and engine sync tests, then review the BAR integration-test approach mentioned in the issue. Define how separate amd64 and arm64 jobs pass structured checksum results to a dependent comparison job. Done means cross-platform and historical checksum mismatches fail CI, with a documented process for intentional golden-checksum updates.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, github-actions
Domain
ci-cd, distributed-systems, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.