EFForg / EFForg/rayhunter

[Feature Request]: A hardware-based CI testing rig for all supported devices

Open
#261 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
5.8k
Forks
490
Avg merge
3d 10h
Merged PRs (30d)
23

Description

### What problem does this feature solve or what does it enhance?

There is no automated testing process right now because it's embedded and there are no emulators. So every new change to rayhunter core code technically needs to be tested manually on each supported device right now.

The lack of proper CI also creates a strong incentive for developers who want to hack on one device to purchase each other support device so they have a working and complete testing environment. While working on #226, I had to strongly consider its impact on orbic. Right now I am asking folks on EFF Mattermost to manually test a particular patch against regressions on the orbic, but I've ordered an orbic myself to the EU just to be able to test my display changes without relying on anybody else.

### Proposed Solution

I'm considering this setup:

* There is a physical server running somewhere, with N devices attached via USB and ADB installed.
* It exposes itself to GitHub actions via SSH
* One test (for now): SSH into the server, run the installer, run the binary for 10-20 seconds and assert there are no errors or crashes

Some random notes:

* A GitHub actions workflow can declare that only one instance of it can run at a time -- we should probably enable that
* devices need to be factory reset between runs, not sure how to deal with that (maybe tear the casing apart and control the reset button via a separate GPIO thing)
* as little code as possible should live on the physical server -- ideally just SSH and ADB
* I don't expect EFF core devs to maintain this entire rig. Those physical servers are community-run/donated, and the availability of one could become a requirement for supporting new devices.

### Alternatives Considered

Keep the status quo. Core developers & contributors start hoarding devices, and as a result have to ship them around the world in order to test anything.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.