Simplify testing infra / create centralized Python script that orchestrates Turnt
- Dominant language
- Rust
- Stars
- 15
- Forks
- 0
- Avg merge
- 6h 20m
- Merged PRs (30d)
- 15
Description
Kevin mentioned he doesn't like our current snapshot testing setup, specifically:
1. How we have so many different file formats (`.prot`, `.tx`, `.rt`)
2. How we need to create duplicate versions of the same `.prot` file in order to supply arguments to Turnt (e.g. the same `.prot` file is present in both the interpreter & monitor subdirectories)
As we add more tests / larger benchmarks, it would be nice if our testing infra was simpler (to avoid duplicate versions of the same `.prot` file from being out of sync with each other). Kevin suggested having a centralized Python script that orchestrates Turnt. That is, for each `.prot` file, the Python test orchestrator figures the right inputs for Turnt for all the different styles of tests (monitor test, interpreter test, round-trip tests) and then actually invokes the right Turnt environment somehow.
Contributor guide
No contributing guide indexed for this repository
Research direction
No concrete file paths or tests are named. Start by inventorying the existing .prot, .tx, and .rt snapshot tests and their Turnt environments, then trace how interpreter, monitor, and round-trip tests are invoked. Done means one centralized Python orchestrator can select the needed inputs and run each test style without duplicated .prot files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100