evstack / evstack/ev-rs

Quint formal verification for Evolve modules

Open
#9 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Summary

Integrate [Quint](https://quint-lang.org/) formal verification into the Evolve SDK to enable formally specified, model-checked, and conformance-tested modules. Quint specs would serve as the canonical source of truth for module behavior, with model-based testing (MBT) ensuring Rust implementations conform to their specifications.

## Background

Quint is an executable specification language built by Informal Systems, semantically based on TLA+ but with a modern typed syntax. It has a proven track record in blockchain projects:

- **Malachite** (Tendermint BFT in Rust) -- spec-first development, MBT conformance testing
- **ChonkyBFT** (zkSync consensus) -- model checking found missing validation bugs before deployment
- **Cosmos Interchain Security** -- TLA+ specs rewritten in Quint for MBT
- **Tendermint Light Client** -- 6 lines of TLA+ generated 500+ lines of integration test vectors
- **Neutron DEX** -- MBT caught a production bug before deployment
- **Alpenglow** (Solana) -- full Votor consensus specification

Quint's mode system (`pure def`, `action`, `temporal`) maps naturally to Evolve's account model (`#[query]`, `#[exec]`, invariants).

## Proposed Architecture

### Quint Specs as Module Specifications

Each Evolve module gets a corresponding `.qnt` spec that defines:
- **State variables** mapping to storage collections (`Item`, `Map`, `Vector`)
- **Actions** mapping to `#[exec]` methods
- **Pure defs** mapping to `#[query]` methods
- **Invariants** (safety properties verified by model checker)
- **Temporal properties** (liveness, fairness)

Example for a token module:

```quint
module Token {
type Addr = str

var balances: Addr -> int
var totalSupply: int
var allowances: (Addr, Addr) -> int

action init: bool = all {
balances' = Map(),
totalSupply' = 0,
allowances' = Map(),
}

action transfer(from: Addr, to: Addr, amount: int): bool = all {
require(amount > 0),
require(balances.getOrElse(from, 0) >= amount),
balances' = balances
.setBy(from, old => old - amount)
.setBy(to, old => old.getOrElse(0) + amount),
totalSupply' = totalSupply,
allowances' = allowances,
}

// Invariants
val noNegativeBalances: bool =
balances.keys().forall(a => balances.get(a) >= 0)

val supplyConserved: bool =
totalSupply == balances.keys().fold(0, (sum, a) => sum + balances.get(a))

// Step for simulation
action step: bool = {
nondet sender = ACCOUNTS.oneOf()
nondet receiver = ACCOUNTS.oneOf()
nondet amount = 1.to(MAX_AMOUNT).oneOf()
any {
transfer(sender, receiver, amount),
approve(sender, receiver, amount),
transferFrom(sender, receiver, sender, amount),
}
}
}
```

### MBT Pipeline

```
┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Quint Spec │────>│ quint run │────>│ ITF JSON Traces │
│ (.qnt files) │ │ --mbt │ │ │
└─────────────────┘ └──────────────┘ └────────┬────────┘

┌──────────────────────────────────────────────┘
v
┌─────────────────┐ ┌──────────────────┐
│ quint-connect / │────>│ Evolve TestApp │
│ itf-rs (Rust) │ │ conformance test │
└─────────────────┘ └──────────────────┘
```

1. **Spec** -- Write Quint specs alongside Rust modules in a `specs/` directory
2. **Verify** -- `quint verify` runs Apalache model checker against invariants
3. **Generate** -- `quint run --mbt --n-traces=N` produces ITF trace files
4. **Conform** -- Rust test driver (using `quint-connect` crate) replays traces against `TestApp`, asserting implementation state matches spec state after each step

### Integration with Evolve Test Infrastructure

The conformance test driver would implement `quint-connect`'s `Driver` trait, mapping:

| Quint Action | Evolve Operation |
|---|---|
| `init` | Genesis setup via `TestApp::new()` |
| `transfer(from, to, amount)` | Submit signed `Transfer` tx to STF |
| `approve(owner, spender, amount)` | Submit signed `Approve` tx to STF |
| State assertions | Query STF state, compare to spec snapshot |

### Repository Structure

```
/specs/
├── README.md # How to write and run Quint specs
├── token.qnt # Token module spec
├── scheduler.qnt # Scheduler module spec
├── stf.qnt # STF-level properties (block validity, tx ordering)
└── common.qnt # Shared types, helpers

/crates/testing/
├── quint-mbt/ # New crate: Quint MBT test driver for Evolve
│ ├── Cargo.toml # deps: quint-connect, itf-rs, evolve-testing
│ └── src/
│ ├── lib.rs # Driver trait impl, state mapping
│ └── drivers/ # Per-module drivers
└── ...
```

## Tasks

- [ ] Add `specs/` directory with initial Quint spec for the token module (`x/token`)
- [ ] Set up `quint` CLI in CI (npm package, or pin binary)
- [ ] Write Quint spec for `x/token` matching current Rust implementation
- [ ] Model check token spec invariants with `quint verify`
- [ ] Create `quint-mbt` testing crate with `quint-connect` integration
- [ ] Implement `Driver` trait mapping ITF actions to `TestApp` operations
- [ ] Generate trace suite and add to `cargo test` pipeline
- [ ] Add `just quint-verify` and `just quint-test` commands to justfile
- [ ] Write Quint spec for STF-level properties (block building, tx validation)
- [ ] Document spec-writing patterns for Evolve module authors

## Key Dependencies

- [quint](https://github.com/informalsystems/quint) (npm / standalone binary) -- spec language + simulator
- [quint-connect](https://quint-lang.org/docs/model-based-testing) (Rust crate) -- MBT framework
- [itf-rs](https://github.com/informalsystems/itf-rs) (Rust crate) -- ITF trace deserialization
- [Apalache](https://github.com/apalache-mc/apalache) (JVM) -- symbolic model checker (auto-downloaded by `quint verify`)

## Open Questions

1. **Scope**: Start with `x/token` only, or also spec the STF and mempool?
2. **CI integration**: Run `quint verify` on every PR (slow, JVM-based) or only on spec changes?
3. **Trace caching**: Pre-generate and commit ITF traces, or generate fresh in CI?
4. **LLM-assisted spec writing**: Use [quint-llm-kit](https://github.com/informalsystems/quint-llm-kit) to bootstrap specs from existing Rust code?
5. **Code generation direction**: Long-term, explore building a Quint-to-Evolve codegen (no prior art exists, significant effort) or stay with the MBT conformance pattern?

## References

- [Quint Language Docs](https://quint-lang.org/docs/lang)
- [Quint Model-Based Testing](https://quint-lang.org/docs/model-based-testing)
- [Quint Connect + Emerald Case Study](https://quint-lang.org/posts/quint_connect_emerald)
- [ChonkyBFT Specification](https://protocols-made-fun.com/consensus/matterlabs/quint/specification/modelchecking/2024/07/29/chonkybft.html)
- [Malachite Consensus Engine](https://github.com/informalsystems/malachite)
- [RVCase25 Paper (Quint for Design and Testing)](https://seanmk.com/rvcase/RVCase25_paper_5.pdf)
- [cosmwasm-to-quint (reverse: Rust -> Quint stubs)](https://github.com/informalsystems/cosmwasm-to-quint)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.