awslabs / awslabs/shuttle

Testing panic handling

Open
#193 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.1k
Forks
59
Avg merge
3d 20h
Merged PRs (30d)
17

Description

Shuttle has been game-changing for us! I spent days trying to construct the exact case where some code would fail in production, only to show that my fix works. That's gone with shuttle :)

The only case that we currently can't test with shuttle is panic handling. Specifically, we want to test that if a thread A waits on a result of thread B and b panics, that A doesn't wait forever. However, this can't be tested with the shuttle today, at least to my knowledge, because the shuttle always thinks that the artificial panic is an error case and that it found a bug.

It would be very nice if shuttle provided a way to test this too. Maybe by having a special sentinel value that a test could throw that shuttle knows to pass through (and not abort with a test failure).

Our non-shuttle test that we would like to shuttle :)

```rust
#[salsa::tracked]
fn query_a(db: &dyn KnobsDatabase) -> u32 {
query_b(db)
}

#[salsa::tracked]
fn query_b(db: &dyn KnobsDatabase) -> u32 {
panic!("cancel!")
}

#[test]
fn execute() {
let db = Knobs::default();

let db_t1 = db.clone();
let t1 = std::thread::spawn(move || query_a(&db_t1));

let db_t2 = db.clone();
let t2 = std::thread::spawn(move || query_b(&db_t2));

// The main thing here is that we don't deadlock.
let (r1, r2) = (t1.join(), t2.join());
assert!(r1.is_err());
assert!(r2.is_err());
}
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the supplied Rust example with Shuttle, then inspect the existing panic-handling path that treats artificial panics as test failures. Done means a panic in thread B can be observed by thread A without a deadlock, and both joins report errors as shown in the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.