`clock.rs:19:29: attempt to subtract with overflow`
- Dominant language
- Rust
- Stars
- 1.1k
- Forks
- 59
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 15
Description
Thanks for your work on shuttle. It helped us to identify (and fix) many multithreading issues in salsa.
I'm now trying to write a new test but I run into an immediate panic. I'm sorry for not having a better repro.
```rust
let mut db = DatabaseImpl::default();
let input = Input::new(&db, 1);
let result = query_b(&db, input);
// assert_eq!(result, MAX);
input.set_value(&mut db).to(2);
let db_t1 = db.clone();
// let db_t2 = db.clone();
let db_t3 = db;
let t1 = thread::spawn(move || query_a(&db_t1, input));
// let t2 = thread::spawn(move || query_b(&db_t2, input));
// let t3 = thread::spawn(move || query_c(&db_t3, input));
let r_t1 = t1.join().unwrap();
// let r_t2 = t2.join().unwrap();
// let r_t3 = t3.join().unwrap();
// assert_eq!((r_t1, r_t2, r_t3), (MAX, MAX, MAX));
```
This test passes with shuttle.
But shuttle panics as soon as I start `t2` or `t3`,
```rust
let mut db = DatabaseImpl::default();
let input = Input::new(&db, 1);
let result = query_b(&db, input);
// assert_eq!(result, MAX);
input.set_value(&mut db).to(2);
let db_t1 = db.clone();
// let db_t2 = db.clone();
let db_t3 = db;
let t1 = thread::spawn(move || query_a(&db_t1, input));
let t2 = thread::spawn(move || query_b(&db_t2, input));
// let t3 = thread::spawn(move || query_c(&db_t3, input));
let r_t1 = t1.join().unwrap();
let r_t2 = t2.join().unwrap();
// let r_t3 = t3.join().unwrap();
// assert_eq!((r_t1, r_t2, r_t3), (MAX, MAX, MAX));
```
But the test also passes if i comment out `query_b`. Now, `query_b` doesn't spawn any threads but we use thread local state, the current thread id and various locks.
```
thread 'cycle_nested_three_threads_changed::the_test' panicked at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/task/clock.rs:19:29:
attempt to subtract with overflow
stack backtrace:
0: rust_begin_unwind
at /rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:695:5
1: core::panicking::panic_fmt
at /rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/core/src/panicking.rs:75:14
2: core::panicking::panic_const::panic_const_sub_overflow
at /rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/core/src/panicking.rs:178:21
3: shuttle::runtime::task::clock::VectorClock::extend
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/task/clock.rs:19:29
4: shuttle::runtime::execution::ExecutionState::spawn_thread::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:428:13
5: shuttle::runtime::execution::ExecutionState::try_with::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:329:26
6: scoped_tls::ScopedKey::with
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/scoped-tls-1.0.1/src/lib.rs:171:13
7: shuttle::runtime::execution::ExecutionState::try_with
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:327:13
8: shuttle::runtime::execution::ExecutionState::with
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:316:9
9: shuttle::runtime::execution::ExecutionState::spawn_thread
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:415:23
10: shuttle::thread::spawn_named_unchecked
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/thread.rs:178:9
11: shuttle::thread::spawn_named
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/thread.rs:158:14
12: shuttle::thread::spawn
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/thread.rs:147:5
13: parallel::cycle_nested_three_threads_changed::the_test::{{closure}}
at ./tests/parallel/cycle_nested_three_threads_changed.rs:84:18
14: shuttle::runtime::runner::Runner::run::{{closure}}::{{closure}}::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/runner.rs:106:101
15: shuttle::thread::thread_fn
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/thread.rs:201:15
16: shuttle::runtime::execution::Execution::run::{{closure}}::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/execution.rs:83:34
17: core::ops::function::FnOnce::call_once{{vtable.shim}}
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
18: as core::ops::function::FnOnce>::call_once
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1976:9
19: shuttle::runtime::thread::continuation::Continuation::new::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/shuttle-0.8.1/src/runtime/thread/continuation.rs:87:21
20: generator::gen_impl::GeneratorImpl::init_code::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/gen_impl.rs:357:21
21: generator::stack::StackBox::call_once
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/stack/mod.rs:137:13
22: generator::stack::Func::call_once
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/stack/mod.rs:119:9
23: generator::detail::gen::gen_init_impl::{{closure}}
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/detail/gen.rs:35:9
24: core::ops::function::FnOnce::call_once
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
25: std::panicking::try::do_call
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:587:40
26: ___rust_try
27: std::panicking::try
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:550:19
28: std::panic::catch_unwind
at /Users/micha/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:358:14
29: generator::detail::gen::catch_unwind_filter
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/detail/gen.rs:23:5
30: generator::detail::gen::gen_init_impl
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/detail/gen.rs:50:25
31: generator::detail::asm::gen_init
at /Users/micha/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/generator-0.8.5/src/detail/aarch64_unix.rs:18:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
Contributor guide
Research direction
Start with runtime/task/clock.rs, especially VectorClock::extend, and trace its call from runtime/execution.rs in spawn_thread. Reproduce the panic with tests/parallel/cycle_nested_three_threads_changed.rs at the thread-spawn location shown in the trace. Done means the reported multi-thread test no longer panics with subtraction overflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100