dfinity / dfinity/sdk

Pocket IC not handling well a devcontainer stopping

Open
#4,264 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-triage
Dominant language
Rust
Stars
204
Forks
98
No merged PRs in 30d

Description

Hi team,

Have made this video for Nathan, but perhaps it's better to share it here directly.

Think there is an edge case not yet covered by dfx or Pocket IC.

I created a video that showcases the error very well:
https://www.loom.com/share/88d792b1da82421589938794a726c7b0

Printscreen:
Image

And here is a repo with a devcontainer that you can easily reproduce. But the issue is on any devcontainer.
Reproducible Repo: https://github.com/tiagoicp/ic-siwe-react-demo-rust

Meta

dfx --version:

26.1 and 27.0
Further log output 

root@codespaces-4eb5c1:/workspaces/ic-siwe-react-demo-rust# dfx start --verbose
Running dfx start for version 0.27.0
There is no project-specific network 'local' defined in /workspaces/ic-siwe-react-demo-rust/dfx.json
Using the default configuration for the local shared network.
Local server configuration:
  bind address: 127.0.0.1:4943
  bitcoin: disabled
  canister http: enabled
    log level: Error
  replica:
    subnet type: Application
    log level: Error
  data directory: /root/.local/share/dfx/network/local
  scope: shared

Starting PocketIC...
Configuring PocketIC server
⠂ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
⠄ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:46283/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠁ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:40871/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠖ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:39345/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠒ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:40747/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠋ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:34001/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠂ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:42453/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠠ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:37767/instances): client error (SendRequest): connection closed before message completed
Starting PocketIC...
Configuring PocketIC server
⠉ Starting local network...                                                                                                                                                          
Stopping PocketIC...
⠤ Starting local network...                                                                                                                                                          thread 'tokio-runtime-worker' panicked at rs/state_machine_tests/src/lib.rs:2584:13:
Critical error mr_non_increasing_batch_time occurred.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at rs/pocket_ic_server/src/state_api/state.rs:756:14:
Failed to create PocketIC instance: JoinError::Panic(Id(21), "Critical error mr_non_increasing_batch_time occurred.", ...)
ERROR: Failed to initialize PocketIC: error sending request for url (http://localhost:41705/instances): client error (SendRequest): connection closed before message completed
Got signal to stop.
Stopped.
All local network processes stopped

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the linked devcontainer repository using dfx start --verbose, then start with rs/state_machine_tests/src/lib.rs and rs/pocket_ic_server/src/state_api/state.rs, which appear in the panic trace. Trace what happens when the devcontainer stops and restarts; done means the reported PocketIC initialization panic no longer occurs in that scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.