stackablectl release install panics

Affected version

1.0.0

Current and expected behavior

When running druid tests, with stackablectl 1.0.0 installed, something deep down panics.

❯ ./scripts/run-tests --test oidc_druid-30.0.1_zookeeper-latest-3.9.3_s3-use-tls-false_openshift-false_oidc-use-tls-false --parallel 1 --skip-delete
INFO:root:Expanding test case id [smoke_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [authorizer_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_opa-1.4.2_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [ingestion-no-s3-ext_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [ingestion-s3-ext_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [s3-deep-storage_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-true_openshift-false]
INFO:root:Expanding test case id [s3-deep-storage_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-false_openshift-false]
INFO:root:Expanding test case id [hdfs-deep-storage_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [resources_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_openshift-false]
INFO:root:Expanding test case id [orphaned-resources_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [tls_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_tls-mode-no-tls_openshift-false]
INFO:root:Expanding test case id [tls_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_tls-mode-internal-and-server-tls_openshift-false]
INFO:root:Expanding test case id [tls_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_tls-mode-internal-and-server-tls-and-tls-client-auth_openshift-false]
INFO:root:Expanding test case id [ldap_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_opa-1.4.2_hadoop-latest-3.4.1_ldap-use-tls-true_ldap-no-bind-credentials-false_openshift-false]
INFO:root:Expanding test case id [ldap_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_opa-1.4.2_hadoop-latest-3.4.1_ldap-use-tls-false_ldap-no-bind-credentials-false_openshift-false]
INFO:root:Expanding test case id [logging_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-3.4.1_openshift-false]
INFO:root:Expanding test case id [cluster-operation_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-latest-3.4.1_openshift-false]
INFO:root:Expanding test case id [oidc_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-true_openshift-false_oidc-use-tls-true]
INFO:root:Expanding test case id [oidc_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-true_openshift-false_oidc-use-tls-false]
INFO:root:Expanding test case id [oidc_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-false_openshift-false_oidc-use-tls-true]
INFO:root:Expanding test case id [oidc_druid-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_s3-use-tls-false_openshift-false_oidc-use-tls-false]
INFO:root:Expanding test case id [overrides_druid-latest-30.0.1,oci.stackable.tech_sdp_druid_30.0.1-stackable25.3.0_zookeeper-latest-3.9.3_hadoop-latest-3.4.1_openshift-false]
ERROR:root:
thread 'tokio-runtime-worker' panicked at src/pb_manager.rs:160:17:
footer progress bar was hidden despite there being pending progress bars
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread 'tokio-runtime-worker' panicked at /tracing-indicatif-0.3.9/src/lib.rs:680:58:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }

thread 'tokio-runtime-worker' panicked at /tracing-indicatif-0.3.9/src/lib.rs:680:58:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }

  18:     0x55f6b332d959 - core::ptr::drop_in_place<tracing::span::Span>::hb64438d77f1747f3
                               at /rustc-1.86.0-src/library/core/src/ptr/mod.rs:523:1
  19:     0x55f6b0a13bf9 - stackable_cockpit::platform::operator::OperatorSpec::install::hdecf19b39a616bd9
                               at /source/src/platform/operator/mod.rs:189:7
  20:     0x55f6b04cd972 - stackable_cockpit::platform::release::spec::ReleaseSpec::install::{{closure}}::{{closure}}::{{closure}}::{{closure}}::h9a4b30713685e976
                               at /source/src/platform/release/spec.rs:120:25
  21:     0x55f6b07ad100 - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h49b075d97703494e
                               at /tracing-0.1.40/src/instrument.rs:321:9
  22:     0x55f6b06b4736 - tokio::runtime::task::core::Core<T,S>::poll::{{closure}}::h788b02bbb66b34bc

The command it appears to run is:

stackablectl release install --release-file tests/release.yaml tests

However, running that myself, it appears to succeed

So I think it might have something to do with getting a lock on stdout (maybe something to do with being executed from inside python)

It does work if I disable indicatif by manually setting the log-level.

diff --git i/scripts/run-tests w/scripts/run-tests
index 31b8e7a..6a64c65 100755
--- i/scripts/run-tests
+++ w/scripts/run-tests
@@ -286,4 +286,5 @@ def maybe_install_release(skip_release: bool, release_file: str) -> None:
         stackablectl_cmd = [
             "stackablectl",
+            "--log-level", "INFO",
             "release",
             "install",

Possible solution

No response

Additional context

No response

Environment

Nix environment (so technically, I'm using the version ahead of 1.0.0 in https://github.com/stackabletech/stackable-cockpit/pull/384)

Would you like to work on fixing this bug?

None

Reproduce the failure with ./scripts/run-tests and the stackablectl release install --release-file tests/release.yaml tests command, comparing default logging with --log-level INFO. Inspect src/pb_manager.rs:160, src/platform/operator/mod.rs:189, and src/platform/release/spec.rs:120 alongside scripts/run-tests. Done means the test-script invocation no longer emits the progress-bar or PoisonError panics.