[ICE]: Found unstable fingerprints for evaluate_obligation(..): Ok(EvaluatedToOk)

I encountered this while working on crates.io. It was reproducible (cargo build failed repeatedly), but in the process of trying to minimize the diff it stopped happening and I haven't been able to get it to come back.

Meta

rustc --version --verbose:

rustc 1.95.0 (59807616e 2026-04-14)
binary: rustc
commit-hash: 59807616e1fa2540724bfbac14d7976d7e4a3860
commit-date: 2026-04-14
host: aarch64-apple-darwin
release: 1.95.0
LLVM version: 22.1.2

Error output

error: internal compiler error: encountered incremental compilation error with evaluate_obligation(b158b1b37df2bdc9-349fa56d7820bc31)
  |
  = note: please follow the instructions below to create a bug report with the provided information
  = note: for incremental compilation bugs, having a reproduction is vital
  = note: an ideal reproduction consists of the code before and some patch that then triggers the bug when applied and compiled again
  = note: as a workaround, you can run `cargo clean -p crates_io` or `cargo clean` to allow your project to compile


thread 'rustc' (29389146) panicked at /rustc-dev/59807616e1fa2540724bfbac14d7976d7e4a3860/compiler/rustc_middle/src/verify_ich.rs:80:9:
Found unstable fingerprints for evaluate_obligation(b158b1b37df2bdc9-349fa56d7820bc31): Ok(EvaluatedToOk)
stack backtrace:
   0:        0x111795dc0 - <<std[bb513d90a5cee88a]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[f63e075e1375d836]::fmt::Display>::fmt
   1:        0x10e9b6c28 - core[f63e075e1375d836]::fmt::write
   2:        0x1117ad5b0 - <std[bb513d90a5cee88a]::sys::stdio::unix::Stderr as std[bb513d90a5cee88a]::io::Write>::write_fmt
   3:        0x11176c48c - std[bb513d90a5cee88a]::panicking::default_hook::{closure#0}
   4:        0x1117882a8 - std[bb513d90a5cee88a]::panicking::default_hook
   5:        0x10f614af0 - std[bb513d90a5cee88a]::panicking::update_hook::<alloc[1519c0c6343f008f]::boxed::Box<rustc_driver_impl[5c335eb20a4751f4]::install_ice_hook::{closure#1}>>::{closure#0}
   6:        0x11178860c - std[bb513d90a5cee88a]::panicking::panic_with_hook
   7:        0x11176c534 - std[bb513d90a5cee88a]::panicking::panic_handler::{closure#0}
   8:        0x111762ed8 - std[bb513d90a5cee88a]::sys::backtrace::__rust_end_short_backtrace::<std[bb513d90a5cee88a]::panicking::panic_handler::{closure#0}, !>
   9:        0x11176db04 - __rustc[b7974e8690430dd9]::rust_begin_unwind
  10:        0x1147a95b8 - core[f63e075e1375d836]::panicking::panic_fmt
  11:        0x114882c74 - rustc_middle[9aaee1f0dbfe40d5]::verify_ich::incremental_verify_ich_failed
  12:        0x110c8b790 - rustc_middle[9aaee1f0dbfe40d5]::verify_ich::incremental_verify_ich::<rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 2usize]>>
  13:        0x110c20208 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_middle[9aaee1f0dbfe40d5]::query::caches::DefaultCache<rustc_type_ir[8d0b5b963df9ae4c]::canonical::CanonicalQueryInput<rustc_middle[9aaee1f0dbfe40d5]::ty::context::TyCtxt, rustc_middle[9aaee1f0dbfe40d5]::ty::ParamEnvAnd<rustc_middle[9aaee1f0dbfe40d5]::ty::predicate::Predicate>>, rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 2usize]>>, true>
  14:        0x110da1c50 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::evaluate_obligation::execute_query_incr::__rust_end_short_backtrace
  15:        0x1114cd744 - <rustc_infer[8e5ac288d7b05e57]::infer::InferCtxt as rustc_trait_selection[2a80b881ce73482c]::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation
  16:        0x1114cdb88 - <rustc_infer[8e5ac288d7b05e57]::infer::InferCtxt as rustc_trait_selection[2a80b881ce73482c]::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation_no_overflow
  17:        0x111445f38 - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor>::process_trait_obligation
  18:        0x1114e64ac - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor as rustc_data_structures[66f2c8724ec626e5]::obligation_forest::ObligationProcessor>::process_obligation
  19:        0x1112bbde4 - <rustc_data_structures[66f2c8724ec626e5]::obligation_forest::ObligationForest<rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::PendingPredicateObligation>>::process_obligations::<rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor>
  20:        0x1114d9180 - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillmentContext<rustc_infer[8e5ac288d7b05e57]::traits::engine::ScrubbedTraitError> as rustc_infer[8e5ac288d7b05e57]::traits::engine::TraitEngine<rustc_infer[8e5ac288d7b05e57]::traits::engine::ScrubbedTraitError>>::try_evaluate_obligations
  21:        0x11156d3cc - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillmentContext<rustc_infer[8e5ac288d7b05e57]::traits::engine::ScrubbedTraitError> as rustc_infer[8e5ac288d7b05e57]::traits::engine::TraitEngine<rustc_infer[8e5ac288d7b05e57]::traits::engine::ScrubbedTraitError>>::evaluate_obligations_error_on_ambiguity
  22:        0x1115cec64 - rustc_traits[894696da730b8b48]::coroutine_witnesses::compute_assumptions::{closure#0}
  23:        0x1115ed958 - rustc_traits[894696da730b8b48]::coroutine_witnesses::coroutine_hidden_types
  24:        0x110da8eac - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::coroutine_hidden_types::invoke_provider_fn::__rust_begin_short_backtrace
  25:        0x110bc91f4 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_middle[9aaee1f0dbfe40d5]::query::caches::DefIdCache<rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 24usize]>>, true>
  26:        0x110da8e0c - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::coroutine_hidden_types::execute_query_incr::__rust_end_short_backtrace
  27:        0x1113bc804 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::confirm_auto_impl_candidate::{closure#0}
  28:        0x111407fd8 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::confirm_candidate
  29:        0x1113b4968 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_candidate::{closure#0}::{closure#0}
  30:        0x1112cf0a8 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluation_probe::<<rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_candidate::{closure#0}::{closure#0}>
  31:        0x1113b5f9c - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_trait_predicate_recursively::{closure#0}::{closure#1}
  32:        0x111457934 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_trait_predicate_recursively
  33:        0x1113b5010 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_predicate_recursively::{closure#0}::{closure#0}
  34:        0x111454d60 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_predicate_recursively
  35:        0x1113aae18 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluation_probe::<<rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_root_obligation::{closure#0}>::{closure#0}
  36:        0x1114522b8 - <rustc_trait_selection[2a80b881ce73482c]::traits::select::SelectionContext>::evaluate_root_obligation
  37:        0x1115ee268 - rustc_traits[894696da730b8b48]::evaluate_obligation::evaluate_obligation
  38:        0x110da1cc4 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::evaluate_obligation::invoke_provider_fn::__rust_begin_short_backtrace
  39:        0x110c201d4 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_middle[9aaee1f0dbfe40d5]::query::caches::DefaultCache<rustc_type_ir[8d0b5b963df9ae4c]::canonical::CanonicalQueryInput<rustc_middle[9aaee1f0dbfe40d5]::ty::context::TyCtxt, rustc_middle[9aaee1f0dbfe40d5]::ty::ParamEnvAnd<rustc_middle[9aaee1f0dbfe40d5]::ty::predicate::Predicate>>, rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 2usize]>>, true>
  40:        0x110da1c50 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::evaluate_obligation::execute_query_incr::__rust_end_short_backtrace
  41:        0x1114cd744 - <rustc_infer[8e5ac288d7b05e57]::infer::InferCtxt as rustc_trait_selection[2a80b881ce73482c]::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation
  42:        0x1114cdb88 - <rustc_infer[8e5ac288d7b05e57]::infer::InferCtxt as rustc_trait_selection[2a80b881ce73482c]::traits::query::evaluate_obligation::InferCtxtExt>::evaluate_obligation_no_overflow
  43:        0x111445f38 - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor>::process_trait_obligation
  44:        0x1114e64ac - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor as rustc_data_structures[66f2c8724ec626e5]::obligation_forest::ObligationProcessor>::process_obligation
  45:        0x1112bbde4 - <rustc_data_structures[66f2c8724ec626e5]::obligation_forest::ObligationForest<rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::PendingPredicateObligation>>::process_obligations::<rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillProcessor>
  46:        0x1114d89ac - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillmentContext<rustc_trait_selection[2a80b881ce73482c]::traits::FulfillmentError> as rustc_infer[8e5ac288d7b05e57]::traits::engine::TraitEngine<rustc_trait_selection[2a80b881ce73482c]::traits::FulfillmentError>>::try_evaluate_obligations
  47:        0x11156d33c - <rustc_trait_selection[2a80b881ce73482c]::traits::fulfill::FulfillmentContext<rustc_trait_selection[2a80b881ce73482c]::traits::FulfillmentError> as rustc_infer[8e5ac288d7b05e57]::traits::engine::TraitEngine<rustc_trait_selection[2a80b881ce73482c]::traits::FulfillmentError>>::evaluate_obligations_error_on_ambiguity
  48:        0x10f9a4f64 - rustc_hir_analysis[3f6a434a01ebfd9b]::check::check::check_coroutine_obligations
  49:        0x110db6524 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::check_coroutine_obligations::invoke_provider_fn::__rust_begin_short_backtrace
  50:        0x110bae35c - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_data_structures[66f2c8724ec626e5]::vec_cache::VecCache<rustc_span[c6548ed10a46efff]::def_id::LocalDefId, rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 1usize]>, rustc_middle[9aaee1f0dbfe40d5]::dep_graph::graph::DepNodeIndex>, true>
  51:        0x110db64b4 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::check_coroutine_obligations::execute_query_incr::__rust_end_short_backtrace
  52:        0x10ef57918 - rustc_borrowck[2268b010a67ebd48]::mir_borrowck
  53:        0x110bb5a50 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_data_structures[66f2c8724ec626e5]::vec_cache::VecCache<rustc_span[c6548ed10a46efff]::def_id::LocalDefId, rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 8usize]>, rustc_middle[9aaee1f0dbfe40d5]::dep_graph::graph::DepNodeIndex>, true>
  54:        0x110d8a0a4 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::mir_borrowck::execute_query_incr::__rust_end_short_backtrace
  55:        0x1106806e0 - rustc_mir_transform[d7c6d90316094693]::mir_drops_elaborated_and_const_checked
  56:        0x110bb5ea0 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_data_structures[66f2c8724ec626e5]::vec_cache::VecCache<rustc_span[c6548ed10a46efff]::def_id::LocalDefId, rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 8usize]>, rustc_middle[9aaee1f0dbfe40d5]::dep_graph::graph::DepNodeIndex>, true>
  57:        0x110dbcf08 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::mir_drops_elaborated_and_const_checked::execute_query_incr::__rust_end_short_backtrace
  58:        0x10ff03368 - <rustc_middle[9aaee1f0dbfe40d5]::ty::context::TyCtxt>::par_hir_body_owners::<rustc_interface[c32df345b14bfa6e]::passes::run_required_analyses::{closure#1}::{closure#0}>::{closure#0}
  59:        0x10ff44f80 - rustc_interface[c32df345b14bfa6e]::passes::analysis
  60:        0x110bdff18 - rustc_query_impl[1f4f3f960cb79e6b]::execution::try_execute_query::<rustc_middle[9aaee1f0dbfe40d5]::query::caches::SingleCache<rustc_middle[9aaee1f0dbfe40d5]::query::erase::ErasedData<[u8; 0usize]>>, true>
  61:        0x110dc0d68 - rustc_query_impl[1f4f3f960cb79e6b]::query_impl::analysis::execute_query_incr::__rust_end_short_backtrace
  62:        0x10f5cea68 - rustc_interface[c32df345b14bfa6e]::passes::create_and_enter_global_ctxt::<core[f63e075e1375d836]::option::Option<rustc_interface[c32df345b14bfa6e]::queries::Linker>, rustc_driver_impl[5c335eb20a4751f4]::run_compiler::{closure#0}::{closure#2}>
  63:        0x10f618958 - rustc_interface[c32df345b14bfa6e]::interface::run_compiler::<(), rustc_driver_impl[5c335eb20a4751f4]::run_compiler::{closure#0}>::{closure#1}
  64:        0x10f60e6f0 - std[bb513d90a5cee88a]::sys::backtrace::__rust_begin_short_backtrace::<rustc_interface[c32df345b14bfa6e]::util::run_in_thread_with_globals<rustc_interface[c32df345b14bfa6e]::util::run_in_thread_pool_with_globals<rustc_interface[c32df345b14bfa6e]::interface::run_compiler<(), rustc_driver_impl[5c335eb20a4751f4]::run_compiler::{closure#0}>::{closure#1}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>
  65:        0x10f61e764 - <std[bb513d90a5cee88a]::thread::lifecycle::spawn_unchecked<rustc_interface[c32df345b14bfa6e]::util::run_in_thread_with_globals<rustc_interface[c32df345b14bfa6e]::util::run_in_thread_pool_with_globals<rustc_interface[c32df345b14bfa6e]::interface::run_compiler<(), rustc_driver_impl[5c335eb20a4751f4]::run_compiler::{closure#0}>::{closure#1}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>::{closure#1} as core[f63e075e1375d836]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  66:        0x11179333c - <std[bb513d90a5cee88a]::sys::thread::unix::Thread>::new::thread_start
  67:        0x198ad7c08 - __pthread_cond_wait

error: the compiler unexpectedly panicked. This is a bug

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.95.0 (59807616e 2026-04-14) running on aarch64-apple-darwin

note: compiler flags: --crate-type lib -C embed-bitcode=no -C debuginfo=2 -C split-debuginfo=unpacked -C incremental=[REDACTED]

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [evaluate_obligation] evaluating trait selection obligation `futures_util::fns::MapErrFn<{async closure@src/controllers/krate/publish.rs:671:50: 671:63}>: futures_util::fns::FnOnce1<core::result::Result<core::option::Option<i64>, crates_io_worker::errors::EnqueueError>>`
#1 [coroutine_hidden_types] looking up the hidden types stored across await points in a coroutine
... and 5 other queries... use `env RUST_BACKTRACE=1` to see the full query stack
error: could not compile `crates_io` (lib) due to 1 previous error

Start with compiler/rustc_middle/src/verify_ich.rs:80 and the evaluate_obligation query shown in the backtrace. Re-run cargo build on the crates.io workload with incremental compilation, using cargo clean -p crates_io only as the reported workaround. Done means obtaining a reproducible case and isolating the change or regression responsible for the unstable fingerprint.