Reached the recursion limit while instantiating (closure)

I tried this code:

https://github.com/andrewbaxter/hammer-of-json/blob/48f37984ee726df6d2f7370f724d4aee9945d1c0/source/src/utils.rs#L175

pub fn search(
    root: bool,
    at: &mut serde_json::Value,
    needle: &serde_json::Value,
    handle_end_found_obj: &mut impl FnMut() -> SearchRes,
    handle_end_found_arr: &mut impl FnMut() -> SearchRes,
    handle_end_root: impl FnOnce() -> SearchRes,
) {
    if root && at == needle {
        match handle_end_root() {
            SearchRes::Replace(value) => *at = value,
            SearchRes::Delete => *at = serde_json::Value::Null,
        }
    } else {
        match &mut *at {
            serde_json::Value::Array(values) => {
                let mut i = 0;
                while i < values.len() {
                    if values[i] == *needle {
                        match handle_end_found_arr() {
                            SearchRes::Delete => {
                                values.remove(i);
                            },
                            SearchRes::Replace(v) => {
                                values[i] = v;
                                i += 1;
                            },
                        }
                    } else {
                        search(
                            false,
                            &mut values[i],
                            &*needle,
                            &mut *handle_end_found_obj,
                            &mut *handle_end_found_arr,
                            || unreachable!(),
                        );
                        i += 1;
                    }
                }
            },
            serde_json::Value::Object(map) => {
                for k in map.keys().cloned().collect::<Vec<_>>() {
                    if map[&k] == *needle {
                        match handle_end_found_obj() {
                            SearchRes::Replace(value) => {
                                map.insert(k, value);
                            },
                            SearchRes::Delete => {
                                map.remove(&k);
                            },
                        }
                    } else {
                        search(
                            false,
                            &mut map[&k],
                            &*needle,
                            &mut *handle_end_found_obj,
                            &mut *handle_end_found_arr,
                            || unreachable!(),
                        );
                    }
                }
            },
            _ => { },
        }
    }
}

I expected to see this happen: Compile success

Instead, this happened: Compile fails with the error:

   Compiling hammer-of-json v0.1.0 (/mnt/home-dev/r/jsonhammer/source)
error: reached the recursion limit while instantiating `search::<{closure@src/search_delete.rs:9:39: 9:41}, {closure@src/search_delete.rs:9:66: 9:68}, {closure@src/utils.rs:210:29: 210:31}>`
   --> src/utils.rs:204:25
    |
204 | /                         search(
205 | |                             false,
206 | |                             &mut values[i],
207 | |                             &*needle,
...   |
210 | |                             || unreachable!(),
211 | |                         );
    | |_________________________^
    |
note: `search` defined here
   --> src/utils.rs:175:1
    |
175 | / pub fn search(
176 | |     root: bool,
177 | |     at: &mut serde_json::Value,
178 | |     needle: &serde_json::Value,
...   |
181 | |     handle_end_root: impl FnOnce() -> SearchRes,
182 | | ) {
    | |_^

error: could not compile `hammer-of-json` (bin "hammer-of-json") due to 1 previous error
Exception: cargo exited with 101

Meta

rustc --version --verbose:

rustc 1.87.0-nightly (794c12416 2025-02-21)

Setting the recursion limit to 1024 didn't help.

I worked around the issue by defining

    fn nil_handle_end() -> SearchRes {
        unreachable!();
    }

then replacing || unreachable!() with nil_handle_end, so I think it's probably related to closures.

This may be "as designed" so feel free to close, but from a user perspective it was fairly unexpected and the line just pointed to a closure with no real indication of why that was causing recursion or anything that would indicate how to solve the issue.

Reproduce the failure from the linked example, focusing on src/utils.rs:175 and the recursive calls around lines 204-210, using the reported nightly toolchain and cargo build. Compare the closure form with the named nil_handle_end function to isolate the generic-instantiation behavior. Done means establishing whether this is expected compiler behavior or improving the compiler diagnostic or handling for this case.