cactus-compute / cactus-compute/needle
[native CLI] `needle --serve` hangs in `needle_complete` spin loop on first /complete request
- Dominant language
- Python
- Stars
- 11k
- Forks
- 710
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 25
Description
## Summary
Running the native macOS CLI (`macos-arm64/needle`) in serve mode against the README's quickstart tools file (`get_weather` + `set_thermostat`) causes the process to enter a spin loop inside `needle_complete` on the very first `POST /complete` request. The connection is never accepted for subsequent requests, no response is written, and the process pegs multiple cores indefinitely.
## Environment
- Binary: `/Users/tom/Models/cactus-needle2/needle` (14,610,568 bytes, matches `Cactus-Compute/needle2` `macos-arm64/needle` byte-for-byte, dated 2026-09-07)
- OS: macOS 27.0 (Darwin 27.0.0), M-series (arm64)
- CLI flags: `--serve --port 8082 --tools /tmp/needle-tools.json`
- Tool set: the exact `get_weather` / `set_thermostat` schemas from the Quickstart section of `README.md` (see #51 for context)
## Reproduction
```sh
cd ~/Models/cactus-needle2
./needle --serve --port 8082 --tools /tmp/needle-tools.json &
# logs only the startup line — nothing further is ever written
curl -X POST http://127.0.0.1:8082/complete -d '{"input":"what is the weather in Lagos"}'
# connection times out, process never returns
```
## Observations
1. The log file (`needle-server-YYYYMMDD.log`) only contains the startup banner — no request lines, so the request never reaches a serialisation path that produces output.
2. `lsof -p ` shows the listener is bound, but the inbound TCP connection is accepted and then sits with zero bytes read/written until I kill the process. There is exactly one closed stale connection in the FD table after several minutes.
3. **RSS stays at ~14.6 MB the entire time** — no model weights are being paged in. This is purely a CPU spin, not an inference pass.
4. `sample ` shows the main thread stuck entirely inside `needle_complete`:
```
1177 Thread_93436044 DispatchQueue_1: com.apple.main-thread (serial)
1177 start (in dyld)
1177 needle_complete (in needle) + 4248
1176 ??? (in needle) load address 0x102988000 + 0xb730
1176 ??? (in needle) load address 0x102988000 + 0x61e8c
1176 ??? (in needle) load address 0x102988000 + 0x447c
781 ??? (in needle) load address 0x102988000 + 0x46944
781 ??? (in needle) load address 0x102988000 + 0x496b8
210 ??? (in needle) load address 0x102988000 + 0x5754c
```
The recursion depth in `needle_complete` and absence of any symbol resolution confirms this is the grammar-constrained decode loop. 1177/1177 samples are inside it — not a request-handling thread.
5. The bug burns ~380% CPU continuously until the process is killed (verified over a 24h run). No timeout, no log line, no way for the operator to know the request will never complete.
## Likely relationship to #51
Issue #51 was a Python-package bug (shared `needle_init` engine with stale tools) that produced a wrong-but-bounded response — the model reasoned about a `description` field that wasn't in the schema, then truncated with `token budget exhausted`. The same `get_weather`/`set_thermostat` schema mismatch that produced wrong-tool reasoning in #51 may be putting the *native* CLI's constrained decode loop into a state with no grammar-accepting continuation, so the engine retries forever instead of falling back to a refusal response like the Python path does.
## Expected
Either:
- a structured `refuse` / `truncated` JSON response matching the Python package's behaviour (preferred, matches #51's resolution), or
- an iteration-cap that emits an error after N decode steps so the request at least terminates.
## Actual
Process spins at ~380% CPU, never logs anything, never writes a response, never returns from `needle_complete`. Killed only by `SIGKILL`.
## Workaround
None observed. The CLI appears unusable with the README's example tools file. Limiting this to the Python package is not a documented constraint.
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the hang with the native macOS CLI, the README quickstart tools file, and the POST /complete request, then inspect the needle_complete path identified by sample. Compare the native behavior with the Python package behavior described in issue #51; done means the request terminates with a refusal, truncated response, or bounded error instead of spinning indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100