MemberJunction / MemberJunction/MJ
AIPromptRunner does not enforce AIPrompt.TimeoutMS on the single-model execution path
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Summary
`AIPrompt.TimeoutMS` is silently ignored when a prompt executes on the single-model path (e.g. `SelectionStrategy=Specific`, or a `params.override` pin). The model call runs unbounded.
## Observed (v5.44.0)
- `@memberjunction/ai-prompts/dist/AIPromptRunner.js` contains **no reference to `TimeoutMS`** anywhere.
- The only timeout in the package is `ParallelExecutionCoordinator`'s `taskTimeoutMS` (default 30000), which applies only to the parallel multi-model path.
- On the single-model path, execution is only bounded when the caller supplies a `cancellationToken` — the runner then does `Promise.race([llm.ChatCompletion(chatParams), ])`. With no token, `ChatCompletion` is awaited with no bound.
- Drivers don't set an HTTP timeout either (checked `@memberjunction/ai-gemini` 5.44.0 — no `httpOptions.timeout` is passed to `@google/genai`), so a hung provider connection never resolves.
## Impact
- Users who set `TimeoutMS` on a prompt reasonably believe they configured an execution bound; they haven't.
- Long-running multimodal prompts (we hit this with interview-audio evaluation, ~minutes per call with multi-MB base64 payloads) can hang indefinitely, pinning large buffers in memory and never surfacing a failure to the caller's retry logic.
## Suggested fix
In the single-model path, when `prompt.TimeoutMS` is set, combine it with any caller-supplied token (e.g. `AbortSignal.any([callerToken, AbortSignal.timeout(prompt.TimeoutMS)])`) and race the model call against it. Ideally also plumb the timeout into the driver's HTTP options so the socket is actually aborted rather than the promise merely abandoned.
## Workaround
Callers can pass `params.cancellationToken = AbortSignal.timeout()` — the existing race honors it. Caveat: this abandons the promise; the underlying HTTP request keeps running.
Contributor guide
Research direction
Start with @memberjunction/ai-prompts/dist/AIPromptRunner.js and compare the single-model path with ParallelExecutionCoordinator's timeout handling. Trace the existing cancellation-token race, then verify that a configured AIPrompt.TimeoutMS bounds single-model execution and that the relevant provider request behavior is covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100