MemberJunction / MemberJunction/MJ

AIPromptRunner does not enforce AIPrompt.TimeoutMS on the single-model execution path

Open
#3,064 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

## Summary

`AIPrompt.TimeoutMS` is silently ignored when a prompt executes on the single-model path (e.g. `SelectionStrategy=Specific`, or a `params.override` pin). The model call runs unbounded.

## Observed (v5.44.0)

- `@memberjunction/ai-prompts/dist/AIPromptRunner.js` contains **no reference to `TimeoutMS`** anywhere.
- The only timeout in the package is `ParallelExecutionCoordinator`'s `taskTimeoutMS` (default 30000), which applies only to the parallel multi-model path.
- On the single-model path, execution is only bounded when the caller supplies a `cancellationToken` — the runner then does `Promise.race([llm.ChatCompletion(chatParams), ])`. With no token, `ChatCompletion` is awaited with no bound.
- Drivers don't set an HTTP timeout either (checked `@memberjunction/ai-gemini` 5.44.0 — no `httpOptions.timeout` is passed to `@google/genai`), so a hung provider connection never resolves.

## Impact

- Users who set `TimeoutMS` on a prompt reasonably believe they configured an execution bound; they haven't.
- Long-running multimodal prompts (we hit this with interview-audio evaluation, ~minutes per call with multi-MB base64 payloads) can hang indefinitely, pinning large buffers in memory and never surfacing a failure to the caller's retry logic.

## Suggested fix

In the single-model path, when `prompt.TimeoutMS` is set, combine it with any caller-supplied token (e.g. `AbortSignal.any([callerToken, AbortSignal.timeout(prompt.TimeoutMS)])`) and race the model call against it. Ideally also plumb the timeout into the driver's HTTP options so the socket is actually aborted rather than the promise merely abandoned.

## Workaround

Callers can pass `params.cancellationToken = AbortSignal.timeout()` — the existing race honors it. Caveat: this abandons the promise; the underlying HTTP request keeps running.

Contributor guide

Open the contributing guide

Research direction

Start with @memberjunction/ai-prompts/dist/AIPromptRunner.js and compare the single-model path with ParallelExecutionCoordinator's timeout handling. Trace the existing cancellation-token race, then verify that a configured AIPrompt.TimeoutMS bounds single-model execution and that the relevant provider request behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.