MemberJunction / MemberJunction/MJ

Add AIPrompt.TimeoutMS column so a prompt can carry its own execution bound

Open
#3,133 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

Split out of #3064.

#3064 was filed as "AIPromptRunner does not enforce `AIPrompt.TimeoutMS`". On investigation, **`AIPrompt` has no `TimeoutMS` column** — the only `TimeoutMS` in the schema belongs to `MJ: Remote Operations`. So there was nothing to enforce; there was simply no way to bound a single-model prompt call at all.

That gap is now closed at the **request** level: `AIPromptParams.timeoutMS` bounds each model call, surfaces a typed retriable `AIPromptTimeoutError`, and composes with any caller-supplied `cancellationToken`. All 19 LLM drivers now forward the signal to their SDK, so a timeout actually tears down the socket.

## What's still missing

A prompt cannot carry its **own** default timeout — every caller must pass one. A long multimodal prompt that reliably needs 5 minutes, and a cheap classification prompt that should never exceed 10 seconds, currently have no way to say so in metadata.

## The work

1. Migration: `ALTER TABLE ${flyway:defaultSchema}.AIPrompt ADD TimeoutMS INT NULL` + `sp_addextendedproperty` describing it.
2. Run CodeGen so `MJAIPromptEntity.TimeoutMS` is generated.
3. One-line change in `AIPromptRunner.getEffectiveTimeoutMS` — the seam is already in place:

```ts
const timeoutMS = prompt.TimeoutMS ?? params.timeoutMS ?? this.DefaultPromptTimeoutMS;
```

Everything downstream (composition with the caller token, the typed error, failover classification, parity between the single-model and parallel paths) already works and needs no further change.

Deliberately **not** shipped with #3064: writing code against a column CodeGen hasn't generated yet would mean reaching for `.Get('TimeoutMS')`, which the project rules forbid — and shipping a dead column would recreate the exact "looks configured, does nothing" bug #3064 reports.

Contributor guide

Open the contributing guide

Research direction

Start with the migration for the AIPrompt table and the AIPromptRunner.getEffectiveTimeoutMS entry point described in the issue. Run CodeGen after adding the column and confirm that MJAIPromptEntity.TimeoutMS is generated. Done means the prompt timeout takes precedence over params.timeoutMS, with the existing fallback behavior preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql, typescript
Domain
ai, backend, databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.