koala73 / koala73/worldmonitor

Stock Analysis panel doesn't disclose when AI commentary is a rules-based fallback template vs LLM-generated

Open Beginner friendly
#5,498 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
86.6k
Forks
13.1k
Avg merge
8h 4m
Merged PRs (30d)
825

Description

## Summary
`analyze-stock.ts`'s AI-overlay builder returns a deterministic templated response (`provider: 'rules'`, `fallback: true`) whenever the LLM call fails or times out, computed from real technical indicators (RSI, MACD, moving averages). The wire response carries `provider`/`fallback` fields, but `StockAnalysisPanel.ts` renders the resulting summary/action text without checking either field anywhere in the file.

## Evidence
- `server/worldmonitor/market/v1/analyze-stock.ts:1132,1134` — fallback overlay path sets `provider: 'rules'` and `fallback: true` (also set at line 1375 for a second call site).
- `src/components/StockAnalysisPanel.ts` — verified via direct grep: zero matches for `fallback` or `provider` anywhere in the file; the panel already calls `Panel.setDataBadge()` elsewhere for its own live/cached state, but that badge is never wired to the AI-overlay's own `provider`/`fallback` fields.

## Why this matters
Users see narrative commentary (summary, "why now", action) with no way to tell whether it's model-generated analysis or a fixed sentence template substituted after an LLM timeout. The underlying quantitative signal (price, RSI, MACD, score) is always genuine — only the surrounding prose framing is ambiguous — but the product markets itself on "AI-synthesized" analysis, and this is a surface where that claim silently degrades to a template with no indication to the reader.

## Attack or failure scenario
The configured LLM provider times out or errors (rate limit, outage, cost-gate) for a given symbol. The rules-based template renders identically to a genuine LLM overlay — same layout, no "template" or "AI unavailable" label — despite the panel already having a badge mechanism it uses for other state.

## Root cause
`provider`/`fallback` are computed and transmitted by the backend but never consumed by the frontend component that would need them to disclose provenance to the user, even though that same component already implements the disclosure pattern (`setDataBadge`) for a different piece of state.

## Recommended fix
- Render a "template" or "AI summary unavailable" indicator in `StockAnalysisPanel.ts` via the panel's existing `setDataBadge()` mechanism when the overlay's `fallback === true` / `provider === 'rules'`.
- Add a test asserting the fallback indicator renders when the overlay's `provider` is `'rules'`.

## Acceptance criteria
- The panel visibly distinguishes rules-based fallback commentary from LLM-generated commentary.
- A test covers this rendering path.

## Suggested labels
bug, area: markets, area: AI/intel

## Priority
P3

## Severity
Low — the numeric data displayed is always genuine; only prose provenance is undisclosed, and a fallback template built from real indicators is materially less misleading than fabricated numbers would be.

## Confidence
Confirmed for the code paths (both fields exist server-side, verified unused client-side); Needs verification for how often the LLM path actually fails in production (exposure frequency isn't measurable from source alone).

Contributor guide

Open the contributing guide

Research direction

Start in src/components/StockAnalysisPanel.ts and trace the overlay response fields from server/worldmonitor/market/v1/analyze-stock.ts, especially provider and fallback. Read the panel's existing Panel.setDataBadge() usage, then add a test covering provider: 'rules'. Done means the panel visibly distinguishes fallback commentary from LLM-generated commentary and the rendering path is tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, frontend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.