future-agi / future-agi/future-agi
model_hub: populate_placeholders silently passes unresolved {{tokens}} to LLM on column resolution failure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 628
- Avg merge
- 1d 50m
- Merged PRs (30d)
- 167
Description
Summary
populate_placeholders() catches all non-media exceptions and returns the original (unsubstituted) messages as a fallback. Unresolved {{column_name}} tokens reach the LLM verbatim with no error surfaced to the caller.
File: futureagi/model_hub/views/run_prompt.py:460-467
except Exception as e:
if media_error:
raise e
else:
traceback.print_exc()
logger.exception(f"Fatal error processing messages: {e}")
# Return original messages as fallback
return messages
The inner per-column loop (lines 403-407) also continues silently on column resolution errors.
Impact
- LLM receives
{{column_name}}as literal text when a column is missing, a cell is null, or column lookup fails. - The prompt response is subtly wrong with no error surfaced to the user.
- Callers cannot distinguish "all resolved" from "fallback due to exception" — both return a list of messages.
- Parallel to simulate issue #312.
Suggested fix
After template rendering, scan for remaining {{...}} tokens in the output messages. If any are found, either:
- Return an error status (preferred), or
- Log at WARNING with the list of unresolved tokens.
See also
ADR 024 in docs/adr/024-populate-placeholders-no-unresolved-validation.md.
Simulate issue #312 for the parallel problem in the simulate module.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in futureagi/model_hub/views/run_prompt.py at populate_placeholders(), especially the per-column loop around lines 403-407 and fallback around lines 460-467. Read docs/adr/024-populate-placeholders-no-unresolved-validation.md and compare the parallel simulate issue #312. Done means unresolved {{...}} tokens cannot reach the LLM silently and the caller receives or can identify the failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100