dotCMS / dotCMS/core

dotAI: Harden RAG prompt construction against injection (always-on data/instruction separation)

Open
#37,152 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : Security Team : Modernization Type : Task
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Note: Part of the dotAI security review. Always-on hardening (not a feature flag).

Description

In CompletionsAPIImpl.buildRequestJson, retrieved content is concatenated into a single flat string and interpolated raw into the prompt template (COMPLETION_TEXT_PROMPT = ... by using only the information in the following text: """ $!{supportingContent} """). There is no real separation between instructions and data: the fenced block and the "use only this text" sentence are the only grounding, and both are just text the model is asked to respect.

This makes stored (indirect) prompt injection possible: text embedded from content (especially if a customer wires a low-trust write path — e.g. a public form — into the embedding pipeline) is treated by the model as instructions and can hijack the response for other users.

We cannot eliminate prompt injection for any LLM, but we should make the default RAG prompt construction as robust as reasonably possible. This is an always-on improvement with negligible downside — not an opt-in flag.

Affected code
  • com.dotcms.ai.api.CompletionsAPIImpl#buildRequestJson(...) — builds supportingContent and the messages.
  • com.dotcms.ai.api.CompletionsAPIImpl#getPrompt / getSystemPrompt / getTextPrompt — Velocity template evaluation.
  • com.dotcms.ai.app.AppKeysCOMPLETION_ROLE_PROMPT, COMPLETION_TEXT_PROMPT defaults.

Acceptance Criteria

  • Retrieved content is wrapped in a clearly-delimited, labeled block and any delimiter-breaking sequences in the content are escaped/neutralized before interpolation.
  • A fixed guard instruction is added to the system prompt: the delimited block is untrusted data, must be used only as reference material, and must never be interpreted as instructions.
  • User query text stays strictly in the user role; retrieved content is never merged into the system role.
  • Behavior is always on (no config flag) and applies to summarize / summarizeStream.
  • Unit test: content containing injection markers (e.g. fake """ fences, "ignore previous instructions", role headers) is escaped/labeled so it cannot break out of the data block.
  • Regression test: normal summarization output quality is preserved for benign content.
  • Documented in release notes as a hardening; pairs with the trust-boundary guidance ("do not auto-embed untrusted input").

Priority

Medium

Additional Context

This mitigates but does not eliminate prompt injection. It complements the RAG permission fix (per-user READ filtering) and the safe-by-default viewtool output hardening. The definitive control for untrusted input remains operational: do not auto-embed content authored through low-trust paths.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CompletionsAPIImpl#buildRequestJson and trace getPrompt, getSystemPrompt, and getTextPrompt, then inspect the default prompts in AppKeys. Check how summarize and summarizeStream construct messages and identify the relevant unit-test locations. Done means untrusted retrieved content remains labeled and delimited, user text stays in the user role, injection-marker and benign-content tests pass, and the hardening is documented in release notes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai-infra-agents, backend, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.