dotAI: Harden RAG prompt construction against injection (always-on data/instruction separation)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Note: Part of the dotAI security review. Always-on hardening (not a feature flag).
Description
In CompletionsAPIImpl.buildRequestJson, retrieved content is concatenated into a single flat string and interpolated raw into the prompt template (COMPLETION_TEXT_PROMPT = ... by using only the information in the following text: """ $!{supportingContent} """). There is no real separation between instructions and data: the fenced block and the "use only this text" sentence are the only grounding, and both are just text the model is asked to respect.
This makes stored (indirect) prompt injection possible: text embedded from content (especially if a customer wires a low-trust write path — e.g. a public form — into the embedding pipeline) is treated by the model as instructions and can hijack the response for other users.
We cannot eliminate prompt injection for any LLM, but we should make the default RAG prompt construction as robust as reasonably possible. This is an always-on improvement with negligible downside — not an opt-in flag.
Affected code
com.dotcms.ai.api.CompletionsAPIImpl#buildRequestJson(...)— buildssupportingContentand the messages.com.dotcms.ai.api.CompletionsAPIImpl#getPrompt / getSystemPrompt / getTextPrompt— Velocity template evaluation.com.dotcms.ai.app.AppKeys—COMPLETION_ROLE_PROMPT,COMPLETION_TEXT_PROMPTdefaults.
Acceptance Criteria
- Retrieved content is wrapped in a clearly-delimited, labeled block and any delimiter-breaking sequences in the content are escaped/neutralized before interpolation.
- A fixed guard instruction is added to the system prompt: the delimited block is untrusted data, must be used only as reference material, and must never be interpreted as instructions.
- User query text stays strictly in the
userrole; retrieved content is never merged into thesystemrole. - Behavior is always on (no config flag) and applies to
summarize/summarizeStream. - Unit test: content containing injection markers (e.g. fake
"""fences, "ignore previous instructions", role headers) is escaped/labeled so it cannot break out of the data block. - Regression test: normal summarization output quality is preserved for benign content.
- Documented in release notes as a hardening; pairs with the trust-boundary guidance ("do not auto-embed untrusted input").
Priority
Medium
Additional Context
This mitigates but does not eliminate prompt injection. It complements the RAG permission fix (per-user READ filtering) and the safe-by-default viewtool output hardening. The definitive control for untrusted input remains operational: do not auto-embed content authored through low-trust paths.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CompletionsAPIImpl#buildRequestJson and trace getPrompt, getSystemPrompt, and getTextPrompt, then inspect the default prompts in AppKeys. Check how summarize and summarizeStream construct messages and identify the relevant unit-test locations. Done means untrusted retrieved content remains labeled and delimited, user text stays in the user role, injection-marker and benign-content tests pass, and the hardening is documented in release notes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ai-infra-agents, backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100