microsoft / microsoft/PyRIT

[Feature] Add RAG retrieval poisoning test scenarios

Open
#2,156 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.5k
Forks
893
Avg merge
3d 50m
Merged PRs (30d)
165

Description

## Summary
Add built-in test scenarios for RAG retrieval poisoning attacks to PyRIT's scenario library.

## Problem
RAG systems are vulnerable to retrieval poisoning where malicious content is injected into the knowledge base. PyRIT currently lacks dedicated scenarios for testing this attack surface.

## Proposed Implementation
- New file: pyrit/scenario/scenarios/rag_retrieval_poisoning.py
- Predefined attack vectors for retrieval poisoning
- Integration with existing attack executor
- Documentation in scenario docstrings

## Use Case
Security teams can use these scenarios to test RAG-based LLM applications against known poisoning attack patterns.

## References
- Related: issue #2085

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing related issue #2085 and the existing scenario library and attack executor. Add pyrit/scenario/scenarios/rag_retrieval_poisoning.py with predefined retrieval-poisoning attack vectors and scenario docstrings. The work is done when the scenarios are integrated with the existing executor and documented in the new file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, security, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.