agentscope-ai / agentscope-ai/agentscope

Proposal: Document a 16-problem RAG and agent failure map for Agentscope users

Abierto
#1,265 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
31.5k
Forks
3.5k
Merge medio
1 d 23 h
PR fusionados (30 d)
95

Descripción

Hi Agentscope team,

I maintain **WFGY ProblemMap**, a small open-source project that provides a **16-problem failure map for RAG and LLM pipelines**, especially for agentic systems.

This map is already adopted in other ecosystems:

- **RAGFlow** – includes it as their official RAG failure modes checklist in the documentation.
- **LlamaIndex** – uses it as a structured failure-mode reference in RAG troubleshooting docs.
- **ToolUniverse (Harvard MIMS Lab)** – wraps it as a triage tool for incident diagnostics in a research agent platform.
- It is also referenced in curated lists such as *Awesome LLM Apps* and *Awesome-AITools*.

Map entry point (MIT-licensed):

- https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

### Why it fits Agentscope

Agentscope is a framework for building agent-oriented applications, which often involve:

- RAG components feeding agents with context.
- Multiple tools and calls between agents.
- Complex failure modes that are not just “the model hallucinated”.

In these kinds of systems, we found that most production failures are variations of a small set of patterns (retrieval issues, chunking, config drift, tool routing problems, eval mismatch, etc.). The ProblemMap encodes those as 16 numbered problems, each with acceptance criteria and suggested structural fixes.

### Proposed small doc addition

I would like to propose adding a short **“Failure modes checklist for agentic workflows (16 problems)”** section to your documentation. That section could:

1. Link to the WFGY ProblemMap poster and README.
2. Explain that users can feed a failing run (conversation, retrieved evidence, prompts, actions) plus the poster into a strong LLM and ask it to classify the failure into one or more of the 16 problems.
3. Encourage labeling recurrent issues by ProblemMap number when debugging complex Agentscope-based systems.

This is entirely optional and does not require any changes to the Agentscope codebase. It is simply a diagnostic lens that some users may find useful, especially as their systems grow more complex.

If this sounds reasonable, I can submit a PR with a draft doc section for review.

Thank you for your time.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.