apache / apache/hugegraph-ai

[Feature] Add optional trace output to the RAG answer API for retrieval debugging

Open
#347 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
143
Forks
89
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [x] I had searched in the [feature](https://github.com/apache/hugegraph-ai/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

### Feature Description (功能描述)

## Feature Description

The `/rag` answer endpoint currently returns final answer fields only. For
retrieval debugging, users need an optional trace/debug response that includes
intermediate graph-recall information such as extracted keywords, matched vertex
IDs, generated Gremlin, graph recall results, and related graph debug fields.

The trace should be opt-in so the default response remains compact and backward
compatible. Vector recall and rerank traces can be handled as follow-up work
after the graph trace contract is stable.

## Current verification

- `/rag` has an explicit TODO saying more response information is needed for users to understand query logic in `hugegraph-llm/src/hugegraph_llm/api/rag_api.py:79`.
- `/rag` currently returns only `query` and requested answer fields in `hugegraph-llm/src/hugegraph_llm/api/rag_api.py:80`.
- `/rag/graph` already returns graph-recall intermediate fields including `query`, `keywords`, `match_vids`, `graph_result_flag`, `gremlin`, `graph_result`, and `vertex_degree_list` in `hugegraph-llm/src/hugegraph_llm/api/rag_api.py:131`.
- `RAGRequest` does not currently define an `include_trace` or similar option.

## Suggested scope

- Add an opt-in request field, for example `include_trace: bool = False`.
- Extend the graph-related RAG flow return contract or add a structured side channel so `/rag` can include graph trace data without parsing logs.
- Reuse existing `/rag/graph` field names where possible.
- Keep vector recall and rerank trace fields out of the required first version unless the implementation already exposes them cleanly.
- Avoid leaking secrets or full prompt contents by default.
- Keep the default `/rag` response compatible for existing users.

## Mermaid reference

```mermaid
flowchart LR
Request[/POST /rag include_trace/]
Request --> RAG[RAG answer flow]
RAG --> Answer[Answer fields]
RAG --> Trace{include_trace?}
Trace -->|false| Compact[Current response shape]
Trace -->|true| Debug[trace: keywords, match_vids, gremlin, graph_result]
Answer --> Compact
Answer --> Debug
```

## Acceptance criteria

- `/rag` without `include_trace` keeps the current response shape.
- `/rag` with `include_trace=true` returns a structured `trace` object.
- Trace includes graph recall fields when `graph_only` or `graph_vector_answer` is used.
- Vector recall and rerank trace fields are optional follow-up fields, not required for the first version.
- Missing unavailable trace sections are omitted or returned as empty structured values consistently.

## Suggested tests

- API contract test for default `/rag` response compatibility.
- API test for `include_trace=true`.
- Flow-level test showing trace data is populated from graph recall.
- Test that sensitive config values are not included in trace.

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.