Dashboard: FAQs and query insights

Context

Once an NGO finalizes their program and releases it to end users in production, they currently have no way to see how it is performing in the field. Within the first 2–4 weeks of a release, program teams should have visibility into how the program is going: what kinds of questions end users are asking, which topics come up most often, and whether the assistant is handling them well.

Today that feedback loop doesn't exist — teams cannot tell whether the assistant's knowledge base covers real user needs, which questions are going unanswered or answered poorly, or how to refine the prompt and content based on actual usage.

What NGOs need

A simple view of their top frequently asked questions over a chosen time window, with query volumes, so they can:

• Spot recurring themes in end-user queries
• Verify the knowledge base has good answers for those questions
• Use this data to improve the assistant (prompt + content) with each iteration

What can be done from our end

Kaapi already persists every user question and response with assistant and timestamp, so the raw data for time-windowed FAQ mining exists today without new capture infrastructure. On top of it:

• Clustering of semantically similar queries (via embeddings or similar techniques) to surface top FAQs with counts
• Aggregation by time period
• Periodic summaries (e.g. a daily rollup)

Since most NGOs operate at modest query volumes, lightweight clustering approaches are viable, keeping the solution simple while giving program teams actionable insight from real production usage.

Status

PRD being drafted; picking up in this iteration.

Start with the PRD being drafted and inspect where Kaapi persists each user question and response, including the assistant and timestamp fields. Define the time-windowed FAQ view, semantic query clustering, counts, aggregation, and periodic summaries; done means the design and implementation meet the agreed PRD requirements.