[New Feature Wanted] Attention Visualize with Gradio

We are wandering if it is possible to use Gradio realize a visualization tools on the top of vicuna original interface.
It can help people write a better prompt, and find the reason of bad case from attention view.

Project Title: LLM Attention Visualization Tool Prototype

Objective:
Design a prototype for an interactive visualization tool that enables users to analyze and understand the attention mechanism of Language Models (LLMs) in a dialogue interface.

Key Features to Visualize:

1. Interactive Dialogue Interface:

   • A dialogue box where users can input text and see the model's completions.
   • Hover-over functionality to display attention weights when the mouse is placed over a word in the completion.

2. Attention Weight Visualization:

   • Color gradients to represent the attention weights associated with the hovered word.
   • Display of attention weights across both input and completion text, correlating to the hovered word.

3. Model Selection:

   • Ability for users to load different pre-trained LLMs like GPT-3, BERT, etc.
   • Support for adjusting model parameters like the number of layers and heads.

Technical Specifications:

• Modern front-end technology stack (e.g., React or Vue).
• Compatibility with major web browsers like Chrome, Firefox, and Safari.

Deliverables:

• Wireframes for each key section of the interface.
• A clickable prototype demonstrating the flow and interactions.
• Annotations explaining the functionality and interactions in each part of the interface.

Inspiration:

• Look into existing attention visualization tools to understand common practices and user expectations.
• Consider how the visualization of attention weights can provide insights into the model's behavior and improve user understanding.

Evaluation Criteria:

• Ease of use and clarity of the visualization.
• Accurate representation of attention weights.
• Responsiveness and compatibility across different browsers and devices.

The issue names the original Vicuna interface and Gradio, but identifies no repository files, tests, or entry points. First clarify the supported models, attention data, and frontend stack, then define the wireframes and clickable prototype scope. Done means an interactive dialogue prototype with attention visualization, model and parameter controls, annotations, and browser/device evaluation.