beehive-lab / beehive-lab/GPULlama3.java

Build GUI Chatbox for GPULlama3.java Inference with embedded GPU resource monitoring

Open
#24 2 comments 0 reactions 0 assignees View on GitHub
good first issue Tooling
Dominant language
Java
Stars
274
Forks
37
Avg merge
5d 1h
Merged PRs (30d)
2

Description

## Description:

We want to create a user-friendly chat-style GUI for running LLM inference via our Java engine (powered by TornadoVM), inspired by the layout shown in the attached screenshot.

## Desired Features

* Prompt Input
* A text box where users can enter prompts for the LLM

* Dropdown Selectors
* Engine: Choose between TornadoVM, JVM, etc.
* Model: Select the model file to use for inference between Llama, Mistral etc

* File Picker
* A "Browse" button to select .java files or model directories

* Run Button
* A button to trigger inference (e.g., using tornado-llama-opencv)

* Output Display
* A read-only area to show model responses and logs
* Optional: System Monitoring Panel
* Live GPU usage via nvtop or nvidia-smi
* Live CPU & memory stats via htop or Java system metrics

Preferably, integrated with the core code base with JavaFX, or external with QT or python QT.

## Example POC:

![Image](https://github.com/user-attachments/assets/ea958a0c-20c9-4045-9fca-0029fce6edb4)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.