[. DEMO] - Benchmarking Framework for Google Gemma Models on the MMLU Dataset.

### In this mini project, I have demonstrated a simple framework to load and evaluate Gemini and other models on the MMLU dataset (10 samples), and check their accuracy

This project is a benchmarking framework that evaluates how well Google's Gemma models perform on MMLU (Massive Multitask Language Understanding) — a popular dataset with multiple-choice questions across many subjects like math, history, law, etc. These questions test a model’s reasoning, knowledge, and problem-solving skills.

What Does the Project Do?
The framework loads questions from the MMLU dataset, formats them into prompts, and feeds them to the Gemma model. It then checks if the model gives the right answer and calculates the accuracy. We’ve also added support for other models like LLaMA, so you can compare how different models perform on the same questions.

What Makes It Cool?
Instead of running everything in one big script, the project is split into modular scripts:

One for loading models

One for loading the MMLU dataset

One for formatting prompts

One for evaluating a single question

One for running evaluation on multiple questions

One script that benchmarks multiple models and shows a bar chart comparison

This makes the project clean, flexible, and easy to update or extend.

Who Is It For?
It’s great for AI researchers, engineers, or students who want to:

Test large language models like Gemma and Llama

Compare performance on academic and general knowledge tasks

Visualize model strength across subjects

ITS A DEMO PROJECT. PLEASE PROVIDE SOME FEEDBACK

- PROJECT LINK: (https://github.com/DEBADAS001KERNEL/BENCHMARK_FRAMEWORK)