[DEMO ] LMLyzer: A CLI-Based Benchmarking Tool for LLM Memory & Time Analysis.

### Title: LMLyzer — A CLI Tool for Benchmarking Memory and Time Efficiency of Language Models
🧠 About the Project
LMLyzer (Language Model + Analyzer) is a lightweight, open-source command-line interface tool designed to benchmark loading time, text generation time, GPU/CPU/memory usage, and overall resource efficiency of various large language models (LLMs).

The motivation is to eliminate the need to write separate scripts for every benchmarking task. With a few simple commands, users and researchers can:

Analyze and compare models

Generate reports and graphs

Identify the most efficient models for specific environments (e.g. low-memory systems)

It’s built for LLM researchers, developers, and AI engineers who want a plug-and-play benchmarking system.

THE LINK OF THE PROJECT : [(https://github.com/DEBADAS001KERNEL/LMLYZER)]

🚀 Note (for### [ GSoC 2025 ](url)Reviewers)
I'm applying to GSoC 2025 with this project as my proposal. This is still a work in progress (60% complete), but I'm actively developing it and pushing regular updates. I welcome guidance and mentorship to:

Improve benchmarking accuracy (e.g., using PyTorch hooks or Hugging Face profiling tools)

Expand CLI command features (multi-model comparison, ranking, and visualizations)

Support multiple model types (HuggingFace, OpenLLM, transformers with custom configs)

Integrate reporting modules (PDF/Markdown export with graphs)

Make it more beginner-friendly (setup guide, sample models, presets)