graniet/llm

View on GitHub

A powerful Rust library and CLI tool to unify and orchestrate multiple LLM, Agent and voice backends (OpenAI, Claude, Gemini, Ollama, ElevenLabs...) with a single, extensible API. Build, chain, evaluate, and serve complex multi-step AI workflows — including speech-to-text, text-to-speech, completions, vision, and reasoning.

Stars: 365
Forks: 90
Open beginner issues: 0
Indexed issues: 17

Dominant language: Rust
License: MIT
Last GitHub push: Jun 6, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

17 open issues indexed

Build fails on WASM as reqwest::blocking is not available Open

graniet/llm#8 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
run Open

graniet/llm#13 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Tool is not Deserialize Open

graniet/llm#29 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Question: Chain & structured output Open

graniet/llm#39 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Issue with Google structured response Open

graniet/llm#44 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Question: mcp support? Open

enhancement

graniet/llm#51 · 2 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
Google Gemini now supports thinking tokens Open

graniet/llm#97 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Unreachable code reach on OpenAI Compatible Open

graniet/llm#108 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Bug: tool_calls messages omit content field Open

graniet/llm#111 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
The crate enables API keys to be leaked in logs Open

graniet/llm#115 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Update to latest reqwests? Open

graniet/llm#118 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Support custom headers and pre-request auth hooks for OpenAI-compatible providers Open

graniet/llm#119 · 2 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
reasoning_content dropped in serialization Open

graniet/llm#123 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature Request: Add mistral.rs Backend Support Open

graniet/llm#125 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feat: Gemini Enterprise Agent Platform (formerly Vertex AI) Open

graniet/llm#127 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Bug: Google Gemini tool calling fails when thinking mode returns `thoughtSignature` Open

graniet/llm#132 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Consider using separate structs for Tool Call requests and Tool Call results Open

graniet/llm#133 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026

graniet/llm

Get new issues in your inbox