simonw / simonw/llm

Design new LLM database schema

Open
#556 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design
Dominant language
Python
Stars
12.5k
Forks
998
Avg merge
6d 12h
Merged PRs (30d)
15

Description

The LLM database schema for storing responses needs to evolve to support some new features:

  • "Mock" responses. Sometimes it's useful to run a prompt that contains several previous messages which didn't actually come from a model - for example, prompting Claude 3 Haiku with a bunch of examples that are input as previous user messages. Differentiating these from actual text returned by a model is important to support LLM's goal of recording responses from models over time so you can analyze them later.
  • Another Claude feature: prompt caching. This is a larger feature to decide how to build, but since one of the uses of prompt caching is for longer conversations it's important to be able to remember at what point in the conversation the "cache_control": {"type": "ephemeral"} rule was injected.
  • Images! For prompts that use images one of the following three needs to be stored: the image URL (for images that were referenced by their URL) the image file path (for images that were a file on disk) or the raw bytes of the image itself (for cases where we want that stored in the database as opposed to living separately on disk)
  • Other multi-modal data: Gemini Pro accepts PDF and audio and video, fer example
  • That Claude trick where you can pre-fill the prefix for the response, passing <!doctype html><html> to ensure an HTML response for example. If that trick is used it should be recorded in the database as well.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by mapping the existing LLM response schema and its storage or migration entry points, then trace how prompts, responses, and multimodal content are represented. The design should cover mock responses, prompt-cache markers, images, other media, and prefilled response prefixes, with tests or migration checks demonstrating that each can be stored and recovered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.