simonw / simonw/llm

Add `--verbose` or similar to return equivalent of log response

Open
#418 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12.5k
Forks
998
Avg merge
3d 13h
Merged PRs (30d)
10

Description

I'd like to be able to pass an option to the chat command to make it return the full logged (json) object, or at least something structured that includes conversation_id and other details. E.g

llm -m 4 "Hello, how are you" --no-stream --verbose
> {
    "id": "",
    "model": "gpt-4",
    "prompt": "Hello, how are you",
    "system": null,
    "prompt_json": {
      "messages": [
        {
          "role": "user",
          "content": "Hello, how are you"
        }
      ]
    },
    "options_json": {},
    "response": "I'm an AI, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?",
    "response_json": {
      "id": "",
      "choices": [
        {
          "finish_reason": "stop",
          "index": 0,
          "message": {
            "content": "I'm an AI, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?",
            "role": "assistant"
          }
        }
      ],
      "created": 000000000,
      "model": "gpt-4-0613",
      "object": "chat.completion",
      "usage": {
        "completion_tokens": 29,
        "prompt_tokens": 12,
        "total_tokens": 41
      }
    },
    "conversation_id": "000000000",
    "duration_ms": 2188,
    "datetime_utc": "2024-01-31T21:48:45.963664",
    "conversation_name": "Hello, how are you",
    "conversation_model": "gpt-4"
 }

I'm running multiple concurrent requests and would like to the option to continue a given conversation by passing the conversation_id. To do that now, I need to get N logs (e.g, llm logs list --json -n 10) and determine which matches based on the response. This works okay, but is error prone and slow.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the chat command and compare its output with the existing llm logs list --json data. Trace how the chat response is logged, then define the verbose output around the logged object and its conversation_id. Done means an option returns structured log details that can be used to continue a conversation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.