simonw / simonw/llm

DB logging crashes when response is empty

Open
#960 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

logging
Dominant language
Python
Stars
12.5k
Forks
991
Avg merge
3d 13h
Merged PRs (30d)
10

Description

When adding some features to the llm-grok plugin, I noticed an issue where if the max tokens is sufficiently low that the model never finishes its reasoning phase and therefore the response is empty, the DB logging will crash.

❯ llm -m grok-3-mini-latest "what color is the sun?" -o max_completion_tokens 5

Traceback (most recent call last):
  File "/Users/daniel/.local/bin/llm", line 10, in <module>
    sys.exit(cli())
             ~~~^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm/cli.py", line 781, in prompt
    response.log_to_db(db)
    ~~~~~~~~~~~~~~~~~~^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm/models.py", line 456, in log_to_db
    "prompt_json": condense_json(self._prompt_json, replacements),
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/condense_json/__init__.py", line 81, in condense_json
    return process(obj)
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/condense_json/__init__.py", line 53, in process
    return {key: process(val) for key, val in value.items()}
                 ~~~~~~~^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/condense_json/__init__.py", line 55, in process
    return [process(item) for item in value]
            ~~~~~~~^^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/condense_json/__init__.py", line 53, in process
    return {key: process(val) for key, val in value.items()}
                 ~~~~~~~^^^^^
  File "/Users/daniel/.local/share/uv/tools/llm/lib/python3.13/site-packages/condense_json/__init__.py", line 67, in process
    replacement_id: str = substr_to_id[matched_text]
                          ~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: ''

This raises a few broader questions of how to control reasoning vs visible tokens, which each provider will have its own approach for. Should reasoning tokens be visible to the user? (Should llm have a flag for that? I am not aware of one currently). If reasoning tokens are not shown to the user, but the max tokens are sufficiently low such that the response is empty, how should that be indicated to the user?

But, more broadly, the DB logging issue seems like a rough edge case.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with llm/models.py in Response.log_to_db(), then reproduce the empty-response case using the reported llm command and max_completion_tokens setting. Trace the condense_json call shown in the traceback and verify that database logging completes without crashing when the response text is empty.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.