Mechanism for recording a different model ID from the one requested
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.5k
- Forks
- 998
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 10
Description
Inspired by this comment in https://simonwillison.net/2024/Apr/22/llama-3/#local-llama-3-70b-instruct-with-llamafile
One warning about this approach: if you use LLM like this then every prompt you run through llamafile will be stored under the same model name in your SQLite logs, even if you try out different llamafile models at different times. You could work around this by registering them with different model_id values in the YAML file.
This made me think: there are other cases where the model that executed the prompt may not be the exact model you specified. Calling OpenAI and asking for gpt-4-turbo and getting a response run against the more recent dated model for example.
It would be useful to have a mechanism for recording the actual model used (as opposed to the model_id requested) in the SQLite logs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the requested model_id reaches the SQLite logging path, using the linked logging documentation for context. Determine where the executed model identity could be recorded separately; the issue names no source files or tests, so repository exploration is required to establish the implementation scope and verification steps.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100