Rethink model.key= pattern to better support shared instances
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.5k
- Forks
- 991
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 10
Description
No wait! This won't work, because the model instance is shared across multiple calls - so one user setting the key correctly could leak through to another user.
Originally posted by @simonw in https://github.com/simonw/datasette-llm-embed/issues/3#issuecomment-1751915001
Noticed this while trying to add an API key feature to the datasette-llm-embed plugin.
Since a single embedding model instance is created and then shared by all code that calls llm.get_embedding_model(model_id) - which is good for in-memory models - code that sets model.key = ... is not compatible with shared usage environments such as Datasette plugins where we don't want keys to leak to other users.
So need to reconsider that model.key = ... pattern entirely.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing get_embedding_model(model_id) and the shared model.key usage described in the issue. Define a design where API keys supplied by one caller cannot leak through a shared embedding model instance, then verify the behavior across multiple callers and the Datasette plugin use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100