Python cannot use custom ollama model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10.5k
- Forks
- 1.2k
- Avg merge
- 4m
- Merged PRs (30d)
- 1
Description
I have created a custom model using the ollama create custom_model -f modelfile. The custom model is based on codellama. Some examples and context are provided in the modelfile. In the CLI interface, the custom model ran well, giving answer based on the examples.
However, when I was trying to use the custom model in python, using ollama.chat(model='custom_model. The same user prompt would give answer as if I am using the original codellama model with no training examples.
I have tried using a wrong model name (custommodel instead of custom_model), and it can be detected that no such model exists. So I think python is picking up the custom model, however, it seems like the context is left out. Also when I run ollama run custom_model, the provided context in the model file always being printed out. Does it mean that the model is being retrained from the original codellama model everytime when I run it? Instead of having the same trained custom model everytime I run the custom model?
It seems to me that all the few shot example provided in the modelfile used to train the custom_model is not provided to the custom model when using ollama.chat. The result seems to be just like using the base model that the custom model is trained on.
Is there any way to use the custom model appropriately in python, just like running it in CLI?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the difference between ollama run custom_model and Python's ollama.chat(model='custom_model'), using the Modelfile examples described in the report. Inspect the Python chat entry point and compare its request with the CLI behavior. Done means the custom model's Modelfile context is consistently honored through the Python API, or the limitation is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ollama, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100