Inconsistent `prompt_eval_count` for Large Prompts in Ollama Python Library

What is the issue?

Inconsistent prompt_eval_count for Large Prompts in Ollama Python Library

For larger prompts, when using the Ollama Python library with the llama3.1:8b-instruct-fp16 model, the prompt_eval_count remains constant at fixed value (1026) tokens, even when the input prompt size varies significantly. This behavior is observed when using the ollama.chat() method.

def classify_incident(row):
    full_prompt = (
        prompt_template + 
        row['user_message'] 
    )

    response = ollama.chat(model=model, options={'temperature' : 0.01}, messages=[
            {
            'role': 'user',
            'content': full_prompt
            }
            ])
    total_token = (response['prompt_eval_count'], response['eval_count'], 
                   response['prompt_eval_count'] + response['eval_count'])
    
    print(f'Tokens: {total_token}\n'
          f'Total_prompt_length: {len(full_prompt)}\n'
          f'{"=" * 50}\n')

Sample output:

Tokens: (1026, 15, 1041)
Total_prompt_length: 57788

Tokens: (1026, 20, 1046)
Total_prompt_length: 57172

Tokens: (1026, 18, 1044)
Total_prompt_length: 57744

Current Behavior

• prompt_eval_count consistently returns same value (1026), regardless of the actual prompt length.
• eval_count (output tokens) varies as expected. (this might also give fixed value once larger text is generated )

Expected Behavior

• prompt_eval_count should accurately reflect the number of tokens in the input prompt.
• The value should change dynamically based on the input size and content.

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.3.9

Start by reproducing the issue through the ollama.chat() entry point with prompts of different lengths using the llama3.1:8b-instruct-fp16 model. Compare prompt_eval_count with the input sizes and confirm that the reported count changes accurately for large prompts; the issue provides no file or test location.