frappe / frappe/flow_client

Flow Provider with Ollama: /api/generate returns empty AI Chat response, /api/chat fails Test Connection with 404

Open
#84 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
69
Forks
28
PR merge metrics
No merged PRs in 30d

Description

### Environment

* Flow Provider: Ollama
* Ollama Model: `gpt-oss:20b`

### Issue

I'm experiencing two different issues when configuring the Ollama Flow Provider.

#### Case 1: Using `/api/generate`

**Endpoint**

```text
http://[OLLAMA-SERVER]/api/generate
```

**Behavior**

* ✅ Test Connection succeeds.
* ❌ AI Chat returns an empty response (no output).

---

#### Case 2: Using `/api/chat`

**Endpoint**

```text
http://[OLLAMA-SERVER]/api/chat
```

**Behavior**

* ❌ Test Connection fails with:

```text
litellm.APIConnectionError: OllamaException - 404 page not found
```

---

### Verification

The same `/api/chat` endpoint works correctly when tested directly using Postman/cURL.

```bash
curl --location 'http://[OLLAMA-SERVER]/api/chat' \
--header 'Content-Type: application/json' \
--data '{
"model": "gpt-oss:20b",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Hello!"
}
],
"stream": false
}'
```

**Response**

```json
{
"model": "gpt-oss:20b",
"created_at": "2026-07-22T08:17:24.694493298Z",
"message": {
"role": "assistant",
"content": "Hello! 👋 How can I help you today?",
"thinking": "The user says \"Hello!\". We should respond with a friendly greeting."
},
"done": true,
"done_reason": "stop",
"total_duration": 837122888,
"load_duration": 282868911,
"prompt_eval_count": 83,
"prompt_eval_duration": 40953169,
"eval_count": 59,
"eval_duration": 450009439
}
```

### Expected Behavior

* When using `http://[OLLAMA-SERVER]/api/chat`, the Test Connection should succeed.
* AI Chat should return the assistant response correctly.
* If `/api/generate` is supported, AI Chat should also produce the generated response instead of returning an empty message.

### Actual Behavior

| Endpoint | Test Connection | AI Chat |
| --------------- | -------------------------------------------------------------------- | ---------------- |
| `/api/generate` | ✅ Success | ❌ Empty response |
| `/api/chat` | ❌ `litellm.APIConnectionError: OllamaException - 404 page not found` | Cannot test |

### Additional Notes

* The Ollama server is reachable.
* The model is loaded and responds correctly.
* Direct requests to the Ollama API using Postman/cURL work as expected.
* The issue appears to be related to how the Flow Provider (or LiteLLM integration) is constructing or validating requests for Ollama endpoints, rather than an issue with the Ollama server itself.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the Flow Provider's Ollama integration and its LiteLLM request handling for /api/generate and /api/chat. Reproduce both cases with the supplied model and curl request, then verify that Test Connection succeeds for /api/chat and that AI Chat returns the assistant content for both supported endpoints.

Written by the indexing model from the issue text.

Assessment

Tech stack
ollama, python
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.