E3SM-Project / E3SM-Project/simboard

[AI]: Run Ollama as a NERSC Spin service for assistant summaries

Open
#200 0 comments 0 reactions 0 assignees View on GitHub
type: ai
Dominant language
Python
Stars
1
Forks
2
Avg merge
25m
Merged PRs (30d)
14

Description

## DevOps Task

### Task Details
Run Ollama as a dedicated NERSC Spin service that SimBoard backend can call for assistant summaries.

Current repo state already treats Ollama as a runtime dependency configured through `ASSISTANT_LLM_PROVIDER`, `ASSISTANT_OLLAMA_BASE_URL`, and `ASSISTANT_OLLAMA_MODEL`, but Spin deployment docs do not define an Ollama workload. The current `http://localhost:11434` example only works when Ollama runs in the same pod/network namespace as the backend. On Spin, we should support a real service URL and an operator-owned deployment path.

Goals:
- deploy Ollama as a Spin workload/service instead of baking Ollama and models into the SimBoard backend image
- expose a stable internal service URL for backend access
- define how model weights are provisioned and persisted across pod restarts
- document the required backend env changes for Spin
- validate at least one supported model path for SimBoard summaries

Suggested starting model path:
- fast/dev: `gemma4:e4b`
- preferred quality target: `gemma4:26b`

Non-goals:
- changing SimBoard API contracts
- adding frontend provider controls
- baking Ollama runtime or model blobs into SimBoard application images

### Steps to Complete
1. Decide deployment shape for Spin:
- dedicated Ollama Deployment + ClusterIP Service
- or sidecar pattern if there is a strong operational reason
2. Define workload requirements:
- container image
- CPU/memory sizing
- storage requirements for model cache
- whether PVC, NGF, or CFS-backed storage is needed
- security context and any NERSC-specific constraints
3. Provision internal service endpoint and update backend config guidance:
- replace `localhost` guidance with service DNS for Spin deployments
- set `ASSISTANT_OLLAMA_BASE_URL` to the Ollama service URL
- set `ASSISTANT_OLLAMA_MODEL` to supported model tag
4. Define model bootstrap workflow:
- how models are pulled initially
- how upgrades/changes to model tag are handled
- how to avoid repeated large downloads on restart
5. Validate end-to-end from SimBoard backend to Ollama on Spin:
- health/connectivity check from backend pod
- successful assistant summary generation using Ollama
- failure behavior when Ollama is unavailable
6. Update deployment/operator docs:
- `docs/deploy/spin.md`
- any CI/CD or runbook docs that should reference the service

### Additional Notes
Relevant repo context:
- `docs/deploy/spin.md` documents backend assistant env vars but not an Ollama workload.
- `backend/app/core/config.py` already supports `ASSISTANT_OLLAMA_BASE_URL` and `ASSISTANT_OLLAMA_MODEL`.
- `docs/ai-196-local-llm/README.md` explicitly keeps shipping Ollama container assets out of scope for repo implementation, which supports treating this as operator/devops work.

Acceptance criteria:
- Ollama runs as a documented Spin workload/service.
- Backend can reach Ollama through a non-`localhost` Spin service URL.
- At least one supported model is successfully exercised from SimBoard on Spin.
- Operator docs explain service deployment, storage/model persistence, and required backend env vars.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.