anomalyco / anomalyco/opencode

[FEATURE]: Document that CPU-only local inference is unsuitable for agent workflows

Open
#47,953 0 comments 0 reactions 1 assignee View on GitHub

@rekram1-node is already working on this.

Since Sep 8, 2026.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Feature hasn been suggested before.

  • I have verified this feature I am about to request has not been suggested before.

Describe the enhancement you want to request

When users configure a local model (Ollama, llama.cpp, etc.) as their opencode provider, there is no guidance about the performance characteristics expected for agent use. Users may spend weeks tuning a CPU-only setup before realizing it is fundamentally too slow.

What I want: Documentation (or a startup check) that warns users when:

  • A local provider is configured with CPU-only inference (no GPU acceleration)
  • Prefill times exceed a reasonable threshold (e.g. >10s)
  • The model has non-disableable thinking/reasoning that adds latency

Concrete suggestion: A note in the docs or a dms doctor-style check:

"CPU-only local models (Ollama without GPU) typically have 30-50s prefill times, making them impractical for agent workflows. Consider using a remote provider or ensuring GPU acceleration (CUDA/ROCm) is configured."

Why it matters: I tested Qwen 30B MoE and Qwen 7B on CPU (Ryzen 7 3800X, 32GB DDR4) — both had 30-50s prefill, and the 30B had non-disableable thinking. After removing Ollama entirely and switching to remote inference, the workflow became usable. This guidance would save others the same trial-and-error.

Environment: CachyOS (Arch), no GPU for local inference (AMD RX 6750 XT not used for LLM), Ollama removed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.