EpistasisLab / EpistasisLab/motoro
Make sentence-transformers an optional extra: it pulls 4.5 GB of CUDA into every consumer
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2m
- Merged PRs (30d)
- 3
Description
## Problem
`sentence-transformers>=5.4.1` is an unconditional dependency (`pyproject.toml:36`).
On Linux it drags in the default PyPI `torch` wheel, which bundles the whole CUDA
stack. In a consumer's venv (ASAREE, `uv sync` against motoro v0.6.0):
| package | size |
|---|---|
| `nvidia/*` | 2.7 GB |
| `torch` 2.13.0 | 1.1 GB |
| `triton` | 689 MB |
| **subtotal** | **~4.5 GB** |
| whole venv | 5.5 GB |
So ~80% of the install is a GPU deep-learning runtime, pulled in transitively:
```
torch v2.13.0 └── sentence-transformers v5.6.1 └── motoro v0.6.0 └── asaree
```
## Why it is dead weight for a consumer that doesn't use local embeddings
The code already tolerates its absence — every path to it is opt-in:
- The import is **lazy**: `from sentence_transformers import SentenceTransformer`
lives inside `_get_local_model()` (`memory/embedding.py:71`), not at module
scope.
- `EmbeddingService` is constructed in exactly one place,
`MemoryService.__init__` (`services/memory_service.py:45`).
- `MemoryService` reaches the engine only as an **injected optional port** —
`memory_service: MemoryServicePort | None = None` (`engine/runtime.py:88`),
and every use is guarded (`runtime.py:402`,
`patterns/orchestrator.py:558`).
A consumer that never injects a memory service therefore never constructs
`EmbeddingService`, never calls `_get_local_model()`, and never imports torch.
ASAREE is exactly that consumer: it has **zero** references to
`MemoryService`/`EmbeddingService`/embeddings anywhere in `src/asaree`, and
none of its 769 tests import torch, transformers, or sentence-transformers.
It downloads 4.5 GB to import nothing.
Worth being precise about the flip side: `settings.embedding_model` defaults to
`sentence-transformers/BAAI/bge-base-en-v1.5` (`config.py:140`), so a consumer
that *does* wire up memory and doesn't override that setting genuinely needs
this dependency. The ask is to make it opt-in, not to drop it.
## Proposed change
Move it to an extra:
```toml
[project.optional-dependencies]
local-embeddings = ["sentence-transformers>=5.4.1"]
```
and turn the lazy import into a directive error, so the failure names the fix
instead of surfacing as a bare `ModuleNotFoundError` from inside the memory
subsystem:
```python
try:
from sentence_transformers import SentenceTransformer
except ImportError as exc: # pragma: no cover
raise RuntimeError(
f"embedding_model is a local sentence-transformers model ({model_name!r}), "
"but sentence-transformers is not installed. Either install "
"motoro[local-embeddings], or set embedding_model to a remote model "
"such as text-embedding-3-small."
) from exc
```
Motoro's own `dev` group should depend on `motoro[local-embeddings]` so the
local backend stays covered by CI, and the default `embedding_model` value
should carry a comment saying it requires the extra.
Follow-on question worth deciding at the same time: with the extra in place,
should the default stay local? A default that requires an extra the default
install doesn't include is a papercut either way — either the default becomes
remote, or `_get_local_model()`'s new error is the documented first-run
experience for anyone enabling memory.
### Alternative, for consumers who do want local embeddings
Pin the CPU-only wheel rather than the CUDA one — drops `nvidia/*` and
`triton` entirely and takes torch to ~200 MB:
```toml
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = { index = "pytorch-cpu" }
```
This is complementary, not a substitute: it belongs in whichever project
actually wants the local backend, and it is the wrong default for motoro
itself to impose on a GPU deployment.
## Impact
- **Consumer installs**: ASAREE's venv goes 5.5 GB → ~1 GB.
- **CI**: ASAREE has no CI yet and is about to get one; its full suite runs in
13s, so the dependency download would dominate the job by two orders of
magnitude. Fixing this is what makes a fast ASAREE CI possible.
- **Docker images**: same 4.5 GB comes off any consumer image that doesn't use
local embeddings.
Requires a release (v0.6.1 or v0.7.0) before ASAREE can benefit, since ASAREE
pins motoro by tag. Not urgent for the current release — ASAREE v0.5.0 ships
fine on v0.6.0 — but it is the single biggest install-size win available.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the dependency at pyproject.toml:36 and inspect the dev group, then read _get_local_model() in memory/embedding.py:71 and the default in config.py:140. Check how MemoryService is wired from services/memory_service.py:45 and guarded in engine/runtime.py:88 and patterns/orchestrator.py:558. Done means the extra and missing-dependency guidance are covered, local embeddings remain available in development, and consumers without memory avoid the large dependency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100