EpistasisLab / EpistasisLab/motoro

Make sentence-transformers an optional extra: it pulls 4.5 GB of CUDA into every consumer

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
0
Forks
0
Avg merge
2m
Merged PRs (30d)
3

Description

## Problem

`sentence-transformers>=5.4.1` is an unconditional dependency (`pyproject.toml:36`).
On Linux it drags in the default PyPI `torch` wheel, which bundles the whole CUDA
stack. In a consumer's venv (ASAREE, `uv sync` against motoro v0.6.0):

| package | size |
|---|---|
| `nvidia/*` | 2.7 GB |
| `torch` 2.13.0 | 1.1 GB |
| `triton` | 689 MB |
| **subtotal** | **~4.5 GB** |
| whole venv | 5.5 GB |

So ~80% of the install is a GPU deep-learning runtime, pulled in transitively:

```
torch v2.13.0 └── sentence-transformers v5.6.1 └── motoro v0.6.0 └── asaree
```

## Why it is dead weight for a consumer that doesn't use local embeddings

The code already tolerates its absence — every path to it is opt-in:

- The import is **lazy**: `from sentence_transformers import SentenceTransformer`
lives inside `_get_local_model()` (`memory/embedding.py:71`), not at module
scope.
- `EmbeddingService` is constructed in exactly one place,
`MemoryService.__init__` (`services/memory_service.py:45`).
- `MemoryService` reaches the engine only as an **injected optional port** —
`memory_service: MemoryServicePort | None = None` (`engine/runtime.py:88`),
and every use is guarded (`runtime.py:402`,
`patterns/orchestrator.py:558`).

A consumer that never injects a memory service therefore never constructs
`EmbeddingService`, never calls `_get_local_model()`, and never imports torch.
ASAREE is exactly that consumer: it has **zero** references to
`MemoryService`/`EmbeddingService`/embeddings anywhere in `src/asaree`, and
none of its 769 tests import torch, transformers, or sentence-transformers.
It downloads 4.5 GB to import nothing.

Worth being precise about the flip side: `settings.embedding_model` defaults to
`sentence-transformers/BAAI/bge-base-en-v1.5` (`config.py:140`), so a consumer
that *does* wire up memory and doesn't override that setting genuinely needs
this dependency. The ask is to make it opt-in, not to drop it.

## Proposed change

Move it to an extra:

```toml
[project.optional-dependencies]
local-embeddings = ["sentence-transformers>=5.4.1"]
```

and turn the lazy import into a directive error, so the failure names the fix
instead of surfacing as a bare `ModuleNotFoundError` from inside the memory
subsystem:

```python
try:
from sentence_transformers import SentenceTransformer
except ImportError as exc: # pragma: no cover
raise RuntimeError(
f"embedding_model is a local sentence-transformers model ({model_name!r}), "
"but sentence-transformers is not installed. Either install "
"motoro[local-embeddings], or set embedding_model to a remote model "
"such as text-embedding-3-small."
) from exc
```

Motoro's own `dev` group should depend on `motoro[local-embeddings]` so the
local backend stays covered by CI, and the default `embedding_model` value
should carry a comment saying it requires the extra.

Follow-on question worth deciding at the same time: with the extra in place,
should the default stay local? A default that requires an extra the default
install doesn't include is a papercut either way — either the default becomes
remote, or `_get_local_model()`'s new error is the documented first-run
experience for anyone enabling memory.

### Alternative, for consumers who do want local embeddings

Pin the CPU-only wheel rather than the CUDA one — drops `nvidia/*` and
`triton` entirely and takes torch to ~200 MB:

```toml
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[tool.uv.sources]
torch = { index = "pytorch-cpu" }
```

This is complementary, not a substitute: it belongs in whichever project
actually wants the local backend, and it is the wrong default for motoro
itself to impose on a GPU deployment.

## Impact

- **Consumer installs**: ASAREE's venv goes 5.5 GB → ~1 GB.
- **CI**: ASAREE has no CI yet and is about to get one; its full suite runs in
13s, so the dependency download would dominate the job by two orders of
magnitude. Fixing this is what makes a fast ASAREE CI possible.
- **Docker images**: same 4.5 GB comes off any consumer image that doesn't use
local embeddings.

Requires a release (v0.6.1 or v0.7.0) before ASAREE can benefit, since ASAREE
pins motoro by tag. Not urgent for the current release — ASAREE v0.5.0 ships
fine on v0.6.0 — but it is the single biggest install-size win available.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the dependency at pyproject.toml:36 and inspect the dev group, then read _get_local_model() in memory/embedding.py:71 and the default in config.py:140. Check how MemoryService is wired from services/memory_service.py:45 and guarded in engine/runtime.py:88 and patterns/orchestrator.py:558. Done means the extra and missing-dependency guidance are covered, local embeddings remain available in development, and consumers without memory avoid the large dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.