kubeflow / kubeflow/docs-agent
pipelines: incremental-pipeline.py and Dockerfile.pipeline still pull full CUDA torch on CPU-only infra
- Dominant language
- Python
- Stars
- 42
- Forks
- 111
- Avg merge
- 6d 23m
- Merged PRs (30d)
- 2
Description
Same root cause as #198, just living in a different file that #199 doesn't touch.
Two spots in `docs-agent-mcp/pipelines/` still install `sentence-transformers`/torch the plain way, which means pip grabs the default CUDA build even though nothing here has a GPU:
- [`incremental-pipeline.py`'s `chunk_and_embed_incremental` component](https://github.com/kubeflow/docs-agent/blob/b93d054c571290f4d81d0b20da237367f69dabf2/docs-agent-mcp/pipelines/incremental-pipeline.py#L156-L181) runs on the full `pytorch/pytorch:...-cuda12.1-cudnn8-runtime` base image and loads the model in-process, then falls back to CPU anyway. This is the live one - it actually ships a multi-GB image every time it runs.
- [`Dockerfile.pipeline`](https://github.com/kubeflow/docs-agent/blob/b93d054c571290f4d81d0b20da237367f69dabf2/docs-agent-mcp/pipelines/Dockerfile.pipeline#L4-L10) has the same issue, just not currently built by any workflow.
What's odd is the other three pipelines already got this right [`kubeflow-pipeline.py`'s `chunk_and_embed`](https://github.com/kubeflow/docs-agent/blob/b93d054c571290f4d81d0b20da237367f69dabf2/docs-agent-mcp/pipelines/kubeflow-pipeline.py#L276-L297) runs on plain `python:3.11-slim` and just calls the shared [TEI embedding service](https://github.com/kubeflow/docs-agent/blob/b93d054c571290f4d81d0b20da237367f69dabf2/docs-agent-mcp/pipelines/utils.py#L93) over HTTP instead of loading a model locally. `code-pipeline.py` and `issues-pipeline.py` do the same. `incremental-pipeline.py` just never got migrated.
Also worth flagging: #199's fix doesn't actually work. Putting `--index-url` on the same line as `torch` in a requirements file is silently ignored by pip I tested it, still pulls the full ~500MB build. It only works as its own line, and then it applies to every package below it, which breaks installing anything not on PyTorch's index. That's a separate problem from this one, but it means #199 shouldn't be merged as a fix for either.
**Proposed fix:** port `chunk_and_embed_incremental` onto the same TEI-service pattern the other three pipelines use drops the CUDA image and all the local model deps entirely. For `Dockerfile.pipeline`, either split the pip install so the CPU index only scopes to torch, or drop the file if `incremental-pipeline.py` is getting retired anyway.
Contributor guide
Assessment
This issue has not been assessed yet.