kubeflow / kubeflow/docs-agent
kagent-feast-mcp: runtime monkey-patch of Feast VARCHAR limit is fragile and breaks across versions
- Dominant language
- Python
- Stars
- 42
- Forks
- 111
- Avg merge
- 6d 23m
- Merged PRs (30d)
- 2
Description
## Summary
The `kagent-feast-mcp` ingestion pipeline uses a runtime monkey-patch to work around Feast's hardcoded `max_length=512` VARCHAR limit in its Milvus online store integration. This approach modifies installed library source code on disk during pipeline execution, which is fragile and opaque.
## Location
`kagent-feast-mcp/pipelines/kubeflow-pipeline.py` in the `store_via_feast` component:
```python
# Patch Feast VARCHAR limit (hardcoded 512 -> 4096) and reload module
import importlib
import feast.infra.online_stores.milvus_online_store.milvus as milvus_mod
src_file = inspect.getfile(milvus_mod)
with open(src_file, "r") as f:
content = f.read()
if "max_length=512" in content:
with open(src_file, "w") as f:
f.write(content.replace("max_length=512", "max_length=4096"))
```
## Problem
1. **Fragile across Feast versions**: If Feast renames `max_length` or changes the file structure, the string replacement silently fails and data gets silently truncated at 512 chars.
2. **Opaque in debugging**: Modifying installed library source at runtime means `pip show feast` still reports the unpatched version. Anyone debugging the pipeline won't see the patch unless they read this specific component's source.
3. **Unnecessary for the use case**: The `kagent-feast-mcp/mcp-server/server.py` already uses `pymilvus` directly (via `MilvusClient`) to query the same Milvus instance without needing Feast at all. This is simpler, thread-safe, and avoids the VARCHAR limitation entirely.
4. **Drop-and-recreate pattern**: The `store_via_feast` component also drops the entire Milvus collection before reinserting (`utility.drop_collection`). If the pipeline fails mid-ingestion (e.g., GitHub API rate limit), the collection is empty and the agent returns nothing. This compounds the Feast layer's fragility.
## Suggested Direction
This supports the ADR-008 direction (pymilvus over Feast) that's being discussed. The MCP server already demonstrates the cleaner pattern:
- **Ingestion**: Use `pymilvus` directly with `MilvusClient.upsert()` keyed on `file_unique_id` + `chunk_index` for idempotent writes (no drop-and-recreate).
- **Serving**: Already done -- `kagent-feast-mcp/mcp-server/server.py` uses `MilvusClient` directly.
- **Mark Feast pipelines as legacy**: Keep them for reference but document that `pymilvus` direct is the production path.
PR freeze is on so just flagging this for architecture discussion. Happy to contribute to the ADR-008 doc.
Related: #181 (content truncation), #63 (model reload), #28 (connection pooling), #72 (codebase cleanup)
Contributor guide
Research direction
Start with kagent-feast-mcp/pipelines/kubeflow-pipeline.py, especially store_via_feast, and compare it with kagent-feast-mcp/mcp-server/server.py's direct MilvusClient usage. Read the ADR-008 discussion before deciding the ingestion scope. Done means the runtime Feast source patch and destructive drop-and-recreate path are replaced or clearly marked legacy, with idempotent direct writes documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering, databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100