GoogleCloudPlatform / GoogleCloudPlatform/cloud-spanner-emulator
Feature Request: Support local embedding services for ML.PREDICT testing
- Dominant language
- C++
- Stars
- 334
- Forks
- 77
- Avg merge
- 8m
- Merged PRs (30d)
- 2
Description
## Feature Request: Local Embedding Service Support for ML.PREDICT
### Problem Statement
The Cloud Spanner Emulator currently generates single-element fingerprint hashes for ML.PREDICT instead of real embeddings. This causes validation failures when testing vector similarity search locally:
```
INVALID_ARGUMENT: Array column embeddings has 1 elements and is less than
the `vector_length` limit: 1024
```
Developers cannot effectively test ML.PREDICT and vector search features locally without connecting to external cloud services (Vertex AI, OpenAI), which requires:
- Network connectivity and API keys
- API costs for every test run
- Slower iteration cycles
### Proposed Solution
Add support for local/self-hosted embedding services in the emulator via a `local://` endpoint prefix convention:
```sql
CREATE MODEL text_embeddings
INPUT (content STRING(MAX))
OUTPUT (embedding ARRAY(vector_length=>1024))
REMOTE OPTIONS (
endpoint = 'local://localhost:8080/embed' -- Points to local service
);
```
When the emulator detects a `local://` endpoint, it would:
1. Make HTTP POST request to the local service with input text
2. Parse the embedding vector response
3. Return properly formatted ARRAY with correct dimensions
4. Validate dimensions match `vector_length` constraints
### Use Cases
**Development & Testing:**
- Test vector similarity search end-to-end without cloud dependencies
- Validate schema constraints (vector_length, distance functions, indexes)
- Run automated tests in CI/CD pipelines offline
**Model Flexibility:**
- Use open-source models (Sentence-Transformers, Qwen, BGE, etc.)
- Test custom fine-tuned embedding models
- Evaluate different embedding dimensions before production deployment
**Cost & Speed:**
- Zero API costs during development
- Faster iteration (no network latency to cloud services)
- Work offline or in restricted network environments
### Example Workflow
```sql
-- 1. Drop and recreate table with FLOAT64
CREATE TABLE documents (
doc_id STRING(MAX),
content STRING(MAX),
embedding ARRAY(vector_length=>1024)
) PRIMARY KEY (doc_id);
-- 2. Create model with FLOAT64 output
CREATE MODEL local_embeddings1
INPUT (content STRING(MAX))
OUTPUT (embedding ARRAY(vector_length=>1024))
REMOTE OPTIONS (endpoint = 'local://localhost:8080/embed'
);
-- 3. Now the INSERT should work
INSERT INTO documents (doc_id, content, embedding)
SELECT doc_id, content, embedding
FROM ML.PREDICT(MODEL local_embeddings1, (
SELECT "doc1" as doc_id, "Cloud Spanner is great" as content
));
```
### Benefits
1. **Improved Developer Experience**: Complete local testing of vector search features
2. **Cost Savings**: No API charges during development/testing phases
3. **Faster Development**: Immediate feedback without network round-trips
4. **CI/CD Integration**: Automated tests can run without external service dependencies
5. **Model Experimentation**: Easy comparison of different embedding models/dimensions
6. **Broader Adoption**: Lowers barrier to entry for vector search development
### Technical Considerations
**Protocol Compatibility:**
Support common embedding service APIs (HuggingFace Text Embeddings Inference, Ollama, etc.):
```json
Request: {"inputs": "text content", "truncate": true}
Response: [[0.334, -0.123, 0.456, ...]]
```
**Backward Compatibility:**
- Existing fingerprint-based behavior remains default
- Only activates with explicit `local://` endpoint
- No breaking changes to query engine or storage layer
- Works with both GoogleSQL and PostgreSQL dialects
### Implementation Reference
A working proof-of-concept implementation is available demonstrating the approach:
- Endpoint detection and routing in ModelEvaluator
- HTTP client for local service communication
- Configuration system and error handling
- Integration with existing ML.PREDICT architecture
Repository: https://github.com/nordanster/cloud-spanner-emulator/tree/feature/local-embeddings
Key changes:
- `backend/query/ml/model_evaluator.{h,cc}` - Endpoint routing logic
- `backend/query/ml/embedding_client.{h,cc}` - HTTP client (new)
- `common/config.{h,cc}` - Configuration flags
- `WORKSPACE` - HTTP library dependency (cpp-httplib)
### Community Impact
This feature would significantly benefit:
- **RAG Application Developers**: Building retrieval-augmented generation systems
- **ML Engineers**: Testing embedding models before production deployment
- **Enterprise Teams**: Using self-hosted models for data privacy/compliance
- **Educational Content**: Tutorials and workshops on Cloud Spanner vector search
- **Open Source Community**: Developers using local/open-source embedding models
### Priority Rationale
Vector embeddings and similarity search are increasingly important for AI/ML workloads. As Cloud Spanner's vector search capabilities mature, enabling effective local development becomes critical for adoption.
Currently, developers must choose between:
1. Testing with fake data (fingerprint hashes) that fail validation
2. Connecting to expensive cloud services for every test
3. Not testing locally at all
This feature eliminates that trade-off and aligns the emulator with modern AI development workflows.
---
**Additional Context:**
- Vector search is a key differentiator for Cloud Spanner in AI workloads
- Local embedding services are standard practice (Ollama, HuggingFace TEI, etc.)
- Similar pattern already exists for PostgreSQL dialect (PgPredict with local endpoints)
- No changes required to production Cloud Spanner (emulator-only feature)
Thank you for considering this feature request!
Contributor guide
Assessment
This issue has not been assessed yet.