spring-projects / spring-projects/spring-ai
OpenSearchVectorStore Does Not Fully Apply Mapping Configuration During Index Initialization
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.5k
- Forks
- 2.9k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 6
Description
When using Spring AI's (version: 1.0.0-M6) OpenSearchVectorStore, the custom mapping configuration is not fully applied during index initialization. Specifically:
loadCustomMapping:
{
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"content": {
"type": "text",
"analyzer": "standard"
},
"metadata": {
"type": "object"
},
"embedding": {
"type": "knn_vector",
"dimension": 1536,
"method": {
"name": "hnsw",
"space_type": "cosinesimil",
"engine": "lucene"
}
}
}
},
"settings": {
"index": {
"knn": true,
"number_of_shards": 1,
"number_of_replicas": 0
}
}
}
@Bean
public VectorStore vectorStore(OpenSearchClient openSearchClient, EmbeddingModel embeddingModel) {
return OpenSearchVectorStore.builder(openSearchClient, embeddingModel)
.index(indexName)
.mappingJson(loadCustomMapping())
.similarityFunction(similarityFunction)
.initializeSchema(initializeSchema)
.batchingStrategy(new TokenCountBatchingStrategy())
.build();
}
- During initial index creation, only basic settings are applied:
"settings": {
"index": {
"replication": {
"type": "DOCUMENT"
},
"number_of_shards": "1",
"provided_name": "mem,
"knn": "true",
"creation_date": "1746330816561",
"number_of_replicas": "1",
"uuid": "43LTNIlUT8Stt9dADOyqmA",
"version": {
"created": "136408027"
}
}
} - After inserting the first document, the
embeddingfield is automatically inferred asfloattype, instead of theknn_vectortype specified in our mapping.json:
"embedding": {
"type": "float"
}
- Our expected configuration (mapping.json) is:
{
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"content": {
"type": "text",
"analyzer": "standard"
},
"metadata": {
"type": "object"
},
"embedding": {
"type": "knn_vector",
"dimension": 1536,
"method": {
"name": "hnsw",
"space_type": "cosinesimil",
"engine": "lucene"
}
}
}
},
"settings": {
"index": {
"knn": true,
"number_of_shards": 1,
"number_of_replicas": 0
}
}
}
The mapping.json configuration should be fully applied when creating the index, specifically:
- The
embeddingfield should be created asknn_vectortype - All KNN-related settings should be applied
- Shards and replicas should be set according to the configuration
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the OpenSearchVectorStore entry point and trace how mappingJson is handled during index initialization. Compare the requested mappings and settings with the created index and first document behavior. Done means the configured knn_vector mapping, KNN settings, shard count, and replica count are applied when the index is created.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100