meilisearch / meilisearch/meilisearch-java
[v1.14] Add composite embedders and pooling for huggingface embedders (experimental)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 245
- Forks
- 152
- PR merge metrics
- No merged PRs in 30d
Description
This issue follows the changes related to Meilisearch v1.14.0 [mega issue](https://github.com/meilisearch/integration-guides/issues/315).
## Description
Update embedders API to be compatible with Meilisearch v1.14.0
For more context, see the related issue in Meilisearch: https://github.com/meilisearch/meilisearch/issues/5343
## Tasks
- [ ] Add new `pooling` parameter for `huggingFace` embedders
- [ ] Add new `composite` source for embedders
For reference, here's how this works using [Meilisearch JS SDK](https://github.com/meilisearch/meilisearch-js):
```ts
client
.index('index_name')
.updateEmbedders({
embedder_name: {
source: "composite",
searchEmbedder: {
source: "huggingFace",
model:
"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
pooling: "useModel",
},
indexingEmbedder: {
source: "huggingFace",
model:
"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
documentTemplate: "{{doc.title}}",
pooling: "useModel",
documentTemplateMaxBytes: 500,
},
},
});
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the Java client's updateEmbedders API and compare its supported embedder configuration with the Meilisearch JS SDK example. Review the linked Meilisearch v1.14 and issue #5343 references; done means supporting the huggingFace pooling parameter and composite source, including the shown nested configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100