meilisearch / meilisearch/meilisearch-java

[v1.14] Add composite embedders and pooling for huggingface embedders (experimental)

Open
#837 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement good first issue
Dominant language
Java
Stars
245
Forks
152
PR merge metrics
No merged PRs in 30d

Description

This issue follows the changes related to Meilisearch v1.14.0 [mega issue](https://github.com/meilisearch/integration-guides/issues/315).

## Description

Update embedders API to be compatible with Meilisearch v1.14.0

For more context, see the related issue in Meilisearch: https://github.com/meilisearch/meilisearch/issues/5343

## Tasks

- [ ] Add new `pooling` parameter for `huggingFace` embedders
- [ ] Add new `composite` source for embedders

For reference, here's how this works using [Meilisearch JS SDK](https://github.com/meilisearch/meilisearch-js):

```ts
client
.index('index_name')
.updateEmbedders({
embedder_name: {
source: "composite",
searchEmbedder: {
source: "huggingFace",
model:
"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
pooling: "useModel",
},
indexingEmbedder: {
source: "huggingFace",
model:
"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
documentTemplate: "{{doc.title}}",
pooling: "useModel",
documentTemplateMaxBytes: 500,
},
},
});
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the Java client's updateEmbedders API and compare its supported embedder configuration with the Meilisearch JS SDK example. Review the linked Meilisearch v1.14 and issue #5343 references; done means supporting the huggingFace pooling parameter and composite source, including the shown nested configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.