meilisearch / meilisearch/meilisearch-java

NDJSON/CSV methods to add and update documents

Open
#256 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement good first issue
Dominant language
Java
Stars
245
Forks
152
PR merge metrics
No merged PRs in 30d

Description

⚠️ This issue is generated, it means the nameing might be done differently in this package (ex: `add_documents_json` instead of `addDocumentsJson`). Keep the already existing way of naming in this package to stay idiomatic with the language and this repository.

📣 We strongly recommend doing multiple PRs to solve all the points of this issue

[MeiliSearch v0.23.0](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.23.1) introduces two changes:
- new valid formats to push data files, additionally to the JSON format: CSV and NDJSON formats.
- it enforces the `Content-type` header for every route requiring a payload (`POST` and `PUT` routes)

---

Here are the expected changes to completely close the issue:

- [ ] Currently, the SDKs always send `Content-Type: application/json` to every request. Only the `POST` and `PUT` requests should send the `Content-Type: application/json` and not the `DELETE` and `GET` ones.

- [ ] Add the following methods and 🔥 **the associated tests** 🔥 to ADD the documents. Depending on the format type (`csv` or `ndjson`) the SDK should send `Content-Type: application/x-dnjson` or `Content-Type: text/csv`)
- [ ] `addDocumentsJson(string docs, string primaryKey)`
- [ ] `addDocumentsCsv(string docs, string primaryKey)`
- [ ] `addDocumentsCsvInBatches(string docs, int batchSize, string primaryKey)`
- [ ] `addDocumentsNdjson(string docs, string primaryKey)`
- [ ] `addDocumentsNdjsonInBatches(string docs, int batchSize, string primaryKey)`

- [ ] Add the following methods and 🔥 **the associated tests** 🔥 to UPDATE the documents. Depending on the format type (`csv` or `ndjson`) the SDK should send `Content-Type: application/x-dnjson` or `Content-Type: text/csv`)
- [ ] `updateDocumentsJson(string docs, string primaryKey)`
- [ ] `updateDocumentsCsv(string docs, string primaryKey)`
- [ ] `updateDocumentsCsvInBatches(string docs, int batchSize, string primaryKey)`
- [ ] `updateDocumentsNdjson(string docs, string primaryKey)`
- [ ] `updateDocumentsNdjsonInBatches(string docs, int batchSize, string primaryKey)`

`docs` are the documents sent as `String`
`primaryKey` is [the primary key of the index](https://docs.meilisearch.com/learn/core_concepts/indexes.html#primary-key)
`batchSize` is the size of the batch. Example: you can send 2000 documents in raw String in `docs` and ask for a `batchSize` of 1000, so your documents will be sent to MeiliSearch in two batches.

Example of PRs:
- in PHP SDK: https://github.com/meilisearch/meilisearch-php/pull/235
- in Python SDK: https://github.com/meilisearch/meilisearch-python/pull/329

----------

Related to: https://github.com/meilisearch/integration-guides/issues/146

If this issue is partially/completely implemented, feel free to let us know.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing document add and update methods and their associated tests, then inspect how request headers are set for POST, PUT, GET, and DELETE calls. Done means the requested JSON, CSV, and NDJSON methods, batching behavior, content types, and tests are implemented without changing the repository's naming conventions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.