meilisearch / meilisearch/meilisearch-java
NDJSON/CSV methods to add and update documents
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 245
- Forks
- 152
- PR merge metrics
- No merged PRs in 30d
Description
⚠️ This issue is generated, it means the nameing might be done differently in this package (ex: `add_documents_json` instead of `addDocumentsJson`). Keep the already existing way of naming in this package to stay idiomatic with the language and this repository.
📣 We strongly recommend doing multiple PRs to solve all the points of this issue
[MeiliSearch v0.23.0](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.23.1) introduces two changes:
- new valid formats to push data files, additionally to the JSON format: CSV and NDJSON formats.
- it enforces the `Content-type` header for every route requiring a payload (`POST` and `PUT` routes)
---
Here are the expected changes to completely close the issue:
- [ ] Currently, the SDKs always send `Content-Type: application/json` to every request. Only the `POST` and `PUT` requests should send the `Content-Type: application/json` and not the `DELETE` and `GET` ones.
- [ ] Add the following methods and 🔥 **the associated tests** 🔥 to ADD the documents. Depending on the format type (`csv` or `ndjson`) the SDK should send `Content-Type: application/x-dnjson` or `Content-Type: text/csv`)
- [ ] `addDocumentsJson(string docs, string primaryKey)`
- [ ] `addDocumentsCsv(string docs, string primaryKey)`
- [ ] `addDocumentsCsvInBatches(string docs, int batchSize, string primaryKey)`
- [ ] `addDocumentsNdjson(string docs, string primaryKey)`
- [ ] `addDocumentsNdjsonInBatches(string docs, int batchSize, string primaryKey)`
- [ ] Add the following methods and 🔥 **the associated tests** 🔥 to UPDATE the documents. Depending on the format type (`csv` or `ndjson`) the SDK should send `Content-Type: application/x-dnjson` or `Content-Type: text/csv`)
- [ ] `updateDocumentsJson(string docs, string primaryKey)`
- [ ] `updateDocumentsCsv(string docs, string primaryKey)`
- [ ] `updateDocumentsCsvInBatches(string docs, int batchSize, string primaryKey)`
- [ ] `updateDocumentsNdjson(string docs, string primaryKey)`
- [ ] `updateDocumentsNdjsonInBatches(string docs, int batchSize, string primaryKey)`
`docs` are the documents sent as `String`
`primaryKey` is [the primary key of the index](https://docs.meilisearch.com/learn/core_concepts/indexes.html#primary-key)
`batchSize` is the size of the batch. Example: you can send 2000 documents in raw String in `docs` and ask for a `batchSize` of 1000, so your documents will be sent to MeiliSearch in two batches.
Example of PRs:
- in PHP SDK: https://github.com/meilisearch/meilisearch-php/pull/235
- in Python SDK: https://github.com/meilisearch/meilisearch-python/pull/329
----------
Related to: https://github.com/meilisearch/integration-guides/issues/146
If this issue is partially/completely implemented, feel free to let us know.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing document add and update methods and their associated tests, then inspect how request headers are set for POST, PUT, GET, and DELETE calls. Done means the requested JSON, CSV, and NDJSON methods, batching behavior, content types, and tests are implemented without changing the repository's naming conventions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100