elastic / elastic/elasticsearch-py
Add option to `bulk` indexer to automatically add nested mapping where appropriate
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 1.2k
- Avg merge
- 17h 17m
- Merged PRs (30d)
- 34
Description
I am using `bulk` to index this JSON dict:
```
{
"project.nested.repeated.addresses":[
{
"status":"current",
"city":"New York",
"zip":33333,
"state":"NY",
"address":"789 Any Avenue",
"numberOfYears":2
},
{
"status":"previous",
"city":"Hoboken",
"zip":44444,
"state":"NJ",
"address":"321 Main Street",
"numberOfYears":3
}
],
"project.nested.repeated.first_name":"Jane",
"project.nested.repeated.last_name":"Doe"
}
```
I would like the addresses field to be [nested in the Elasticsearch sense](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html).
I have code that [manually crawls the schema and adds nested mappings](https://github.com/DataBiosphere/data-explorer-indexers/blob/e0aace1c166aa9c7eb97e87d8ed791f0d5630fb8/bigquery/indexer.py#L96).
It would be nice if I didn't have to manually put `nested` everywhere. This seems like a common use case:
If there is a nested JSON field, make the corresponding Elasticsearch field nested.
Contributor guide
Research direction
Start by reviewing the bulk indexer implementation and the schema-crawling example in bigquery/indexer.py around the linked lines. Define how nested JSON fields should be detected and mapped, then add coverage showing that bulk indexing automatically creates Elasticsearch nested mappings where appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, python
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100