django-haystack / django-haystack/django-haystack

Elastic Search backend tries to update mapping for each request

Open
#1,027 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

backend : elasticsearch needs review
Dominant language
Python
Stars
3.7k
Forks
1.3k
Avg merge
3h 18m
Merged PRs (30d)
3

Description

  • Haystack 2.1.1-dev
  • Elastic Search 1.0.1

Elastic Search returns unexpected format of mapping.
Example of dataset from get_mapping call (after rebuild_index):

{u'modelresult': {u'_boost': {u'name': u'boost', u'null_value': 1.0}, u'properties': {u'django_ct': {u'include_in_all': False, u'index': u'not_analyzed', u'type': u'string'}, u'text': {u'type': u'string', u'analyzer': u'snowball'}, u'django_id': {u'include_in_all': False, u'index': u'not_analyzed', u'type': u'string'}, u'id': {u'type': u'string'}, u'time_created': {u'type': u'date', u'format': u'dateOptionalTime'}}}}

Expected mapping dataset by Haystack:

{u'modelresult': {u'_boost': {u'name': u'boost', u'null_value': 1.0}, u'properties': {u'django_ct': {u'include_in_all': False, u'index': u'not_analyzed', u'type': u'string'}, 'text': {u'type': u'string', u'analyzer': u'snowball'}, u'django_id': {u'include_in_all': False, u'index': u'not_analyzed', u'type': u'string'}, 'time_created': {u'type': u'date'}}}}

Take a look at code:
https://github.com/toastdriven/django-haystack/blob/master/haystack/backends/elasticsearch_backend.py#L131

In current implementation current_mapping will never be equal to self.existing_mapping. Because Haystack input dict is always different from ES dict retrieved by get_mapping call.

How it works right now? By default 'SILENTLY_FAIL': False. So Haystack tries to update mapping and silently fails.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at haystack/backends/elasticsearch_backend.py around line 131 and inspect how get_mapping output is compared with the Haystack mapping. Reproduce the mapping shown in the issue after rebuild_index; done means unchanged mappings no longer trigger an update attempt on each request.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, python
Domain
backend, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.