django-haystack / django-haystack/django-haystack

rebuild index command will not involve the router

Open
#1,138 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs review
Dominant language
Python
Stars
3.7k
Forks
1.3k
Avg merge
3h 18m
Merged PRs (30d)
3

Description

I have serveral db models that require to build index in serveral cores.

Thus I defined serveral haystack connections as below:

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
        'URL': 'http://127.0.0.1:8983/solr/overall',
    },
    'model1': {
        'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
        'URL': 'http://127.0.0.1:8983/solr/model1',
    },
    'model2': {
        'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
        'URL': 'http://127.0.0.1:8983/solr/model2',
    },
}

And I create a customized router to handle the index connection.

class MulitCoresHayStackRouter(DefaultRouter):
    def for_read(self, **hints):
        return DEFAULT_ALIAS

    def for_write(self, **hints):
        alis = DEFAULT_ALIAS

        if 'instance' in hints:
            instance_class = hints['instance'].__module__ + "." + hints['instance'].__class__.__name__
            if instance_class in settings.HAYSTACK_CONNECTIONS:
                alis = instance_class
                logger.debug('Get the connection alis: %s with instance_class: %s' % (alis, instance_class))

        return alis

However, when I run the rebuild_index command, it will index all db models's data to all connections.

Is there any fixing or walkaround solution ?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the rebuild_index command and how it selects connections when multiple HAYSTACK_CONNECTIONS entries and a custom router are configured. Reproduce the reported behavior with the shown settings and router, then verify that rebuilding indexes each model only in its intended connection rather than every connection.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.