manticoresoftware / manticoresoftware/manticoresearch

Request: Add to config values from parent config

Open
#463 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
12k
Forks
642
Avg merge
3d 12h
Merged PRs (30d)
47

Description

Is your feature request related to a problem? Please describe.

Can use source/index inheritance in the config files to cut down on duplication etc. But when (re)define a multi-value config variable (like sql_query_pre, or sql_attr_multi) in the child config, it completely overrides the values defined in the parent.

This is particularly common with sql_query_pre, where a a 'template' index can define variables like 'query_cache_type' and 'max_statement_time' etc.

... but means when a child index needs to add a sql_query_pre, (eg to setup a counter in a main+delta setup), then needs to duplicate all the queries from the template index as well.

Describe the solution you'd like

Most universal would be a way explicitly add to existing, rather than starting again, Perhaps something like

sql_query_pre           += REPLACE INTO sph_counter SELECT .... 

(ie += to explicitly to add to value(s) from the parent, rather than starting again in the child)

Describe alternatives you've considered

In the case of main+delta, could perhaps move the counter update into a _post, particully using the $maxid macro. (rather than adding to _pre). But its not a universal solution, as some _pre do other stuff, like creating a temporary table, or setting up locks needed for some specific indexes.

Another perhaps simpler idea would be TWO sets of sql_query_pre queries. E.g. have both sql_query_setup and sql_query_pre, both simply run querie(s) before main index query. Means sql_query_setup (for example) could be defined in the template, and sql_query_pre would be in the child indexes.

... in fact this 'sql_query_setup' might be useful, as it could ALSO be run when reconnect to run the 'sql_attr_multi' queries. Ie sql_query_setup is run for ALL mysql connections, but sql_query_pre is only run on the 'main' index connection.

Additional context

This is not a particularly important request, as it all it would do is simplify config files, but it could really help on complex setups.

https://github.com/geograph-project/geograph-project/blob/british-isles/system/docker/manticore/etc/sphinxsearch/sphinx.conf.d/sample8E.conf
is an example index, that needs to redefine all the sql_query_pre, which already defined in the parent index.
https://github.com/geograph-project/geograph-project/blob/british-isles/system/docker/manticore/etc/sphinxsearch/sphinx.conf.d/21-database.conf

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the inheritance and repeated sql_query_pre definitions in sample8E.conf and 21-database.conf linked in the issue. Compare the proposed += behavior with the sql_query_setup alternative; the work is done when a child configuration can add to parent multi-value settings without duplicating them and the resulting query execution behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.