GitHub#72 ⁃ mem_limit not being honoured

Nobody has claimed this yet.

# Describe the environment
**2.6.2:**

**Centos 6:**

**Packaged from source:**

# Describe the problem
mem_limit is set to 256M but the indexer process is using over 3GB of RAM

**Steps to reproduce:**

```
searchd
{
listen = /var/run/manticore/manticore.sock:sphinx
listen = /var/run/manticore/mysql41.sock:mysql41
log = /var/log/manticore/searchd.log
query_log = /var/log/manticore/query.log
read_timeout = 5
max_children = 30
pid_file = /var/run/manticore/searchd.pid
seamless_rotate = 1
preopen_indexes = 0
unlink_old = 1
workers = threads
collation_server = utf8_general_ci
rt_flush_period = 3600
binlog_path = /var/lib/manticore
}

indexer
{
mem_limit = 256M
max_iops = 30
max_iosize = 4M
}

common
{
lemmatizer_base = /usr/share/manticore/dicts
}

source base
{
type = odbc
odbc_dsn = DSN=postgres-database-ssl
}

source messages : base
{
sql_query_range = SELECT MIN(id), MAX(id) FROM messages WHERE ts < (SELECT maxts FROM indexer_counters WHERE tablename='messages' AND hostname='search.example.com')
sql_range_step = 10000
sql_query_pre = INSERT INTO indexer_counters (tablename, hostname, maxts) \
VALUES('messages', 'search.example.com', NOW()) \
ON CONFLICT (tablename,hostname) DO UPDATE SET maxts=excluded.maxts
sql_query = SELECT id, messageid, subject, CRC32(from_address) AS from_addr, CRC32(to_address) AS \
to_addr, CRC32(from_domain) AS from_dom, CRC32(to_domain) AS to_dom, headers, \
hostname, UNIX_TIMESTAMP(timestamp) AS timestamp, isquarantined FROM messages \
WHERE ts < (SELECT maxts FROM indexer_counters WHERE tablename='messages' AND hostname='search.example.com') \
AND id >= $start AND id <= $end
sql_query_post = INSERT INTO indexer_counters (tablename, hostname, maxts) \
VALUES('messages_tmp', 'search.example.com', \
(SELECT maxts FROM indexer_counters WHERE tablename='messages' AND hostname='search.example.com')) \
ON CONFLICT (tablename,hostname) DO UPDATE SET maxts=excluded.maxts
sql_query_post_index = DELETE FROM indexer_counters WHERE tablename='messages' AND hostname='search.example.com'
sql_query_post_index = UPDATE indexer_counters SET tablename='messages' WHERE tablename='messages_tmp' AND hostname='search.example.com'
sql_query_post_index = DELETE FROM indexer_killlist WHERE ts < (SELECT maxts FROM indexer_counters WHERE tablename='messages' \
AND hostname='search.example.com') \
AND tablename='messages'
sql_column_buffers = headers=64K
sql_attr_uint = from_addr
sql_attr_uint = to_addr
sql_attr_uint = from_dom
sql_attr_uint = to_dom
sql_attr_timestamp = timestamp
sql_attr_bool = isquarantined
}

index messages
{
source = messages
path = /var/lib/manticore/messages
docinfo = extern
morphology = stem_en
min_infix_len = 3
index_exact_words = 1
ondisk_attrs = 1
}
```

When indexing top shows the following

top

```
top - 10:06:00 up 23 days, 16:42, 1 user, load average: 2.24, 1.97, 2.35
Tasks: 146 total, 2 running, 144 sleeping, 0 stopped, 0 zombie
Cpu(s): 16.2%us, 1.0%sy, 0.0%ni, 82.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8193520k total, 7443064k used, 750456k free, 13480k buffers
Swap: 8208380k total, 770004k used, 7438376k free, 1952980k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19800 manticor 20 0 3594m 3.1g 3412 R 16.0 39.8 2:30.06 indexer
```

As you can see resident memory usage is 3.1G.

There are about 75k records in the table.

**Messsages from log files:**

No errors.

Start at the indexer entry point and trace how the configured mem_limit of 256M is applied during the supplied source and index setup. Reproduce the configuration while watching the indexer with top; done means the process stays within the intended memory limit or the remaining limitation is clearly documented and covered by a regression test.

Written by the indexing model from the issue text.