manticoresoftware / manticoresoftware/manticoresearch

The html_remove_elements configuration option does not correctly handle self-closing HTML tags.

Open
#280 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

low priority
Dominant language
C++
Stars
12k
Forks
642
Avg merge
3d 12h
Merged PRs (30d)
47

Description

Manticore 2.8.2 on CentOS 7

~ $ searchd -v
Manticore 2.8.2 4e81114@190402 release
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2019, Manticore Software LTD (http://manticoresearch.com)

# Describe the problem

The `html_remove_elements` configuration option does not correctly handle self-closing HTML tags.

**Steps to reproduce:**

With a configuration like this:
```conf
html_remove_elements = style, script, img, meta
```
If we try to index this HTML:
```html





Tokens that should be correctly indexed


```

manticore won't index any token. If the meta tag is closed like this `` it will work as expected. We think that Manticore is removing all the HTML after the meta tag, not detecting that meta is a self-closing tag, so it doesn't need a tag.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue using the shown html_remove_elements configuration and HTML sample, comparing an unclosed meta tag with a self-closing one. Trace the implementation of html_remove_elements and verify that tokens after a self-closing tag are indexed while the configured element is removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, html
Domain
databases, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.