derrickoswald / derrickoswald/things

improve search-engine

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

There are a number of issues with the search system as it now stands, which is:
- [couchdb-lucene](https://github.com/rnewson/couchdb-lucene)
- executing as a [external process](http://docs.couchdb.org/en/1.6.1/externals.html)
- running [lucene](http://lucene.apache.org/)

The issues are:
- The external process is executed by CouchDB (i.e. it has the ppid of CouchDB) but is not shut down when CouchDB is restarted. Instead it reverts to ppid 1 and another process is started by CouchDB, leading to multiple processes in memory.
- The credentials to access CouchDB, i.e. the admin username and password, need to be placed in the couchdb-lucene configuration file - in plain text - rather than using [proxy authentication](http://docs.couchdb.org/en/1.6.1/api/server/authn.html#proxy-authentication) and reading the secret key from the CouchDB configuration (see the user_manager code for how this is handled).
- The logging from couchdb-lucene to the console (as configured in conf/log4j.xml) does not conform to the logging format expected by the stdin reader of the CouchDB externals code, leading to messages in the log like Ignoring OS daemon request: {{error,garbage_after_value}.
- Each database is indexed separately with the result that there are multiple queries required to do a search.
- The installation of couchdb-lucene requires compilation from scratch, with the inherent overhead of installing maven and all the crap it downloads.
- The Lucene install is nearly 60MB for some simple functionality.
- It is unclear how search can be extended to other systems. So far all the documents need to be local, and querying other _things_ systems would require either CORS \* or be done server side with another nodejs process

This is probably because couchdb-lucene was written before the externals API, and was probably not intended to run as a external process. There isn't anything wrong with Lucene per se, but the bolting on isn't working very well and could be improved.

This enhancement would probably involve either improving the couchdb-lucene integration or finding a better alternative to using the Lucene search engine.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.