greenelab / greenelab/tribe

Use Postgres built-in search functionalities instead of Elasticsearch?

Open
#43 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Postgres has been improving full text search and trigram search a lot since version 9.6. I wonder whether it is possible to use them to replace Elasticsearch. If we can, the backend architecture (and deployment) can be greatly simplified. With the `GIN` or `GIST` indexes on search fields, we don't have to worry about the index updates (which invoke `celery` jobs right now).

Right now, Elasticsearch is being used to search genes and genesets. We have **312,983 genes** and **408,237 genesets** in Postgres backend database.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the current Elasticsearch search implementation and Celery index-update jobs, then assess the existing gene and geneset search behavior against the proposed PostgreSQL indexes. Done would require a concrete feasibility decision and an agreed migration scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, postgres
Domain
backend-api-design, databases, search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.