apache / apache/airavata-data-catalog

Evaluate the overhead (absolute and relative) of loading JSONB documents with and without a GIN index

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
10
Forks
6
PR merge metrics
No merged PRs in 30d

Description

This came up in the design review. From Amila:

> 1. The performance -- each time we insert/update documents the inverted index needs to be re-build. If we are are re-building the inverted index synchronously, the api call might need to wait more time. Asyncrhounous updates might be more suitable.

Perhaps we can do asynchronous updates, but first I think it would be good to quantify what overhead there is with updating the GIN index each time a JSONB document (i.e., the `metadata` column) is inserted into the data_product table.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the data_product table and its metadata JSONB column, then identify how inserts and updates can be measured with and without a GIN index. Done means reporting the absolute and relative overhead of maintaining the index.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.