OpenFn / OpenFn/lightning

Search not returning results during spikes in project activity

Open
#1,916 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

@jyeshe has identified a problem whereby search results may be inaccurate if searches are executing while the DB is processing thousands of new dataclips.

We haven't been able to replicate on other DBs yet, but when Roger searched both input_dataclip.body and log_lines he was not seeing the right results. (Nothing appeared, despite there being a record in the DB that should have matched.)

Not flagging this is a bug yet but opening in case anyone else sees it.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing how searches cover input_dataclip.body and log_lines while the database is processing thousands of new dataclips. Try to reproduce the missing-result report during a comparable activity spike, then confirm whether results remain accurate or document that the issue cannot be reproduced on other databases.

Written by the indexing model from the issue text.

Assessment

Domain
databases, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.