apache / apache/lucene

another idea for updatable fields [LUCENE-4272]

Open
#5,341 11 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I've been reviewing the ideas for updatable fields and have an alternative
proposal that I think would address my biggest concern:

- not slowing down searching

When I look at what Solr and Elasticsearch do here, by basically reindexing from stored fields, I think they solve a lot of the problem: users don't have to "rebuild" their document from scratch just to update one tiny piece.

But I think we can do this more efficiently: by avoiding reindexing of the unaffected fields.

The basic idea is that we would require term vectors for this approach (as the already store a serialized indexed version of the doc), and so we could just take the other pieces from the existing vectors for the doc.

I dont think we should discard the idea because vectors are slow/big today, this seems like something we could fix.

Personally I like the idea of not slowing down search performance to solve the problem, I think we should really start from that angle and work towards making the indexing side more efficient, not vice-versa.

---
Migrated from [LUCENE-4272](https://issues.apache.org/jira/browse/LUCENE-4272) by Robert Muir (@rmuir), 4 votes, updated May 02 2013
Linked issues:
- #5664
- #5328
- #2963

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by reviewing the linked issues and the proposal's requirement for term vectors, then determine whether the updatable-fields design has been accepted; completion criteria are not defined in this issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.