Improve the search service index management
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 274
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 103
Description
Referencing: #5503 ([docs-only] search service readme)
While documenting the search service, some points raised that need improvement:
It is more or less handling the index - from different POV's.
1. After the discussion about a service user, we also need to update the command line, means removing
the `--user` argument:
`ocis search index --space $SPACE_ID --user $USER_ID` -- >
`ocis search index --space $SPACE_ID`
2. We do not have a mechanism to fully recreate an index from scratch, either because it was corrupt or not present (like missed restore from backup). Note that the current mechanism to update on change is not an option as it updates items ONLY if the path has changed like when renaming or moving a path component:
`ocis search index --purge`
`ocis search index --recreate` (recreate is an automatic iteration over all spaces)
3. It is possible that _content_ extraction was added at a later time or the extractor did not do its job. Because there is a missing gap in indexing, a manual trigger for "reindexing" the missing is necessary. The gap can be identified via an internal query and then executed:
`ocis search index --update`
**Item 4 is solved via #5503**
4. From the discussion, there is a state named `Resource Trashed` stating that the index entry will not get purged though the file got trashed. IMHO there is a difference in `Resource Trashed` (trashbin) vs `Resource Purged` (finally gone):
a. Either implement a `Resource Purged` state which removes the index entry or
b. Implement `ocis search index --cleanup` to remove orphaned index entries
5. As space needed for the index is always growing, a command is necessary to return the current index size used and, if feasable, how much space is remaining on the filesystem used:
`ocis search index --size`
6. We need to have a mechanism to check if the index aligns to real files present and in addition an option to purge those index items that have no match:
`ocis search index --check -purge`
@fschade @aduffeck @micbar @butonic @dragotin
Contributor guide
Assessment
This issue has not been assessed yet.