hashgraph / hashgraph/guardian
Indexer: advanced search
- Dominant language
- TypeScript
- Stars
- 146
- Forks
- 186
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 116
Description
### Problem description
At present Indexer supports 'direct' or 'flat' searches for data, i.e. users can specify the target document and a data value but anything beyond that, e.g. cross-referencing across documents or using values from from the found document in the search, is not possible - it requires multiple searches/steps, perhaps even noting down values and various IDs (on paper), and more importantly a deep understanding of the underlying low-level data relationships structure from everyone who is doing the searching. This makes many practical tasks complex and slow, very repetitive and/or otherwise impossible for non-technical business users.
### Requirements
Introduce ability to configure complex global multi-document searches in the Indexer such that:
- any combination of document types, properties, attributes, fields, datatypes, and their values can be used to search for a set of items (documents, tokens)
- search conditions (for values) allow advanced matching for strings, dates, numbers, and other datatypes:
- exact match for single values and for 'in the set' matching from the lists of values
- ranges (i.e. the value is within a range)
- regexp pattern matching
- '>', '<', etc
- arithmetical expressions, e.g. `(value - 2)/10 > 5` finds all documents for which this conditions resolves to true - i.e. all documents in which `value` is bigger than `52`
- arithmetical expression can be used on values across all found documents as a search criteria, e.g. find only items where the `SUM` of all issued tokens for a period of time (across all the minting VCs) is > 10K.
- multiple search steps can be configured in the search, where subsequent searches can be performed based on the values from fields of the documents found in the previous steps. E.g. the following hypothetical sequence: find all VCs from a specific Standard Registry (SR) where `name` is equal to `Monitoring Report` and `methodology` is `Verra 5`, and then find all minting VCs which have the value of `project ID` field equal to the value of p`roject number `field from the 'monitoring report' VCs.
- the display of the results of the search can be configured to show a selection of fields and their values in the resulting grid
- the grid results can be sorted based on any or all of the fields
- the grid results can be grouped based on the same value or values range (based on conditions) - see above for the conditions types
- once a search is configured, the search URL would contain everything needed to specify/execute this same search again. I.e. if a user then saves the URL/search in their browser bookmarks, clicking on the bookmark at a later time would bring up the same search UI with the same parameters configured, allow user to modify any of the parameters, and if executed display the up-to-date results
### Definition of done
- Indexer search capabilities are enhanced as specified above
- Documentation is updated accordingly
- Examples covering use-cases in the attached document are documented, with recommendations on further customisations
### Acceptance criteria
- Use-cases described in the attached document are covered by the advanced Indexer search functionality
- https://github.com/hashgraph/guardian/issues/5021 is covered
- https://github.com/hashgraph/guardian/issues/5019 is covered
[Potential Indexer Use Cases_12-4-24.xlsx](https://github.com/user-attachments/files/20792898/Potential.Indexer.Use.Cases_12-4-24.xlsx)
Contributor guide
Research direction
Start with the attached Potential Indexer Use Cases spreadsheet and the acceptance-criteria issues 5021 and 5019. The work is done when those use cases are supported, searches can be saved and rerun from their URLs, and the required documentation and examples are updated.
Written by the indexing model from the issue text.
Assessment
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100