typesense / typesense/typesense

Can't get RegEx fields working

Open
#1,091 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug nested
Dominant language
C++
Stars
26.6k
Forks
973
Avg merge
18h 45m
Merged PRs (30d)
4

Description

Sorry if this is a silly question, but I really can't seem to get RegEx fields working.

Background

I have documents like this. Basically a set of fields on the document that are associated with any (variable/dynamic) field (tenant ID in this case).

{
  ...
  "tenants": {
    "tenantA": {
      "balance": 0,
      ...
    },
    "tenantB": {
      "balance": 12.5,
      ...
    },
    "anyTenantId": {
      "balance": -3,
      ...
    },
  }
}

I want to be able to query by an exact defined (but dynamic) tenant ID. For examlpe, specifically tenant tenantAbc123 (or any other specific ID): ?filter_by=tenants.tenantAbc123.balance:>0, or ?sort_by=tenants.tenantDef456:desc.

I can't flatten the balance fields into an array, because I loose the association with the tenant ID.

What I've tried

With a schema like this:

{ "name": "tenants..*.balance", type: "float" }

I always get Cannot find a field named 'tenants.tenantAbc123.balance' in the schema. I've also tried flattening the fields like tenantAbc123_balance, but same issue.

The only thing that I got to work was { "name": "tenants", type: "object" } along with enable_nested_fields: true, but I run in to problems with type auto-detection, because it infers int64 from initial integer values and then fails when it comes accross a float value. I know exactly the types of the fields, don't need auto-detection, I just need a wildcard in the field name but can't make it work?

On a side note

The documentation about RegEx fields rather sparse, it mentions "you can use any RegEx expression to define a field name", but is that really the case? Does it support anchors and character classes, alternations etc, or is it just .* (because that's the only example provided)? How are single dots (.) treated in a field name of a normal nested field (e.g. nested_field.field1)? Do they match any character and need to be escaped?

Typesense Version: 0.24.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by investigating schema handling for tenants..*.balance, nested fields with enable_nested_fields, and the type auto-detection behavior described in the issue. Done would require a clear decision on wildcard and regex semantics, support for querying dynamic tenant IDs with declared types, and documentation covering regex syntax and dots in nested field names.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.