[Question] Left join behavior: include_fields with $ref requires filter to activate

Hello,

I have been tinkering with join operations for the past two days. I have finally found a way to achieve a left join, but I would like to know if there is a better approach.

I have a collection of roughly 7 million records that I need to join based on a tenant_id.

The collections are:

// Companies: mainly public, but can also be created
// inside a tenant, hence the nullable tenant_id
companies {
  id: uuid,
  tenant_id: nullable uuid,
  name: string,
  city: string,
  age: number,
  revenue: number,
  coordinates: GeoPoint,
}

// Entities: enrich companies on a per-tenant basis
entities {
  id: uuid,
  companyId: uuid,
  tenantId: uuid,
  tags: string[],
}

What I need to do is, in theory, straightforward:

1. Search across all companies that are either public or belong to the same tenant as the user.
2. If a company has an associated entity within the user's tenant, enrich the result by including the entity's tags.

The first part is simple — I combine a scoped API key with a basic search.

Scoped key filter:

organizationId:[{organizationId},~~]

~~ is used as a "null" value and is indexed accordingly.

Search:

{
  "q": "my target company",
  "query_by": "name, city",
  "query_by_weights": "10, 2",
  "filter_by": "age:[10, 20] && revenue:[-1, 200000]",
  "sort_by": "revenue:desc",
  "page": 1,
  "page_size": 10
}

The problem

The problem arises when including the tags.

If I add "include_fields": "$entities(tags, strategy:nest_array)", nothing happens — the tags do not appear unless $entities is also referenced inside a filter.

This might be acceptable on its own, but the filter must be duplicated in order to replicate left join behavior:

(age:[10, 20] && revenue:[-1, 200000])
  || ((age:[10, 20] && revenue:[-1, 200000]) && $entities(organizationId:{organizationId}))

This is because:

• Adding && $entities(organizationId:{organizationId}) returns only companies that have a matching entity — effectively an inner join, not a left join.
• Adding || $entities(organizationId:{organizationId}) always includes all companies belonging to the tenant, ignoring the other filters.

Setting the filter at the scoped key level also results in inner join behavior:

organizationId:[{organizationId},~~] && $entities(organizationId:{organizationId})

Duplicating the filter is unfortunately very slow, as it essentially doubles the search time.

Current workaround

The only performant workaround I have found is to use a multi-search: the first query includes the $entities filter, the second does not. The results are then merged and deduplicated.

Question

Is there a better way to achieve left join semantics — specifically, having include_fields with a $ref attach joined data when available, without requiring the join reference in the filter?

The payload names no source files, tests, or entry points. Begin by reproducing the supplied include_fields and $entities queries, comparing them with the multi-search workaround; the issue is resolved when matching entity tags are attached without duplicating the company filter or changing left-join results.