quickwit-oss / quickwit-oss/quickwit

How to Do SubQuery Like SQL

Open
#5,066 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tutorial
Dominant language
Rust
Stars
11.7k
Forks
597
Avg merge
2d 22h
Merged PRs (30d)
37

Description

I'm implementing tempo's traceql in ltbridge, and I've meet a problem.

Here's a traceql example:

{foo.bar="1" && duration > 20ms} && {a.b="hello" && attributes.service.name="redis"}

Each curly brackets pair means searching for spans which must satisfy all the conditions inside the curly brackets. Just like select * from spans_table where foo.bar="1" and duration>20ms.

Logical Operator such as && || between curly brackets (eg: {A} && {B}), means Inner Join spans from A and B on the same trace_id. In human language, this search can be something like:

I want to search for traces in which it contains at least one span which satisfy A and contains another span which satisfy B and ...

The result should be the form of:

trace_id_a: # there're many spans in a trace, but just keep spans that matches one of the query conditions
  span_id_a,
  span_id_b,
trace_id_b:
  span_id_c,
  span_id_d
..

Here's a conversion from traceql to sql
traceql:

{resource.app="camp" && serviceName="fooSvc"} && ({span.qwe="qqq"} || {foo>10})

sql:

    SELECT * FROM spans sp
    WHERE sp.span_id IN
        (SELECT span_id
        FROM (
                (SELECT span_id,
                        trace_id
                  FROM spans
                  WHERE (resource_attributes['app'] = 'camp'
                        AND service_name = 'fooSvc'))
              UNION
                (SELECT span_id,
                        trace_id
                  FROM spans
                  WHERE span_attributes['qwe'] = 'qqq')
              UNION
                (SELECT span_id,
                        trace_id
                  FROM spans
                  WHERE (span_attributes['foo'] > 10 OR resource_attributes['foo']>10))) AS sub
        WHERE (sub.trace_id IN
                      (SELECT trace_id
                        FROM spans
                        WHERE (resource_attributes['app'] = 'camp'
                              AND service_name = 'fooSvc'))
                    AND (sub.trace_id IN
                            (SELECT trace_id
                            FROM spans
                            WHERE span_attributes['qwe'] = 'qqq')
                          OR sub.trace_id IN
                            (SELECT trace_id
                            FROM spans
                            WHERE (span_attributes['foo'] > 10 OR resource_attributes['foo']>10)))))

I don't know the equivalent quickwit or elasticsearch query, and need your help

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SQL and TraceQL examples in the issue, then review Quickwit and Elasticsearch query documentation for filtering spans by trace_id and combining conditions across spans. Determine whether an equivalent query can express the requested same-trace semantics, and document the supported query or the missing capability.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.