typelevel / typelevel/grackle

Adding LIMIT clause to query results in assertion failure

Open
#243 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Scala
Stars
189
Forks
32
Avg merge
18h 31m
Merged PRs (30d)
37

Description

The following query

Select(
  "programs", 
  Nil, 
  Filter(
    And(
      Or(
        Contains(ListPath(List("users", "userId")), Const("u-107")), 
        Eql(UniquePath(List("piUserId")), Const("u-107"))
      ), 
      Eql(UniquePath(List("existence")), Const(Present))
    ), 
    Group(
      List(
        Select("id", List(), Empty),  
        Select("name", List(), Empty)
      )
    )
  )
)

produces the following SQL.

SELECT
  t_program.c_name,
  t_program.c_program_id
FROM
  t_program
  INNER JOIN (
    SELECT
      DISTINCT ON ((t_program.c_program_id COLLATE "C"))(t_program.c_program_id COLLATE "C")
    FROM
      t_program
      LEFT JOIN t_program_user ON (
        t_program_user.c_program_id = t_program.c_program_id
      )
    WHERE
      (
        (
          (
            (t_program_user.c_user_id = $1)
            OR (t_program.c_pi_user_id = $2)
          )
          AND (t_program.c_existence = $3)
        )
      )
      AND (t_program.c_program_id IS NOT NULL)
    ORDER BY
      (t_program.c_program_id COLLATE "C") NULLS LAST
  ) AS t_program_pred ON (
    t_program_pred.c_program_id = t_program.c_program_id
  )

Adding a Limit to the child query, thus

Select(
  "programs",
  Nil,
  Filter(
    And(
      Or(
        Contains(ListPath(List("users", "userId")), Const("u-107")),
        Eql(UniquePath(List("piUserId")), Const("u-107"))
      ),
      Eql(UniquePath(List("existence")), Const(Present))
    ),
    Limit(
      1000,
      Group(
        List(
          Select("id", Nil, Empty), 
          Select("name", Nil, Empty)
        )
      )
    )
  )
)

results in

java.lang.AssertionError: assertion failed
	at scala.Predef$.assert(Predef.scala:264)
	at edu.gemini.grackle.sql.SqlMapping$SqlQuery$SqlSelect.addFilterOrderByOffsetLimit(SqlMapping.scala:1717)
        ...

due to (I assume) the already-present ORDER BY caused by the joins ... ?

assert(orders.isEmpty && offset.isEmpty && limit.isEmpty && !isDistinct)

Relevant [abbreviated] definitions:

create table t_user (
  c_user_id     d_user_id     primary key not null,
  ...
);

create table t_program (
  c_program_id  d_program_id  not null primary key,
  c_existence   e_existence   not null,
  c_pi_user_id  d_user_id,
  c_name        text,
  ...
);

create table t_program_user (
  c_program_id  d_program_id  not null,
  c_user_id     d_user_id     not null,
  ...
);
object Program extends TableDef("t_program") {
  val Id        = col("c_program_id", program_id)
  val PiUserId  = col("c_pi_user_id", user_id)
  val Existence = col("c_existence", existence)
  val Name      = col("c_name", text_nonempty.opt)
}
object ProgramUser extends TableDef("t_program_user") {
  val ProgramId = col("c_program_id", program_id)
  val UserId    = col("c_user_id", user_id)
  val Role      = col("c_role", program_user_role)
}
object User extends TableDef("t_user") {
  val Id = col("c_user_id", user_id)
}
ObjectMapping(
  tpe = QueryType,
  fieldMappings = List(
    SqlRoot("programs"),
    SqlRoot("program"),
  )
),
ObjectMapping(
  tpe = ProgramType,
  fieldMappings = List(
    SqlField("id", Program.Id, key = true),
    SqlField("existence", Program.Existence, hidden = true),
    SqlField("name", Program.Name),
    SqlField("piUserId", Program.PiUserId, hidden = true),
    SqlObject("pi", Join(Program.PiUserId, User.Id)),
    SqlObject("users", Join(Program.Id, ProgramUser.ProgramId)),
  ),
),
ObjectMapping(
  tpe = ProgramUserType,
  fieldMappings = List(
    SqlField("programId", ProgramUser.ProgramId, hidden = true, key = true),
    SqlField("userId", ProgramUser.UserId, key = true),
    SqlField("role", ProgramUser.Role),
    SqlObject("user", Join(ProgramUser.UserId, User.Id))
  ),
),
LeafMapping[lucuma.core.model.User.Id](UserIdType),
LeafMapping[lucuma.core.model.Program.Id](ProgramIdType),
LeafMapping[ProgramUserRole](ProgramUserRoleType),

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at SqlMapping.scala:1717 and reproduce the provided Scala query with the child Limit applied. Inspect addFilterOrderByOffsetLimit and how the existing ORDER BY from joins interacts with the limit. Done means the query no longer triggers the assertion and produces valid SQL retaining the requested limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, sql
Domain
backend, database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.