typelevel / typelevel/skunk

Support query pipelineing in a single session

Open
#912 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
1.7k
Forks
175
Avg merge
5d 14h
Merged PRs (30d)
9

Description

This follows from a discussion on discord.

When submitting multiple queries concurrently, in the same session, you'll get a portal_n does not exist. Here's a minimal example from repo:

  def run(args: List[String]): IO[ExitCode] =
    session.use { s =>
      import cats.syntax.all._
      (makeQuery(s), makeQuery(s)).parMapN{ case _ => ExitCode.Success }
    }

  def makeQuery(session: Session[IO]) =
    for {
      p <- session.prepare(sql"select current_date".query(date))
      d <- p.unique(Void)
    } yield ()
2023-06-26 15:59:37.731 UTC [154] DEBUG:  parse statement_1: select current_date
2023-06-26 15:59:37.731 UTC [154] DEBUG:  StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2023-06-26 15:59:37.734 UTC [154] DEBUG:  parse statement_2: select current_date
2023-06-26 15:59:37.747 UTC [154] DEBUG:  bind portal_3 to statement_1
2023-06-26 15:59:37.748 UTC [154] DEBUG:  bind portal_4 to statement_2
2023-06-26 15:59:37.756 UTC [154] DEBUG:  CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2023-06-26 15:59:37.760 UTC [154] ERROR:  portal "portal_3" does not exist

This can be gotten around by increasing the connection pool size, or by using a separate session for each makeQuery call.

PostgreSQL seems to say that this should work however; it's documented here. But perhaps I've misunderstood this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the concurrent Session example using prepare, p.unique, and parMapN, then compare its behavior with PostgreSQL's protocol pipelining documentation. Trace how the shared session handles the two queries and verify that both complete without a portal-does-not-exist error when finished.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgres, scala
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.