nextcloud / nextcloud/deck

More than 1000 expressions in a list are not allowed on Oracle

Open
#6,099 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

backend bug feature: tags technical debt
Dominant language
JavaScript
Stars
1.4k
Forks
354
Avg merge
1d 10h
Merged PRs (30d)
43

Description

What happens: When the Deck app builds a calendar view, it asks the database for labels assigned to a large number of cards all at once. The database call fails with the error "More than 1000 expressions in a list are not allowed on Oracle." As a result the calendar/Caldav request fails and an error is written to the logs.

Important note about logging: we see and log this same error message even on non‑Oracle systems. The application constructs an IN(...) list with >1000 placeholders and the DB access layer (Doctrine/DBAL) raises a QueryException with that message. Because the exception message is generated/recorded by the DB/DBAL layer, it appears in logs regardless of whether the backend is actually Oracle.

I saw today the same error on a Nextcloud 28.0.7.4 on cadav requests:

Assistent
Bildgenerirung{
  "reqId": "h4GkaBMxZBbaYGvGCNf6",
  "level": 3,
  "time": "2024-07-12T17:16:45+00:00",
  "remoteAddr": "2a0f:6480:1::2",
  "user": "abb8c2f2-26c2-1034-821f-c3ab67df07dc",
  "app": "core",
  "method": "REPORT",
  "url": "/remote.php/caldav/calendars/abb8c2f2-26c2-1034-821f-c3ab67df07dc/app-generated--deck--board-27/",
  "message": "More than 1000 expressions in a list are not allowed on Oracle.",
  "userAgent": "Evolution/3.46.4",
  "version": "28.0.7.4",
  "exception": {
    "Exception": "Doctrine\\DBAL\\Query\\QueryException",
    "Message": "More than 1000 expressions in a list are not allowed on Oracle.",
    "Code": 0,
    "Trace": [
      {
        "file": "/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
        "line": 293,
        "function": "execute",
        "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php",
        "line": 335,
        "function": "executeQuery",
        "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/Db/LabelMapper.php",
        "line": 92,
        "function": "findEntities",
        "class": "OCP\\AppFramework\\Db\\QBMapper",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/Service/CardService.php",
        "line": 146,
        "function": "findAssignedLabelsForCards",
        "class": "OCA\\Deck\\Db\\LabelMapper",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/Service/CardService.php",
        "line": 207,
        "function": "enrichCards",
        "class": "OCA\\Deck\\Service\\CardService",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/DAV/DeckCalendarBackend.php",
        "line": 85,
        "function": "findCalendarEntries",
        "class": "OCA\\Deck\\Service\\CardService",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/DAV/Calendar.php",
        "line": 152,
        "function": "getChildren",
        "class": "OCA\\Deck\\DAV\\DeckCalendarBackend",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/DAV/Calendar.php",
        "line": 135,
        "function": "getBackendChildren",
        "class": "OCA\\Deck\\DAV\\Calendar",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/deck/lib/DAV/Calendar.php",
        "line": 90,
        "function": "getChildren",
        "class": "OCA\\Deck\\DAV\\Calendar",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Plugin.php",
        "line": 598,
        "function": "calendarQuery",
        "class": "OCA\\Deck\\DAV\\Calendar",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Plugin.php",
        "line": 258,
        "function": "calendarQueryReport",
        "class": "Sabre\\CalDAV\\Plugin",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "report",
        "class": "Sabre\\CalDAV\\Plugin",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
        "line": 690,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "httpReport",
        "class": "Sabre\\DAV\\CorePlugin",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 472,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 253,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 321,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/dav/appinfo/v1/caldav.php",
        "line": 123,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/remote.php",
        "line": 172,
        "args": [
          "/var/www/nextcloud/apps/dav/appinfo/v1/caldav.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
    "Line": 263,
    "message": "More than 1000 expressions in a list are not allowed on Oracle.",
    "query": "SELECT `l`.*, `card_id` FROM `*PREFIX*deck_labels` `l` INNER JOIN `*PREFIX*deck_assigned_labels` `al` ON l.id = al.label_id WHERE `card_id` IN (:dcValue1) ORDER BY `l`.`id` ASC",
    "exception": {},
    "CustomMessage": "More than 1000 expressions in a list are not allowed on Oracle."
  }
}

Originally posted by @schiessle in https://github.com/nextcloud/server/issues/32593#issuecomment-2226008846

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in apps/deck/lib/Db/LabelMapper.php at line 92, where findAssignedLabelsForCards reaches the query builder, then follow CardService.php and DeckCalendarBackend.php. Reproduce with a CalDAV calendar request containing more than 1000 cards and inspect the generated IN query. Done means large card lists no longer fail and the request completes without this logged exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.