nextcloud / nextcloud/maps

[Bug]: modifying filters on geolocation renders database unresponsive

Open
#832 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage bug
Dominant language
JavaScript
Stars
598
Forks
113
PR merge metrics
No merged PRs in 30d

Description

⚠️ This issue respects the following points: ⚠️
Bug description

Hello everyone,

as soon as I load up maps, you can see the strain on my database setup, changing settings like the time-slider will overwhelm the database completely.

I have around 15k geotagged images in my installation, that is where the load is coming from.
It seems, as maps tries to load all these pictures and there thumbnails with its own connection, and therefore quickly flooding the server.

When is happens, MariaDB errors out all the connections with:

[Warning] Aborted connection 0 to db: 'unconnected' user: 'unauthenticated' host: 'connecting host' (Too many connections)
At that point, the whole Nextcloud installation stalls for a few seconds and throws error for everyone.

That's about it, maps is using up 200 Allowed database-connections.
When I make them 300, maps will consume 300, it does not change a thing.

Steps to reproduce
  1. Open maps application on nextcloud
  2. move the timeslider 2-3 times without waiting for the result
  3. watch the database-logs and look for "too many connections"-type messages
Expected behavior

maps should utilize a fixed amount of database connections, pool them or have some kind of overflow-protection

Installation method

Official Docker image

Operating system

Other

PHP engine version

PHP 8.0

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install of the server?

No response

Is this bug present after an update or on a fresh install of the app?

Fresh Nextcloud Maps install (never installed before)

Are you using the Nextcloud Server Encryption module?

No response

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report
{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "<SOME_IP>:3443",
            "<SOME_TLD>"
        ],
        "dbtype": "mysql",
        "version": "24.0.4.1",
        "overwrite.cli.url": "https:\/\/<SOME_IP>:3443",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "loglevel": 0,
        "enable_previews": true,
        "allow_local_remote_servers": true,
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\PDF",
            "2": "OC\\Preview\\Image",
            "3": "OC\\Preview\\Photoshop",
            "4": "OC\\Preview\\TIFF",
            "5": "OC\\Preview\\SVG",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\MKV",
            "8": "OC\\Preview\\MP4",
            "9": "OC\\Preview\\AVI"
        },
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpport": "465",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "preview_use_imagick": true,
        "preview_interpolate": false,
        "preview_thread_limit": 8
    }
}
List of activated Apps
Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - breezedark: 24.0.2
  - bruteforcesettings: 2.4.0
  - calendar: 3.4.2
  - camerarawpreviews: 0.8.0
  - circles: 24.0.1
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.2.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_external: 1.16.1
  - files_fulltextsearch: 24.0.1
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - fulltextsearch: 24.0.0
  - fulltextsearch_elasticsearch: 24.0.1
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - mail: 1.13.8
  - maps: 0.2.0
  - nextcloud_announcements: 1.13.0
  - notifications: 2.12.0
  - oauth2: 1.12.0
  - password_policy: 1.14.0
  - photos: 1.6.0
  - previewgenerator: 5.0.0
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - recommendations: 1.3.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - tasks: 0.14.4
  - text: 3.5.1
  - theming: 1.15.0
  - twofactor_backupcodes: 1.13.0
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - video_converter: 1.0.5
  - viewer: 1.8.0
  - weather_status: 1.4.0
  - workflowengine: 2.6.0
Disabled:
  - admin_audit
  - contactsinteraction: 1.2.0
  - documentserver_community: 0.1.12
  - encryption
  - onlyoffice: 7.5.4
  - unsplash: 1.2.5
  - user_ldap
Nextcloud Signing status
No errors have been found.
Nextcloud Logs
{"reqId":"Xw4TiOkrgNnsrMoToKfR","level":3,"time":"2022-08-21T08:01:58+00:00","remoteAddr":"<SOME_CLIENT_IP>","user":"--","app":"core","method":"GET","url":"/core/preview?fileId=3172199&x=341&y=256&a=1","message":"Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [1040] Too many connections","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36","version":"24.0.4.1","exception":{"Exception":"Doctrine\\DBAL\\Exception","Message":"Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [1040] Too many connections","Code":1040,"Trace":[{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1519,"function":"connect","class":"OC\\DB\\Connection","type":"->"},{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1041,"function":"getWrappedConnection","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/config/www/nextcloud/lib/private/DB/Connection.php","line":261,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/config/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":345,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/config/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":281,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppConfig.php","line":411,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppConfig.php","line":176,"function":"loadConfigValues","class":"OC\\AppConfig","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppConfig.php","line":367,"function":"getApps","class":"OC\\AppConfig","type":"->"},{"file":"/config/www/nextcloud/lib/private/legacy/OC_App.php","line":983,"function":"getValues","class":"OC\\AppConfig","type":"->"},{"file":"/config/www/nextcloud/lib/private/Server.php","line":713,"function":"getAppVersions","class":"OC_App","type":"::"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":162,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":122,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":129,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/config/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":57,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":184,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":162,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":118,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":129,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/config/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":57,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/Server.php","line":1105,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":162,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":122,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":129,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/config/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":57,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/Server.php","line":2082,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/Files/View.php","line":122,"function":"getLockingProvider","class":"OC\\Server","type":"->"},{"file":"/config/www/nextcloud/lib/private/Server.php","line":454,"function":"__construct","class":"OC\\Files\\View","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":162,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":122,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":129,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/config/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":57,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/config/www/nextcloud/lib/private/Server.php","line":1445,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/config/www/nextcloud/lib/base.php","line":602,"function":"boot","class":"OC\\Server","type":"->"},{"file":"/config/www/nextcloud/lib/base.php","line":1104,"function":"init","class":"OC","type":"::"},{"file":"/config/www/nextcloud/index.php","line":34,"args":["/config/www/nextcloud/lib/base.php"],"function":"require_once"}],"File":"/config/www/nextcloud/lib/private/DB/Connection.php","Line":139,"CustomMessage":"--"},"id":"6301ed4dc3215"}
Browser Console

No response

Additional info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue in the Maps application by moving the time slider repeatedly while watching the MariaDB logs and the Nextcloud preview errors. The report names no source files or tests, so the relevant request path and connection usage must first be traced; done means repeated filter changes no longer exhaust database connections or stall the installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, javascript, mariadb, nginx, php
Domain
backend, database, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.