nextcloud / nextcloud/backup

100% CPU by many database process, too many connections

Open
#513 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
272
Forks
37
No merged PRs in 30d

Description

Hello, I have 100% CPU used by many DB process when I enable NC Backup background jobs.

image

When I disable these background jobs in the app config panel and reboot, the pb is solved.

I use NC 25.0.5. , backup app v1.2.0 on debian / apache / mariaDB on Intel Nuc hardware.
It still works, but a bit slower than usual, and desktop sync fails.

According to logs, I wonder if there is something wrong with the SQLDump ?
Is there a way to raise the app log level ? any idea to troubleshoot deeper ?

ERROR 1:

[no app in context] Erreur: OCP\Files\NotFoundException: /Local/backupapp/20230322225619-differential-rIoiVQ95QBexwKR/sqldump/sqldump-9f050013-9ca3-46ae-baf4-7bcd1c78c4e9/sqldump-9f050013-9ca3-46ae-baf4-7bcd1c78c4e9.zip has no owner at <<closure>>

 0. /var/www/nextcloud/lib/private/Files/Filesystem.php line 805
    OC\Files\View->getOwner()
 1. /var/www/nextcloud/apps/files_versions/lib/Storage.php line 112
    OC\Files\Filesystem::getOwner()
 2. /var/www/nextcloud/apps/files_versions/lib/Storage.php line 247
    OCA\Files_Versions\Storage::getUidAndFilename()
 3. /var/www/nextcloud/apps/files_versions/lib/Hooks.php line 82
    OCA\Files_Versions\Storage::markDeletedFile()
 4. /var/www/nextcloud/lib/private/legacy/OC_Hook.php line 106
    OCA\Files_Versions\Hooks::pre_remove_hook()
 5. /var/www/nextcloud/lib/private/Files/View.php line 1294
    OC_Hook::emit()
 6. /var/www/nextcloud/lib/private/Files/View.php line 1164
    OC\Files\View->runHooks()
 7. /var/www/nextcloud/lib/private/Files/View.php line 727
    OC\Files\View->basicOperation()
 8. /var/www/nextcloud/lib/private/Files/Node/File.php line 131
    OC\Files\View->unlink()
 9. /var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFile.php line 133
    OC\Files\Node\File->delete()
10. /var/www/nextcloud/apps/backup/lib/Service/ChunkService.php line 957
    OC\Files\SimpleFS\SimpleFile->delete()
11. /var/www/nextcloud/apps/backup/lib/Service/PackService.php line 200
    OCA\Backup\Service\ChunkService->removeChunkFile()
12. /var/www/nextcloud/apps/backup/lib/Cron/Manage.php line 173
    OCA\Backup\Service\PackService->packPoint()
13. /var/www/nextcloud/apps/backup/lib/Cron/Manage.php line 119
    OCA\Backup\Cron\Manage->manage()
14. /var/www/nextcloud/lib/private/BackgroundJob/Job.php line 54
    OCA\Backup\Cron\Manage->run()
15. /var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php line 60
    OC\BackgroundJob\Job->execute()
16. /var/www/nextcloud/cron.php line 152
    OC\BackgroundJob\TimedJob->execute()

at 2023-03-23T22:11:40+00:00

ERROR 2 :

[no app in context] Erreur: Sabre\DAV\Exception\BadRequest: Taille du fichier attendue : 7420045 octets mais taille du fichier lue (depuis le client Nextcloud) et écrit (dans le stockage Nextcloud) : 3014656 octets. Cela peut être un problème de réseau au niveau du client ou un problème de stockage au niveau du serveur. at <<closure>>

0. /var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php line 151
   OCA\DAV\Connector\Sabre\File->put()
1. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 1098
   OCA\DAV\Connector\Sabre\Directory->createFile()
2. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 504
   Sabre\DAV\Server->createFile()
3. /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
   Sabre\DAV\CorePlugin->httpPut()
4. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 472
   Sabre\DAV\Server->emit()
5. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 253
   Sabre\DAV\Server->invokeMethod()
6. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 321
   Sabre\DAV\Server->start()
7. /var/www/nextcloud/apps/dav/lib/Server.php line 360
   Sabre\DAV\Server->exec()
8. /var/www/nextcloud/apps/dav/appinfo/v2/remote.php line 35
   OCA\DAV\Server->exec()
9. /var/www/nextcloud/remote.php line 171
   require_once("/var/www/nextcl ... p")

PUT /remote.php/dav/files/ValouH/Photos%20Valentin/2023/Malaisie%202023/4%20-%20Penang%20Fev%202023/IMG_20230223_120737.jpg
from 1.32.17.81 by ValouH at 2023-03-24T08:33:28+00:00

ERROR 3:


[core] Erreur: Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [1040] Too many connections at <<closure>>

 0. /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php line 1519
    OC\DB\Connection->connect()
 1. /var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php line 1041
    Doctrine\DBAL\Connection->getWrappedConnection()
 2. /var/www/nextcloud/lib/private/DB/Connection.php line 261
    Doctrine\DBAL\Connection->executeQuery()
 3. /var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php line 345
    OC\DB\Connection->executeQuery()
 4. /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php line 281
    Doctrine\DBAL\Query\QueryBuilder->execute()
 5. /var/www/nextcloud/lib/private/AppConfig.php line 418
    OC\DB\QueryBuilder\QueryBuilder->execute()
 6. /var/www/nextcloud/lib/private/AppConfig.php line 183
    OC\AppConfig->loadConfigValues()
 7. /var/www/nextcloud/lib/private/AppConfig.php line 374
    OC\AppConfig->getApps()
 8. /var/www/nextcloud/lib/private/legacy/OC_App.php line 967
    OC\AppConfig->getValues()
 9. /var/www/nextcloud/lib/private/Server.php line 725
    OC_App::getAppVersions()
10. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 162
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
11. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
12. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 129
    Pimple\Container->offsetGet()
13. /var/www/nextcloud/lib/private/ServerContainer.php line 136
    OC\AppFramework\Utility\SimpleContainer->query()
14. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 57
    OC\ServerContainer->query()
15. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 184
    OC\AppFramework\Utility\SimpleContainer->get()
16. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 162
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
17. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 118
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
18. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 129
    Pimple\Container->offsetGet()
19. /var/www/nextcloud/lib/private/ServerContainer.php line 136
    OC\AppFramework\Utility\SimpleContainer->query()
20. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 57
    OC\ServerContainer->query()
21. /var/www/nextcloud/lib/private/Server.php line 1119
    OC\AppFramework\Utility\SimpleContainer->get()
22. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 162
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
23. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
24. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 129
    Pimple\Container->offsetGet()
25. /var/www/nextcloud/lib/private/ServerContainer.php line 136
    OC\AppFramework\Utility\SimpleContainer->query()
26. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 57
    OC\ServerContainer->query()
27. /var/www/nextcloud/lib/private/Server.php line 2070
    OC\AppFramework\Utility\SimpleContainer->get()
28. /var/www/nextcloud/lib/private/Files/View.php line 117
    OC\Server->getLockingProvider()
29. /var/www/nextcloud/lib/private/Server.php line 462
    OC\Files\View->__construct()
30. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 162
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
31. /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
32. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 129
    Pimple\Container->offsetGet()
33. /var/www/nextcloud/lib/private/ServerContainer.php line 136
    OC\AppFramework\Utility\SimpleContainer->query()
34. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 57
    OC\ServerContainer->query()
35. /var/www/nextcloud/lib/private/Server.php line 1472
    OC\AppFramework\Utility\SimpleContainer->get()
36. /var/www/nextcloud/lib/base.php line 617
    OC\Server->boot()
37. /var/www/nextcloud/lib/base.php line 1144
    OC::init()
38. /var/www/nextcloud/index.php line 34
    require_once("/var/www/nextcloud/lib/base.php")

GET /index.php/apps/systemtags/lastused
from 192.168.1.1 at 2023-03-25T20:01:03+00:00

ERROR 4:

[no app in context] Erreur: OCP\AppFramework\QueryException: Could not resolve trashManager! Class "trashManager" does not exist at <<closure>>

 0. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 133
    OC\AppFramework\Utility\SimpleContainer->resolve()
 1. /var/www/nextcloud/lib/private/ServerContainer.php line 161
    OC\AppFramework\Utility\SimpleContainer->query()
 2. /var/www/nextcloud/lib/private/AppFramework/DependencyInjection/DIContainer.php line 440
    OC\ServerContainer->query()
 3. /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php line 57
    OC\AppFramework\DependencyInjection\DIContainer->query()
 4. /var/www/nextcloud/lib/private/AppFramework/Bootstrap/FunctionInjector.php line 57
    OC\AppFramework\Utility\SimpleContainer->get()
 5. <<closure>>
    OC\AppFramework\Bootstrap\FunctionInjector->OC\AppFramework\Bootstrap\{closure}("*** sensitive parameters replaced ***")
 6. /var/www/nextcloud/lib/private/AppFramework/Bootstrap/FunctionInjector.php line 67
    array_map()
 7. /var/www/nextcloud/lib/private/AppFramework/Bootstrap/BootContext.php line 51
    OC\AppFramework\Bootstrap\FunctionInjector->injectFn()
 8. /var/www/nextcloud/apps/files_trashbin/lib/AppInfo/Application.php line 61
    OC\AppFramework\Bootstrap\BootContext->injectFn()
 9. /var/www/nextcloud/lib/private/AppFramework/Bootstrap/Coordinator.php line 190
    OCA\Files_Trashbin\AppInfo\Application->boot()
10. /var/www/nextcloud/lib/private/legacy/OC_App.php line 208
    OC\AppFramework\Bootstrap\Coordinator->bootApp()
11. /var/www/nextcloud/lib/private/legacy/OC_App.php line 141
    OC_App::loadApp()
12. /var/www/nextcloud/apps/dav/lib/AppInfo/Application.php line 205
    OC_App::loadApps()
13. /var/www/nextcloud/lib/private/AppFramework/Bootstrap/Coordinator.php line 190
    OCA\DAV\AppInfo\Application->boot()
14. /var/www/nextcloud/lib/private/legacy/OC_App.php line 208
    OC\AppFramework\Bootstrap\Coordinator->bootApp()
15. /var/www/nextcloud/lib/private/legacy/OC_App.php line 141
    OC_App::loadApp()
16. /var/www/nextcloud/lib/private/Files/SetupManager.php line 132
    OC_App::loadApps()
17. /var/www/nextcloud/lib/private/Files/SetupManager.php line 340
    OC\Files\SetupManager->setupBuiltinWrappers()
18. /var/www/nextcloud/lib/private/Files/SetupManager.php line 380
    OC\Files\SetupManager->setupRoot()
19. /var/www/nextcloud/lib/private/Files/Mount/Manager.php line 95
    OC\Files\SetupManager->setupForPath()
20. /var/www/nextcloud/lib/private/Files/View.php line 1390
    OC\Files\Mount\Manager->find()
21. /var/www/nextcloud/lib/private/Files/Node/Root.php line 205
    OC\Files\View->getFileInfo()
22. <<closure>>
    OC\Files\Node\Root->get()
23. /var/www/nextcloud/lib/private/Files/Node/LazyFolder.php line 72
    call_user_func_array()
24. /var/www/nextcloud/lib/private/Files/Node/LazyFolder.php line 149
    OC\Files\Node\LazyFolder->__call()
25. /var/www/nextcloud/lib/private/Files/AppData/AppData.php line 132
    OC\Files\Node\LazyFolder->get()
26. /var/www/nextcloud/lib/private/Template/JSCombiner.php line 88
    OC\Files\AppData\AppData->getFolder()
27. /var/www/nextcloud/lib/private/Template/JSResourceLocator.php line 125
    OC\Template\JSCombiner->process()
28. /var/www/nextcloud/lib/private/Template/JSResourceLocator.php line 77
    OC\Template\JSResourceLocator->cacheAndAppendCombineJsonIfExist()
29. /var/www/nextcloud/lib/private/Template/ResourceLocator.php line 78
    OC\Template\JSResourceLocator->doFind()
30. /var/www/nextcloud/lib/private/TemplateLayout.php line 379
    OC\Template\ResourceLocator->find()
31. /var/www/nextcloud/lib/private/TemplateLayout.php line 211
    OC\TemplateLayout::findJavascriptFiles()
32. /var/www/nextcloud/lib/private/legacy/OC_Template.php line 184
    OC\TemplateLayout->__construct()
33. /var/www/nextcloud/lib/private/Template/Base.php line 132
    OC_Template->fetchPage()
34. /var/www/nextcloud/lib/private/legacy/OC_Template.php line 333
    OC\Template\Base->printPage()
35. /var/www/nextcloud/index.php line 74
    OC_Template::printExceptionErrorPage()

GET /index.php/apps/recommendations/api/recommendations
from 192.168.1.1 at 2023-03-25T20:01:04+00:00

here is my Backup App config :
image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the backup stack-trace entry points in apps/backup/lib/Service/ChunkService.php, apps/backup/lib/Service/PackService.php, and apps/backup/lib/Cron/Manage.php, then compare behavior with background jobs enabled. Reproduce the high-CPU and “Too many connections” symptoms while checking the SQLDump path and related logs; done requires identifying a specific backup failure and a verified fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
apache, mariadb, php
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.