`files:scan-app-data` database optimization
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
I found something that really speed up files:scan-app-data cron
function getUsersToScan in ./lib/BackgroundJob/ScanFiles.php generarate this SQL:
SELECT MAX(`user_id`) FROM `oc_filecache` `f` INNER JOIN `oc_mounts` `m` ON `storage_id` = `storage` WHERE `size` < 0 GROUP BY `storage_id`;
but it use fs_size index on oc_filecache, and I think is wrong
Better will work to use fs_storege_size index
For my instance it took about 10 minutes on fs_size index, but on fs_storage_size it takes 2-3 seconds!
So my suggestion is change function to:
private function getUsersToScan(): array {
$query = $this->connection->getQueryBuilder();
$query->select($query->func()->max('user_id'))
->from('filecache FORCE INDEX(`fs_storage_size`)', 'f')
->innerJoin('f', 'mounts', 'm', $query->expr()->eq('storage_id', 'storage'))
->where($query->expr()->lt('size', $query->createNamedParameter(0, IQueryBuilder::PARAM_INT)))
->groupBy('storage_id')
->setMaxResults(self::USERS_PER_SESSION);
return $query->execute()->fetchAll(\PDO::FETCH_COLUMN);
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in lib/BackgroundJob/ScanFiles.php at getUsersToScan() and inspect the query builder call and the fs_size and fs_storage_size indexes mentioned in the issue. Run the files:scan-app-data cron against a comparable database and verify that the generated query uses the intended index and improves scan time without changing the returned users.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, sql
- Domain
- backend, database
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100