fire-eggs / fire-eggs/Danbooru2021

FilterView performance is slow

Open
#58 0 comments 0 reactions 0 assignees View on GitHub
bug database
Dominant language
Python
Stars
42
Forks
2
PR merge metrics
No merged PRs in 30d

Description

The query when using FilterView is not performant. Mainly because the use of `like` statements requires scanning the tags table (no useful index possible).

E.g. the following query:
```
select image_id from images where is_deleted=0 and is_banned=0 and image_id in
(select image_id from imageTags where tag_id in (select tag_id from tags where name like '%girl%')) order by image_id
```
takes over 22 seconds to execute (returning 6,260,963 rows).

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the FilterView implementation and reproduce the reported query against the SQLite dataset. Profile the tag lookup and filtering path before deciding on an approach. Done means FilterView returns the same results with a measurable improvement over the reported 22-second query.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
databases, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.