hydrusnetwork / hydrusnetwork/hydrus

Duplicates filtering sorting and filter

Open
#424 1 comment 22 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request system:duplicates system:searching-sorting
Dominant language
Python
Stars
3.2k
Forks
207
PR merge metrics
No merged PRs in 30d

Description

Some sorting and filter settings to make dupe work a bit easier.

Dupe sorting

  • Pixel dupes
  • Most similar/smallest difference
  • Least similar/biggest difference
  • Group potential duplicate groups so you can deal with all alternates and dupes of large groups at once instead of doing them pieacemeal
    Would probably need to pre-calculate score and store pixel dupes and similarity status in db if it's not already done.

Some users would prefer dealing with one typ of dupes before another. Maybe do pixel dupes or very similar files first and then coast through the pairs where one is obviously superior to the other or clearly alternates.

Dupe filter

  • Show only pixel perfect dupe pairs
  • Show only same/different filetype pairs
  • Show only same/different filesize pairs
  • Show only same/different resolution pairs
  • Show only same/different aspect ratio pairs
  • Show only same/different URL/domain
  • Filter by distance

Similar to the dupe sorting options but more specific. With better filter settings there's perhaps decent hope for auto-resolving some dupe decisions, or making them easier at least.

At the very least, better sorting and filters would make dupe working a bit more predictable than the current, apparently very random, way where you don't quite know what's coming up next.

Filtering by distance would let a user do a full search up to whatever distance they ever feel like dealing with and yet only have to dupe filter what they feel like doing right then.

A lot of dupes that are alternates comes from the same gallery URL such as Pixiv albums while similarily something gotten from Twitter and then Pixiv are likely to different quality duplicates. Much like the other filter settings this would make things more predictable. With access to post time (#263) you could use domain filter to make it easier to tell if you're dealing with revised images.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Begin by locating the duplicate sorting and filtering flow, including the existing pixel-perfect and distance filters; done would require an agreed scope and tests for the selected options.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.