WeblateOrg / WeblateOrg/weblate
Reverse Screenshot Association: Add `Auto-OCR` and `Find Screenshot` Features
- Dominant language
- Python
- Stars
- 6.1k
- Forks
- 1.4k
- Avg merge
- 9h 53m
- Merged PRs (30d)
- 395
Description
### Describe the problem
The current feature for associating strings with screenshots via OCR is quite nice and useful.
But it still has some limitations:
- The process of association is still a lot of work
I was able to create 100 screenshots in half an hour, but associating strings is still a time-consuming task
- It is per component
The strings in those 100 screenshots can come from multiple components - even the strings on a single screenshot can be sourced from multiple components. Currently, you need to go through all screenshots in each component, so when it's 5 components like in my example, you need to do OCR and assignments 500 times
- It often fails to recognize strings going over multiple lines on screen
This is difficult to handle, especially when you don't in which component a certain string is contained - you cannot do so many manual searches
- Screenshots without assuciation are causing alerts
It's suffitient to flag them, but this should not cause an alert. It is extremely annoyiing to delete dozens of screenshots just to clear the alert (I had to add those 100 screenshots to all 5 components in the firtst place)
- RELATED BUG
When you have screenshots without associations and you sort by "number of linked strings", those with 0 don't get properly sorted to the start or the end
### Solution brainstorm
- Most of the time, translators do not need a screenshot
- Scrreenshots still are **very useful in specific cases**
- But you do not know in advance which cases these might be
=> this kind of forces you to create associatings for ALL strings, even though just VERY FEW are really needed
## The Idea: Reverse the Procedure
### 1. Automatically Auto-OCR Screenshots when added
OCR-ing a screenshot is a light-weight operation. It can easily be done automatically in the background when a screenshot is added to a repository. The detected strings are stored in a database table with full-text search enabled.
(tesseract can also give you the location of each word as rectangle. Storing this would allow hightlighting - see below)
=> **This also improves the current procedure for assigning strings:** You don't need to click the buttion anymore for each screenshot
### 2. Add a "Find Screenshot" button to Translation Views
- In the sidebar, where screenshots are shown, add a "Find Screenshot" button.
- This will open a dialog which allows searching for screenshots by entering words.
- The search box is pre-populated with the current source-language phrase
Below, a list of screenshots is shown which have the searched term. If containment rects are stored, the search terms could even be highlighted in each screenshot image).
Finally, the dialog should provide the opportunity to create an association so that other translators won't need to search for it afterwards.
Contributor guide
Assessment
This issue has not been assessed yet.