WeblateOrg / WeblateOrg/weblate
"Similar Strings" Tab in Translation View
- Dominant language
- Python
- Stars
- 6.1k
- Forks
- 1.4k
- Avg merge
- 9h 53m
- Merged PRs (30d)
- 395
Description
### Describe the problem
Currently, Weblate offers a "Similar keys" tab, which is very useful for projects where translation keys (`msgid`) carry semantic meaning. However, for many projects (especially those using Gettext or similar formats), the keys are the source strings themselves or opaque identifiers.
This leaves a gap in functionality: there is no easy way for a translator to find and reference other source strings that are textually similar to the one they are currently working on. This can lead to inconsistencies in translation, for example:
* Slight variations of a phrase (e.g., "Click the button" vs. "Please click the button") might be translated differently by different translators or at different times.
* A translator might not remember the specific terminology used for a similar concept in another part of the project.
* Verb conjugations or plural forms are translated without context of their base form.
The current workaround is to use the global search, but this is inefficient as it removes the translator from their current context and doesn't provide a ranked list of similarity.
### Solution brainstorm
I propose adding a new tab in the translation interface, right next to "Nearby strings," "Similar keys," "History," etc., called **"Similar strings."**
This feature would function analogously to "Similar keys," but would perform a similarity search based on the *content of the source string* instead of its key.
**Example:**
If a translator is working on the English string **"A quick brown fox"**, the "Similar strings" tab would show other source strings from the same component like:
* "The quick brown fox jumps over the lazy dog." (High similarity)
* "A quick brown dog" (Medium similarity)
* "A slow red fox" (Lower similarity)
**Technical Implementation Idea:**
1. **Backend Logic (Search):**
* **Recommended Approach (Scalable):** Implement a database-backed Full-Text Search (FTS). For PostgreSQL, this can be achieved efficiently using the `pg_trgm` extension and `TrigramSimilarity`. This would involve:
* Adding a `SearchVectorField` to the `String` model in `weblate.trans.models`.
* Creating a database migration and a mechanism to keep the vector updated.
* A new function that uses `TrigramSimilarity` or `SearchRank` to find and rank similar strings within the same component.
* **Alternative Approach (Simpler):** Utilize the existing `rapidfuzz` library (already a Weblate dependency). A function could iterate over strings in the component and calculate a `fuzz.ratio`. This would be less performant on very large components but easier to implement for a proof-of-concept.
2. **API Endpoint:**
* Create a new API endpoint, for example: `/api/units//similar-strings/`.
* This endpoint would asynchronously fetch the list of similar strings when the user clicks the tab, preventing a performance hit on the initial page load. It would follow the pattern used by the "History" or "Machinery" tabs.
3. **Frontend UI:**
* Modify the translation view template (`weblate/templates/trans/translate.html`) to add the new tab and a corresponding panel.
* Use JavaScript (e.g., in `weblate/static/js/trans.js`) to handle the AJAX call to the new API endpoint upon tab activation.
* The JavaScript would then render the returned list of similar strings, with each item linking to its respective translation unit.
### Describe alternatives you have considered
1. **Manual Search:** As mentioned, a translator can manually use the project-wide search. This is disruptive to the workflow, requires leaving the translation view, and doesn't provide a ranked list of similarity focused on the current component.
2. **Translation Memory (TM):** The TM is excellent for exact or high-percentage matches of full strings. However, it's not designed to find strings that are just *conceptually* or *partially* similar. For instance, TM wouldn't typically link "Log in to your account" with "Please sign in to continue," whereas a textual similarity search would likely find this connection valuable for maintaining consistency.
This feature would complement the TM by providing a different axis of similarity, focused on the source text itself.
### Screenshots
I envision the new tab appearing here in the user interface:
The new **"Similar strings"** tab would be located alongside the existing tabs:
`[ Nearby strings ] [ Similar keys ] [ Similar strings ] [ Other languages ] [ History ]`
### Additional context
This feature would significantly enhance translator consistency and efficiency, especially in large projects with many similar but not identical strings. It leverages existing technologies in Weblate (`rapidfuzz`, powerful database backends) and follows established UI patterns within the application. It would be a powerful addition to the translator's toolkit, sitting nicely between the "Nearby strings" and "Translation Memory" features.
Contributor guide
Research direction
Start by reviewing the existing Similar keys, History, and Machinery tab flows, then inspect weblate.trans.models, weblate/templates/trans/translate.html, and weblate/static/js/trans.js. Clarify the search and ranking approach before implementation. Done means a Similar strings tab can load ranked source strings from the same component through the new API and link to their translation units.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, javascript, postgresql, python
- Domain
- backend, databases, frontend, localization
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100