deezer / deezer/weslang

Issue detecting languages in non latin languages

Open
#4 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
54
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Hello,
I've compiled the cld2 lib and built the Java project. When I try detecting some texts, it seems to work for latin languages (Dutch, Spanish, French, English) but when I feed it Arabic or Hebrew, the Result always returns "UNKNOWN".

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the Java project's language detection with Arabic and Hebrew text, then compare the input and results with the working Latin-language cases. Inspect the Java integration with the compiled cld2 library, and consider the issue done when non-Latin samples return their detected languages instead of UNKNOWN.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.