clips / clips/pattern

Suggestion: Use UDPipe for support for 40 languages

Open
#154 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.9k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Context: I really like the look of pattern, but I'm looking for something that supports swedish. But since this isn't one of the big languages I doubt it will be added any time soon, looking at how much manual job there seems to be adding a new language to pattern.

A couple of days ago I found out about udpipe, I library that does tokenization, tagging, lemmatization and dependency parsing of 40 languages, with one API. It builds on open data, has a permissive license, and anyone can download free pre-trained models.

Idea: What would happen if you used udpipe to add the 35 languages you don't yet support? I understand this is a big project, but looking at open issues I see that a lot of the open issues are related to language specific things; things that you could get for free if you depended in a bigger library.

So basically, my suggestion is: integrate udpipe to pattern as a way to support more languages.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests; first locate Pattern’s existing language-support entry points and review how they compare with UDPipe’s tokenization, tagging, lemmatization, and dependency-parsing API. Done would require an agreed integration design and verified support for the proposed languages without breaking existing language behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.