bigscience-workshop / bigscience-workshop/pii_processing

add backtrans for regex pipeline

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9
Forks
6
PR merge metrics
No merged PRs in 30d

Description

in test_regex.py, add ability to do backtrans, first trans to english, perform ner, then translate back to target_lang, and match the ner entity between back transed text and original text.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in test_regex.py and read the existing regex pipeline to identify its translation and NER entry points. Trace the current test flow, then verify that it translates to English, performs NER, translates back to target_lang, and matches entities between the back-translated and original text.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.