EleutherAI / EleutherAI/dps

[ja] replace Japanese PII

Open
#49 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
93
Forks
26
PR merge metrics
No merged PRs in 30d

Description

# Background
- Seems that we don't have to implement a lot of pre-processing to replace Japanese PII
- because there are already some PII pre-processing in language agnostic processing.
- But we might need to implement additionally to replace some Japanese specific PII.

# TODOs
- [ ] replace Japanese phone number
- But we might be able to just re-use the Korean one
- cf. https://github.com/EleutherAI/dps/blob/master/dps/spark/utils/korean_utils.py#L182
- [ ] replace Japanese bank account number

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.