IronLanguages / IronLanguages/ironpython3

Update unicodedata module

Open
#691 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
2.8k
Forks
316
Avg merge
1d 9h
Merged PRs (30d)
1

Description

Module unicodedata needs to be updated:

  • It currently provides Unicode Data 5.2.0. Python 3.4 provides Unicode Data 6.3.0, so the data needs to be updated.
  • The script to convert Unicode Data downloaded from the Unicode Consortium's website to an embedded resource (genunicodedata.py) is written in Python 2. Needs to be converted to Python 3.
  • The project needs to contain the Copyright and Permission Notice from Unicode Consortium, to satisfy the usage and distribution requirements. Probably including the clause in the Documentation folder will be sufficient.
  • Python also provided Unicode Data 3.2.0, for those applications that need to conform to IDNA 2003. The current IronPython unicodedata module simply aliases the 5.2 database as 3.2. This needs to be corrected and the proper 3.2.0 database needs to be provided. The Unicode Data 3.2.0 is in a different format than 6.3.0, so a new conversion script needs to be written.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading genunicodedata.py and the existing unicodedata module to understand how Unicode Data 5.2.0 is converted and embedded. Compare the required 6.3.0 and 3.2.0 source formats, convert the script to Python 3, add the Unicode Consortium notice in the Documentation folder, and verify that both databases are provided rather than aliasing 3.2 to 5.2.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.