IronLanguages / IronLanguages/ironpython3
CJK codecs are incomplete.
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 2.8k
- Forks
- 316
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 1
Description
Description
For CJK codecs (cp932 for example), the "encoding" sysmodule requires some complements and they are slightly complex.
In CPython, these complements are implemented in C.
- Modules/cjkcodecs
- e.g.) _codecs_jp.c for japanese locale (cp932, euc-jp, ...)
Steps to Reproduce
- Run ipy.exe without arguments.
Expected behavior:
IronPython.Runtime.Operations.PythonOps.LookupEncoding() does not throw any exceptions.
This method is called by importing "site" module.
Actual behavior:
IronPython.Runtime.Operations.PythonOps.LookupEncoding() throws following exception:
PythonOps.LookupError("unknown encoding: {0}", encoding)
Versions
IronPython 3.4.0a0 DEBUG (3.4.0.0000)
[.NETFramework,Version=v4.5 on .NET Framework 4.8.4075.0 (64-bit)] on win32
$ git rev-parse HEAD
342e49f69d9a967606d49394df51a059a55a4381
Workarounds (for developers)
The immediate issue is that make.ps1 test-all fails on CJK environment, but this can be workaround by (locally) applying patch as follows:
- return "cp" + CultureInfo.CurrentCulture.TextInfo.ANSICodePage.ToString();
+ return "cp" + (new CultureInfo("en-US", false)).TextInfo.ANSICodePage.ToString();
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at IronPython.Runtime.Operations.PythonOps.LookupEncoding() and reproduce the failure by running ipy.exe without arguments, then run make.ps1 test-all in a CJK environment. Compare the relevant CPython Modules/cjkcodecs files, including _codecs_jp.c, to identify the missing complements. Done means importing site and the test suite no longer fail with an unknown encoding error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, csharp, python
- Domain
- backend, internationalization
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100