json-schema-org / json-schema-org/JSON-Schema-Test-Suite

jsonschema_suite check fails with UnicodeDecodeError on Windows (non-UTF-8 default locale)

Open Beginner friendly
#1,009 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
750
Forks
307
Avg merge
3d 17h
Merged PRs (30d)
24

Description

While running the sanity check script on Windows, I ran into an issue where python bin/jsonschema_suite check fails to parse test files that contain non-ASCII Unicode characters, such as idn-hostname.json and non-bmp-regex.json.

What happened?

It looks like path.read_text() is being called without an explicit encoding.

On Windows, Python uses the system's default locale encoding (in my case, cp1252) instead of UTF-8. Since these test files are UTF-8 encoded, this causes a UnicodeDecodeError when Python encounters certain characters.

Environment
  • OS: Windows 10
  • Python Version: Python 3.12
  • Command: python bin/jsonschema_suite check
Proposed Fix

We can pass encoding="utf-8" to all read_text() calls in bin/jsonschema_suite so that files are always read as UTF-8, regardless of the host OS locale:

json.loads(path.read_text(encoding="utf-8"))
Screenshots
Image Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in bin/jsonschema_suite by locating every read_text() call and check how the suite loads files such as idn-hostname.json and non-bmp-regex.json. Run python bin/jsonschema_suite check on Windows or a non-UTF-8 locale; done means the check parses these UTF-8 files without UnicodeDecodeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.