HaxeFoundation / HaxeFoundation/haxe

Specify handling of invalid bytes in Bytes.getString/toString

Open
#12,598 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haxe
Stars
6.9k
Forks
715
Avg merge
2d 2h
Merged PRs (30d)
11

Description

Currently eval throws on invalid utf8, but only for Bytes.getString: #10623. Other targets have inconsistent behaviour.

This has caused issues in the past on lua, because getString allowed an invalid string which broke String.length on certain versions of lua-utf8: #11088.

Quoting @Apprentice-Alchemist:
> one solution is to decode as UTF-8 and replace invalid sequences with U+FFFD.
This is already done by JS and Python.

If it's not too costly on some targets, it seems worth going with this behaviour.

Also related:
#7904

Contributor guide

Open the contributing guide

Research direction

Compare Bytes.getString and toString implementations for the targets mentioned, beginning with existing invalid-UTF-8 handling in JS and Python. Review related issues #10623, #11088, and #7904 for prior decisions. Done means the behavior for invalid bytes is consistent across the affected targets and the chosen handling is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, lua, python
Domain
compilers
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.