Document iconv implementation quirks
Nobody has claimed this yet.
- Dominant language
- XML
- Stars
- 596
- Forks
- 890
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 55
Description
tl;dr document some if the problems I ran into in https://github.com/php/php-src/issues/22413
PHP's iconv() function mostly defers to the system's iconv_open() and iconv() routines. POSIX 2024 covers these:
- https://pubs.opengroup.org/onlinepubs/9799919799/functions/iconv_open.html
- https://pubs.opengroup.org/onlinepubs/9799919799/functions/iconv.html
As you can see, there are three magic suffixes that can be appended to the "to" charset:
- //IGNORE
- //TRANSLIT
- //NON_IDENTICAL_DISCARD
In practice, support for these varies widely, and when they are supported they may not conform to POSIX. For example, musl does not support //IGNORE, and (man 3 iconv) BSD/Solaris replace unknown translations with ? rather than raise an error that would require //TRANSLIT... so they do not have //TRANSLIT.
The details are sordid, and we should not try to capture them in the PHP documentation. Instead, we should simply state that support for any iconv() feature can vary widely depending on your implementation. Right now the PHP iconv docs mention //TRANSLIT and //IGNORE, but say only that //TRANSLIT is unreliable.
- We should also mention that
//IGNOREis implementation-dependent. PHP has a workaround for iconv implementations (glibc) that return an error upon ignoring an invalid sequence, which helps standardize PHP's behavior, but//IGNOREmay not be supported at all (musl), and there is no workaround for that. //NON_IDENTICAL_DISCARDneeds a mention.- The "to" and "from" charsets are also implementation-defined. Musl for example only supports a few, and will fall back to UTF-8 if you pass in something it doesn't understand.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the PHP iconv() manual page linked in the issue and compare its current //TRANSLIT and //IGNORE notes with the cited POSIX iconv_open() and iconv() specifications. Update the documentation to cover implementation-dependent suffix support, //NON_IDENTICAL_DISCARD, and implementation-defined charsets, while preserving the page’s existing guidance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100