Perl-Toolchain-Gang / Perl-Toolchain-Gang/Module-Metadata
handling encodings
Open
@karenetheridge is already working on this.
Since Mar 20, 2015.
enhancement
- Dominant language
- Perl
- Stars
- 8
- Forks
- 17
- PR merge metrics
- No merged PRs in 30d
Description
Here is my (very early in the morning) assessment about what we should do regarding scanning the file and encodings (related issue: https://rt.cpan.org/Ticket/Display.html?id=78434)
- we need different interfaces for new-from-string, new-from-file, new-from-handle. new-from-string will assume decoded characters. new-from-file takes an encoding parameter, and uses binmode(":encoding($encoding)") on the handle. new-from-handle assumes that the caller has already set encoding layers appropriately.
- new-from-file may not have been provided an encoding parameter. in this case we start reading assuming ascii. If we encounter a 'use utf8;' directive, we back up and start over with the correct encoding applied. We will also flag a warning (error?) if we encounter a 'use utf8' directive when we were explicitly passed a different encoding parameter than 'utf8' (UTF-8?).
- pod can have a different encoding than the code. when starting to read pod, and we don't already have an encoding supplied (can we assume that new-from-string is decoded and therefore exempt ourself from this step?) and we encounter an '=encoding ...' directive, we decode the extracted pod lines as they are read. Test cases need to consider corpus examples where the code is in a different encoding than the (interleaved) pod (can this happen in any situation where the code is not ascii, which is utf8-compatible?)
Am I missing anything?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.