The codecs module mostly but not fully supports non str<->bytes codecs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
There's in the codecs module an option for charbuffertype which allows bytes<->bytes codecs, but if there were a type for the encoded value, it would allow cleaner implementation of str<->str codecs, or more...
Most codecs in CPython use decoded str and encoded bytes, there are also codecs that have decoded bytes and encoded bytes, like base64 (note that a base64 codec arguably could be one that considers decoded bytes and encoded str).
I had been more or less perverting the codecs module to implement things like COBS (fundamentally bytes<->bytes), bubblebabble (I did bytes<->bytes but it should encode to str), and recently ANSI escape filtering, as a str<->str codec that works by implementing a virtual terminal, with a screen buffer. To make it work I only had to do self.bytebuffer = "" in the StreamReader constructor, but since there's a charbuffertype, maybe we could use a bytesbuffertype, and open the flood gates to all kinds of creative abuse.
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the codecs module and inspect the StreamReader constructor, including the existing charbuffertype behavior. Determine the intended model for encoded and decoded value types, then define what broader non-str/bytes codec support should cover; the work is done only once that scope and its expected behavior are established and implemented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100