Codec error handling callback should return newpos > start
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 36k
- PR merge metrics
- PR metrics pending
Description
According to PEP 293, the codec error handling callback should either raise an exception or return a pair (replacement, newpos). newpos specifies a new position within object, where the encoder/decoder will continue encoding/decoding. Negative values for newpos are treated as being relative to end of object. PEP 293 does not specify the bound of newpos, and this is a problem.
If newpos == start, the codec will not advance. If replacement is empty, it will just hang in an infinite loop, calling the callback again and again. If replacement is not empty, it will additionally add it in an internal buffer, and finally, after long time, you will get a MemoryError. If newpos < start, the behavior may be even more weird.
Before fixing other errors in #81000 it even caused a crash, but now it is only an infinite loop.
I propose to limit the returning value by range [start+1, len(object)].
@malemburg, @doerwalter, @vstinner
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading PEP 293 and tracing the encoder/decoder error-callback behavior described in the issue. Investigate how returned newpos values are handled when they are equal to or below start, then add coverage for the reported looping and invalid-position cases. Done means these callbacks no longer cause non-advancing processing or unbounded buffering.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100