asyncio SSL loses received data when `get_buffer()` returns a `bytearray`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
Over TLS, a BufferedProtocol returning a bytearray from get_buffer() gets NULs instead of data - with the byte count still reported as correct.
Repro (run from a CPython directory root):
import asyncio
import ssl
D = "Lib/test/certdata/"
class Buffered(asyncio.BufferedProtocol):
buf = bytearray(100)
def get_buffer(self, hint): return self.buf
def buffer_updated(self, n): print("BufferedProtocol:", bytes(self.buf[:n]))
class Plain(asyncio.Protocol):
def data_received(self, data): print("Protocol :", data)
async def h(r, w):
w.write(b"A"*30); await w.drain(); w.write(b"B"*30)
async def main():
srv_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER); srv_ctx.load_cert_chain(D + "keycert3.pem")
cli_ctx = ssl.create_default_context(cafile=D + "pycacert.pem")
srv = await asyncio.start_server(h, "localhost", 0, ssl=srv_ctx)
port = srv.sockets[0].getsockname()[1]
loop = asyncio.get_running_loop()
for proto in (Buffered, Plain):
await loop.create_connection(proto, "localhost", port, ssl=cli_ctx)
await asyncio.sleep(0.3)
asyncio.run(main())
The output would be, Protocol is here to have an example of proper behavior
BufferedProtocol: b'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
Protocol : b'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBB'
Doc says (asyncio-protocol.rst - 620)
get_buffer() must return an object implementing the buffer protocol.
Proposed fix is to take memoryview(buf) once before the loop and read into view[offset:]
I have a fix ready for that
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-156276
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the asyncio protocol documentation at asyncio-protocol.rst around line 620 and reproduce the TLS BufferedProtocol example from the issue. Then inspect linked PR gh-156276 and the relevant asyncio SSL transport tests; done means bytearray buffers preserve received TLS data and the regression is covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100