HTTP client gives incorrect Content-Length for automatically decompressed responses

The application below has two handlers, the first of which serves a gzipped tarball at /hello.tar.gz, and the second of which tries to proxy /hello2.tar.gz to the first. But the second actually raises tornado.httputil.HTTPOutputError: Tried to write more data than Content-Length.

The reason is that the Tornado HTTP client has decompressed the Content-Encoding: gzip without adjusting the Content-Length field, resulting in an invalid response. A workaround is setting decompress_response=False.

When decompress_response is on, Tornado should adjust or remove the Content-Length field of decompressed responses, similarly to how it already removes the Content-Encoding field.

I would also argue that decompress_response is unexpected behavior that should really be off by default, even if it’s fixed, as it changes the semantic meaning of the response: Content-Encoding is not Transfer-Encoding.

import tarfile
import io
import tornado.ioloop
import tornado.web
import tornado.httpclient

class TgzHandler(tornado.web.RequestHandler):
    def get(self):
        b = io.BytesIO()
        with tarfile.open(fileobj=b, mode="w:gz") as tar:
            contents = b"Hello, world!\n"
            info = tarfile.TarInfo("hello.txt")
            info.size = len(contents)
            tar.addfile(info, io.BytesIO(contents))
        self.set_header("Content-Type", "application/x-tar")
        self.set_header("Content-Encoding", "gzip")
        self.write(b.getvalue())

class ProxyHandler(tornado.web.RequestHandler):
    async def get(self):
        resp = await tornado.httpclient.AsyncHTTPClient().fetch(
            "http://localhost:8888/hello.tar.gz"
        )
        for k, v in resp.headers.get_all():
            self.add_header(k, v)
        self.write(resp.body)

app = tornado.web.Application(
    [(r"/hello.tar.gz", TgzHandler), (r"/hello2.tar.gz", ProxyHandler)]
)
app.listen(8888)
tornado.ioloop.IOLoop.current().start()

Start at AsyncHTTPClient.fetch with decompress_response enabled and reproduce the supplied proxy example. Verify that a decompressed response no longer retains an invalid Content-Length, so forwarding resp.headers and resp.body does not raise HTTPOutputError; the default-behavior question is a separate concern.