Incorrect (too long) header from server causes Twist... | twisted/twisted

Nobody has claimed this yet.

	@pawelmhm reported
Trac ID	trac#8570
Type	enhancement
Created	2016-07-07 12:58:22Z

If Twisted Agent receives HTTP header with value that exceeds size limits defined in protocol it will fail and crash.

Expected behavior: if server sends incorrect header client should not fail but simply ignore this header.

To reproduce, create following Twisted server sending header with 100 000 characters in value:

#
import sys
from twisted.python import log
from twisted.web import server, resource
from twisted.internet import reactor

class Simple(resource.Resource):
    isLeaf = True

    def render_GET(self, request):
        request.setHeader("Set-Cookie", "a" * 100000)
        return "<html>Hello, world!</html>"

site = server.Site(Simple())
reactor.listenTCP(8080, site)
log.startLogging(sys.stdout)
reactor.run()

Twisted client trying to reach above server:

from twisted.internet import reactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers

agent = Agent(reactor)

d = agent.request(
    'GET',
    'http://localhost:8080',
    Headers({'User-Agent': ['Twisted Web Client Example']}),
    None)


def cbResponse(failure):
    print(failure)


d.addBoth(cbResponse)


def cbShutdown(ignored):
    reactor.stop()

d.addBoth(cbShutdown)

reactor.run()

Output is:

python client.py 
[Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.ResponseFailed'>: [<twisted.python.failure.Failure exceptions.AttributeError: 'TransportProxyProducer' object has no attribute 'loseConnection'>]

Searchable metadata

trac-id__8570 8570
type__enhancement enhancement
reporter__pawelmhm pawelmhm
priority__normal normal
milestone__None None
branch__ 
branch_author__ 
status__new new
resolution__None None
component__core core
keywords__http__Agent__webclient http, Agent, webclient
time__1467896302835926 1467896302835926
changetime__1511740897981113 1511740897981113
version__None None
owner__None None

The reproduction uses twisted.web.server/resource and twisted.web.client.Agent; start by running the supplied server and client to confirm the failure. Trace the Agent response path around the reported ResponseFailed and TransportProxyProducer error, then add a regression test for the oversized response header. Done means the client handles that malformed header without failing.

Written by the indexing model from the issue text.

Incorrect (too long) header from server causes Twisted Agent to fail

Description

Contributor guide

First steps

Research direction

Assessment