TLS: Twisted should buffer transport writes before encryption

	@oberstet reported
Trac ID	trac#6899
Type	enhancement
Created	2014-01-15 17:46:09Z

When using plain TCP, as far as I understand, multiple smallish writes to the transport will be buffered up inside Twisted, and only then flattened out and written to the socket. This results in large TCP segments and hence efficient use (low TCP overhead compared to aggregate payload).

However, with TLS, I have done measurements that lead me to believe, each and every call to transport.write will result in a new TLS record being started. The individual TLS records again seem to get buffered up, and largish TCP segment written to the socket, but the overhead induced by excessive number of TLS records can get big.

Now, applications using TLS can of course buffer up stuff themselves, and only call transport.write once in a while, but this seems wrong.

What seems right is buffering up payloads from transport.write before encrypting (as in the non-TLS case), and only call into OpenSSL with the flattened and concatenated payload for encrypting later inside Twisted. And then write the encrypted stuff to the socket.

From what I understand (which might be wrong), this should induce less TLS records, and hence incur less wire-level overhead.

Note: I understand that the transport.write interface makes no guarantees in which way it will produce TLS records when the transport runs over TLS. Nevertheless I think it would be nice if above optimization would be available.

Downsides:

• Since with above proposal, unencrypted payload is buffer inside Twisted, this might be seen as "less secure" than today. However, I'd argue, that an attacker that has made it either into the Python process running Twisted or has succeeded to read arbitrary machine memory can compromise the application in any case.

• With above proposal, the sending of data to socket does not happen immediately - same as with today. The difference is that the encryption is postponed. Today, encryption happens at transport.write time. With the proposal, encryption happens just before socket.send. Not sure what the implications are.

trac-id__6899 6899
type__enhancement enhancement
reporter__oberstet oberstet
priority__normal normal
milestone__ 
branch__ 
branch_author__ 
status__new new
resolution__None None
component__core core
keywords__ 
time__1389807969000000 1389807969000000
changetime__1389807969000000 1389807969000000
version__None None
owner__ 

The issue does not name implementation files, tests, or entry points. Start by tracing Twisted's transport.write path for plain TCP and TLS, then compare where payloads are buffered and encrypted; done means a reviewed design and tests demonstrating the intended buffering behavior without changing the transport.write interface.