twisted.names.server never truncates datagram response messages

	@wallrj reported
Trac ID	trac#6949
Type	defect
Created	2014-02-06 17:19:11Z
Branch	https://github.com/twisted/twisted/tree/names-truncate-udp-6949

Until it supports EDNS0, twisted.names.server ought to limit the size of
its response messages to 512B.

But because of the way dns.Message.decode sets the maxSize=0 and because of the
way server re-uses the request message for its responses, it never actually
truncates large messages.

This probably means that some responses will be dropped or fragmented on their
route across the Internet, which means that clients have to wait for a timeout
before re-issuing the query over TCP.

If instead, the response was properly truncated, the client would receive the
response and immediately issue a TCP query.

Here's a demo using a pyzone containing more A records than will fit in a 512B
response.

$ twistd -n dns --port 10053 --pyzone=docs/projects/names/howto/listings/names/example-domain.com

$ dig -p 10053 @localhost ns1.example-domain.com A +norecurse +notcp +ignore
...
;; flags: qr aa ad; QUERY: 1, ANSWER: 229, AUTHORITY: 0, ADDITIONAL: 0
...

ns1.example-domain.com.	3600	IN	A	127.0.0.1
ns1.example-domain.com.	3600	IN	A	127.0.0.1
ns1.example-domain.com.	3600	IN	A	127.0.0.2

;; Query time: 9 msec
;; SERVER: 127.0.0.1#10053(127.0.0.1)
;; WHEN: Thu Feb 06 17:09:45 GMT 2014
;; MSG SIZE  rcvd: 3704

Note that the datagram response was 3704B and contained the complete RRSET (ending in 127.0.0.2)

Now if we try again with the following patch applied

Index: twisted/names/dns.py
===================================================================
--- twisted/names/dns.py	(revision 41582)
+++ twisted/names/dns.py	(working copy)
@@ -2139,7 +2139,7 @@


     def decode(self, strio, length=None):
-        self.maxSize = 0
+#        self.maxSize = 0
         header = readPrecisely(strio, self.headerSize)
         r = struct.unpack(self.headerFmt, header)
         self.id, byte3, byte4, nqueries, nans, nns, nadd = r

$ dig -p 10053 @localhost ns1.example-domain.com A +norecurse +notcp +ignore
;; Warning: Message parser reports malformed message packet.

; <<>> DiG 9.9.3-rl.13207.22-P2-RedHat-9.9.3-14.P2.fc19 <<>> -p 10053 @localhost ns1.example-domain.com A +norecurse +notcp +ignore
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16064
;; flags: qr aa tc ad; QUERY: 1, ANSWER: 229, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: Messages has 6 extra bytes at end

...
ns1.example-domain.com.	3600	IN	A	127.0.0.1
ns1.example-domain.com.	3600	IN	A	127.0.0.1
ns1.example-domain.com.	3600	IN	A	127.0.0.1
ns1.example-domain.com.	3600	IN	A	127.0.0.1

;; Query time: 8 msec
;; SERVER: 127.0.0.1#10053(127.0.0.1)
;; WHEN: Thu Feb 06 17:13:53 GMT 2014
;; MSG SIZE  rcvd: 512

We see that the message has been limited to 512B and the TC flag has been set.

But we also see another problem; that the message hasn't been truncated
properly, just chopped off at 512B instead of at a record boundary....but that's
a separate issue.

I noticed this while working on #6645 because instead of using the request
message I'm generating a new response message whose maxSize defaults to 512.

Attachments:

• ticket#6949.patch (3923 bytes) - added by philippe.goetz on 2014-06-13 09:38:01Z -

trac-id__6949 6949
type__defect defect
reporter__rwall rwall
priority__high high
milestone__ 
branch__branches_names_truncate_udp_6949 branches/names-truncate-udp-6949
branch_author__jml jml
status__new new
resolution__None None
component__names names
keywords__ 
time__1391707151000000 1391707151000000
changetime__1413475793813915 1413475793813915
version__None None
owner__philippe_goetz philippe.goetz
cc__free.ekanayaka

Start in twisted/names/dns.py by reading dns.Message.decode and the server path that reuses the request message for responses. Reproduce the issue with the documented twistd and dig commands against the example pyzone, then verify that oversized datagram responses stop at 512B, set TC, and end at a record boundary.