TCP MD5 hash calculated incorrectly

Brief description

The curent code does not correctly compute the hash of the TCP MD5 option.

According to RFC2385, tcpdump, and the Linux kernel, the TCP options must be skipped when hashing the packet, while their lengths still affect the result through the segment length in the TCP pseudo header and data offset in the TCP header, both of which are part of the hash input.

Scapy version

251daecbd3f4e91f437908332cc17a517305674e

Python version

3.14.7

Operating system

Arch Linux 7.2.3-arch1-3

Additional environment information

No response

How to reproduce

• tcpdump with -M ... configuring the key used to verify the MD5 checksum

  sudo tcpdump -tlnni lo -M 12345678 tcp port 9999
  

• Python3/scapy to forge and send packets with the same key (12345678) used above

  pkt = IPv6(dst="::1")/TCP(dport=9999)
  sign_tcp_md5(pkt[TCP], b"12345678")
  send(pkt)
  
  pkt = IP(dst="127.0.0.1")/TCP(dport=9999)
  sign_tcp_md5(pkt[TCP], b"12345678")
  send(pkt)
  

Actual result

IP6 ::1.20 > ::1.9999: Flags [S], seq 0, win 8192, options [md5  (invalid),eol], length 0                                                                                                                                                                                                  
IP 127.0.0.1.20 > 127.0.0.1.9999: Flags [S], seq 0, win 8192, options [md5  (invalid),eol], length 0

The md5 (invalid) is the problem.

Expected result

IP6 ::1.20 > ::1.9999: Flags [S], seq 0, win 8192, options [md5 valid,eol], length 0
IP 127.0.0.1.20 > 127.0.0.1.9999: Flags [S], seq 0, win 8192, options [md5 valid,eol], length 0

The md5 valid is what you should see.

Related resources

No response

Start at the sign_tcp_md5 entry point used in the reproduction and compare its packet hashing inputs with RFC2385, tcpdump, and the Linux kernel behavior described in the issue. Verify the change by sending both IPv4 and IPv6 packets with the documented key and confirming tcpdump reports the TCP MD5 option as valid.