GNS3 / GNS3/dynamips

Device links fail. Source IP is wrong.

Open
#76 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
405
Forks
99
Avg merge
3m
Merged PRs (30d)
1

Description

I've got an issue with dynamips 0.2.11 and 0.2.16 on an AWS instance of Ubuntu 14.04. All the links between simulated devices (ethernet and serial) are unable to pass any traffic. The key difference I see between this instance and a working instance on my Mac is that the UDP tunneled traffic on the loopback uses the wrong source IP.

Tcpdump on loopback for working instance on Mac OS X, pinging R2 FA0/0 from R1 FA0/0. All traffic is sourced from 127.0.0.1:

13:05:28.908787 IP (tos 0x0, ttl 64, id 7909, offset 0, flags [none], proto UDP (17), length 88, bad cksum 0 (->5dae)!)
    127.0.0.1.10000 > 127.0.0.1.10001: [bad udp cksum 0xfe57 -> 0xb439!] UDP, length 60
    0x0000:  0200 0000 4500 0058 1ee5 0000 4011 0000  ....E..X....@...
    0x0010:  7f00 0001 7f00 0001 2710 2711 0044 fe57  ........'.'..D.W
    0x0020:  cc02 6b01 0000 cc02 6b01 0000 9000 0000  ..k.....k.......
    0x0030:  0100 0000 0000 0000 0000 0000 0000 0000  ................
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000 0000 0000 0000 0000 0000            ............
13:05:29.708890 IP (tos 0x0, ttl 64, id 35228, offset 0, flags [none], proto UDP (17), length 88, bad cksum 0 (->f2f6)!)
    127.0.0.1.10001 > 127.0.0.1.10000: [bad udp cksum 0xfe57 -> 0xb435!] UDP, length 60
    0x0000:  0200 0000 4500 0058 899c 0000 4011 0000  ....E..X....@...
    0x0010:  7f00 0001 7f00 0001 2711 2710 0044 fe57  ........'.'..D.W
    0x0020:  cc03 6b02 0000 cc03 6b02 0000 9000 0000  ..k.....k.......
    0x0030:  0100 0000 0000 0000 0000 0000 0000 0000  ................
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000 0000 0000 0000 0000 0000            ............

Broken instance on AWS Ubuntu 14.04, pinging R2 from R1. All pings and some other traffic are sourced from the external IP address of the AWS instance:

15:38:30.435262 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 102: (tos 0x0, ttl 64, id 62743, offset 0, flags [DF], proto UDP (17), length 88)
    <Eth0's external IP>.10000 > 127.0.0.1.10001: [bad udp cksum 0x9e5d -> 0xb83a!] UDP, length 60
    0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
    0x0010:  0058 f517 4000 4011 a775 0a63 14a4 7f00  .X..@.@..u.c....
    0x0020:  0001 2710 2711 0044 9e5d ffff ffff ffff  ..'.'..D.]......
    0x0030:  ca00 2c5e 0006 0806 0001 0800 0604 0001  ..,^............
    0x0040:  ca00 2c5e 0006 ac14 0001 0000 0000 0000  ..,^............
    0x0050:  ac14 0002 0000 0000 0000 0000 0000 0000  ................
    0x0060:  0000 0000 0000                           ......
15:38:30.435288 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 130: (tos 0xc0, ttl 64, id 27801, offset 0, flags [none], proto ICMP (1), length 116)
    127.0.0.1 > 127.0.0.1: ICMP 127.0.0.1 udp port 10001 unreachable, length 96
    (tos 0x0, ttl 64, id 62743, offset 0, flags [DF], proto UDP (17), length 88)
    127.0.0.1.10000 > 127.0.0.1.10001: [bad udp cksum 0x3e63 -> 0x5840!] UDP, length 60
    0x0000:  0000 0000 0000 0000 0000 0000 0800 45c0  ..............E.
    0x0010:  0074 6c99 0000 4001 0f2e 7f00 0001 7f00  .tl...@.........
    0x0020:  0001 0303 1532 0000 0000 4500 0058 f517  .....2....E..X..
    0x0030:  4000 4011 477b 7f00 0001 7f00 0001 2710  @.@.G{........'.
    0x0040:  2711 0044 3e63 ffff ffff ffff ca00 2c5e  '..D>c........,^
    0x0050:  0006 0806 0001 0800 0604 0001 ca00 2c5e  ..............,^
    0x0060:  0006 ac14 0001 0000 0000 0000 ac14 0002  ................
    0x0070:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0080:  0000                                     ..

On the broken instance I see these logs:

Oct 04 16:32:57.300 HYPERVISOR: exec_cmd: hypervisor version 
Oct 04 16:32:57.300 HYPERVISOR: exec_cmd: hypervisor reset 
Oct 04 16:32:57.300 GENERAL: reset done.
Oct 04 16:32:57.300 HYPERVISOR: exec_cmd: hypervisor working_dir /<snip!>/dynagen/simple_2_routers 
Oct 04 16:32:57.300 GENERAL: working_dir=/home/cvazquez/dynagen/simple_2_routers
Oct 04 16:32:57.300 HYPERVISOR: exec_cmd: vm create R1 0 c7200 
Oct 04 16:32:57.302 VM: VM R1 created.
Oct 04 16:32:57.302 HYPERVISOR: exec_cmd: vm set_con_tcp_port R1 2000 
Oct 04 16:32:57.302 HYPERVISOR: exec_cmd: c7200 set_npe R1 npe-400 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm set_ram R1 160 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: c7200 set_npe R1 npe-400 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm set_ios R1 /<snip!>/c7200-adventerprisek9-mz.124-20.T.bin 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm set_sparse_mem R1 0 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm set_blk_direct_jump R1 0 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm set_idle_pc R1 0x6061aa54 
Oct 04 16:32:57.303 HYPERVISOR: exec_cmd: vm create R2 1 c7200 
Oct 04 16:32:57.305 VM: VM R2 created.
Oct 04 16:32:57.305 HYPERVISOR: exec_cmd: vm set_con_tcp_port R2 2001 
Oct 04 16:32:57.305 HYPERVISOR: exec_cmd: c7200 set_npe R2 npe-400 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm set_ram R2 160 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: c7200 set_npe R2 npe-400 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm set_ios R2 /<snip!>/c7200-adventerprisek9-mz.124-20.T.bin 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm set_sparse_mem R2 0 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm set_blk_direct_jump R2 0 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm set_idle_pc R2 0x6061aa54 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm slot_add_binding R1 0 0 C7200-IO-2FE 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: nio create_tap nio_tap0 tap0 
Oct 04 16:32:57.306 HYPERVISOR: exec_cmd: vm slot_add_nio_binding R1 0 0 nio_tap0 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: vm slot_add_binding R2 0 0 C7200-IO-2FE 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: nio create_udp nio_udp0 10000 127.0.0.1 10001 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: nio create_udp nio_udp1 10001 127.0.0.1 10000 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: vm slot_add_nio_binding R1 0 1 nio_udp0 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: vm slot_add_nio_binding R2 0 1 nio_udp1 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: vm slot_add_binding R1 1 0 PA-8T 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: vm slot_add_binding R2 1 0 PA-8T 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: nio create_udp nio_udp2 10002 127.0.0.1 10003 
Oct 04 16:32:57.307 HYPERVISOR: exec_cmd: nio create_udp nio_udp3 10003 127.0.0.1 10002 
Oct 04 16:32:57.308 HYPERVISOR: exec_cmd: vm slot_add_nio_binding R1 1 0 nio_udp2 
Oct 04 16:32:57.308 HYPERVISOR: exec_cmd: vm slot_add_nio_binding R2 1 0 nio_udp3 
Oct 04 16:32:57.308 HYPERVISOR: exec_cmd: vm start R1
Oct 04 16:32:57.598 HYPERVISOR: exec_cmd: vm start R2 
Oct 04 16:40:13.667 HYPERVISOR: exec_cmd: hypervisor reset 
Oct 04 16:40:13.774 VM: VM R2 shutdown.
Oct 04 16:40:13.774 VM: VM R2 destroyed.
Oct 04 16:40:13.914 VM: VM R1 shutdown.
Oct 04 16:40:13.914 VM: VM R1 destroyed.
Oct 04 16:40:13.915 GENERAL: reset done.

And these Established UDP connections:

netstat -an | grep udp
udp        0      0 127.0.0.1:10000         127.0.0.1:10001         ESTABLISHED
udp        0      0 127.0.0.1:10001         127.0.0.1:10000         ESTABLISHED
udp        0      0 127.0.0.1:10002         127.0.0.1:10003         ESTABLISHED
udp        0      0 127.0.0.1:10003         127.0.0.1:10002         ESTABLISHED

Appreciate any help, and I'm happy to produce more logs or test results,
Chris Vazquez

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the nio create_udp path shown in the hypervisor logs and reproduce the UDP tunnel on Ubuntu 14.04, comparing it with the working macOS capture. Investigate why loopback traffic uses the instance's external source IP; done means simulated Ethernet and serial links pass traffic with UDP packets sourced from 127.0.0.1.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, c, linux, ubuntu
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.