[ovs-discuss] Tracing packets between switches?
Jason Kary (jkary)
jkary at cisco.com
Fri Jul 18 14:30:53 UTC 2014
Hi folks,
I have what I hope is a basic question. I am running OpenStack Icehouse using the OVS switch for my networking. The typical setup of OVS for OpenStack is to create a br-int and br-tun on a compute host (assuming you are using GRE tunnels). There appears to be network connectivity between vm on the same host however all pings are lost when trying to ping VMs between compute hosts. Looking at the br-int switch with TCPDUMP shows a bunch of arp requests. Looking at the br-tun switch show no traffic.
Pinging the default router or VMs on different CPU hosts does not work. Using TCPDUMP -i BR-INT I see the following:
tcpdump -i br-int
=============
root at nmtg-cpu003:~# tcpdump -i br-int
tcpdump: WARNING: br-int: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-int, link-type EN10MB (Ethernet), capture size 65535 bytes
10:10:45.016358 ARP, Request who-has 10.3.0.1 tell 10.3.0.14, length 28
10:10:46.016382 ARP, Request who-has 10.3.0.1 tell 10.3.0.14, length 28
10:10:47.020425 ARP, Request who-has 10.3.0.1 tell 10.3.0.14, length 28
10:10:48.020346 ARP, Request who-has 10.3.0.1 tell 10.3.0.14, length 28
The flow table and the switch show for br-int is:
ovs-ofctl dump-flows br-int
====================
root at nmtg-cpu003:~# ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=3022205.956s, table=0, n_packets=236627, n_bytes=60752190, idle_age=1, hard_age=65534, priority=1 actions=NORMAL
ovs-ofctl show br-int
================
root at nmtg-cpu003:~# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00009e2b04a37f40
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(patch-tun): addr:12:79:4e:dc:05:04
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
37(qvoacb7937a-de): addr:f6:e1:b9:a4:7e:ed
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
40(qvobeedd792-67): addr:86:a3:b0:6f:e0:34
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
43(qvo73c3897d-09): addr:c6:d0:a9:cd:7a:13
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
LOCAL(br-int): addr:9e:2b:04:a3:7f:40
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
root at nmtg-cpu003:~#
This all looks good to my amateur eyes and if I am reading the output correctly there are no flow rules so we have a dumb hub. Patch-tun *should* be forwarding traffic to br-tun however when I try to sniff packets on br-tun then I see nothing even though I know there are ARPs happening on br-int.
The br-tun switch looks as follows:
ovs-ofctl dump-flows br-tun
=====================
root at nmtg-cpu003:~# ovs-ofctl dump-flows br-tun
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=3022525.764s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1,in_port=3 actions=resubmit(,2)
cookie=0x0, duration=3022526.484s, table=0, n_packets=236918, n_bytes=60762378, idle_age=0, hard_age=65534, priority=1,in_port=1 actions=resubmit(,1)
cookie=0x0, duration=3022525.879s, table=0, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1,in_port=2 actions=resubmit(,2)
cookie=0x0, duration=3022526.446s, table=0, n_packets=6, n_bytes=468, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=3022526.407s, table=1, n_packets=3, n_bytes=180, idle_age=65534, hard_age=65534, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
cookie=0x0, duration=3022526.366s, table=1, n_packets=236915, n_bytes=60762198, idle_age=0, hard_age=65534, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21)
cookie=0x0, duration=70705.91s, table=2, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1,tun_id=0x7 actions=mod_vlan_vid:9,resubmit(,10)
cookie=0x0, duration=3022526.325s, table=2, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=3022526.285s, table=3, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=0 actions=drop
cookie=0x0, duration=3022526.245s, table=10, n_packets=0, n_bytes=0, idle_age=65534, hard_age=65534, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
cookie=0x0, duration=3022526.205s, table=20, n_packets=3, n_bytes=180, idle_age=65534, hard_age=65534, priority=0 actions=resubmit(,21)
cookie=0x0, duration=70705.961s, table=21, n_packets=67286, n_bytes=2838643, idle_age=0, hard_age=65534, dl_vlan=9 actions=strip_vlan,set_tunnel:0x7,output:2,output:3
cookie=0x0, duration=3022526.164s, table=21, n_packets=186, n_bytes=15388, idle_age=2391, hard_age=65534, priority=0 actions=drop
ovs-ofctl show br-tun
================
root at nmtg-cpu003:~# ovs-ofctl show br-tun
OFPT_FEATURES_REPLY (xid=0x2): dpid:00004224b3644344
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(patch-int): addr:f6:51:48:8a:6d:e8
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
2(gre-0a060203): addr:76:5b:56:ae:e8:d4
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
3(gre-0a060205): addr:8a:30:6d:be:15:ca
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
LOCAL(br-tun): addr:42:24:b3:64:43:44
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
root at nmtg-cpu003:~#
Shouldn’t I be seeing ARP packets on the BR-TUN via the patch-int i/f? I can’t see how else packets get from br-int over to br-tun. It looks like I am getting some traffic over to the BR-TUN as I can see flow packet counters increasing.
Can someone help me figure out what my next steps are in debugging this issue? Is it possible to snoop a particular port?
Take Care
Jason Kary
More information about the discuss
mailing list