[ovs-discuss] Unable to Ping Bridge IP address over VXLAN Tunnels

Jesse Pendergrass pendergrassjesse at gmail.com
Thu Jul 16 23:53:51 UTC 2020


Hello,

I have three physical hosts and I have installed openvswitch on each. The
networking for each host is set up as follows:

eno1 - slave to bond0
enp4s0f0 - slave to bond0
bond0
br0 - bridge using bond0, with 192.168.1.0/24 IP address
br1 - internal OVS bridge, with 172.16.0.0/24 IP address

root at host1:~# ip link show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
> DEFAULT group default qlen 1
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8950 qdisc mq master
> bond0 state UP mode DEFAULT group default qlen 1000
>     link/ether 68:05:ca:00:1d:de brd ff:ff:ff:ff:ff:ff
> 6: enp4s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8950 qdisc
> pfifo_fast master bond0 state UP mode DEFAULT group default qlen 1000
>     link/ether 68:05:ca:00:1d:de brd ff:ff:ff:ff:ff:ff
> 8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8950 qdisc noqueue
> master br0 state UP mode DEFAULT group default qlen 1000
>     link/ether 68:05:ca:00:1d:de brd ff:ff:ff:ff:ff:ff
> 9: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state UP
> mode DEFAULT group default qlen 1000
>     link/ether 68:05:ca:00:1d:de brd ff:ff:ff:ff:ff:ff
> 10: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT group default qlen 1
>     link/ether 32:d7:d7:09:2a:a8 brd ff:ff:ff:ff:ff:ff
> 11: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc noqueue state
> UNKNOWN mode DEFAULT group default qlen 1
>     link/ether ba:7e:9c:1a:b9:46 brd ff:ff:ff:ff:ff:ff
> 12: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65485 qdisc
> noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
>     link/ether da:34:8d:73:d3:72 brd ff:ff:ff:ff:ff:ff
>

VXLAN tunnel ports are configured on br1 to set up a star topology, so each
host has a tunnel to the others.

I haven't set up any flows manually, so each switch is just using L2
forwarding at the moment.

root at host1:~# ovs-ofctl dump-flows br1
> NXST_FLOW reply (xid=0x4):
>  cookie=0x0, duration=2182.022s, table=0, n_packets=374, n_bytes=16020,
> idle_age=575, priority=0 actions=NORMAL
>

Here's my issue. I'm on host1, and I'm trying to ping the bridge IP
addresses on the remote nodes. For instance, I ping the br1 IP of host2
(172.16.0.101) from host1 (172.16.0.100) can see ARP requests going out
over the bond0 interface, and on the remote hosts I can see them being
received on both remote nodes via the vxlan_sys_4789 iface.

HOST1

> root at host1:~# tcpdump -e -vvvni bond0 not port 22 and host 192.168.1.101
> tcpdump: listening on bond0, link-type EN10MB (Ethernet), capture size
> 262144 bytes
> 23:33:43.519699 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 4985, offset 0, flags [DF], proto
> UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
> ba:7e:9c:1a:b9:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42:
> Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.0.101 tell
> 172.16.0.100, length 28
> 23:33:44.519673 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 5217, offset 0, flags [DF], proto
> UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
> ba:7e:9c:1a:b9:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42:
> Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.0.101 tell
> 172.16.0.100, length 28
> 23:33:45.519649 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 5276, offset 0, flags [DF], proto
> UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
>

 HOST2

> root at host2:~# tcpdump -e -vvvni bond0 not port 22 and host 192.168.1.100
> tcpdump: listening on bond0, link-type EN10MB (Ethernet), capture size
> 262144 bytes
> 23:35:25.729416 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 29060, offset 0, flags [DF],
> proto UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
> ba:7e:9c:1a:b9:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42:
> Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.0.101 tell
> 172.16.0.100, length 28
> 23:35:26.727622 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 29086, offset 0, flags [DF],
> proto UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
> ba:7e:9c:1a:b9:46 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42:
> Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.0.101 tell
> 172.16.0.100, length 28
> 23:35:27.727691 68:05:ca:00:1d:de > 90:e2:ba:2d:f8:1a, ethertype IPv4
> (0x0800), length 92: (tos 0x0, ttl 64, id 29268, offset 0, flags [DF],
> proto UDP (17), length 78)
>     192.168.1.100.55555 > 192.168.1.101.4789: [no cksum] VXLAN, flags [I]
> (0x08), vni 3084014
>

root at host2:~# tcpdump -vvvni vxlan_sys_4789
> tcpdump: listening on vxlan_sys_4789, link-type EN10MB (Ethernet), capture
> size 262144 bytes
> 23:36:31.042292 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has
> 172.16.0.101 tell 172.16.0.100, length 28
> 23:36:32.039645 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has
> 172.16.0.101 tell 172.16.0.100, length 28
> 23:36:33.039686 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has
> 172.16.0.101 tell 172.16.0.100, length 28
> 23:36:34.057901 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has
> 172.16.0.101 tell 172.16.0.100, length 28
>

But I get no responses back from host2. I see no packets arriving on br1. I
see these messages in /var/log/openvswitch/ovs-vswitchd.log.

2020-07-16T23:43:21.802Z|00034|ofproto_dpif_upcall(handler45)|INFO|Dropped
> 36 log messages in last 152 seconds (most recently, 116 seconds ago) due to
> excessive rate
> 2020-07-16T23:43:21.802Z|00035|ofproto_dpif_upcall(handler45)|INFO|received
> packet on unassociated datapath port 2
> 2020-07-16T23:43:22.459Z|00001|tunnel(revalidator63)|WARN|receive tunnel
> port not found
> (arp,tun_id=0x2f0eee,tun_src=192.168.1.100,tun_dst=192.168.1.101,tun_ipv6_src=::,tun_ipv6_dst=::,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,tun_flags=key,in_port=2,vlan_tci=0x0000,dl_src=ba:7e:9c:1a:b9:46,dl_dst=ff:ff:ff:ff:ff:ff,arp_spa=172.16.0.100,arp_tpa=172.16.0.101,arp_op=1,arp_sha=ba:7e:9c:1a:b9:46,arp_tha=00:00:00:00:00:00)
>

I've tried to add ARP entries manually but this hasn't had any impact. This
is present on each host:

root at host:~# ovs-appctl tnl/arp/show
> IP                                            MAC                 Bridge
> ==========================================================================
> 172.16.0.101                                  da:cb:57:b6:92:46   br1
> 172.16.0.100                                  ba:7e:9c:1a:b9:46   br1
> 172.16.0.102                                  22:b8:00:be:25:4f   br1
>

I'm not sure what I'm doing wrong. Does anyone have any guidance on what
might be wrong with my setup? Is what I'm trying to do even possible?

Thanks!

Jesse Pendergrass
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200716/0fdfa7f9/attachment-0001.html>


More information about the discuss mailing list