[ovs-discuss] Native tunnel routing failed error for VXLAN between ovs bridges

Alan Kayahan hsykay at gmail.com
Sat Nov 3 03:48:08 UTC 2018


Solved this one, and here is what I have done in case it may be
helpful to someone.

My setup is a derivative of
http://docs.openvswitch.org/en/latest/howto/userspace-tunneling/,
where I am using a single ovs bridge per node and connect the ovs
bridges of each node via VETH pairs instead. The guide states that
"This additional bridge is required when running Open vSwitch in
userspace rather than kernel-based Open vSwitch. The purpose of this
bridge is to allow use of the kernel network stack for routing and ARP
resolution.". Getting rid of this bridge means that I had to take care
of the tunnel related ARP by myself.

First https://mail.openvswitch.org/pipermail/ovs-discuss/2016-April/040618.html
fixed the packet_in ARP request issue. I had to avoid .NORMAL or
.FLOOD actions in ARP handling because ARP requests looped between the
VXLAN ports. So my workaround first install 2 rules;
arp_tpa="bridgeIP" actions=LOCAL and nw_dst="bridgeIP" actions=LOCAL.
Then use ovs-appctl tnl/arp/set for "bridgeIP" to "bridgeMAC".
On Tue, Oct 30, 2018 at 4:10 PM Alan Kayahan <hsykay at gmail.com> wrote:
>
> After reading some example deployments, I removed the IP address
> assignments from VETH ends. Instead, I assigned the IP addresses to
> bridges themselves (all 3 in the same subnet) and set the link status
> for the bridges to UP. Now I see the following.
>
> bridge("SW-00")
> ---------------
>  0. in_port=5, priority 10
>     output:9
>      -> output to native tunnel
>      -> tunneling to 192.168.0.2 via SW-00
>      -> neighbor cache miss for 192.168.0.2 on bridge SW-00, sending ARP request
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,in_port=5,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
> Datapath actions: drop
>
> However I don't see any packet_in from the bridge at the controller
> for such ARP request. Any ideas?
> On Mon, Oct 29, 2018 at 6:54 AM Alan Kayahan <hsykay at gmail.com> wrote:
> >
> > Hello,
> >
> > I have ovs bridges SW-00 and SW-01 operating in their respective
> > docker containers as following
> >
> > -------------Container A--------------
> > 2: ovs-netdev: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state
> > DOWN group default qlen 1000
> >     link/ether 96:56:12:93:cc:59 brd ff:ff:ff:ff:ff:ff
> > 3: SW-00: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state DOWN
> > group default qlen 1000
> >     link/ether e6:1e:5e:86:4a:42 brd ff:ff:ff:ff:ff:ff
> > 394: eth0 at if395: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > noqueue state UP group default
> >     link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> >     inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
> >        valid_lft forever preferred_lft forever
> > 401: VETH-00_01 at if400: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
> > qdisc noqueue state UP group default qlen 1000
> >     link/ether c6:90:ee:5d:bb:8a brd ff:ff:ff:ff:ff:ff link-netnsid 1
> >     inet 192.168.0.1/24 scope global VETH-00_01
> >        valid_lft forever preferred_lft forever
> >
> >
> > e67f7443-fbb1-488e-9884-4c755a61f6ab
> >     Bridge "SW-00"
> >         Controller "tcp:172.17.0.1:6653"
> >             is_connected: true
> >         Port "VXLAN-00_01"
> >             Interface "VXLAN-00_01"
> >                 type: vxlan
> >                 options: {key="1", local_ip="192.168.0.1",
> > remote_ip="192.168.0.2"}
> >         Port "SW-00"
> >             Interface "SW-00"
> >                 type: internal
> >     ovs_version: "2.10.90"
> >
> >  cookie=0x0, duration=1853.837s, table=0, n_packets=7, n_bytes=124,
> > priority=10,in_port=10 actions=output:"VXLAN-00_01"
> >
> >
> > -------------Container B--------------
> > 2: ovs-netdev: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state
> > DOWN group default qlen 1000
> >     link/ether c2:aa:82:0b:9e:f6 brd ff:ff:ff:ff:ff:ff
> > 3: SW-01: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop state DOWN
> > group default qlen 1000
> >     link/ether 2a:f7:3d:b8:06:48 brd ff:ff:ff:ff:ff:ff
> > 396: eth0 at if397: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> > noqueue state UP group default
> >     link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
> >     inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
> >        valid_lft forever preferred_lft forever
> > 400: VETH-01_00 at if401: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
> > qdisc noqueue state UP group default qlen 1000
> >     link/ether 76:6b:e4:02:18:7f brd ff:ff:ff:ff:ff:ff link-netnsid 1
> >     inet 192.168.0.2/24 scope global VETH-01_00
> >        valid_lft forever preferred_lft forever
> >
> >
> > 8fceec3b-e926-40e5-92ee-0d6ffe7e458a
> >     Bridge "SW-01"
> >         Controller "tcp:172.17.0.1:6653"
> >             is_connected: true
> >         Port "VXLAN-01_00"
> >             Interface "VXLAN-01_00"
> >                 type: vxlan
> >                 options: {key="1", local_ip="192.168.0.2",
> > remote_ip="192.168.0.1"}
> >         Port "SW-01"
> >             Interface "SW-01"
> >                 type: internal
> >     ovs_version: "2.10.90"
> >
> >  cookie=0x0, duration=1882.835s, table=0, n_packets=0, n_bytes=0,
> > priority=10,in_port="VXLAN-01_00" actions=output:"5"
> >
> > /-----------------------------------------------------
> >
> > So the two containers are connected via VETH pair and can ping.
> > However the VXLAN traffic is not passing any traffic; the packets are
> > hitting the rule on SW-00, however they are not making it to SW-01.
> >
> > ovs-appctl ofproto/trace SW-00 in_port=10
> > Flow: in_port=10,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
> >
> > bridge("SW-00")
> > ---------------
> >  0. in_port=10, priority 10
> >     output:2
> >      -> output to native tunnel
> >      >> native tunnel routing failed
> >
> > Final flow: unchanged
> > Megaflow: recirc_id=0,eth,in_port=10,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
> > Datapath actions: drop
> >
> > ovs-appctl ovs/route/show
> > Route Table:
> > Cached: 127.0.0.1/32 dev lo SRC 127.0.0.1 local
> > Cached: 172.17.0.2/32 dev eth0 SRC 172.17.0.2 local
> > Cached: 192.168.0.1/32 dev VETH-00_01 SRC 192.168.0.1 local
> > Cached: 127.0.0.0/8 dev lo SRC 127.0.0.1 local
> > Cached: 192.168.0.0/24 dev VETH-00_01 SRC 192.168.0.1
> > Cached: 172.17.0.0/16 dev eth0 SRC 172.17.0.2
> > Cached: 0.0.0.0/0 dev eth0 GW 172.17.0.1 SRC 172.17.0.2
> >
> > The ofproto trace reports "native tunnel routing failed", despite
> > there exists an appropriate entry in ovs/route/show. I'd appreciate
> > any input on why this could be happening.
> >
> > Regards,
> > Alan


More information about the discuss mailing list