[ovs-discuss] Issue when using local_ip with VXLAN tunnels in OVS

Gregory Rose gvrose8192 at gmail.com
Mon Nov 19 16:18:42 UTC 2018


On 11/19/2018 7:50 AM, Siva Teja ARETI wrote:
>
>
> On Fri, Nov 16, 2018 at 4:52 PM Gregory Rose <gvrose8192 at gmail.com 
> <mailto:gvrose8192 at gmail.com>> wrote:
>
>     On 11/6/2018 8:51 AM, Siva Teja ARETI wrote:
>>     Hi Greg,
>>
>>     Thanks for looking into this.
>>
>>     I have two VMs in my setup each with two interfaces. Trying to
>>     setup the VXLAN tunnels across these interfaces which are in
>>     different subnets. A docker container is attached to ovs bridge
>>     using ovs-docker utility on each VM and doing a ping from one
>>     container to another.
>>
>>     *VM1 details:*
>>
>>     [root at vm1 ~]# ip a
>>     .......
>>     3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     pfifo_fast state UP qlen 1000
>>         link/ether 52:54:00:b8:05:be brd ff:ff:ff:ff:ff:ff
>>         inet 30.30.0.59/24 <http://30.30.0.59/24> brd 30.30.0.255
>>     scope global dynamic eth1
>>      valid_lft 3002sec preferred_lft 3002sec
>>         inet6 fe80::5054:ff:feb8:5be/64 scope link
>>      valid_lft forever preferred_lft forever
>>     4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     pfifo_fast state UP qlen 1000
>>         link/ether 52:54:00:f0:64:37 brd ff:ff:ff:ff:ff:ff
>>         inet 20.20.0.183/24 <http://20.20.0.183/24> brd 20.20.0.255
>>     scope global dynamic eth2
>>      valid_lft 3248sec preferred_lft 3248sec
>>         inet6 fe80::5054:ff:fef0:6437/64 scope link
>>      valid_lft forever preferred_lft forever
>>     .......
>
>     Hi Siva,
>
>     I have a question.  Are you able to ping between the two
>     interfaces on VM1 with this command?:
>
>     # ping 20.20.0.183 -I eth1
>
>     thanks,
>
>     - Greg
>
> Hi Greg,
>
> Sorry for the late reply.
>
> Yes, I am able to ping between two interfaces.
>
> [root at localhost ~]# ovs-appctl dpif/show
> system at ovs-system: hit:2799 missed:198775
>         testbr0:
>                 a0769422cfc04_l 2/3: (system)
>                 testbr0 65534/1: (internal)
>                 vxlan0 10/2: (vxlan: local_ip=30.30.0.193, 
> remote_ip=20.20.0.183)
> [root at localhost ~]# ping 20.20.0.183 -I 30.30.0.193
> PING 20.20.0.183 (20.20.0.183) from 30.30.0.193 : 56(84) bytes of data.
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=1 ttl=64 
> time=0.470 ms
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=2 ttl=64 
> time=0.657 ms
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=3 ttl=64 
> time=0.685 ms
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=4 ttl=64 
> time=0.721 ms
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=5 ttl=64 
> time=0.630 ms
> 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=6 ttl=64 
> time=0.629 ms
> ^C

Well that's probably where my setup isn't configured right.  What is the 
output of 'ip route' on that system?

Thanks,

- Greg

> --- 20.20.0.183 ping statistics ---
> 6 packets transmitted, 6 received, 0% packet loss, time 5000ms
> rtt min/avg/max/mdev = 0.470/0.632/0.721/0.079 ms
> [root at localhost ~]#
>
>  Siva Teja.
>
>>     [root at vm1 ~]# ovs-vsctl show
>>     ff70c814-d1b0-4018-aee8-8b635187afee
>>         Bridge "testbr0"
>>             Port "gre0"
>>     Interface "gre0"
>>     type: gre
>>     options: {local_ip="20.20.0.183", remote_ip="30.30.0.193"}
>>             Port "testbr0"
>>     Interface "testbr0"
>>     type: internal
>>             Port "2cfb62a9b0f04_l"
>>     Interface "2cfb62a9b0f04_l"
>>     ovs_version: "2.9.2"
>>     [root at vm1 ~]# ip rule list
>>     0:      from all lookup local
>>     32765:  from 20.20.0.183 lookup siva
>>     32766:  from all lookup main
>>     32767:  from all lookup default
>>     [root at vm1 ~]# ip route show table siva
>>     default dev eth2 scope link src 20.20.0.183
>>     [root at vm1 ~]# ######################### A docker container is
>>     attached to ovs bridge using ovs-docker utility
>>     [root at vm1 ~]# docker ps
>>     CONTAINER ID   IMAGE  COMMAND  CREATED  STATUS PORTS  NAMES
>>     be4ab434db99   busybox  "sh" 5 days ago Up 5 days  admiring_euclid
>>     [root at vm1 ~]# nsenter -n -t `docker inspect be4
>>     --format={{.State.Pid}}` -- ip a
>>     1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
>>     UNKNOWN qlen 1
>>         link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>         inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>>            valid_lft forever preferred_lft forever
>>         inet6 ::1/128 scope host
>>            valid_lft forever preferred_lft forever
>>     2: gre0 at NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
>>         link/gre 0.0.0.0 brd 0.0.0.0
>>     3: gretap0 at NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state
>>     DOWN qlen 1000
>>         link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
>>     9: eth0 at if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     noqueue state UP qlen 1000
>>         link/ether 22:98:41:0f:e8:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0
>>         inet 70.70.0.10/24 <http://70.70.0.10/24> scope global eth0
>>            valid_lft forever preferred_lft forever
>>         inet6 fe80::2098:41ff:fe0f:e850/64 scope link
>>            valid_lft forever preferred_lft forever
>>
>>
>>     *VM2 details:*
>>     *
>>     *
>>     [root at vm2 ~]# ip a
>>     ........
>>     3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     pfifo_fast state UP qlen 1000
>>         link/ether 52:54:00:79:ef:92 brd ff:ff:ff:ff:ff:ff
>>         inet 30.30.0.193/24 <http://30.30.0.193/24> brd 30.30.0.255
>>     scope global dynamic eth1
>>      valid_lft 2406sec preferred_lft 2406sec
>>         inet6 fe80::5054:ff:fe79:ef92/64 scope link
>>      valid_lft forever preferred_lft forever
>>     4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     pfifo_fast state UP qlen 1000
>>         link/ether 52:54:00:05:93:7c brd ff:ff:ff:ff:ff:ff
>>         inet 20.20.0.64/24 <http://20.20.0.64/24> brd 20.20.0.255
>>     scope global dynamic eth2
>>      valid_lft 2775sec preferred_lft 2775sec
>>         inet6 fe80::5054:ff:fe05:937c/64 scope link
>>      valid_lft forever preferred_lft forever
>>     .......
>>     [root at vm2 ~]# ovs-vsctl show
>>     b85514db-3f29-4f7a-9001-37d70adfca34
>>         Bridge "testbr0"
>>             Port "gre0"
>>     Interface "gre0"
>>     type: gre
>>     options: {local_ip="30.30.0.193", remote_ip="20.20.0.183"}
>>             Port "a0769422cfc04_l"
>>     Interface "a0769422cfc04_l"
>>             Port "testbr0"
>>     Interface "testbr0"
>>     type: internal
>>     ovs_version: "2.9.2"
>>     [root at vm2 ~]# ip rule list
>>     0:      from all lookup local
>>     32766:  from all lookup main
>>     32767:  from all lookup default
>>     [root at vm2 ~]# ######################### A docker container is
>>     attached to ovs bridge using ovs-docker utility
>>     [root at vm2 ~]# docker ps
>>     CONTAINER ID     IMAGE      COMMAND        CREATED        
>>      STATUS           PORTS            NAMES
>>     86214f0d99e8 busybox:latest   "sh"     5 days ago       Up 5 days
>>      peaceful_snyder
>>     [root at vm2 ~]# nsenter -n -t `docker inspect 862
>>     --format={{.State.Pid}}` -- ip a
>>     1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
>>     UNKNOWN qlen 1
>>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>         inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>>      valid_lft forever preferred_lft forever
>>         inet6 ::1/128 scope host
>>      valid_lft forever preferred_lft forever
>>     2: gre0 at NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
>>         link/gre 0.0.0.0 brd 0.0.0.0
>>     3: gretap0 at NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state
>>     DOWN qlen 1000
>>         link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
>>     9: eth0 at if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>     noqueue state UP qlen 1000
>>         link/ether ae:ac:14:7a:40:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0
>>         inet 70.70.0.20/24 <http://70.70.0.20/24> scope global eth0
>>      valid_lft forever preferred_lft forever
>>         inet6 fe80::acac:14ff:fe7a:405f/64 scope link
>>      valid_lft forever preferred_lft forever
>>
>>     With this configuration, if I do a ping from docker container on
>>     VM1 to docker container on VM2 it works.
>>
>>     [root at vm1 ~]# nsenter -n -t `docker inspect be4
>>     --format={{.State.Pid}}` -- ping 70.70.0.20
>>     PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
>>     64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=1 ttl=64
>>     time=0.831 ms
>>     64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=2 ttl=64
>>     time=0.933 ms
>>     64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=3 ttl=64
>>     time=0.564 ms
>>     ^C
>>     --- 70.70.0.20 ping statistics ---
>>     3 packets transmitted, 3 received, 0% packet loss, time 2001ms
>>     rtt min/avg/max/mdev = 0.564/0.776/0.933/0.155 ms
>>
>>     And the traffic is as expected on VM2.
>>
>>     [root at vm2 ~]# tcpdump -n -i any host 20.20.0.183
>>     tcpdump: verbose output suppressed, use -v or -vv for full
>>     protocol decode
>>     listening on any, link-type LINUX_SLL (Linux cooked), capture
>>     size 262144 bytes
>>     16:37:32.262553 IP 20.20.0.183 > 30.30.0.193
>>     <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
>>     70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
>>     1, length 64
>>     16:37:32.262835 IP 30.30.0.193 > 20.20.0.183
>>     <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
>>     70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 1,
>>     length 64
>>     16:37:33.263211 IP 20.20.0.183 > 30.30.0.193
>>     <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
>>     70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
>>     2, length 64
>>     16:37:33.263374 IP 30.30.0.193 > 20.20.0.183
>>     <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
>>     70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 2,
>>     length 64
>>     16:37:34.264159 IP 20.20.0.183 > 30.30.0.193
>>     <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
>>     70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
>>     3, length 64
>>     16:37:34.264252 IP 30.30.0.193 > 20.20.0.183
>>     <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
>>     70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 3,
>>     length 64
>>
>>     But when I change the tunnel type to vxlan, ping fails.
>>
>>     [root at vm1 ~]# ovs-vsctl del-port testbr0 gre0
>>     [root at vm1 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface
>>     vxlan0 type=vxlan options:local_ip=20.20.0.183
>>     options:remote_ip=30.30.0.193 options:dst_port=4789
>>     [root at vm1 ~]# ovs-vsctl show
>>     ff70c814-d1b0-4018-aee8-8b635187afee
>>         Bridge "testbr0"
>>             Port "testbr0"
>>     Interface "testbr0"
>>     type: internal
>>             Port "vxlan0"
>>     Interface "vxlan0"
>>     type: vxlan
>>     options: {dst_port="4789", local_ip="20.20.0.183",
>>     remote_ip="30.30.0.193"}
>>             Port "2cfb62a9b0f04_l"
>>     Interface "2cfb62a9b0f04_l"
>>     ovs_version: "2.9.2"
>>
>>     [root at vm2 ~]# ovs-vsctl del-port testbr0 gre0
>>     [root at vm2 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface
>>     vxlan0 type=vxlan options:local_ip=30.30.0.193
>>     options:remote_ip=20.20.0.183 options:dst_port=4789
>>     [root at vm2 ~]# ovs-vsctl show
>>     b85514db-3f29-4f7a-9001-37d70adfca34
>>         Bridge "testbr0"
>>             Port "a0769422cfc04_l"
>>     Interface "a0769422cfc04_l"
>>             Port "vxlan0"
>>     Interface "vxlan0"
>>     type: vxlan
>>     options: {dst_port="4789", local_ip="30.30.0.193",
>>     remote_ip="20.20.0.183"}
>>             Port "testbr0"
>>     Interface "testbr0"
>>     type: internal
>>     ovs_version: "2.9.2"
>>
>>     Ping fails with this setup
>>
>>     [root at vm1 ~]# nsenter -n -t `docker inspect be4
>>     --format={{.State.Pid}}` -- ping 70.70.0.20
>>     PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
>>     ^C
>>     --- 70.70.0.20 ping statistics ---
>>     6 packets transmitted, 0 received, 100% packet loss, time 4999ms
>>
>>     Expected traffic is not seen on VM2
>>
>>     [root at vm2 ~]# tcpdump -n -i any host 20.20.0.183
>>     tcpdump: verbose output suppressed, use -v or -vv for full
>>     protocol decode
>>     listening on any, link-type LINUX_SLL (Linux cooked), capture
>>     size 262144 bytes
>>     ^C
>>     0 packets captured
>>     0 packets received by filter
>>     0 packets dropped by kernel
>>
>>     Kindly let me know if you need more information.
>>
>>     Siva Teja.
>>
>>     On Tue, Nov 6, 2018 at 10:49 AM Gregory Rose
>>     <gvrose8192 at gmail.com <mailto:gvrose8192 at gmail.com>> wrote:
>>
>>
>>         On 11/5/2018 6:10 PM, Siva Teja ARETI wrote:
>>>         Hi,
>>>
>>>         I am trying to use local_ip option for a VXLAN tunnel using
>>>         ovs but it does not seem to work. The same works when I use
>>>         GRE tunnel. I also found a previous discussion from another
>>>         user who tried the exact same approach. Here is the link to
>>>         the discussion
>>>
>>>         _https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html_
>>>
>>>         I am unable to find any working resolution at the end of
>>>         this discussion. Could you please help?
>>
>>         I looked into that but was never able to set up a
>>         configuration like the one in that discussion and could
>>         not repro the bug.
>>
>>         Please provide some details on your usage, configuration and
>>         steps to repro and I can look into it.
>>
>>         Thanks,
>>
>>         - Greg
>>
>>>
>>>         I am using ovs 2.9.2
>>>
>>>         [root at localhost ~]# ovs-vsctl --version
>>>         ovs-vsctl (Open vSwitch) 2.9.2
>>>         DB Schema 7.15.1
>>>
>>>         Thanks,
>>>         Siva Teja.
>>>
>>>
>>>         _______________________________________________
>>>         discuss mailing list
>>>         discuss at openvswitch.org <mailto:discuss at openvswitch.org>
>>>         https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20181119/e09b1587/attachment-0001.html>


More information about the discuss mailing list