<div dir="ltr"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, Nov 16, 2018 at 4:52 PM Gregory Rose <<a href="mailto:gvrose8192@gmail.com">gvrose8192@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
On 11/6/2018 8:51 AM, Siva Teja ARETI wrote:<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi Greg,
<div><br>
</div>
<div>Thanks for looking into this.</div>
<div><br>
</div>
<div>I have two VMs in my setup
each with two interfaces. Trying
to setup the VXLAN tunnels
across these interfaces which
are in different subnets. A
docker container is attached to
ovs bridge using ovs-docker
utility on each VM and doing a
ping from one container to
another.</div>
<div><br>
</div>
<div><b>VM1 details:</b></div>
<div><br>
</div>
<div>
<div>[root@vm1 ~]# ip a</div>
<div>.......</div>
<div>3: eth1:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast
state UP qlen 1000<br>
</div>
<div> link/ether
52:54:00:b8:05:be brd
ff:ff:ff:ff:ff:ff</div>
<div> inet <a href="http://30.30.0.59/24" target="_blank">30.30.0.59/24</a>
brd 30.30.0.255 scope global
dynamic eth1</div>
<div> valid_lft 3002sec
preferred_lft 3002sec</div>
<div> inet6
fe80::5054:ff:feb8:5be/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
<div>4: eth2:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast
state UP qlen 1000</div>
<div> link/ether
52:54:00:f0:64:37 brd
ff:ff:ff:ff:ff:ff</div>
<div> inet <a href="http://20.20.0.183/24" target="_blank">20.20.0.183/24</a>
brd 20.20.0.255 scope global
dynamic eth2</div>
<div> valid_lft 3248sec
preferred_lft 3248sec</div>
<div> inet6
fe80::5054:ff:fef0:6437/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
</div>
<div>.......</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Hi Siva,<br>
<br>
I have a question. Are you able to ping between the two interfaces
on VM1 with this command?:<br>
<br>
# ping 20.20.0.183 -I eth1<br>
<br>
thanks,<br>
<br>
- Greg<br><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote></div></blockquote><div>Hi Greg,</div><div><br></div><div>Sorry for the late reply. </div><div><br></div><div>Yes, I am able to ping between two interfaces.</div><div><br></div><div><div>[root@localhost ~]# ovs-appctl dpif/show</div><div>system@ovs-system: hit:2799 missed:198775</div><div> testbr0:</div><div> a0769422cfc04_l 2/3: (system)</div><div> testbr0 65534/1: (internal)</div><div> vxlan0 10/2: (vxlan: local_ip=30.30.0.193, remote_ip=20.20.0.183)</div><div>[root@localhost ~]# ping 20.20.0.183 -I 30.30.0.193</div><div>PING 20.20.0.183 (20.20.0.183) from 30.30.0.193 : 56(84) bytes of data.</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=1 ttl=64 time=0.470 ms</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=2 ttl=64 time=0.657 ms</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=3 ttl=64 time=0.685 ms</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=4 ttl=64 time=0.721 ms</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=5 ttl=64 time=0.630 ms</div><div>64 bytes from <a href="http://20.20.0.183">20.20.0.183</a>: icmp_seq=6 ttl=64 time=0.629 ms</div><div>^C</div><div>--- 20.20.0.183 ping statistics ---</div><div>6 packets transmitted, 6 received, 0% packet loss, time 5000ms</div><div>rtt min/avg/max/mdev = 0.470/0.632/0.721/0.079 ms</div><div>[root@localhost ~]#</div></div><div><br></div><div> Siva Teja.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div>[root@vm1 ~]# ovs-vsctl
show</div>
<div>ff70c814-d1b0-4018-aee8-8b635187afee</div>
<div> Bridge "testbr0"</div>
<div> Port "gre0"</div>
<div> Interface
"gre0"</div>
<div> type: gre</div>
<div> options:
{local_ip="20.20.0.183",
remote_ip="30.30.0.193"}</div>
<div> Port "testbr0"</div>
<div> Interface
"testbr0"</div>
<div> type:
internal</div>
<div> Port
"2cfb62a9b0f04_l"</div>
<div> Interface
"2cfb62a9b0f04_l"</div>
<div> ovs_version: "2.9.2"</div>
</div>
<div>
<div>[root@vm1 ~]# ip rule list</div>
<div>0: from all lookup
local </div>
<div>32765: from 20.20.0.183
lookup siva </div>
<div>32766: from all lookup
main </div>
<div>32767: from all lookup
default </div>
</div>
<div>
<div>[root@vm1 ~]# ip route show
table siva</div>
<div>default dev eth2 scope link
src 20.20.0.183 </div>
</div>
<div>[root@vm1 ~]#
######################### A
docker container is attached to
ovs bridge using ovs-docker
utility</div>
<div>[root@vm1 ~]# docker ps</div>
<div>CONTAINER ID IMAGE
COMMAND
CREATED STATUS
PORTS
NAMES</div>
<div>be4ab434db99 busybox
"sh" 5
days ago Up 5 days
admiring_euclid</div>
<div>[root@vm1 ~]# nsenter -n -t
`docker inspect be4
--format={{.State.Pid}}` -- ip a</div>
<div>1: lo:
<LOOPBACK,UP,LOWER_UP> mtu
65536 qdisc noqueue state
UNKNOWN qlen 1</div>
<div> link/loopback
00:00:00:00:00:00 brd
00:00:00:00:00:00</div>
<div> inet <a href="http://127.0.0.1/8" target="_blank">127.0.0.1/8</a>
scope host lo</div>
<div> valid_lft forever
preferred_lft forever</div>
<div> inet6 ::1/128 scope host </div>
<div> valid_lft forever
preferred_lft forever</div>
<div>2: gre0@NONE: <NOARP>
mtu 1476 qdisc noop state DOWN
qlen 1</div>
<div> link/gre 0.0.0.0 brd
0.0.0.0</div>
<div>3: gretap0@NONE:
<BROADCAST,MULTICAST> mtu
1462 qdisc noop state DOWN qlen
1000</div>
<div> link/ether
00:00:00:00:00:00 brd
ff:ff:ff:ff:ff:ff</div>
<div>9: eth0@if10:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc noqueue state UP
qlen 1000</div>
<div> link/ether
22:98:41:0f:e8:50 brd
ff:ff:ff:ff:ff:ff link-netnsid 0</div>
<div> inet <a href="http://70.70.0.10/24" target="_blank">70.70.0.10/24</a>
scope global eth0</div>
<div> valid_lft forever
preferred_lft forever</div>
<div> inet6
fe80::2098:41ff:fe0f:e850/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
<div> <br>
</div>
<div><br>
</div>
<div><b>VM2 details:</b></div>
<div><b><br>
</b></div>
<div>
<div>[root@vm2 ~]# ip a</div>
<div>........<br>
</div>
<div>3: eth1:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast
state UP qlen 1000<br>
</div>
<div> link/ether
52:54:00:79:ef:92 brd
ff:ff:ff:ff:ff:ff</div>
<div> inet <a href="http://30.30.0.193/24" target="_blank">30.30.0.193/24</a>
brd 30.30.0.255 scope global
dynamic eth1</div>
<div> valid_lft 2406sec
preferred_lft 2406sec</div>
<div> inet6
fe80::5054:ff:fe79:ef92/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
<div>4: eth2:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast
state UP qlen 1000</div>
<div> link/ether
52:54:00:05:93:7c brd
ff:ff:ff:ff:ff:ff</div>
<div> inet <a href="http://20.20.0.64/24" target="_blank">20.20.0.64/24</a>
brd 20.20.0.255 scope global
dynamic eth2</div>
<div> valid_lft 2775sec
preferred_lft 2775sec</div>
<div> inet6
fe80::5054:ff:fe05:937c/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
</div>
<div>.......</div>
<div>
<div>[root@vm2 ~]# ovs-vsctl
show</div>
<div>b85514db-3f29-4f7a-9001-37d70adfca34</div>
<div> Bridge "testbr0"</div>
<div> Port "gre0"</div>
<div> Interface
"gre0"</div>
<div> type: gre</div>
<div> options:
{local_ip="30.30.0.193",
remote_ip="20.20.0.183"}</div>
<div> Port
"a0769422cfc04_l"</div>
<div> Interface
"a0769422cfc04_l"</div>
<div> Port "testbr0"</div>
<div> Interface
"testbr0"</div>
<div> type:
internal</div>
<div> ovs_version: "2.9.2"</div>
</div>
<div>
<div>[root@vm2 ~]# ip rule list</div>
<div>0: from all lookup
local </div>
<div>32766: from all lookup
main </div>
<div>32767: from all lookup
default </div>
</div>
<div>[root@vm2 ~]#
######################### A
docker container is attached to
ovs bridge using ovs-docker
utility<br>
</div>
<div>
<div>[root@vm2 ~]# docker ps</div>
<div>CONTAINER ID IMAGE
COMMAND
CREATED STATUS
PORTS
NAMES</div>
<div>86214f0d99e8
busybox:latest "sh"
5 days ago
Up 5 days
peaceful_snyder</div>
<div>[root@vm2 ~]# nsenter -n -t
`docker inspect 862
--format={{.State.Pid}}` -- ip
a</div>
<div>1: lo:
<LOOPBACK,UP,LOWER_UP>
mtu 65536 qdisc noqueue state
UNKNOWN qlen 1</div>
<div> link/loopback
00:00:00:00:00:00 brd
00:00:00:00:00:00</div>
<div> inet <a href="http://127.0.0.1/8" target="_blank">127.0.0.1/8</a>
scope host lo</div>
<div> valid_lft forever
preferred_lft forever</div>
<div> inet6 ::1/128 scope
host </div>
<div> valid_lft forever
preferred_lft forever</div>
<div>2: gre0@NONE: <NOARP>
mtu 1476 qdisc noop state DOWN
qlen 1</div>
<div> link/gre 0.0.0.0 brd
0.0.0.0</div>
<div>3: gretap0@NONE:
<BROADCAST,MULTICAST>
mtu 1462 qdisc noop state DOWN
qlen 1000</div>
<div> link/ether
00:00:00:00:00:00 brd
ff:ff:ff:ff:ff:ff</div>
<div>9: eth0@if10:
<BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc noqueue state
UP qlen 1000</div>
<div> link/ether
ae:ac:14:7a:40:5f brd
ff:ff:ff:ff:ff:ff link-netnsid
0</div>
<div> inet <a href="http://70.70.0.20/24" target="_blank">70.70.0.20/24</a>
scope global eth0</div>
<div> valid_lft forever
preferred_lft forever</div>
<div> inet6
fe80::acac:14ff:fe7a:405f/64
scope link </div>
<div> valid_lft forever
preferred_lft forever</div>
</div>
<div><br>
</div>
<div>With this configuration, if I
do a ping from docker container
on VM1 to docker container on
VM2 it works.</div>
<div>
<div><br>
</div>
<div>[root@vm1 ~]# nsenter -n -t
`docker inspect be4
--format={{.State.Pid}}` --
ping 70.70.0.20</div>
<div>PING 70.70.0.20
(70.70.0.20) 56(84) bytes of
data.</div>
<div>64 bytes from <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
icmp_seq=1 ttl=64 time=0.831
ms</div>
<div>64 bytes from <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
icmp_seq=2 ttl=64 time=0.933
ms</div>
<div>64 bytes from <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
icmp_seq=3 ttl=64 time=0.564
ms</div>
<div>^C</div>
<div>--- 70.70.0.20 ping
statistics ---</div>
<div>3 packets transmitted, 3
received, 0% packet loss, time
2001ms</div>
<div>rtt min/avg/max/mdev =
0.564/0.776/0.933/0.155 ms</div>
</div>
<div><br>
</div>
<div>And the traffic is as
expected on VM2.</div>
<div>
<div><br>
</div>
<div>[root@vm2 ~]# tcpdump -n -i
any host 20.20.0.183
</div>
<div>tcpdump: verbose output
suppressed, use -v or -vv for
full protocol decode</div>
<div>listening on any, link-type
LINUX_SLL (Linux cooked),
capture size 262144 bytes</div>
<div>16:37:32.262553 IP
20.20.0.183 > <a href="http://30.30.0.193" target="_blank">30.30.0.193</a>:
GREv0, length 102: IP
70.70.0.10 > <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
ICMP echo request, id 28158,
seq 1, length 64</div>
<div>16:37:32.262835 IP
30.30.0.193 > <a href="http://20.20.0.183" target="_blank">20.20.0.183</a>:
GREv0, length 102: IP
70.70.0.20 > <a href="http://70.70.0.10" target="_blank">70.70.0.10</a>:
ICMP echo reply, id 28158, seq
1, length 64</div>
<div>16:37:33.263211 IP
20.20.0.183 > <a href="http://30.30.0.193" target="_blank">30.30.0.193</a>:
GREv0, length 102: IP
70.70.0.10 > <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
ICMP echo request, id 28158,
seq 2, length 64</div>
<div>16:37:33.263374 IP
30.30.0.193 > <a href="http://20.20.0.183" target="_blank">20.20.0.183</a>:
GREv0, length 102: IP
70.70.0.20 > <a href="http://70.70.0.10" target="_blank">70.70.0.10</a>:
ICMP echo reply, id 28158, seq
2, length 64</div>
<div>16:37:34.264159 IP
20.20.0.183 > <a href="http://30.30.0.193" target="_blank">30.30.0.193</a>:
GREv0, length 102: IP
70.70.0.10 > <a href="http://70.70.0.20" target="_blank">70.70.0.20</a>:
ICMP echo request, id 28158,
seq 3, length 64</div>
<div>16:37:34.264252 IP
30.30.0.193 > <a href="http://20.20.0.183" target="_blank">20.20.0.183</a>:
GREv0, length 102: IP
70.70.0.20 > <a href="http://70.70.0.10" target="_blank">70.70.0.10</a>:
ICMP echo reply, id 28158, seq
3, length 64</div>
</div>
<div><br>
</div>
<div>But when I change the tunnel
type to vxlan, ping fails.</div>
<div><br>
</div>
<div>
<div>[root@vm1 ~]# ovs-vsctl
del-port testbr0 gre0</div>
<div>[root@vm1 ~]# ovs-vsctl
add-port testbr0 vxlan0 -- set
interface vxlan0 type=vxlan
options:local_ip=20.20.0.183
options:remote_ip=30.30.0.193
options:dst_port=4789</div>
<div>[root@vm1 ~]# ovs-vsctl
show</div>
<div>ff70c814-d1b0-4018-aee8-8b635187afee</div>
<div> Bridge "testbr0"</div>
<div> Port "testbr0"</div>
<div> Interface
"testbr0"</div>
<div> type:
internal</div>
<div> Port "vxlan0"</div>
<div> Interface
"vxlan0"</div>
<div> type: vxlan</div>
<div> options:
{dst_port="4789",
local_ip="20.20.0.183",
remote_ip="30.30.0.193"}</div>
<div> Port
"2cfb62a9b0f04_l"</div>
<div> Interface
"2cfb62a9b0f04_l"</div>
<div> ovs_version: "2.9.2"</div>
</div>
<div>
<div><br>
</div>
<div>[root@vm2 ~]# ovs-vsctl
del-port testbr0 gre0</div>
<div>[root@vm2 ~]# ovs-vsctl
add-port testbr0 vxlan0 -- set
interface vxlan0 type=vxlan
options:local_ip=30.30.0.193
options:remote_ip=20.20.0.183
options:dst_port=4789
</div>
<div>[root@vm2 ~]# ovs-vsctl
show</div>
<div>b85514db-3f29-4f7a-9001-37d70adfca34</div>
<div> Bridge "testbr0"</div>
<div> Port
"a0769422cfc04_l"</div>
<div> Interface
"a0769422cfc04_l"</div>
<div> Port "vxlan0"</div>
<div> Interface
"vxlan0"</div>
<div> type: vxlan</div>
<div> options:
{dst_port="4789",
local_ip="30.30.0.193",
remote_ip="20.20.0.183"}</div>
<div> Port "testbr0"</div>
<div> Interface
"testbr0"</div>
<div> type:
internal</div>
<div> ovs_version: "2.9.2"</div>
</div>
<div><br>
</div>
<div>Ping fails with this setup</div>
<div><br>
</div>
<div>
<div>[root@vm1 ~]# nsenter -n -t
`docker inspect be4
--format={{.State.Pid}}` --
ping 70.70.0.20</div>
<div>PING 70.70.0.20
(70.70.0.20) 56(84) bytes of
data.</div>
<div>^C</div>
<div>--- 70.70.0.20 ping
statistics ---</div>
<div>6 packets transmitted, 0
received, 100% packet loss,
time 4999ms</div>
<div><br>
</div>
</div>
<div>Expected traffic is not seen
on VM2</div>
<div>
<div><br>
</div>
<div>[root@vm2 ~]# tcpdump -n -i
any host 20.20.0.183
</div>
<div>tcpdump: verbose output
suppressed, use -v or -vv for
full protocol decode</div>
<div>listening on any, link-type
LINUX_SLL (Linux cooked),
capture size 262144 bytes</div>
<div>^C</div>
<div>0 packets captured</div>
<div>0 packets received by
filter</div>
<div>0 packets dropped by kernel</div>
</div>
<div><br>
</div>
<div>Kindly let me know if you
need more information.</div>
<div><br>
</div>
<div>Siva Teja.</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Nov 6, 2018 at 10:49 AM Gregory Rose
<<a href="mailto:gvrose8192@gmail.com" target="_blank">gvrose8192@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
<div class="gmail-m_-2500359003436686162m_5639563742210253641moz-cite-prefix">On
11/5/2018 6:10 PM, Siva Teja ARETI wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>
<div>Hi,</div>
<div><br>
</div>
<div>I am trying to use local_ip option for a
VXLAN tunnel using ovs but it does not seem to
work. The same works when I use GRE tunnel. I
also found a previous discussion from another
user who tried the exact same approach. Here is
the link to the discussion</div>
<div><br>
</div>
<div><font color="#1155cc"><u><a href="https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html" target="_blank">https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html</a></u></font><br>
</div>
<div><br>
</div>
<div>I am unable to find any working resolution at
the end of this discussion. Could you please
help?</div>
</div>
</div>
</div>
</blockquote>
<br>
I looked into that but was never able to set up a
configuration like the one in that discussion and could<br>
not repro the bug.<br>
<br>
Please provide some details on your usage, configuration
and steps to repro and I can look into it.<br>
<br>
Thanks,<br>
<br>
- Greg<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>
<div><br>
</div>
<div>I am using ovs 2.9.2</div>
<div><br>
</div>
<div>
<div>[root@localhost ~]# ovs-vsctl --version</div>
<div>ovs-vsctl (Open vSwitch) 2.9.2</div>
<div>DB Schema 7.15.1</div>
</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Siva Teja.</div>
</div>
</div>
</div>
<br>
<fieldset class="gmail-m_-2500359003436686162m_5639563742210253641mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
discuss mailing list
<a class="gmail-m_-2500359003436686162m_5639563742210253641moz-txt-link-abbreviated" href="mailto:discuss@openvswitch.org" target="_blank">discuss@openvswitch.org</a>
<a class="gmail-m_-2500359003436686162m_5639563742210253641moz-txt-link-freetext" href="https://mail.openvswitch.org/mailman/listinfo/ovs-discuss" target="_blank">https://mail.openvswitch.org/mailman/listinfo/ovs-discuss</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote></div></div></div>