<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <div class="moz-cite-prefix">On 11/13/2018 1:44 PM, Siva Teja ARETI
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAD6T32b5zmSXrOVBV6Be7s8S-vwjTtDVWzZbDQ1W0cFxO9i+ug@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=utf-8">
      <div dir="ltr">Hi Greg,
        <div><br>
        </div>
        <div>Did you happen to get a chance to investigate this further?</div>
      </div>
    </blockquote>
    <br>
    Unfortunately not.  The IT team replaced a switch in the lab over
    the weekend and my access to the<br>
    test machines is down.<br>
    I have a ticket in to get it fixed and will resume debugging then.<br>
    <br>
    Sorry for the delay.<br>
    <br>
    - Greg<br>
    <br>
    <blockquote type="cite"
cite="mid:CAD6T32b5zmSXrOVBV6Be7s8S-vwjTtDVWzZbDQ1W0cFxO9i+ug@mail.gmail.com">
      <div dir="ltr">
        <div><br>
        </div>
        <div>Siva Teja.</div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr">On Fri, Nov 9, 2018 at 1:26 PM Gregory Rose &lt;<a
            href="mailto:gvrose8192@gmail.com" moz-do-not-send="true">gvrose8192@gmail.com</a>&gt;
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div text="#000000" bgcolor="#FFFFFF"> <br>
            <div class="m_-6304102931384336444moz-cite-prefix">On
              11/8/2018 4:16 PM, Gregory Rose wrote:<br>
            </div>
            <blockquote type="cite"> On 11/8/2018 3:48 PM, Siva Teja
              ARETI wrote:<br>
              <blockquote type="cite">
                <div dir="ltr">
                  <div dir="ltr"><br>
                    <br>
                    <div class="gmail_quote">Siva,<br>
                      <blockquote class="gmail_quote" style="margin:0px
                        0px 0px 0.8ex;border-left:1px solid
                        rgb(204,204,204);padding-left:1ex">
                        <div bgcolor="#FFFFFF"> <br>
                          When you see the error condition with the
                          local_ip option on vxlan can you provide me
                          the output of<br>
                          this command?<br>
                          <br>
                          <font size="-1" face="Courier New, Courier,
                            monospace"><b><font color="#ff0000"># ip -s
                                link show vxlan_sys_4789</font></b><br>
                            70: vxlan_sys_4789:
                            &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu
                            65470 qdisc noqueue master ovs-system state
                            UNKNOWN mode DEFAULT group default qlen 1000<br>
                                link/ether 0e:9b:58:4a:6e:44 brd
                            ff:ff:ff:ff:ff:ff<br>
                                RX: bytes  packets  errors  dropped
                            overrun mcast<br>
                                0          0        0       0      
                            0       0<br>
                                TX: bytes  packets  errors  dropped
                            carrier collsns<br>
                                0          0        99      8      
                            99      0<br>
                          </font><br>
                        </div>
                      </blockquote>
                      <div>Hi Greg,</div>
                      <div><br>
                      </div>
                      <div>Here is the output. </div>
                      <div><br>
                      </div>
                      <div>[root@vm1 ~]# ip -s link show vxlan_sys_4789</div>
                      <div>27: vxlan_sys_4789:
                        &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu
                        65000 qdisc noqueue master ovs-system state
                        UNKNOWN mode DEFAULT qlen 1000</div>
                      <div>    link/ether ca:8f:0d:13:08:1f brd
                        ff:ff:ff:ff:ff:ff</div>
                      <div>    RX: bytes  packets  errors  dropped
                        overrun mcast</div>
                      <div>    0          0        0       0       0   
                           0</div>
                      <div>    TX: bytes  packets  errors  dropped
                        carrier collsns</div>
                      <div>    3666796    130957   0       0       0   
                           0</div>
                      <div><br>
                      </div>
                      <div> Siva Teja.</div>
                      <blockquote class="gmail_quote" style="margin:0px
                        0px 0px 0.8ex;border-left:1px solid
                        rgb(204,204,204);padding-left:1ex">
                        <div bgcolor="#FFFFFF"> It will help me
                          understand which error you're encountering.<br>
                          <br>
                          Thanks!<br>
                          <br>
                          - Greg<br>
                        </div>
                      </blockquote>
                    </div>
                  </div>
                </div>
              </blockquote>
              <br>
              Well then obviously I still have errors in my own setup.<br>
              <br>
              Back to the drawing board but I think it's a routing issue
              in my case.<br>
              <br>
              Thanks!<br>
              <br>
            </blockquote>
            <br>
            Siva,<br>
            <br>
            I've made progress.  I misconfigured my network which led to
            the errors you were seeing.  Now I've got that fixed up and
            I think I'm reproducing the error you are seeing.  When
            adding the local IP option the packets are getting<br>
            delivered to the VXLAN port but not getting delivered over
            to the bridge with the local ip address.<br>
            <br>
            I have two machines A and B.  They are bare metal running
            OVS with kvm virtual machines.  Here is the config:<br>
            <br>
            A) IP 10.172.208.214<br>
            <font size="-1" face="Courier New, Courier, monospace">   
              Bridge test-vxlan   &lt;-------- ip=10.1.1.3<br>
                      Port test-vxlan<br>
                          Interface test-vxlan<br>
                              type: internal<br>
                      Port "vxlan0"<br>
                          Interface "vxlan0"<br>
                              type: vxlan<br>
                              options: {key="100", local_ip="10.1.1.3",
              remote_ip="10.172.208.215"}<br>
                      Port "vnet4"<br>
                          Interface "vnet4"  &lt;-------- VM 1 with IP
              10.1.1.1<br>
            </font><br>
            <br>
            B) IP 10.172.208.215<br>
            <font size="-1" face="Courier New, Courier, monospace">   
              Bridge test-vxlan &lt;--------- ip=10.1.1.4<br>
                      Port "vxlan0"<br>
                          Interface "vxlan0"<br>
                              type: vxlan<br>
                              options: {key="100", local_ip="10.1.1.4",
              remote_ip="10.172.208.214"}<br>
                      Port "vnet6"<br>
                          Interface "vnet6"  &lt;-------- VM 2 with IP
              10.1.1.2<br>
                      Port test-vxlan<br>
                          Interface test-vxlan<br>
                              type: internal<br>
            </font><br>
            From VM 2 on machine B I start a ping from 10.1.1.2 -&gt;
            10.1.1.1<br>
            <br>
            <font size="-1" face="Courier New, Courier, monospace">roseg@ubuntu-1604-base:~$
              ping 10.1.1.1<br>
              PING 10.1.1.1 (10.1.1.1) 56(84) bytes of data.<br>
              From 10.1.1.2 icmp_seq=1 Destination Host Unreachable<br>
              From 10.1.1.2 icmp_seq=2 Destination Host Unreachable<br>
              From 10.1.1.2 icmp_seq=3 Destination Host Unreachable<br>
              From 10.1.1.2 icmp_seq=4 Destination Host Unreachable<br>
            </font><br>
            On machine B we can see the vxlan_sys_4789 tx counter
            increasing:<br>
            <br>
            <font size="-1" face="Courier New, Courier, monospace">[root@sc2-hs2-b2515
              ~]# ip -s link show vxlan_sys_4789<br>
              76: vxlan_sys_4789:
              &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 65470 qdisc
              noqueue master ovs-system state UNKNOWN mode DEFAULT group
              default qlen 1000<br>
                  link/ether f2:3a:d4:fd:b3:46 brd ff:ff:ff:ff:ff:ff<br>
                  RX: bytes  packets  errors  dropped overrun mcast<br>
                  0          0        0       0       0       0<br>
                  TX: bytes  packets  errors  dropped carrier collsns<br>
                  4200       150      0       8       0       0<br>
            </font><br>
            On machine A we can see the vxlan_sys_4789 rx counter
            increasing:<br>
            <br>
            <font face="Courier New, Courier, monospace"><font size="-1">53:
                vxlan_sys_4789: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt;
                mtu 65470 qdisc noqueue ma<br>
                ster ovs-system state UNKNOWN mode DEFAULT group default
                qlen 1000<br>
                    link/ether 06:4b:21:d8:af:8b brd ff:ff:ff:ff:ff:ff<br>
                    RX: bytes  packets  errors  dropped overrun mcast<br>
                    4200       150      0       0       0       0<br>
                    TX: bytes  packets  errors  dropped carrier collsns<br>
                    0          0        0       8       0       0</font><br>
            </font><br>
            However, even though there is no indication of drops the
            packets are not getting over to the test-vxlan bridge<br>
            which has the local 10.1.1.3 ip address:<br>
            <br>
            <font size="-1" face="Courier New, Courier, monospace">35:
              test-vxlan: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu
              1500 qdisc noqueue state UNKNOWN mode DEFAULT group
              default qlen 1000<br>
                  link/ether 86:9b:1f:ae:ba:42 brd ff:ff:ff:ff:ff:ff<br>
                  RX: bytes  packets  errors  dropped overrun mcast<br>
                  0          0        0       0       0       0<br>
                  TX: bytes  packets  errors  dropped carrier collsns<br>
                  0          0        0       0       0       0<br>
            </font><br>
            They're just not seen at all - none of the counters are
            increasing.  When I remove the local_ip option from<br>
            the vxlan tunnels then the ping between the VMs works as
            expected which you have shown:<br>
            <br>
            <font size="-1" face="Courier New, Courier, monospace">roseg@ubuntu-1604-base:~$
              ping 10.1.1.1<br>
              PING 10.1.1.1 (10.1.1.1) 56(84) bytes of data.<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=1 ttl=64
              time=2.04 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=2 ttl=64
              time=0.366 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=3 ttl=64
              time=0.332 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=4 ttl=64
              time=0.335 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=5 ttl=64
              time=0.367 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=6 ttl=64
              time=0.319 ms<br>
              64 bytes from <a href="http://10.1.1.1" target="_blank"
                moz-do-not-send="true">10.1.1.1</a>: icmp_seq=7 ttl=64
              time=0.272 ms<br>
              .<br>
              .<br>
              .<br>
            </font><br>
            So now that I have a reproduction I'm debugging why they're
            not getting delivered... <br>
            <br>
            I'll continue to update as the investigation continues.<br>
            <br>
            - Greg<br>
            <br>
          </div>
        </blockquote>
      </div>
    </blockquote>
    <br>
  </body>
</html>