[ovs-discuss] Bridge traffic stops flowing after ovs-vtep run

Guru Shetty guru at ovn.org
Thu Nov 26 16:07:58 UTC 2015


On 26 November 2015 at 02:58, Mikhail Bagrov <mbagrov at cloudlinux.com> wrote:

> Hi Guru, Thanks much. I've added the internal bridge ports, started the
> vtep-emulator, added a docker container on all of my 3 nodes.
> ====================
> [root at ovs-1 ovs]# ovs-vsctl show
> 63796e59-59dd-4f8d-9ad2-2721257e1b60
>     Bridge "br0"
>         Port "p0"
>             Interface "p0"
>                 type: internal
>         Port "p1"
>             Interface "p1"
>                 type: internal
>         Port "1ab7473b25934_l"
>             Interface "1ab7473b25934_l"
>         Port "0000-p0-p"
>             Interface "0000-p0-p"
>                 type: patch
>                 options: {peer="0000-p0-l"}
>         Port "br0"
>             Interface "br0"
>                 type: internal
>     Bridge vtep_bfd
>         Port vtep_bfd
>             Interface vtep_bfd
>                 type: internal
>         Port "bfd10.66.66.35"
>             Interface "bfd10.66.66.35"
>                 type: vxlan
>                 options: {remote_ip="10.66.66.35"}
>         Port "bfd10.66.66.36"
>             Interface "bfd10.66.66.36"
>                 type: vxlan
>                 options: {remote_ip="10.66.66.36"}
>         Port "bfd10.66.66.37"
>             Interface "bfd10.66.66.37"
>                 type: vxlan
>                 options: {remote_ip="10.66.66.37"}
>     Bridge "vtep_ls1"
>         Port "vx5"
>             Interface "vx5"
>                 type: vxlan
>                 options: {key="0", remote_ip="10.66.66.37"}
>         Port "vx4"
>             Interface "vx4"
>                 type: vxlan
>                 options: {key="0", remote_ip="10.66.66.36"}
>         Port "vx6"
>             Interface "vx6"
>                 type: vxlan
>                 options: {key="0", remote_ip="10.66.66.35"}
>         Port "vtep_ls1"
>             Interface "vtep_ls1"
>                 type: internal
>         Port "0000-p0-l"
>             Interface "0000-p0-l"
>                 type: patch
>                 options: {peer="0000-p0-p"}
> ======================
> Interface "1ab7473b25934_l" is a docker eth1 port
> ======================
> also added container's MAC to VTEP-IP bindings
> ======================
> [root at ovs-1 ovs]# vtep-ctl list-remote-macs ls0
> ucast-mac-remote
>   0a:34:43:e1:5a:45 -> vxlan_over_ipv4/10.66.66.35
>   3e:88:e6:eb:71:0a -> vxlan_over_ipv4/10.66.66.36
>   e6:bd:d8:2c:39:2d -> vxlan_over_ipv4/10.66.66.37
>
> mcast-mac-remote
> ======================
> When I try to ping between containers, it fails, but ovs-ofctl dump-flows
> increments
> ======================
> [root at ovs-1 ovs]# ovs-ofctl dump-flows br0
> NXST_FLOW reply (xid=0x4):
>  cookie=0x0, duration=11227.914s, table=0, n_packets=8, n_bytes=648,
> idle_age=11183, in_port=1 actions=output:3
>  cookie=0x0, duration=11227.912s, table=0, n_packets=0, n_bytes=0,
> idle_age=11227, in_port=3 actions=output:1
> [root at ovs-1 ovs]# ovs-ofctl dump-flows br0
> NXST_FLOW reply (xid=0x4):
>  cookie=0x0, duration=11230.203s, table=0, n_packets=8, n_bytes=648,
> idle_age=11185, in_port=1 actions=output:3
>  cookie=0x0, duration=11230.201s, table=0, n_packets=0, n_bytes=0,
> idle_age=11230, in_port=3 actions=output:1
> ======================
> [root at ovs-1 ovs]# ovs-ofctl dump-ports br0 1ab7473b25934_l
> OFPST_PORT reply (xid=0x4): 1 ports
>   port  4: rx pkts=7223, bytes=303678, drop=0, errs=0, frame=0, over=0,
> crc=0
>            tx pkts=8, bytes=648, drop=0, errs=0, coll=0
> [root at ovs-1 ovs]# ovs-ofctl dump-ports br0 1ab7473b25934_l
> OFPST_PORT reply (xid=0x4): 1 ports
>   port  4: rx pkts=7226, bytes=303804, drop=0, errs=0, frame=0, over=0,
> crc=0
>            tx pkts=8, bytes=648, drop=0, errs=0, coll=0
>


There is a discrepancy above. As you can see the openflow flows in br0 has
two ofports - 3 and 1. But your container interface has a ofport of '4'. So
your container traffic will go nowhere.

If I had to guess, you have not added your container interface as a port to
physical switch using vtep-ctl. If you actually added that, then you have
not restarted your ovs-vtep python daemon.


> ======================
> and tcpdump on docker bridge captures only ARP requests, but no answers.
> ======================
> [root at ovs-1 ovs]# tcpdump -i 1ab7473b25934_l
> tcpdump: WARNING: 1ab7473b25934_l: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on 1ab7473b25934_l, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 10:35:28.282498 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> 10:35:29.285339 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> 10:35:30.287307 ARP, Request who-has 10.111.111.113 tell 10.111.111.111,
> length 28
> =======================
> Logs look fine except /var/log/openvswitch/ovs-vswitchd.log
> =======================
> 2015-11-26T10:35:30.781Z|00754|ofproto|WARN|vtep_bfd: cannot configure bfd
> on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00755|ofproto|WARN|vtep_bfd: cannot configure
> LLDP on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00756|ofproto|WARN|vtep_bfd: cannot configure bfd
> on nonexistent port 65535
> 2015-11-26T10:35:30.781Z|00757|ofproto|WARN|vtep_bfd: cannot configure
> LLDP on nonexistent port 65535
> 2015-11-26T10:36:11.109Z|00758|ofproto|WARN|Dropped 59 log messages in
> last 60 seconds (most recently, 5 seconds ago) due to excessive rate
> 2015-11-26T10:36:11.109Z|00759|ofproto|WARN|vtep_bfd: cannot get STP stats
> on nonexistent port 65535
> 2015-11-26T10:37:11.120Z|00760|ofproto|WARN|Dropped 35 log messages in
> last 60 seconds (most recently, 5 seconds ago) due to excessive rate
> 2015-11-26T10:37:11.120Z|00761|ofproto|WARN|vtep_bfd: cannot get STP stats
> on nonexistent port 65535
> =======================
> Looks like I've missed something....
>
> Thanks in advance.
>
> On Wed, Nov 25, 2015 at 8:04 PM, Guru Shetty <guru at ovn.org> wrote:
>
>>
>>
>> On 23 November 2015 at 09:11, Mikhail Bagrov <mbagrov at cloudlinux.com>
>> wrote:
>>
>>> Hi. I'm trying to implement the VTEP Emulator from
>>> https://github.com/openvswitch/ovs/blob/master/vtep/README.ovs-vtep.md
>>> I've built OVS from source on Centos7, trying to interconnect docker
>>> containers between different and within a single docker node. Still no
>>> traffic flows between containers.
>>>
>>
>> Debugging a VTEP emulator is a step by step process. How familiar are you
>> with openflow flows? i.e. do you understand what 'ovs-ofctl dump-flows br0'
>> says? When you ping from your docker container do you see a particular flow
>> stats getting incremented ?
>>
>>
>>>
>>> I have:
>>> [root at ovs11 ovs]# vtep-ctl list-remote-macs ls0
>>> ucast-mac-remote
>>>   7a:2c:ba:c0:62:16 -> vxlan_over_ipv4/10.2.2.102
>>>   c2:fd:9f:ea:3f:e3 -> vxlan_over_ipv4/10.2.2.101
>>>   de:eb:3c:62:65:fc -> vxlan_over_ipv4/10.2.2.103
>>> a mac to ip mapping table populated on each host.
>>> [root at ovs11 ovs]# ovs-vsctl show
>>> 82434185-c196-45dd-bcf1-1260e33ceae3
>>>     Bridge vtep_bfd
>>>         Port "bfd10.2.2.101"
>>>             Interface "bfd10.2.2.101"
>>>                 type: vxlan
>>>                 options: {remote_ip="10.2.2.101"}
>>>         Port "bfd10.2.2.102"
>>>             Interface "bfd10.2.2.102"
>>>                 type: vxlan
>>>                 options: {remote_ip="10.2.2.102"}
>>>         Port "bfd10.2.2.103"
>>>             Interface "bfd10.2.2.103"
>>>                 type: vxlan
>>>                 options: {remote_ip="10.2.2.103"}
>>>         Port vtep_bfd
>>>             Interface vtep_bfd
>>>                 type: internal
>>>     Bridge "vtep_ls1"
>>>         Port "vx2"
>>>             Interface "vx2"
>>>                 type: vxlan
>>>                 options: {key="0", remote_ip="10.2.2.102"}
>>>         Port "vx1"
>>>             Interface "vx1"
>>>                 type: vxlan
>>>                 options: {key="0", remote_ip="10.2.2.101"}
>>>         Port "0000-p0-l"
>>>             Interface "0000-p0-l"
>>>                 type: patch
>>>                 options: {peer="0000-p0-p"}
>>>         Port "vx3"
>>>             Interface "vx3"
>>>                 type: vxlan
>>>                 options: {key="0", remote_ip="10.2.2.103"}
>>>         Port "vtep_ls1"
>>>             Interface "vtep_ls1"
>>>                 type: internal
>>>     Bridge "br0"
>>>         Port "p1"
>>>             Interface "p1"
>>>         Port "br0"
>>>             Interface "br0"
>>>                 type: internal
>>>         Port "0000-p0-p"
>>>             Interface "0000-p0-p"
>>>                 type: patch
>>>                 options: {peer="0000-p0-l"}
>>>         Port "1d53fed7f9284_l"
>>>             Interface "1d53fed7f9284_l"
>>>         Port "p0"
>>>             Interface "p0"
>>>
>>> Port "1d53fed7f9284_l" is a docker container's port.
>>> Settings look fine, but still no traffic flows. Thanks in advance.
>>>
>>>
>>> --
>>> Regards, Mikhail Bagrov
>>> DevOps at Kuberdock
>>>
>>>
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> http://openvswitch.org/mailman/listinfo/discuss
>>>
>>>
>>
>
>
> --
> Regards, Mikhail Bagrov
> DevOps at Kuberdock
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151126/154da49c/attachment-0002.html>


More information about the discuss mailing list