[ovs-discuss] FW: OVS 2.9.0 native firewall drops empty payload TCP packets continued

Darrell Ball dlu998 at gmail.com
Fri May 3 15:55:15 UTC 2019


couple corrections inline

On Fri, May 3, 2019 at 8:52 AM Darrell Ball <dlu998 at gmail.com> wrote:

>
>
> On Fri, May 3, 2019 at 8:29 AM Zhang, Jing C. (Nokia - CA/Ottawa) <
> jing.c.zhang at nokia.com> wrote:
>
>>
>>    1. This issue is with native OVS firewall where the data flows are
>>    subject to conntrack rules, there is no issue for hybrid firewall
>>
>>
> 1/ Does 'native OVS firewall' mean either kernel datapath or userpace
> datapath ?
> 2/ Pls define 'hybrid datapath' in your context ?
>

2/ Pls define 'hybrid firewall' in your context ?


>
>
>>
>>    1.
>>
>>
>>
>>    1. Below is from DPDK compute:
>>
>>
>>
>> # ovs-vsctl --no-wait get Open_vSwitch . other_config
>>
>
> 3/ dpdk is not initialized
>
> An info log is also present when dpdk is initialized: "DPDK Enabled -
> initialized"
>
> btw: '--no-wait' is needed for get commands
>

btw: '--no-wait' is NOT needed for get commands


>
>
>
>> # ovs-vsctl -- list bridge br-int | grep datapath
>>
>> datapath_id         : "00001a9b5b9ec94e"
>>
>> datapath_type       : netdev
>>
>> datapath_version    : "<built-in>"
>>
>
> 4/ You are using userspace datapath on this particular node without dpdk
> support
>     Is that intentional ?
>
>
>
>>
>>
>> *From:* Darrell Ball <dlu998 at gmail.com>
>> *Sent:* Friday, May 3, 2019 11:24 AM
>> *To:* Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com>
>> *Cc:* Han Zhou <zhouhan at gmail.com>; ovs-discuss at openvswitch.org
>> *Subject:* Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty
>> payload TCP packets continued
>>
>>
>>
>> The node you are displaying below is running kernel datapath
>>
>>
>>
>> fyi: The fix Han pointed you to is for userspace datapath/conntrack
>>
>>
>>
>>
>>
>>
>>
>> On Fri, May 3, 2019 at 8:14 AM Zhang, Jing C. (Nokia - CA/Ottawa) <
>> jing.c.zhang at nokia.com> wrote:
>>
>> We have both OVS and OVS-dpdk computes.
>>
>>
>>
>> Below is from OVS compute:
>>
>>
>>
>> # ovs-vsctl --no-wait get Open_vSwitch . other_config
>>
>> {}
>>
>> # ovs-vsctl -- list bridge br-int | grep datapath
>>
>> datapath_id         : "0000aaf62aaf3546"
>>
>> datapath_type       : system
>>
>> datapath_version    : "<unknown>"
>>
>>
>>
>> *From:* Darrell Ball <dlu998 at gmail.com>
>> *Sent:* Friday, May 3, 2019 12:19 AM
>> *To:* Han Zhou <zhouhan at gmail.com>; Zhang, Jing C. (Nokia - CA/Ottawa) <
>> jing.c.zhang at nokia.com>; ovs-discuss at openvswitch.org
>> *Subject:* Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty
>> payload TCP packets continued
>>
>>
>>
>> What do the following commands yield ?
>>
>>
>>
>> sudo ovs-vsctl -- get bridge <bridge name> datapath_type
>>
>>
>>
>> sudo ovs-vsctl --no-wait get Open_vSwitch . other_config
>>
>>
>>
>>
>>
>>
>>
>> *From: *<ovs-discuss-bounces at openvswitch.org> on behalf of Han Zhou <
>> zhouhan at gmail.com>
>> *Date: *Thursday, May 2, 2019 at 7:12 PM
>> *To: *"Zhang, Jing C. (Nokia - CA/Ottawa)" <jing.c.zhang at nokia.com>
>> *Cc: *"ovs-discuss at openvswitch.org" <ovs-discuss at openvswitch.org>
>> *Subject: *Re: [ovs-discuss] OVS 2.9.0 native firewall drops empty
>> payload TCP packets continued
>>
>>
>>
>>
>>
>> On Thu, May 2, 2019 at 6:04 PM Zhang, Jing C. (Nokia - CA/Ottawa) <
>> jing.c.zhang at nokia.com> wrote:
>> >
>> > We (our VNFs) continue to observe the same empty payload TCP (ACK)
>> packet drop with native firewall (see original post below) after upgrading
>> to Centos 7.6. This packet drop results in unacceptable TCP performance, by
>> that native firewall still can not be enabled in product.
>> >
>> >
>> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-August/047263.html
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fpipermail%2Fovs-discuss%2F2018-August%2F047263.html&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524642583&sdata=ihTE%2BcOA9d8yNflCbqJYHXOJWhFuqvq4yJmu7H9lGwo%3D&reserved=0>
>> >
>> > $ uname -a
>> > Linux overcloud-sriovperformancecompute-0 3.10.0-957.10.1.el7.x86_64 #1
>> SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>> >
>> > $ ovs-vswitchd --version
>> > ovs-vswitchd (Open vSwitch) 2.9.0
>> > DPDK 17.11.0
>> >
>> > The scenario: OVS provider VLAN network is used
>> >
>> >
>> > in physical interface of ovs compute zero length tcp payload packet
>> arrives as padded to 64 bytes (and vlan tag is included in ethernet header)
>> > same packet does not appear anymore in the tcpdump taken from tap-xyz
>> interface (once vlan tag is removed and packet is cut by 4 bytes to 60
>> bytes)
>> >
>> >
>> > Tcpdump on physical port:
>> >
>> > 00:25:24.468423 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q
>> (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
>> 6893, offset 0, flags [DF], proto TCP (6), length 2656)
>> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
>> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>> > 00:25:24.468593 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q
>> (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
>> 56318, offset 0, flags [DF], proto TCP (6), length 40)
>> >     192.168.10.60.57576 > 192.168.10.52.80: Flags [.], cksum 0x1d34
>> (correct), seq 78, ack 11577, win 391, length 0
>> > 00:25:24.475848 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q
>> (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
>> 56319, offset 0, flags [DF], proto TCP (6), length 40)
>> >     192.168.10.60.57576 > 192.168.10.52.80: Flags [F.], cksum 0x1d33
>> (correct), seq 78, ack 11577, win 391, length 0
>> > 00:25:24.480337 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q
>> (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
>> 6894, offset 0, flags [DF], proto TCP (6), length 2656)
>> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
>> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>> >
>> > Tcpdump on vm tap interface:
>> >
>> > 00:25:24.468419 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4
>> (0x0800), length 2670: (tos 0x0, ttl 64, id 6893, offset 0, flags [DF],
>> proto TCP (6), length 2656)
>> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
>> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>> > 00:25:24.480331 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4
>> (0x0800), length 2670: (tos 0x0, ttl 64, id 6894, offset 0, flags [DF],
>> proto TCP (6), length 2656)
>> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
>> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>> >
>> > Very straightforward to see the issue:
>> >
>> >
>> > Configure neutron OVS agent to use native firewall
>> > Create a pair of VMs on separate computes on provider vLAN
>> > Disable TCP timestamp inside the VMs
>> > Exchange TCP traffic between the VMs, e.g. http download.
>> > Tcpdump on the physical and vm port, and compare.
>> >
>> >
>> > I wonder why such obvious issue is not widely discussed?
>> >
>> > Jing
>> >
>>
>>
>>
>> Maybe it's fixed by:
>>
>>
>> https://github.com/openvswitch/ovs/commit/9171c63532ee9cbc63bb8cfae364ab071f44389b
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenvswitch%2Fovs%2Fcommit%2F9171c63532ee9cbc63bb8cfae364ab071f44389b&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524652585&sdata=OLr263fKPdgKA5nga2UlGNF0WtB6srXSIg9a7aHkf44%3D&reserved=0>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190503/27428a53/attachment-0001.html>


More information about the discuss mailing list