[ovs-discuss] FW: OVS 2.9.0 native firewall drops empty payload TCP packets continued

Zhang, Jing C. (Nokia - CA/Ottawa) jing.c.zhang at nokia.com
Fri May 3 17:51:13 UTC 2019


Sorry, I overlooked your questions on dpdk computes. DPDK itself is find, VMs are up and running.

Jing

From: Darrell Ball <dlu998 at gmail.com>
Sent: Friday, May 3, 2019 11:55 AM
To: Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com>
Cc: Han Zhou <zhouhan at gmail.com>; ovs-discuss at openvswitch.org
Subject: Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty payload TCP packets continued

couple corrections inline

On Fri, May 3, 2019 at 8:52 AM Darrell Ball <dlu998 at gmail.com<mailto:dlu998 at gmail.com>> wrote:


On Fri, May 3, 2019 at 8:29 AM Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>> wrote:

  1.  This issue is with native OVS firewall where the data flows are subject to conntrack rules, there is no issue for hybrid firewall

1/ Does 'native OVS firewall' mean either kernel datapath or userpace datapath ?
2/ Pls define 'hybrid datapath' in your context ?

2/ Pls define 'hybrid firewall' in your context ?



  1.


  1.  Below is from DPDK compute:

# ovs-vsctl --no-wait get Open_vSwitch . other_config

3/ dpdk is not initialized

An info log is also present when dpdk is initialized: "DPDK Enabled - initialized"

btw: '--no-wait' is needed for get commands

btw: '--no-wait' is NOT needed for get commands



# ovs-vsctl -- list bridge br-int | grep datapath
datapath_id         : "00001a9b5b9ec94e"
datapath_type       : netdev
datapath_version    : "<built-in>"

4/ You are using userspace datapath on this particular node without dpdk support
    Is that intentional ?



From: Darrell Ball <dlu998 at gmail.com<mailto:dlu998 at gmail.com>>
Sent: Friday, May 3, 2019 11:24 AM
To: Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>>
Cc: Han Zhou <zhouhan at gmail.com<mailto:zhouhan at gmail.com>>; ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>
Subject: Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty payload TCP packets continued

The node you are displaying below is running kernel datapath

fyi: The fix Han pointed you to is for userspace datapath/conntrack



On Fri, May 3, 2019 at 8:14 AM Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>> wrote:
We have both OVS and OVS-dpdk computes.

Below is from OVS compute:

# ovs-vsctl --no-wait get Open_vSwitch . other_config
{}
# ovs-vsctl -- list bridge br-int | grep datapath
datapath_id         : "0000aaf62aaf3546"
datapath_type       : system
datapath_version    : "<unknown>"

From: Darrell Ball <dlu998 at gmail.com<mailto:dlu998 at gmail.com>>
Sent: Friday, May 3, 2019 12:19 AM
To: Han Zhou <zhouhan at gmail.com<mailto:zhouhan at gmail.com>>; Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>>; ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>
Subject: Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty payload TCP packets continued

What do the following commands yield ?

sudo ovs-vsctl -- get bridge <bridge name> datapath_type

sudo ovs-vsctl --no-wait get Open_vSwitch . other_config



From: <ovs-discuss-bounces at openvswitch.org<mailto:ovs-discuss-bounces at openvswitch.org>> on behalf of Han Zhou <zhouhan at gmail.com<mailto:zhouhan at gmail.com>>
Date: Thursday, May 2, 2019 at 7:12 PM
To: "Zhang, Jing C. (Nokia - CA/Ottawa)" <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>>
Cc: "ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>" <ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>>
Subject: Re: [ovs-discuss] OVS 2.9.0 native firewall drops empty payload TCP packets continued



On Thu, May 2, 2019 at 6:04 PM Zhang, Jing C. (Nokia - CA/Ottawa) <jing.c.zhang at nokia.com<mailto:jing.c.zhang at nokia.com>> wrote:
>
> We (our VNFs) continue to observe the same empty payload TCP (ACK) packet drop with native firewall (see original post below) after upgrading to Centos 7.6. This packet drop results in unacceptable TCP performance, by that native firewall still can not be enabled in product.
>
> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-August/047263.html<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fpipermail%2Fovs-discuss%2F2018-August%2F047263.html&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524642583&sdata=ihTE%2BcOA9d8yNflCbqJYHXOJWhFuqvq4yJmu7H9lGwo%3D&reserved=0>
>
> $ uname -a
> Linux overcloud-sriovperformancecompute-0 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>
> $ ovs-vswitchd --version
> ovs-vswitchd (Open vSwitch) 2.9.0
> DPDK 17.11.0
>
> The scenario: OVS provider VLAN network is used
>
>
> in physical interface of ovs compute zero length tcp payload packet arrives as padded to 64 bytes (and vlan tag is included in ethernet header)
> same packet does not appear anymore in the tcpdump taken from tap-xyz interface (once vlan tag is removed and packet is cut by 4 bytes to 60 bytes)
>
>
> Tcpdump on physical port:
>
> 00:25:24.468423 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 6893, offset 0, flags [DF], proto TCP (6), length 2656)
>     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013 (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> 00:25:24.468593 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 56318, offset 0, flags [DF], proto TCP (6), length 40)
>     192.168.10.60.57576 > 192.168.10.52.80: Flags [.], cksum 0x1d34 (correct), seq 78, ack 11577, win 391, length 0
> 00:25:24.475848 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 56319, offset 0, flags [DF], proto TCP (6), length 40)
>     192.168.10.60.57576 > 192.168.10.52.80: Flags [F.], cksum 0x1d33 (correct), seq 78, ack 11577, win 391, length 0
> 00:25:24.480337 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 6894, offset 0, flags [DF], proto TCP (6), length 2656)
>     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013 (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>
> Tcpdump on vm tap interface:
>
> 00:25:24.468419 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4 (0x0800), length 2670: (tos 0x0, ttl 64, id 6893, offset 0, flags [DF], proto TCP (6), length 2656)
>     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013 (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> 00:25:24.480331 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4 (0x0800), length 2670: (tos 0x0, ttl 64, id 6894, offset 0, flags [DF], proto TCP (6), length 2656)
>     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013 (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
>
> Very straightforward to see the issue:
>
>
> Configure neutron OVS agent to use native firewall
> Create a pair of VMs on separate computes on provider vLAN
> Disable TCP timestamp inside the VMs
> Exchange TCP traffic between the VMs, e.g. http download.
> Tcpdump on the physical and vm port, and compare.
>
>
> I wonder why such obvious issue is not widely discussed?
>
> Jing
>

Maybe it's fixed by:
https://github.com/openvswitch/ovs/commit/9171c63532ee9cbc63bb8cfae364ab071f44389b<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenvswitch%2Fovs%2Fcommit%2F9171c63532ee9cbc63bb8cfae364ab071f44389b&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524652585&sdata=OLr263fKPdgKA5nga2UlGNF0WtB6srXSIg9a7aHkf44%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190503/a765c560/attachment-0001.html>


More information about the discuss mailing list