[ovs-discuss] FW: OVS 2.9.0 native firewall drops empty payload TCP packets continued

[Darrell Ball]

The node you are displaying below is running kernel datapath

fyi: The fix Han pointed you to is for userspace datapath/conntrack



On Fri, May 3, 2019 at 8:14 AM Zhang, Jing C. (Nokia - CA/Ottawa) <
jing.c.zhang at nokia.com> wrote:

> We have both OVS and OVS-dpdk computes.
>
>
>
> Below is from OVS compute:
>
>
>
> # ovs-vsctl --no-wait get Open_vSwitch . other_config
>
> {}
>
> # ovs-vsctl -- list bridge br-int | grep datapath
>
> datapath_id         : "0000aaf62aaf3546"
>
> datapath_type       : system
>
> datapath_version    : "<unknown>"
>
>
>
> *From:* Darrell Ball <dlu998 at gmail.com>
> *Sent:* Friday, May 3, 2019 12:19 AM
> *To:* Han Zhou <zhouhan at gmail.com>; Zhang, Jing C. (Nokia - CA/Ottawa) <
> jing.c.zhang at nokia.com>; ovs-discuss at openvswitch.org
> *Subject:* Re: FW: [ovs-discuss] OVS 2.9.0 native firewall drops empty
> payload TCP packets continued
>
>
>
> What do the following commands yield ?
>
>
>
> sudo ovs-vsctl -- get bridge <bridge name> datapath_type
>
>
>
> sudo ovs-vsctl --no-wait get Open_vSwitch . other_config
>
>
>
>
>
>
>
> *From: *<ovs-discuss-bounces at openvswitch.org> on behalf of Han Zhou <
> zhouhan at gmail.com>
> *Date: *Thursday, May 2, 2019 at 7:12 PM
> *To: *"Zhang, Jing C. (Nokia - CA/Ottawa)" <jing.c.zhang at nokia.com>
> *Cc: *"ovs-discuss at openvswitch.org" <ovs-discuss at openvswitch.org>
> *Subject: *Re: [ovs-discuss] OVS 2.9.0 native firewall drops empty
> payload TCP packets continued
>
>
>
>
>
> On Thu, May 2, 2019 at 6:04 PM Zhang, Jing C. (Nokia - CA/Ottawa) <
> jing.c.zhang at nokia.com> wrote:
> >
> > We (our VNFs) continue to observe the same empty payload TCP (ACK)
> packet drop with native firewall (see original post below) after upgrading
> to Centos 7.6. This packet drop results in unacceptable TCP performance, by
> that native firewall still can not be enabled in product.
> >
> >
> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-August/047263.html
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fpipermail%2Fovs-discuss%2F2018-August%2F047263.html&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524642583&sdata=ihTE%2BcOA9d8yNflCbqJYHXOJWhFuqvq4yJmu7H9lGwo%3D&reserved=0>
> >
> > $ uname -a
> > Linux overcloud-sriovperformancecompute-0 3.10.0-957.10.1.el7.x86_64 #1
> SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> >
> > $ ovs-vswitchd --version
> > ovs-vswitchd (Open vSwitch) 2.9.0
> > DPDK 17.11.0
> >
> > The scenario: OVS provider VLAN network is used
> >
> >
> > in physical interface of ovs compute zero length tcp payload packet
> arrives as padded to 64 bytes (and vlan tag is included in ethernet header)
> > same packet does not appear anymore in the tcpdump taken from tap-xyz
> interface (once vlan tag is removed and packet is cut by 4 bytes to 60
> bytes)
> >
> >
> > Tcpdump on physical port:
> >
> > 00:25:24.468423 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q
> (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
> 6893, offset 0, flags [DF], proto TCP (6), length 2656)
> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> > 00:25:24.468593 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q
> (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
> 56318, offset 0, flags [DF], proto TCP (6), length 40)
> >     192.168.10.60.57576 > 192.168.10.52.80: Flags [.], cksum 0x1d34
> (correct), seq 78, ack 11577, win 391, length 0
> > 00:25:24.475848 fa:16:3e:ff:dd:29 > fa:16:3e:d7:bb:2c, ethertype 802.1Q
> (0x8100), length 60: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
> 56319, offset 0, flags [DF], proto TCP (6), length 40)
> >     192.168.10.60.57576 > 192.168.10.52.80: Flags [F.], cksum 0x1d33
> (correct), seq 78, ack 11577, win 391, length 0
> > 00:25:24.480337 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype 802.1Q
> (0x8100), length 2674: vlan 3837, p 0, ethertype IPv4, (tos 0x0, ttl 64, id
> 6894, offset 0, flags [DF], proto TCP (6), length 2656)
> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> >
> > Tcpdump on vm tap interface:
> >
> > 00:25:24.468419 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4
> (0x0800), length 2670: (tos 0x0, ttl 64, id 6893, offset 0, flags [DF],
> proto TCP (6), length 2656)
> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> > 00:25:24.480331 fa:16:3e:d7:bb:2c > fa:16:3e:ff:dd:29, ethertype IPv4
> (0x0800), length 2670: (tos 0x0, ttl 64, id 6894, offset 0, flags [DF],
> proto TCP (6), length 2656)
> >     192.168.10.52.80 > 192.168.10.60.57576: Flags [P.], cksum 0xa013
> (incorrect -> 0x772d), seq 8961:11577, ack 78, win 210, length 2616: HTTP
> >
> > Very straightforward to see the issue:
> >
> >
> > Configure neutron OVS agent to use native firewall
> > Create a pair of VMs on separate computes on provider vLAN
> > Disable TCP timestamp inside the VMs
> > Exchange TCP traffic between the VMs, e.g. http download.
> > Tcpdump on the physical and vm port, and compare.
> >
> >
> > I wonder why such obvious issue is not widely discussed?
> >
> > Jing
> >
>
>
>
> Maybe it's fixed by:
>
>
> https://github.com/openvswitch/ovs/commit/9171c63532ee9cbc63bb8cfae364ab071f44389b
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenvswitch%2Fovs%2Fcommit%2F9171c63532ee9cbc63bb8cfae364ab071f44389b&data=02%7C01%7Cdball%40vmware.com%7Cd358d5a23b2640ebd28708d6cf6cc524%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636924463524652585&sdata=OLr263fKPdgKA5nga2UlGNF0WtB6srXSIg9a7aHkf44%3D&reserved=0>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190503/831d66ae/attachment.html>