[ovs-dev] [PATCH v4 0/3] Add support for TSO with DPDK

[Loftus, Ciara]

> -----Original Message-----
> From: Flavio Leitner <fbl at sysclose.org>
> Sent: Thursday 16 January 2020 17:01
> To: dev at openvswitch.org
> Cc: Stokes, Ian <ian.stokes at intel.com>; Loftus, Ciara
> <ciara.loftus at intel.com>; Ilya Maximets <i.maximets at ovn.org>;
> yangyi01 at inspur.com; txfh2007 <txfh2007 at aliyun.com>; Flavio Leitner
> <fbl at sysclose.org>
> Subject: [PATCH v4 0/3] Add support for TSO with DPDK
> 
> Abbreviated as TSO, TCP Segmentation Offload is a feature which enables
> the network stack to delegate the TCP segmentation to the NIC reducing
> the per packet CPU overhead.
> 
> A guest using vhost-user interface with TSO enabled can send TCP packets
> much bigger than the MTU, which saves CPU cycles normally used to break
> the packets down to MTU size and to calculate checksums.
> 
> It also saves CPU cycles used to parse multiple packets/headers during
> the packet processing inside virtual switch.
> 
> If the destination of the packet is another guest in the same host, then
> the same big packet can be sent through a vhost-user interface skipping
> the segmentation completely. However, if the destination is not local,
> the NIC hardware is instructed to do the TCP segmentation and checksum
> calculation.
> 
> The first 2 patches are not really part of TSO support, but they are
> required to make sure everything works.
> 
> There are good improvements sending to or receiving from veth pairs or
> tap devices as well. See the iperf3 results below:
> 
> [*] veth with ethtool tx off.
> 
> VM sending to:          Default           Enabled     Enabled/Default
>    Local BR             3 Gbits/sec     23 Gbits/sec      7x
>    Net NS (veth)        3 Gbits/sec[*]  22 Gbits/sec      7x
>    VM (same host)     2.5 Gbits/sec     24 Gbits/sec      9x
>    Ext Host            10 Gbits/sec     35 Gbits/sec      3x

I re-ran my tests and observed similar (slightly better, actually) improvements as I reported for the v3:
VM -> VM (same host): +5.5x
VM -> Ext Host: +4.1x

Tested-by: Ciara Loftus <ciara.loftus at intel.com>

Thanks,
Ciara

>    Ext Host (vxlan)   8.8 Gbits/sec     (not supported)
> 
>   Using VLAN:
>    Local BR             3 Gbits/sec     23 Gbits/sec      7x
>    VM (same host)     2.5 Gbits/sec     21 Gbits/sec      8x
>    Ext Host           6.4 Gbits/sec     34 Gbits/sec      5x
> 
>   Using IPv6:
>    Net NS (veth)      2.7 Gbits/sec[*]  22 Gbits/sec      8x
>    VM (same host)     2.6 Gbits/sec     21 Gbits/sec      8x
>    Ext Host           8.7 Gbits/sec     34 Gbits/sec      4x
> 
>   Conntrack:
>    No packet changes: 1.41 Gbits/sec    33 Gbits/sec      23x
> 
> VM receiving from:
>    Local BR           2.5 Gbits/sec     2.4 Gbits/sec     1x
>    Net NS (veth)      2.5 Gbits/sec[*]  9.3 Gbits/sec     3x
>    VM (same host)     4.9 Gbits/sec      25 Gbits/sec     5x
>    Ext Host           9.7 Gbits/sec     9.4 Gbits/sec     1x
>    Ext Host (vxlan)   5.5 Gbits/sec     (not supported)
> 
>   Using VLAN:
>    Local BR           2.4 Gbits/sec     2.4 Gbits/sec     1x
>    VM (same host)     3.8 Gbits/sec      24 Gbits/sec     8x
>    Ext Host           9.5 Gbits/sec     9.5 Gbits/sec     1x
> 
>   Using IPv6:
>    Net NS (veth)      2.2 Gbits/sec[*]   9 Gbits/sec      4x
>    VM (same host)     4.5 Gbits/sec     24 Gbits/sec      5x
>    Ext Host           8.9 Gbits/sec    8.9 Gbits/sec      1x
> 
> Used iperf3 -u to test UDP traffic limited at default 1Mbits/sec
> and noticed no change with the exception for tunneled packets (not
> supported).
> 
> Travis, AppVeyor, and Cirrus-ci passed.
> 
> Flavio Leitner (3):
>   dp-packet: preserve headroom when cloning a pkt batch
>   vhost: Disable multi-segmented buffers
>   netdev-dpdk: Add TCP Segmentation Offload support
> 
>  Documentation/automake.mk              |   1 +
>  Documentation/topics/index.rst         |   1 +
>  Documentation/topics/userspace-tso.rst |  98 +++++++
>  NEWS                                   |   1 +
>  lib/automake.mk                        |   2 +
>  lib/conntrack.c                        |  29 +-
>  lib/dp-packet.h                        | 192 +++++++++++-
>  lib/ipf.c                              |  32 +-
>  lib/netdev-dpdk.c                      | 355 ++++++++++++++++++++---
>  lib/netdev-linux-private.h             |   5 +
>  lib/netdev-linux.c                     | 386 ++++++++++++++++++++++---
>  lib/netdev-provider.h                  |   9 +
>  lib/netdev.c                           |  78 ++++-
>  lib/userspace-tso.c                    |  48 +++
>  lib/userspace-tso.h                    |  23 ++
>  vswitchd/bridge.c                      |   2 +
>  vswitchd/vswitch.xml                   |  17 ++
>  17 files changed, 1154 insertions(+), 125 deletions(-)
>  create mode 100644 Documentation/topics/userspace-tso.rst
>  create mode 100644 lib/userspace-tso.c
>  create mode 100644 lib/userspace-tso.h
> 
> --
> 2.24.1