[ovs-dev] 答复: [PATCH v4 0/3] Add support for TSO with DPDK

=?gb2312?B?WWkgWWFuZyAo0e6gRCkt1Ma3/s7xvK/NxQ==?= yangyi01 at inspur.com
Thu Feb 20 10:10:36 UTC 2020


Hi, Flavio

I find this tso feature doesn't work normally on my Ubuntu 16.04, here is my
result. My kernel version is 

$ uname -a
Linux cmp008 4.15.0-55-generic #60~16.04.2-Ubuntu SMP Thu Jul 4 09:03:09 UTC
2019 x86_64 x86_64 x86_64 GNU/Linux
$

$ ./run-iperf3.sh
Connecting to host 10.15.1.3, port 5201
[  4] local 10.15.1.2 port 56466 connected to 10.15.1.3 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-10.00  sec  7.05 MBytes  5.91 Mbits/sec  2212   5.66 KBytes
[  4]  10.00-20.00  sec  7.67 MBytes  6.44 Mbits/sec  2484   5.66 KBytes
[  4]  20.00-30.00  sec  7.77 MBytes  6.52 Mbits/sec  2500   5.66 KBytes
[  4]  30.00-40.00  sec  7.77 MBytes  6.52 Mbits/sec  2490   5.66 KBytes
[  4]  40.00-50.00  sec  7.76 MBytes  6.51 Mbits/sec  2500   5.66 KBytes
[  4]  50.00-60.00  sec  7.79 MBytes  6.54 Mbits/sec  2504   5.66 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec  45.8 MBytes  6.40 Mbits/sec  14690
sender
[  4]   0.00-60.00  sec  45.7 MBytes  6.40 Mbits/sec
receiver

Server output:
Accepted connection from 10.15.1.2, port 56464
[  5] local 10.15.1.3 port 5201 connected to 10.15.1.2 port 56466
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.00  sec  6.90 MBytes  5.79 Mbits/sec
[  5]  10.00-20.00  sec  7.71 MBytes  6.47 Mbits/sec
[  5]  20.00-30.00  sec  7.73 MBytes  6.48 Mbits/sec
[  5]  30.00-40.00  sec  7.79 MBytes  6.53 Mbits/sec
[  5]  40.00-50.00  sec  7.79 MBytes  6.53 Mbits/sec
[  5]  50.00-60.00  sec  7.79 MBytes  6.54 Mbits/sec


iperf Done.
$

But it does work for tap, I'm not sure if it is a kernel issue, which kernel
version are you using? I didn't use tpacket_v3 patch. Here is my local ovs
info.

$ git log
commit 1223cf123ed141c0a0110ebed17572bdb2e3d0f4
Author: Ilya Maximets <i.maximets at ovn.org>
Date:   Thu Feb 6 14:24:23 2020 +0100

    netdev-dpdk: Don't enable offloading on HW device if not requested.

    DPDK drivers has different implementations of transmit functions.
    Enabled offloading may cause driver to choose slower variant
    significantly affecting performance if userspace TSO wasn't requested.

    Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
    Reported-by: David Marchand <david.marchand at redhat.com>
    Acked-by: David Marchand <david.marchand at redhat.com>
    Acked-by: Flavio Leitner <fbl at sysclose.org>
    Acked-by: Kevin Traynor <ktraynor at redhat.com>
    Signed-off-by: Ilya Maximets <i.maximets at ovn.org>

commit 73858f9dbe83daf8cc8d4b604acc23eb62cc3f52
Author: Flavio Leitner <fbl at sysclose.org>
Date:   Mon Feb 3 18:45:50 2020 -0300

    netdev-linux: Prepend the std packet in the TSO packet

    Usually TSO packets are close to 50k, 60k bytes long, so to
    to copy less bytes when receiving a packet from the kernel
    change the approach. Instead of extending the MTU sized
    packet received and append with remaining TSO data from
    the TSO buffer, allocate a TSO packet with enough headroom
    to prepend the std packet data.

    Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
    Suggested-by: Ben Pfaff <blp at ovn.org>
    Signed-off-by: Flavio Leitner <fbl at sysclose.org>
    Signed-off-by: Ben Pfaff <blp at ovn.org>

commit 2297cbe6cc25b6b1862c499ce8f16f52f75d9e5f
Author: Flavio Leitner <fbl at sysclose.org>
Date:   Mon Feb 3 11:22:22 2020 -0300

    netdev-linux-private: fix max length to be 16 bits

    The dp_packet length is limited to 16 bits, so document that
    and fix the length value accordingly.

    Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
    Signed-off-by: Flavio Leitner <fbl at sysclose.org>
    Signed-off-by: Ben Pfaff <blp at ovn.org>

commit 3d6a6f450af5b7eaf4b532983cb14458ae792b72
Author: David Marchand <david.marchand at redhat.com>
Date:   Tue Feb 4 22:28:26 2020 +0100

    netdev-dpdk: Fix port init when lacking Tx offloads for TSO.

    The check on TSO capability did not ensure ip checksum, tcp checksum and
    TSO tx offloads were available which resulted in a port init failure
    (example below with a ena device):

    *2020-02-04T17:42:52.976Z|00084|dpdk|ERR|Ethdev port_id=0 requested Tx
    offloads 0x2a doesn't match Tx offloads capabilities 0xe in
    rte_eth_dev_configure()*

    Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")

    Reported-by: Ravi Kerur <rkerur at gmail.com>
    Signed-off-by: David Marchand <david.marchand at redhat.com>
    Acked-by: Kevin Traynor <ktraynor at redhat.com>
    Acked-by: Flavio Leitner <fbl at sysclose.org>
    Signed-off-by: Ilya Maximets <i.maximets at ovn.org>

commit 8e371aa497aa95e3562d53f566c2d634b4b0f589
Author: Kirill A. Kornilov <kirill at tp>
Date:   Mon Jan 13 12:29:10 2020 +0300

    vswitchd: Add serial number configuration.

    Signed-off-by: Kirill A. Kornilov <kornilov at zelax.ru>
    Signed-off-by: Ben Pfaff <blp at ovn.org>

I applied your tap patch.

$ git diff
diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c
index c6f3d27..74a5728 100644
--- a/lib/netdev-linux.c
+++ b/lib/netdev-linux.c
@@ -1010,6 +1010,23 @@ netdev_linux_construct_tap(struct netdev *netdev_)
         goto error_close;
     }

+    if (userspace_tso_enabled()) {
+        /* Old kernels don't support TUNSETOFFLOAD. If TUNSETOFFLOAD is
+         * available, it will return EINVAL when a flag is unknown.
+         * Therefore, try enabling offload with no flags to check
+         * if TUNSETOFFLOAD support is available or not. */
+        if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, 0) == 0 || errno !=
EINVAL) {
+            unsigned long oflags = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6;
+
+            if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, oflags) == -1) {
+                VLOG_WARN("%s: enabling tap offloading failed: %s", name,
+                          ovs_strerror(errno));
+                error = errno;
+                goto error_close;
+            }
+        }
+    }
+
     netdev->present = true;
     return 0;

$

Here is performance result for tap.

$ ./run-iperf3.sh
Connecting to host 10.15.1.3, port 5201
[  4] local 10.15.1.2 port 56480 connected to 10.15.1.3 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-10.00  sec  19.4 GBytes  16.7 Gbits/sec    0   3.05 MBytes
[  4]  10.00-20.00  sec  18.3 GBytes  15.7 Gbits/sec    0   3.05 MBytes
[  4]  20.00-30.00  sec  17.6 GBytes  15.1 Gbits/sec    0   3.05 MBytes
[  4]  30.00-40.00  sec  19.3 GBytes  16.6 Gbits/sec    0   3.05 MBytes
[  4]  40.00-50.00  sec  18.8 GBytes  16.1 Gbits/sec    0   3.05 MBytes
[  4]  50.00-60.00  sec  17.9 GBytes  15.4 Gbits/sec    0   3.05 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec   111 GBytes  15.9 Gbits/sec    0             sender
[  4]   0.00-60.00  sec   111 GBytes  15.9 Gbits/sec
receiver

Server output:
Accepted connection from 10.15.1.2, port 56478
[  5] local 10.15.1.3 port 5201 connected to 10.15.1.2 port 56480
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.00  sec  19.3 GBytes  16.6 Gbits/sec
[  5]  10.00-20.00  sec  18.3 GBytes  15.7 Gbits/sec
[  5]  20.00-30.00  sec  17.6 GBytes  15.1 Gbits/sec
[  5]  30.00-40.00  sec  19.3 GBytes  16.6 Gbits/sec
[  5]  40.00-50.00  sec  18.8 GBytes  16.1 Gbits/sec
[  5]  50.00-60.00  sec  17.9 GBytes  15.4 Gbits/sec


iperf Done.
$


More information about the dev mailing list