[ovs-dev] 答复: [PATCH v4 0/3] Add support for TSO with DPDK
Flavio Leitner
fbl at sysclose.org
Thu Feb 20 13:41:09 UTC 2020
On Thu, Feb 20, 2020 at 10:10:36AM +0000, Yi Yang (杨�D)-云服务集团 wrote:
> Hi, Flavio
>
> I find this tso feature doesn't work normally on my Ubuntu 16.04, here is my
> result. My kernel version is
>
> $ uname -a
> Linux cmp008 4.15.0-55-generic #60~16.04.2-Ubuntu SMP Thu Jul 4 09:03:09 UTC
> 2019 x86_64 x86_64 x86_64 GNU/Linux
> $
I tested with 4.15.0 upstream and it worked. Can you do the same?
> $ ./run-iperf3.sh
> Connecting to host 10.15.1.3, port 5201
> [ 4] local 10.15.1.2 port 56466 connected to 10.15.1.3 port 5201
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 4] 0.00-10.00 sec 7.05 MBytes 5.91 Mbits/sec 2212 5.66 KBytes
> [ 4] 10.00-20.00 sec 7.67 MBytes 6.44 Mbits/sec 2484 5.66 KBytes
> [ 4] 20.00-30.00 sec 7.77 MBytes 6.52 Mbits/sec 2500 5.66 KBytes
> [ 4] 30.00-40.00 sec 7.77 MBytes 6.52 Mbits/sec 2490 5.66 KBytes
> [ 4] 40.00-50.00 sec 7.76 MBytes 6.51 Mbits/sec 2500 5.66 KBytes
> [ 4] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec 2504 5.66 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-60.00 sec 45.8 MBytes 6.40 Mbits/sec 14690
> sender
> [ 4] 0.00-60.00 sec 45.7 MBytes 6.40 Mbits/sec
> receiver
That looks like TSO packets are being dropped and the traffic is
basically TCP retransmissions of MTU size.
fbl
>
> Server output:
> Accepted connection from 10.15.1.2, port 56464
> [ 5] local 10.15.1.3 port 5201 connected to 10.15.1.2 port 56466
> [ ID] Interval Transfer Bandwidth
> [ 5] 0.00-10.00 sec 6.90 MBytes 5.79 Mbits/sec
> [ 5] 10.00-20.00 sec 7.71 MBytes 6.47 Mbits/sec
> [ 5] 20.00-30.00 sec 7.73 MBytes 6.48 Mbits/sec
> [ 5] 30.00-40.00 sec 7.79 MBytes 6.53 Mbits/sec
> [ 5] 40.00-50.00 sec 7.79 MBytes 6.53 Mbits/sec
> [ 5] 50.00-60.00 sec 7.79 MBytes 6.54 Mbits/sec
>
>
> iperf Done.
> $
>
> But it does work for tap, I'm not sure if it is a kernel issue, which kernel
> version are you using? I didn't use tpacket_v3 patch. Here is my local ovs
> info.
>
> $ git log
> commit 1223cf123ed141c0a0110ebed17572bdb2e3d0f4
> Author: Ilya Maximets <i.maximets at ovn.org>
> Date: Thu Feb 6 14:24:23 2020 +0100
>
> netdev-dpdk: Don't enable offloading on HW device if not requested.
>
> DPDK drivers has different implementations of transmit functions.
> Enabled offloading may cause driver to choose slower variant
> significantly affecting performance if userspace TSO wasn't requested.
>
> Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
> Reported-by: David Marchand <david.marchand at redhat.com>
> Acked-by: David Marchand <david.marchand at redhat.com>
> Acked-by: Flavio Leitner <fbl at sysclose.org>
> Acked-by: Kevin Traynor <ktraynor at redhat.com>
> Signed-off-by: Ilya Maximets <i.maximets at ovn.org>
>
> commit 73858f9dbe83daf8cc8d4b604acc23eb62cc3f52
> Author: Flavio Leitner <fbl at sysclose.org>
> Date: Mon Feb 3 18:45:50 2020 -0300
>
> netdev-linux: Prepend the std packet in the TSO packet
>
> Usually TSO packets are close to 50k, 60k bytes long, so to
> to copy less bytes when receiving a packet from the kernel
> change the approach. Instead of extending the MTU sized
> packet received and append with remaining TSO data from
> the TSO buffer, allocate a TSO packet with enough headroom
> to prepend the std packet data.
>
> Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
> Suggested-by: Ben Pfaff <blp at ovn.org>
> Signed-off-by: Flavio Leitner <fbl at sysclose.org>
> Signed-off-by: Ben Pfaff <blp at ovn.org>
>
> commit 2297cbe6cc25b6b1862c499ce8f16f52f75d9e5f
> Author: Flavio Leitner <fbl at sysclose.org>
> Date: Mon Feb 3 11:22:22 2020 -0300
>
> netdev-linux-private: fix max length to be 16 bits
>
> The dp_packet length is limited to 16 bits, so document that
> and fix the length value accordingly.
>
> Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
> Signed-off-by: Flavio Leitner <fbl at sysclose.org>
> Signed-off-by: Ben Pfaff <blp at ovn.org>
>
> commit 3d6a6f450af5b7eaf4b532983cb14458ae792b72
> Author: David Marchand <david.marchand at redhat.com>
> Date: Tue Feb 4 22:28:26 2020 +0100
>
> netdev-dpdk: Fix port init when lacking Tx offloads for TSO.
>
> The check on TSO capability did not ensure ip checksum, tcp checksum and
> TSO tx offloads were available which resulted in a port init failure
> (example below with a ena device):
>
> *2020-02-04T17:42:52.976Z|00084|dpdk|ERR|Ethdev port_id=0 requested Tx
> offloads 0x2a doesn't match Tx offloads capabilities 0xe in
> rte_eth_dev_configure()*
>
> Fixes: 29cf9c1b3b9c ("userspace: Add TCP Segmentation Offload support")
>
> Reported-by: Ravi Kerur <rkerur at gmail.com>
> Signed-off-by: David Marchand <david.marchand at redhat.com>
> Acked-by: Kevin Traynor <ktraynor at redhat.com>
> Acked-by: Flavio Leitner <fbl at sysclose.org>
> Signed-off-by: Ilya Maximets <i.maximets at ovn.org>
>
> commit 8e371aa497aa95e3562d53f566c2d634b4b0f589
> Author: Kirill A. Kornilov <kirill at tp>
> Date: Mon Jan 13 12:29:10 2020 +0300
>
> vswitchd: Add serial number configuration.
>
> Signed-off-by: Kirill A. Kornilov <kornilov at zelax.ru>
> Signed-off-by: Ben Pfaff <blp at ovn.org>
>
> I applied your tap patch.
>
> $ git diff
> diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c
> index c6f3d27..74a5728 100644
> --- a/lib/netdev-linux.c
> +++ b/lib/netdev-linux.c
> @@ -1010,6 +1010,23 @@ netdev_linux_construct_tap(struct netdev *netdev_)
> goto error_close;
> }
>
> + if (userspace_tso_enabled()) {
> + /* Old kernels don't support TUNSETOFFLOAD. If TUNSETOFFLOAD is
> + * available, it will return EINVAL when a flag is unknown.
> + * Therefore, try enabling offload with no flags to check
> + * if TUNSETOFFLOAD support is available or not. */
> + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, 0) == 0 || errno !=
> EINVAL) {
> + unsigned long oflags = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6;
> +
> + if (ioctl(netdev->tap_fd, TUNSETOFFLOAD, oflags) == -1) {
> + VLOG_WARN("%s: enabling tap offloading failed: %s", name,
> + ovs_strerror(errno));
> + error = errno;
> + goto error_close;
> + }
> + }
> + }
> +
> netdev->present = true;
> return 0;
>
> $
>
> Here is performance result for tap.
>
> $ ./run-iperf3.sh
> Connecting to host 10.15.1.3, port 5201
> [ 4] local 10.15.1.2 port 56480 connected to 10.15.1.3 port 5201
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 4] 0.00-10.00 sec 19.4 GBytes 16.7 Gbits/sec 0 3.05 MBytes
> [ 4] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec 0 3.05 MBytes
> [ 4] 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec 0 3.05 MBytes
> [ 4] 30.00-40.00 sec 19.3 GBytes 16.6 Gbits/sec 0 3.05 MBytes
> [ 4] 40.00-50.00 sec 18.8 GBytes 16.1 Gbits/sec 0 3.05 MBytes
> [ 4] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec 0 3.05 MBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec 0 sender
> [ 4] 0.00-60.00 sec 111 GBytes 15.9 Gbits/sec
> receiver
>
> Server output:
> Accepted connection from 10.15.1.2, port 56478
> [ 5] local 10.15.1.3 port 5201 connected to 10.15.1.2 port 56480
> [ ID] Interval Transfer Bandwidth
> [ 5] 0.00-10.00 sec 19.3 GBytes 16.6 Gbits/sec
> [ 5] 10.00-20.00 sec 18.3 GBytes 15.7 Gbits/sec
> [ 5] 20.00-30.00 sec 17.6 GBytes 15.1 Gbits/sec
> [ 5] 30.00-40.00 sec 19.3 GBytes 16.6 Gbits/sec
> [ 5] 40.00-50.00 sec 18.8 GBytes 16.1 Gbits/sec
> [ 5] 50.00-60.00 sec 17.9 GBytes 15.4 Gbits/sec
>
>
> iperf Done.
> $
--
fbl
More information about the dev
mailing list