[ovs-dev] [PATCH v1 00/23] dpif-netdev: Parallel offload processing

David Marchand david.marchand at redhat.com
Mon Mar 15 09:38:51 UTC 2021


On Wed, Feb 10, 2021 at 4:07 PM Gaetan Rivet <grive at u256.net> wrote:
>
> This patch series aims to improve the performance of the management
> of hw-offloads in dpif-netdev. In the current version, some setup
> will experience high memory usage and poor latency between a flow
> decision and its execution regarding hardware offloading.
>
> This series starts by measuring key metrics regarding both issues
> Those patches are introduced first to compare the current status
> with each improvements introduced.
> Offloads enqueued and inserted, as well as the latency
> from queue insertion to hardware insertion is measured. A new
> command 'ovs-appctl dpctl/offload-stats-show' is introduced
> to show the current measure.
>
> In my current performance test setup I am measuring an
> average latency hovering between 1~2 seconds.
> After the optimizations, it is reduced to 500~900 ms.
> Finally when using multiple threads and with proper driver
> support[1], it is measured in the order of 1 ms.
>
> A few modules are introduced:
>
>   * An ID pool with reduced capabilities, simplifying its
>     operations and allowing better performances in both
>     single and multi-thread setup.
>
>   * A lockless queue between PMDs / revalidators and
>     offload thread(s). As the number of PMDs increases,
>     contention can be high on the shared queue.
>     This queue is designed to serve as message queue
>     between threads.
>
>   * A bounded lockless MPMC ring and some helpers for
>     calculating moving averages.
>
>   * A moving average module for Cumulative and Exponential
>     moving averages.
>
> The netdev-offload-dpdk module is made thread-safe.
> Internal maps are made per-netdev instead, and locks are
> taken for shorter critical sections within the module.

Hey Gaetan,

This looks like an interesting series, I'll be looking too at it in
the coming weeks.
Just a first and easy update, I noticed an assert is triggered when
stopping OVS (at least).
2021-03-11T20:32:01.928Z|00350|util|EMER|lib/netdev-offload.c:479:
assertion thread_is_hw_offload || thread_is_rcu failed in
netdev_offload_thread_init()

I did not check yet where the issue is, but I noticed it too half way
of the series, and it was not on stop in this case.
You can probably catch it easily.

I have hw-offloads enabled, 2 pf ports, 2 representors ports in a
single bridge, 2 pmds, no additional configuration.
I usually have bi directional traffic running through OVS while I restart.


-- 
David Marchand



More information about the dev mailing list