[ovs-dev] [PATCH v1 00/23] dpif-netdev: Parallel offload processing
david.marchand at redhat.com
Mon Mar 15 09:38:51 UTC 2021
On Wed, Feb 10, 2021 at 4:07 PM Gaetan Rivet <grive at u256.net> wrote:
> This patch series aims to improve the performance of the management
> of hw-offloads in dpif-netdev. In the current version, some setup
> will experience high memory usage and poor latency between a flow
> decision and its execution regarding hardware offloading.
> This series starts by measuring key metrics regarding both issues
> Those patches are introduced first to compare the current status
> with each improvements introduced.
> Offloads enqueued and inserted, as well as the latency
> from queue insertion to hardware insertion is measured. A new
> command 'ovs-appctl dpctl/offload-stats-show' is introduced
> to show the current measure.
> In my current performance test setup I am measuring an
> average latency hovering between 1~2 seconds.
> After the optimizations, it is reduced to 500~900 ms.
> Finally when using multiple threads and with proper driver
> support, it is measured in the order of 1 ms.
> A few modules are introduced:
> * An ID pool with reduced capabilities, simplifying its
> operations and allowing better performances in both
> single and multi-thread setup.
> * A lockless queue between PMDs / revalidators and
> offload thread(s). As the number of PMDs increases,
> contention can be high on the shared queue.
> This queue is designed to serve as message queue
> between threads.
> * A bounded lockless MPMC ring and some helpers for
> calculating moving averages.
> * A moving average module for Cumulative and Exponential
> moving averages.
> The netdev-offload-dpdk module is made thread-safe.
> Internal maps are made per-netdev instead, and locks are
> taken for shorter critical sections within the module.
This looks like an interesting series, I'll be looking too at it in
the coming weeks.
Just a first and easy update, I noticed an assert is triggered when
stopping OVS (at least).
assertion thread_is_hw_offload || thread_is_rcu failed in
I did not check yet where the issue is, but I noticed it too half way
of the series, and it was not on stop in this case.
You can probably catch it easily.
I have hw-offloads enabled, 2 pf ports, 2 representors ports in a
single bridge, 2 pmds, no additional configuration.
I usually have bi directional traffic running through OVS while I restart.
More information about the dev