[ovs-dev] [PATCH v11 0/7] OVS-DPDK flow offload with rte_flow
Shahaf Shuler
shahafs at mellanox.com
Thu Jun 28 11:22:49 UTC 2018
Hi Ian, Flavio,
Thursday, June 28, 2018 12:14 PM, Ian Stokes:
> Subject: Re: [PATCH v11 0/7] OVS-DPDK flow offload with rte_flow
>
> On 6/25/2018 2:21 PM, Shahaf Shuler wrote:
> > Hi,
> >
> > Here is a joint work from Mellanox and Napatech, to enable the flow hw
> > offload with the DPDK generic flow interface (rte_flow).
> >
> > The basic idea is to associate the flow with a mark id (a unit32_t number).
> > Later, we then get the flow directly from the mark id, which could
> > bypass some heavy CPU operations, including but not limiting to mini
> > flow extract, emc lookup, dpcls lookup, etc.
> >
> > The association is done with CMAP in patch 1. The CPU workload
> > bypassing is done in patch 2. The flow offload is done in patch 3,
> > which mainly does two things:
> >
> > - translate the ovs match to DPDK rte flow patterns
> > - bind those patterns with a RSS + MARK action.
> >
> > Patch 5 makes the offload work happen in another thread, for leaving
> > the datapath as light as possible.
> >
> > A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1
> > million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than
> > 260% performance boost.
> >
> > Note that it's disabled by default, which can be enabled by:
> >
> > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> >
> > For Mellanox NICs to be tested with the latest stable 17.11.3.
> >
>
> Thanks for the latest revision Shahaf, Just a quick query, I've proposed
> moving from DPDK 17.11.2 to 17.11.3 in OVS. There are a number of patches
> related to Mellanox cards in that release from what I can see, have you
> validated this series with 17.11.3, I guess I'm just wondering are there any
> patches in 17.11.3 that are required?
Yes the series was validated and tested against this version.
There are some important fixes for mlx5 PMD to make the OVS-DPDK HWOL work like:
f75e68a net/mlx5: fix flow validation
(this one I manually gave Flavio when reported an issue).
>
> Ian
> > v11:
> > - Moved flow mark finding to happen after the next packet prefetch
> > and the metadata initialization. This to support with CT action
> > and to improve the performance.
Flavio - following your comment on the performance. I am still not able to reproduce it. Please check the above change which supposed to help, and the series w/ 17.11.3
If this still reproduce then I suggest we will have a debug session to locate the root cause.
> > - Don't search for flow mark in re-circulation. This is to address
> > the VXLAN issue reported.
> > - Set packet offsets as part of the tcp parsing. This is to address
> > the VXLAN issue reported.
> > - Fixed travis compilation issue.
> > - Fixed VLAN flow item to discard the VLAN_CFI bit.
> > - Fixed brackets location.
> > - rebased on top of master.
> >
> > v10:
> > - rebased on top of master.
> >
> > v9: - introduced IP packet sanity checks in a seperate commit
> > - moved structs and enums definition to the begining of the file
> > - fixed sparse compilation error (error is assumed to be fixed, issues
> > on Travis CI preventing from fully testing it.
> >
> > v8: - enhanced documentation with more details on supported protocols
> > - fixed VLOG to start with capital letter
> > - fixed compilation issues
> > - fixed coding style
> > - addressed the rest of Ian's comments
> >
> > v7: - fixed wrong hash for mark_to_flow that has been refactored in v6
> > - set the rss_conf for rss action to NULL, to workaround a mlx5 change
> > in DPDK v17.11. Note that it will obey the rss settings OVS-DPDK has
> > set in the beginning. Thus, nothing should be effected.
> >
> > v6: - fixed a sparse warning
> > - added documentation
> > - used hash_int to compute mark to flow hash
> > - added more comments
> > - added lock for pot lookup
> > - rebased on top of the latest code
> >
> > v5: - fixed an issue that it took too long if we do flow add/remove
> > repeatedly.
> > - removed an unused mutex lock
> > - turned most of the log level to DBG
> > - rebased on top of the latest code
> >
> > v4: - use RSS action instead of QUEUE action with MARK
> > - make it work with multiple queue (see patch 1)
> > - rebased on top of latest code
> >
> > v3: - The mark and id association is done with array instead of CMAP.
> > - Added a thread to do hw offload operations
> > - Removed macros completely
> > - dropped the patch to set FDIR_CONF, which is a workround some
> > Intel NICs.
> > - Added a debug patch to show all flow patterns we have created.
> > - Misc fixes
> >
> > v2: - workaround the queue action issue
> > - fixed the tcp_flags being skipped issue, which also fixed the
> > build warnings
> > - fixed l2 patterns for Intel nic
> > - Converted some macros to functions
> > - did not hardcode the max number of flow/action
> > - rebased on top of the latest code
> >
> > Thanks.
> >
> > ---
> >
> > Finn Christensen (1):
> > netdev-dpdk: implement flow offload with rte flow
> >
> > Yuanhan Liu (6):
> > dpif-netdev: associate flow with a mark id
> > flow: Introduce IP packet sanity checks
> > dpif-netdev: retrieve flow directly from the flow mark
> > netdev-dpdk: add debug for rte flow patterns
> > dpif-netdev: do hw flow offload in a thread
> > Documentation: document ovs-dpdk flow offload
> >
> > Documentation/howto/dpdk.rst | 22 ++
> > NEWS | 2 +-
> > lib/dp-packet.h | 13 +
> > lib/dpif-netdev.c | 499 +++++++++++++++++++++++++-
> > lib/flow.c | 168 +++++++--
> > lib/flow.h | 1 +
> > lib/netdev-dpdk.c | 734
> +++++++++++++++++++++++++++++++++++++-
> > lib/netdev.h | 6 +
> > 8 files changed, 1405 insertions(+), 40 deletions(-)
> >
More information about the dev
mailing list