[ovs-dev] [PATCH v11 0/7] OVS-DPDK flow offload with rte_flow

Ian Stokes ian.stokes at intel.com
Thu Jun 28 09:13:54 UTC 2018


On 6/25/2018 2:21 PM, Shahaf Shuler wrote:
> Hi,
> 
> Here is a joint work from Mellanox and Napatech, to enable the flow hw
> offload with the DPDK generic flow interface (rte_flow).
> 
> The basic idea is to associate the flow with a mark id (a unit32_t number).
> Later, we then get the flow directly from the mark id, which could bypass
> some heavy CPU operations, including but not limiting to mini flow extract,
> emc lookup, dpcls lookup, etc.
> 
> The association is done with CMAP in patch 1. The CPU workload bypassing
> is done in patch 2. The flow offload is done in patch 3, which mainly does
> two things:
> 
> - translate the ovs match to DPDK rte flow patterns
> - bind those patterns with a RSS + MARK action.
> 
> Patch 5 makes the offload work happen in another thread, for leaving the
> datapath as light as possible.
> 
> A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1
> million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than 260%
> performance boost.
> 
> Note that it's disabled by default, which can be enabled by:
> 
>      $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> 
> For Mellanox NICs to be tested with the latest stable 17.11.3.
> 

Thanks for the latest revision Shahaf, Just a quick query, I've proposed 
moving from DPDK 17.11.2 to 17.11.3 in OVS. There are a number of 
patches related to Mellanox cards in that release from what I can see, 
have you validated this series with 17.11.3, I guess I'm just wondering 
are there any patches in 17.11.3 that are required?

Ian
> v11:
>      - Moved flow mark finding to happen after the next packet prefetch
>        and the metadata initialization. This to support with CT action
>        and to improve the performance.
>      - Don't search for flow mark in re-circulation. This is to address
>        the VXLAN issue reported.
>      - Set packet offsets as part of the tcp parsing. This is to address
>        the VXLAN issue reported.
>      - Fixed travis compilation issue.
>      - Fixed VLAN flow item to discard the VLAN_CFI bit.
>      - Fixed brackets location.
>      - rebased on top of master.
> 
> v10:
>      - rebased on top of master.
> 
> v9: - introduced IP packet sanity checks in a seperate commit
>      - moved structs and enums definition to the begining of the file
>      - fixed sparse compilation error (error is assumed to be fixed, issues
>        on Travis CI preventing from fully testing it.
> 
> v8: - enhanced documentation with more details on supported protocols
>      - fixed VLOG to start with capital letter
>      - fixed compilation issues
>      - fixed coding style
>      - addressed the rest of Ian's comments
> 
> v7: - fixed wrong hash for mark_to_flow that has been refactored in v6
>      - set the rss_conf for rss action to NULL, to workaround a mlx5 change
>        in DPDK v17.11. Note that it will obey the rss settings OVS-DPDK has
>        set in the beginning. Thus, nothing should be effected.
> 
> v6: - fixed a sparse warning
>      - added documentation
>      - used hash_int to compute mark to flow hash
>      - added more comments
>      - added lock for pot lookup
>      - rebased on top of the latest code
> 
> v5: - fixed an issue that it took too long if we do flow add/remove
>        repeatedly.
>      - removed an unused mutex lock
>      - turned most of the log level to DBG
>      - rebased on top of the latest code
> 
> v4: - use RSS action instead of QUEUE action with MARK
>      - make it work with multiple queue (see patch 1)
>      - rebased on top of latest code
> 
> v3: - The mark and id association is done with array instead of CMAP.
>      - Added a thread to do hw offload operations
>      - Removed macros completely
>      - dropped the patch to set FDIR_CONF, which is a workround some
>        Intel NICs.
>      - Added a debug patch to show all flow patterns we have created.
>      - Misc fixes
> 
> v2: - workaround the queue action issue
>      - fixed the tcp_flags being skipped issue, which also fixed the
>        build warnings
>      - fixed l2 patterns for Intel nic
>      - Converted some macros to functions
>      - did not hardcode the max number of flow/action
>      - rebased on top of the latest code
> 
> Thanks.
> 
> ---
> 
> Finn Christensen (1):
>    netdev-dpdk: implement flow offload with rte flow
> 
> Yuanhan Liu (6):
>    dpif-netdev: associate flow with a mark id
>    flow: Introduce IP packet sanity checks
>    dpif-netdev: retrieve flow directly from the flow mark
>    netdev-dpdk: add debug for rte flow patterns
>    dpif-netdev: do hw flow offload in a thread
>    Documentation: document ovs-dpdk flow offload
> 
>   Documentation/howto/dpdk.rst |  22 ++
>   NEWS                         |   2 +-
>   lib/dp-packet.h              |  13 +
>   lib/dpif-netdev.c            | 499 +++++++++++++++++++++++++-
>   lib/flow.c                   | 168 +++++++--
>   lib/flow.h                   |   1 +
>   lib/netdev-dpdk.c            | 734 +++++++++++++++++++++++++++++++++++++-
>   lib/netdev.h                 |   6 +
>   8 files changed, 1405 insertions(+), 40 deletions(-)
> 



More information about the dev mailing list