[ovs-dev] [PATCH v11 0/7] OVS-DPDK flow offload with rte_flow
Ian Stokes
ian.stokes at intel.com
Fri Jun 29 10:23:15 UTC 2018
On 6/28/2018 6:39 PM, Flavio Leitner wrote:
> On Thu, Jun 28, 2018 at 11:22:49AM +0000, Shahaf Shuler wrote:
>> Hi Ian, Flavio,
>
> Hi!
>
>> Thursday, June 28, 2018 12:14 PM, Ian Stokes:
>>> Subject: Re: [PATCH v11 0/7] OVS-DPDK flow offload with rte_flow
>>>
>>> On 6/25/2018 2:21 PM, Shahaf Shuler wrote:
>>>> Hi,
>>>>
>>>> Here is a joint work from Mellanox and Napatech, to enable the flow hw
>>>> offload with the DPDK generic flow interface (rte_flow).
>>>>
>>>> The basic idea is to associate the flow with a mark id (a unit32_t number).
>>>> Later, we then get the flow directly from the mark id, which could
>>>> bypass some heavy CPU operations, including but not limiting to mini
>>>> flow extract, emc lookup, dpcls lookup, etc.
>>>>
>>>> The association is done with CMAP in patch 1. The CPU workload
>>>> bypassing is done in patch 2. The flow offload is done in patch 3,
>>>> which mainly does two things:
>>>>
>>>> - translate the ovs match to DPDK rte flow patterns
>>>> - bind those patterns with a RSS + MARK action.
>>>>
>>>> Patch 5 makes the offload work happen in another thread, for leaving
>>>> the datapath as light as possible.
>>>>
>>>> A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1
>>>> million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than
>>>> 260% performance boost.
>>>>
>>>> Note that it's disabled by default, which can be enabled by:
>>>>
>>>> $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
>>>>
>>>> For Mellanox NICs to be tested with the latest stable 17.11.3.
>>>>
>>>
>>> Thanks for the latest revision Shahaf, Just a quick query, I've proposed
>>> moving from DPDK 17.11.2 to 17.11.3 in OVS. There are a number of patches
>>> related to Mellanox cards in that release from what I can see, have you
>>> validated this series with 17.11.3, I guess I'm just wondering are there any
>>> patches in 17.11.3 that are required?
>>
>> Yes the series was validated and tested against this version.
>> There are some important fixes for mlx5 PMD to make the OVS-DPDK HWOL work like:
>> f75e68a net/mlx5: fix flow validation
>>
>> (this one I manually gave Flavio when reported an issue).
>
> I had a chance to give a quick try with 17.11.3 and master + v11 and
> I could reproduce the same issue. However, looks like the performance
> now is pretty much the same as without HWOF enabled and that's a
> significant improvement. I'd like to double check everything though.
>
Ok, thanks for testing Flavio, It looks like DPDK 17.11.3 is a
pre-requisite for the HWOL case. I have a patch proposed to move to it,
https://mail.openvswitch.org/pipermail/ovs-dev/2018-June/348419.html
It's just waiting on an ack :), if one you of you could assist there I'd
be happy to apply both to the next dpdk_merge.
Thanks
Ian
>
>>> Ian
>>>> v11:
>>>> - Moved flow mark finding to happen after the next packet prefetch
>>>> and the metadata initialization. This to support with CT action
>>>> and to improve the performance.
>>
>> Flavio - following your comment on the performance. I am still not
>> able to reproduce it. Please check the above change which supposed
>> to help, and the series w/ 17.11.3
>> If this still reproduce then I suggest we will have a debug session
>> to locate the root cause.
>
> Ian, I am afraid that the issue might be local and we most probably can
> fix with a follow up patch later, so if others are happy with the patchset,
> please go ahead. I can see this only in the mid of next week.
>
> Thanks,
> fbl
>
>
>>
>>>> - Don't search for flow mark in re-circulation. This is to address
>>>> the VXLAN issue reported.
>>>> - Set packet offsets as part of the tcp parsing. This is to address
>>>> the VXLAN issue reported.
>>>> - Fixed travis compilation issue.
>>>> - Fixed VLAN flow item to discard the VLAN_CFI bit.
>>>> - Fixed brackets location.
>>>> - rebased on top of master.
>>>>
>>>> v10:
>>>> - rebased on top of master.
>>>>
>>>> v9: - introduced IP packet sanity checks in a seperate commit
>>>> - moved structs and enums definition to the begining of the file
>>>> - fixed sparse compilation error (error is assumed to be fixed, issues
>>>> on Travis CI preventing from fully testing it.
>>>>
>>>> v8: - enhanced documentation with more details on supported protocols
>>>> - fixed VLOG to start with capital letter
>>>> - fixed compilation issues
>>>> - fixed coding style
>>>> - addressed the rest of Ian's comments
>>>>
>>>> v7: - fixed wrong hash for mark_to_flow that has been refactored in v6
>>>> - set the rss_conf for rss action to NULL, to workaround a mlx5 change
>>>> in DPDK v17.11. Note that it will obey the rss settings OVS-DPDK has
>>>> set in the beginning. Thus, nothing should be effected.
>>>>
>>>> v6: - fixed a sparse warning
>>>> - added documentation
>>>> - used hash_int to compute mark to flow hash
>>>> - added more comments
>>>> - added lock for pot lookup
>>>> - rebased on top of the latest code
>>>>
>>>> v5: - fixed an issue that it took too long if we do flow add/remove
>>>> repeatedly.
>>>> - removed an unused mutex lock
>>>> - turned most of the log level to DBG
>>>> - rebased on top of the latest code
>>>>
>>>> v4: - use RSS action instead of QUEUE action with MARK
>>>> - make it work with multiple queue (see patch 1)
>>>> - rebased on top of latest code
>>>>
>>>> v3: - The mark and id association is done with array instead of CMAP.
>>>> - Added a thread to do hw offload operations
>>>> - Removed macros completely
>>>> - dropped the patch to set FDIR_CONF, which is a workround some
>>>> Intel NICs.
>>>> - Added a debug patch to show all flow patterns we have created.
>>>> - Misc fixes
>>>>
>>>> v2: - workaround the queue action issue
>>>> - fixed the tcp_flags being skipped issue, which also fixed the
>>>> build warnings
>>>> - fixed l2 patterns for Intel nic
>>>> - Converted some macros to functions
>>>> - did not hardcode the max number of flow/action
>>>> - rebased on top of the latest code
>>>>
>>>> Thanks.
>>>>
>>>> ---
>>>>
>>>> Finn Christensen (1):
>>>> netdev-dpdk: implement flow offload with rte flow
>>>>
>>>> Yuanhan Liu (6):
>>>> dpif-netdev: associate flow with a mark id
>>>> flow: Introduce IP packet sanity checks
>>>> dpif-netdev: retrieve flow directly from the flow mark
>>>> netdev-dpdk: add debug for rte flow patterns
>>>> dpif-netdev: do hw flow offload in a thread
>>>> Documentation: document ovs-dpdk flow offload
>>>>
>>>> Documentation/howto/dpdk.rst | 22 ++
>>>> NEWS | 2 +-
>>>> lib/dp-packet.h | 13 +
>>>> lib/dpif-netdev.c | 499 +++++++++++++++++++++++++-
>>>> lib/flow.c | 168 +++++++--
>>>> lib/flow.h | 1 +
>>>> lib/netdev-dpdk.c | 734
>>> +++++++++++++++++++++++++++++++++++++-
>>>> lib/netdev.h | 6 +
>>>> 8 files changed, 1405 insertions(+), 40 deletions(-)
>>>>
>>
>
More information about the dev
mailing list