[ovs-dev] Considering the possibility of integrating DPDK generic classifier APIs in OVS.

[Gray, Mark D]

> Hi Dev Team,
> 
> I submitted a RFC patch few months ago to optimize the tunneling
> performance in OVS-DPDK (specifically VxLAN performance) using Intel
> Fortville Flow director feature. More details about the patch can be found
> below.
> 
> http://openvswitch.org/pipermail/dev/2016-March/067988.html
> 
> The proposal is not accepted by the community, though the patch gives very
> significant tunneling performance improvement. One of the major concern
> was the implementation is too hardware specific.
> Now DPDK is working on a generic classifier API sets to expose these
> hardware packet classification/flow in more generic way. More details on this
> proposal can be found in the following link.
> 
> http://dpdk.org/ml/archives/dev/2016-July/043365.html
> 
[Gray, Mark D] 
The key thing for me is that the DPDK community will provide a performant software
solution for all functionality in case the hardware doesn't support it.

> I feel that OVS should definitely leverage these feature because this will be
> big advantage for the customers who are using OVS +smart NICs.
> The early engagement on this proposal can influence the design to co-exists
> with OVS architecture natively. I had already provided few comments based
> on the feedback that I received from OVS community on tunneling
> optimization patch.
>  Please feel free to provide more comments if I have missed out any.
> 
> This time I am thinking of a more generic design to accommodate these APIs
> in OVS. At very high level, this design involves changes in OVS control plane
> as well as the data plane, something like below,

 [Gray, Mark D] I think we should focus on one or two use cases rather than
a general offload like you discuss below. A general offload involves a huge amount of code
churn and there are a lot of difficulties, some that you have highlighted below.
A more focused implementation will flush out any issues with the API. In particular,
the VxLAN use case that you mentioned above and perhaps the offload of the hash
calculation (but the generic filtering api would also need to support generation of hashes)
could be two targets for this DPDK api.

> 
> 1) Control plane modification for ,
> 	*)  All the rules cannot offload to the hardware. So its important to
> identify which one can offload and which cannot. The possible options can
> be,
> 		1) Let the user choose a rule can be hardware/only software
> 		2) Let the control plane decide a hardware rule based on the
> flow parameters.
> 		Once the rule is decided as hardware rule, next is choose
> right type of hardware rule(for eg: ID or Queue or RSS or chained) .
> 		This is going to be little bit complicated because there are
> multiple parameters need to consider before decide a type. Or we could use
> the ID flow type for all the hardware flows. Any suggestions?

[Gray, Mark D] 
This is a general problem with hardware offload of vswitch functionality. You need
to consider the capabilities of the offload target but also the capacity of the hardware.

> 	*) The control plane modification to program the rule only in
> hardware, only in software, or in both (A hardware rule need to handle
> further in software as well).
> 	The hardware flow insertion is something like below,
> 	 	+ Insert the software fallback rule in OVS. (This is the normal
> software rule to handle the specific type packets)
> 		+ Insert the hardware rule, and corresponding post software
> rule to handle hardware processed packets.
> 		+ Remove the software fallback rule from OVS.
> 	Similarly the hardware flow deletion can be
> 		+ Delete the software fallback path if exists any,
> 		+  Remove the post-software rule for the corresponding
> hardware rule.
> 		+ Remove the hardware rule.
> 
> Please note, the hardware rules can be used for long lived rules.

[Gray, Mark D] 
Yes, if you can determine what is long lived. Also, you need to consider statistics,
revalidation, flow deletion, - all the corner cases.

I have always thought for a full offload like what you are discussing, a double-dpif
model would make more sense - i.e. implement a dpif for a particular hardware 
target and allow two dpifs to act as back ends for a bridge with hardware ports associated
with the hardware dpif and software ports with the software dpif. It may also
be useful for ovs-kernel and ovs-dpdk to interact in the same way.
> 
> 2) Dataplane changes are for
> 	*) Miniflow extract change to handle hardware processed rules. I
> feel in most of the cases we could skip miniflow extract on packets hit by a
> hardware rule.
> 	*) changes in emc_insert, dpcls_insert, emc_evict, dpcls_evict to
> handle the hardware rules.
> 3) Changes for manage hardware flow stats
> 
> 
> This is just an initial thought to enable these generic APIs in OVS.  Ofcourse I
> might have missed out many points. The intention of this mail is to invite the
> community attention to the proposal and provide the comments and
> suggestions at the design stage itself.
> 

[Gray, Mark D] I think it is a good discussion. I would like to see them used to
allow us to take advantage of more of these types of capabilities in NICs.
> 
> 
> Regards
> _Sugesh