[ovs-dev] [PATCH RFC] netdev-dpdk: Expand RSS hash calculation by MAC addresses

Wojciechowicz, RobertX robertx.wojciechowicz at intel.com
Wed Apr 13 12:42:13 UTC 2016


Hi,

> -----Original Message-----
> From: Jesse Gross [mailto:jesse at kernel.org]
> Sent: Friday, March 25, 2016 7:57 PM
> To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
> Cc: ovs dev <dev at openvswitch.org>
> Subject: Re: [ovs-dev] [PATCH RFC] netdev-dpdk: Expand RSS hash
> calculation by MAC addresses
> 
> On Fri, Mar 25, 2016 at 7:45 AM, Wojciechowicz, RobertX
> <robertx.wojciechowicz at intel.com> wrote:
> > Hi Jesse,
> >
> > please find my answers inline.
> >
> >> -----Original Message-----
> >> From: Jesse Gross [mailto:jesse at kernel.org]
> >> Sent: Thursday, March 24, 2016 11:20 PM
> >> To: Wojciechowicz, RobertX <robertx.wojciechowicz at intel.com>
> >> Cc: ovs dev <dev at openvswitch.org>
> >> Subject: Re: [ovs-dev] [PATCH RFC] netdev-dpdk: Expand RSS hash
> >> calculation by MAC addresses
> >>
> >> On Thu, Mar 24, 2016 at 7:30 AM, Robert Wojciechowicz
> >> <robertx.wojciechowicz at intel.com> wrote:
> >> > This patch configures RSS hash filter in DPDK in order to include
> >> > source and destination MAC addresses into RSS hash calculation.
> >> > This configuration considerably improves performance for flows
> >> > with overlapping IP address spaces and non-overlapping Ethernet
> >> > address spaces.
> >> > Here is the example test result (throughput in frames per second)
> >> > I have done for 16 flows (fixed IP addresses and changing source
> >> > MAC addresses) on X710 for 10GbE SFP+:
> >> >
> >> > 1. original hash calculation:                    4870667 fps
> >> > 2. expanded hash calculation (with this patch): 12740258 fps
> >> >
> >> > Signed-off-by: Robert Wojciechowicz
> <robertx.wojciechowicz at intel.com>
> >>
> >> This is somewhat of an unusual configuration - I guess as evidenced by
> >> the fact that it is not supported by ixgbe.
> >>
> >> How much of a difference does this make in non-synthetic benchmarks? I
> >> would expect in most cases that the use of L4 ports would provide
> >> sufficient entropy even in cases where there are overlapping IP
> >> addresses.
> >>
> >
> > In most cases you're probably right, but there are use cases where
> > this overlapping exists. For example in openstack configurations
> > with many tenants.
> > The idea of this patch is to make this configuration possible
> > for people who really need it.
> 
> I understand that that there can be configurations where there are
> overlapping IP addresses, however, I'm not sure that really means that
> the L4 ports aren't able to still spread the load. Have you seen real
> world situations where this patch makes a difference in practice, as
> opposed to just in theory?
> 

We contacted our customer regarding this real world test, but unfortunately
we're still waiting for response. However synthetic benchmarks could be useful
also, e.g. for comparison with other vswitches.
I discussed this issue also with openstack team in more detail. 
It seems that overlapping ip and mac address spaces can happen for vlan 
or other tunnels they use a lot in openstack setups. So there is suggestion 
to expand this hash by tunnel id instead of mac addresses. 
I will try to test this scenario also.

> The reason why some of the RSS modes you are enabling here aren't on
> by default is usually because they can introduce reordering. So I'm
> not really eager to go down this path unless you can show that there
> are specific reasons why the assumptions behind the default
> configuration don't apply here.
> 

To be honest I'm not sure how expanding rss hash may introduce reordering.
My understanding is that hash filter is a mechanism to statistically distribute received 
packets into several receive queues. By default there is used 5-tuple (src_addr,
dst_addr, protocol, src_port, dst_port). This patch tries to expand
this tuple by mac or tunnel_id in order to reduce collisions for some traffic flows
and improve packets distribution into queues.
Could you please explain why it may introduce reordering?

> >> Two other concerns:
> >>  * This appears to enable hashing for a wide range of packets,
> >> including IP fragments, which would typically allow the possibility of
> >> reordering. I think we probably want to be a little more careful in
> >> considering what to include.
> >>  * I think we need to more gracefully handle NICs where this
> >> configuration is not supported. I don't like that we should what looks
> >> like an error message in normal conditions.
> >
> > Unfortunately DPDK returns EINVAL error also for unsupported operations,
> > so I couldn't handle it in more reasonable way. I wanted to make it
> > minimally invasive for other hardware than XL710, so I report all errors
> > as info messages.
> > Basically I'm not sure if there is some recommended approach
> > for providing functionality to OVS, which is hardware specific?
> > If there is any I could of course reimplement this patch appropriately.
> 
> I don't think that we have any hardware specific configuration in OVS.
> The best way to handle this is to fix DPDK or expose an API that
> abstracts out the hardware specific pieces.

Ok, so DPDK patch could be the next step in this work.

Br,
Robert


More information about the dev mailing list