[ovs-discuss] Open vSwitch performance with UDP traffic

[Ben Pfaff]

On Wed, Jan 22, 2014 at 08:17:05PM +0000, McGarvey, Kevin wrote:
> 
> 
> On 1/22/14 12:44 PM, "Ben Pfaff" <blp at nicira.com> wrote:
> 
> >On Wed, Jan 22, 2014 at 09:39:14AM -0800, Ben Pfaff wrote:
> >> On Wed, Jan 22, 2014 at 05:35:40PM +0000, McGarvey, Kevin wrote:
> >> > 
> >> > 
> >> > On 1/21/14 6:17 PM, "Ben Pfaff" <blp at nicira.com> wrote:
> >> > >I'd expect a dramatic drop in CPU consumption in that case.  There
> >>are
> >> > >a few special cases where the upgrade wouldn't help.  One is if
> >> > >in-band control is in use, another is if NetFlow is turned on, a
> >>third
> >> > >is if LACP bonds with L4 port based hashing are turned on, and there
> >> > >are probably a few others that don't come to mind immediately.
> >> > 
> >> > I plan to rerun the test to rule out some mistake on my part.
> >> > 
> >> > Could you provide more information about the nature of the change
> >>made in
> >> > 1.11 that improves performance for this type of traffic?  Is the
> >>kernel
> >> > module able to forward UDP DNS packets without sending them to
> >>userspace,
> >> > or was it an optimization of the userspace processing?  What roughly
> >>is
> >> > the level of performance I should see?
> >> 
> >> In 1.11 and later, for simple OpenFlow tables (I don't think you
> >> mentioned whether you are using a controller or which one), Open
> >> vSwitch can set up only a single kernel flow that covers many possible
> >> flows, for example all possible UDP destination ports, rather than
> >> setting up an individual kernel flow for each UDP packet.  When that
> >> works, it eliminates most of the kernel/userspace traffic, improving
> >> performance.  Version 2.0 is better at analyzing OpenFlow flow tables
> >> to see when this is possible, so it can better take advantage of the
> >> ability.
> >
> >I see that I didn't answer your question about performance.
> >
> >When this optimization kicks in fully, I guess that the performance
> >should be about the same as for traffic with long flows (like the
> >netperf TCP_STREAM test, for example) in terms of packets per second.
> 
> Thanks.  This is encouraging.  The only question is why isn't the
> optimization kicking in?
> 
> 
> I repeated the test, and under a load of 10K DNS requests/responses per
> second ovs-vswitchd is using 82% of a core.
> 
> I wasn't sure whether in-band control was on or off by default, so I
> disabled it with the command below and restarted openvswitch, but the cpu
> consumption didn't change:
> 
> ovs-vsctl set bridge <bridge> other-config:disable-in-band=true
> 
> I did not set up the configuration, but as far as I can tell Netflow is
> not turned on.  The output of 'ovsdb-tool -show-log | grep -i netflow' is
> empty.
> 
> There are no bonded interfaces.  The 2 NICs used for DNS traffic are
> associated with separate bridges.
> 
> We are not using a controller.
> 
> In your response you mentioned that for simple OpenFlow tables Open
> vSwitch can set up a single kernel flow that covers many possible flows.
> I think this is exactly what I need.  Do I need to add a flow using
> ovs-ofctl?  

No.  With the settings you describe, it should kick in automatically.

Here is an experiment that might help.  Take one of the flows that
"ovs-dpctl dump-flows" prints, then feed that flow back into
"ovs-appctl ofproto/trace", and show us the results.  (You might have
to spend a few minutes reading the ovs-vswitchd manpage to learn the
ofproto/trace syntax, if you don't already know it.)