[ovs-discuss] Scaling up ovs-dpdk

[Traynor, Kevin]

> -----Original Message-----
> From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of Gabe
> Black
> Sent: Thursday, September 10, 2015 4:08 PM
> To: discuss at openvswitch.org
> Subject: [ovs-discuss] Scaling up ovs-dpdk
> 
> So I have been able to profile performance with ovs-dpdk on multi-node and
> single node servers.  However, when I scale up the number of VMs on the
> compute nodes, I see a corresponding drop in individual VM performance.
> 
> I launched ovs-dpdk with a single GB of memory on the first numa node
> (instructions seem to indicate memory should only be allocated to the first
> numa node... not sure why).  When I tried increasing the number of RX queues
> (ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=2), performance got
> significantly worse.  I was wondering what is the proper way to scale up the
> performance of ovs-dpdk when scaling up the number of VMs that each are using
> significant network bandwidth?

Probably the best way to scale is to add more cores with pmd-cpu-mask.
You won't get much by adding multiple rxqs unless you do this, as the same
pmd (core) will be polling all the rxqs. 

I submitted a patch with additional information about performance tuning,
hopefully that will help explain.
http://openvswitch.org/pipermail/dev/2015-September/059806.html

> 
> On a related note, it seems with the default config of ovs-dpdk that it was
> capped at around 6 Gb/s (with 1500 byte packets).  It seemed to be an
> artificial limit as looking at the dpdk stats of where it was spending time,
> the majority of the time was in the idle/poll loop rather than the processing
> packets loop.  I thought perhaps that it was the VMs, so I had two instances
> splitting the traffic, and again the aggregate b/w seemed to be capped at 6
> Gb/s with each pair of VMs doing 3 Gb/s.  Is this an artificial limit?  Or is
> it related to performance tuning I might have to do?

No artificial limits in OVS with DPDK. It may just be that you need to do some
performance tuning to increase the throughput.

> 
> Thanks for any recommendations!
> Gabriel Black
> 
> 
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss