[ovs-discuss] OVS DPDK with no DPDK nics on NUMA 0

[O'Reilly, Darragh]

> -----Original Message-----
> From: Stokes, Ian [mailto:ian.stokes at intel.com]
> Sent: 17 January 2017 14:45
> To: O'Reilly, Darragh <darragh.oreilly at hpe.com>; ovs-
> discuss at openvswitch.org
> Subject: RE: OVS DPDK with no DPDK nics on NUMA 0
> 
> > This article https://software.intel.com/en-us/articles/using-open-vswitch-
> > and-dpdk-with-neutron-in-devstack says: "If you have the NICs installed
> > entirely on a NUMA Node other than 0, you will encounter a bug that will
> > prevent correct OVS setup. You may wish to move your NIC device to a
> > different PCIe slot."
> >
> > Anyone know what this bug is, and is it in OVS-DPDK?  "correct OVS setup"
> > suggests that it might just be a bug in the devstack script.
> 
> I think this bug is related to the fact that the ovs PMD coremask value defined in
> the local.conf is 0x4 by default, which is a cpu 3 on socket 0 in this case. If the
> NIC is attached to NUMA node 1 then it will fail to initialize I believe.

Yes. I just tested with OVS 2.5 by moving the NIC to a NUMA 1 slot, and it did not work. Adding NUMA 1 cores to the PMD mask made it work.

> I think this is still an issue today for OVS-DPDK, but there are plans to enable
> cross numa pmd configurations (although one should note that there would be a
> performance penalty for crossing the numa nodes).
> 
> There is a work around mentioned in the guide under the 'Additional OVS/DPDK
> Options of Note', essentially you may change the pmd coremask to be a cpu on
> numa node 1.
> 
> Thanks
> Ian