[ovs-dev] dpif-netdev: Assign ports to pmds on non-local numa node.

Wed Mar 29 17:34:36 UTC 2017

I see there is warning log added for the non-local case.
I wonder if during setup, there will be no issues with basic connectivity seen.

Then some long time later at high PPS usage, unexpected low performance will be seen.
I am not sure whether the log will be immediately consulted to check for this, as the reasons
for low PPS performance would likely be elsewhere; this is pretty subtle.

I think there needs to be documentation updates associated with this change explaining this.
Ideally the warning would be referenced in the documentation as clear cross reference.

Thanks
Darrell

On 3/9/17, 7:16 AM, "ovs-dev-bounces at openvswitch.org on behalf of O Mahony, Billy" <ovs-dev-bounces at openvswitch.org on behalf of billy.o.mahony at intel.com> wrote:

    Hi Jan,

    Thanks. That is another valid point if you don't - currently if you have dual-socket (numa) system then you have to spend at least 2 cores on your ovs-dpdk vswitch. With this patch it will allow such systems to operate on just one core (with reduced switching capacity of course).

    I'd be interested to hear any other comments from the community on this patch.

    Regards,

    /Billy

    > -----Original Message-----

    > From: Jan Scheurich [mailto:jan.scheurich at ericsson.com]

    > Sent: Monday, March 6, 2017 8:25 AM

    > To: O Mahony, Billy <billy.o.mahony at intel.com>; Ilya Maximets

    > <i.maximets at samsung.com>; dev at openvswitch.org

    > Subject: RE: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local numa

    > node.

    > 

    > I support Billy's point here. There are a number of constraints/trade-offs that

    > can lead to an OVS deployments in OpenStack with e.g. a single PMD.

    > Without this change, a dual socket system is effectively rendered use-less. At

    > least one socket is lost for running VMs.

    > 

    > BR, Jan

    > 

    > > -----Original Message-----

    > > From: ovs-dev-bounces at openvswitch.org

    > > [mailto:ovs-dev-bounces at openvswitch.org] On Behalf Of O Mahony, Billy

    > > Sent: Tuesday, 28 February, 2017 14:59

    > > To: Ilya Maximets <i.maximets at samsung.com>; dev at openvswitch.org

    > > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local

    > numa node.

    > >

    > > Hi Ilya,

    > >

    > > Comments below.

    > >

    > > BR,

    > > Billy.

    > >

    > > > -----Original Message-----

    > > > From: Ilya Maximets [mailto:i.maximets at samsung.com]

    > > > Sent: Tuesday, February 28, 2017 12:49 PM

    > > > To: O Mahony, Billy <billy.o.mahony at intel.com>; dev at openvswitch.org

    > > > Cc: Daniele Di Proietto <diproiettod at ovn.org>

    > > > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on

    > > > non-local numa node.

    > > >

    > > > On 28.02.2017 14:43, O Mahony, Billy wrote:

    > > > > Hi Ilya,

    > > > >

    > > > > Thanks for the quick response. You make some good points.

    > > > >

    > > > > It's important to point out that local pmds are still chosen when

    > available.

    > > > > The change only operates to avoid totally non-operational/

    > > > > non-polled

    > > > ports.

    > > > >

    > > > > This is something we came across when deploying DPDK-enabled OVS

    > > > > in OpenStack environments (OPNFV project). Where we had remote

    > > > > (both physically and administratively) multi-node labs already

    > > > > wired up and would have much preferred to have sub-optimal

    > > > > operation that a non-

    > > > operational OpenStack environment.

    > > > >

    > > > > Some further comments below.

    > > > >

    > > > > Best Regards,

    > > > > Billy

    > > > >

    > > > >> -----Original Message-----

    > > > >> From: Ilya Maximets [mailto:i.maximets at samsung.com]

    > > > >> Sent: Tuesday, February 28, 2017 11:21 AM

    > > > >> To: O Mahony, Billy <billy.o.mahony at intel.com>;

    > > > >> dev at openvswitch.org

    > > > >> Cc: Daniele Di Proietto <diproiettod at ovn.org>

    > > > >> Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on

    > > > >> non-local numa node.

    > > > >>

    > > > >> Hello.

    > > > >>

    > > > >> On 28.02.2017 13:12, Billy O'Mahony wrote:

    > > > >>> From: billyom <billy.o.mahony at intel.com>

    > > > >>>

    > > > >>> Previously if there is no available (non-isolated) pmd on the

    > > > >>> numa node for a port then the port is not polled at all. This

    > > > >>> can result in a non-operational system until such time as nics

    > > > >>> are physically repositioned.

    > > > >>

    > > > >> Why you can't just reconfigure your pmd-cpu-mask after NICs'

    > > > repositioning?

    > > > > [[BO'M]] The idea is to avoid having to repositioning NICs as they

    > > > > may be in remote data-centers administered by other organisations.

    > > > > Also this can be related to multi-node clusters where it won't be

    > > > > just one NIC on one system that needs to be moved but several

    > > > > NICS/nodes

    > > > affected.

    > > > >

    > > > >

    > > > >> Maybe you can use pmd-rxq-affinity to assign port on another NUMA

    > > > node?

    > > > > [[BO'M]] The low-performance assignment will only occur if there

    > > > > is no available PMD on the local NUMA node. Ie if possible the

    > > > > normal highly performant assignment is made. It is only when it is

    > > > > a choice between lower performance and total non-performance that

    > > > > the lesser of two evils

    > > > is chosen.

    > > >

    > > > Maybe you can include at least one core from each node in pmd-cpu-

    > mask?

    > >

    > > [[BO'M]] Ideally everyone using and developing  the many OpenStack

    > > deployment tools would be aware of the optimal settings but we can

    > > expect that this will not the case in many situations. Poor

    > > configurations will occur. A warning message is included in the VLOG

    > > in this case to give users a hint that performance will not be optimal but

    > without imposing the penalty of a broken system - which is a heavy penalty if

    > you don’t even have direct access to the pmd-cpu-mask as you are driving

    > the ovs configuration via an OpenStack installation tool such as Fuel, RDO,

    > etc.

    > >

    > > >

    > > > >> The main concern here is that this 'remote' port will degrade

    > > > >> performance of other ports served by chosen PMD thread

    > significantly.

    > > > > [[BO'M]] That is a good point that there are second-order

    > > > > consequences on

    > > > other ports.

    > > > > But certainly with a many cloud systems even if one port is

    > > > > non-operational it means the entire node is effectively down - for

    > > > > instance an OpenStack compute node with a non-working provider

    > > > network

    > > > > for it's tenant VMs is useless even though the administration and

    > > > > control

    > > > networks are still working.

    > > > >>

    > > > >> Best regards, Ilya Maximets.

    > > _______________________________________________

    > > dev mailing list

    > > dev at openvswitch.org

    > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=GNRRfh6vBdD0g81qjkYyoE4hWv80KC2GRbQro6HR_go&s=0nEw3lSrDPhqOGsgc7cUQhvvyh90VzJgqRbMXaEG-B8&e= 

    _______________________________________________
    dev mailing list
    dev at openvswitch.org
    https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=GNRRfh6vBdD0g81qjkYyoE4hWv80KC2GRbQro6HR_go&s=0nEw3lSrDPhqOGsgc7cUQhvvyh90VzJgqRbMXaEG-B8&e= 