[ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Bodireddy, Bhanuprakash]

>-----Original Message-----
>From: Aaron Conole [mailto:aconole at redhat.com]
>Sent: Tuesday, October 25, 2016 2:36 PM
>To: Bodireddy, Bhanuprakash <bhanuprakash.bodireddy at intel.com>
>Cc: Christian Ehrhardt <christian.ehrhardt at canonical.com>; ychen
><ychen103103 at 163.com>; dev at openvswitch.org
>Subject: Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master
>lcore?
>
>"Bodireddy, Bhanuprakash" <bhanuprakash.bodireddy at intel.com> writes:
>
>>>
>>>++ pidof ovs-vswitchd
>>>+ ps -To tid,pid,psr,comm -p 25932
>>>  TID   PID PSR COMMAND
>>>25932 25932   1 ovs-vswitchd
>>>25934 25932   0 eal-intr-thread
>>>25935 25932   1 dpdk_watchdog1
>>>25936 25932   2 vhost_thread2
>>>25937 25932   1 pdump-thread
>>>25938 25932   8 urcu3
>>>25939 25932   2 ovs-vswitchd
>>>25959 25932   1 ct_clean4
>>>26013 25932   3 handler18
>>>26014 25932   9 handler17
>>>26015 25932   3 handler20
>>>26016 25932  11 handler19
>>>26022 25932   4 handler21
>>>26028 25932   9 handler22
>>>26033 25932  10 handler23
>>>26037 25932  10 handler24
>>>26042 25932   9 revalidator25
>>>26045 25932  10 revalidator26
>>>26046 25932   0 revalidator27
>>>26054 25932   2 revalidator28
>>>26143 25932   5 pmd29
>>>26144 25932   4 pmd30
>>
>> This is for the reason that 'dpdk-lcore-mask' hasn't been specified in
>> the above case.
>> If one specify dpdk-lcore-mask=<mask>, all the
>> threads(revalidator/handler/ovs-vswitchd/urc) will automatically be
>> pinned to the lowest core of the 'mask'.
>>
>> For example with 'ovs-vsctl --no-wait set Open_vSwitch .
>> other_config:dpdk-lcore-mask=f00', I would expect the threads to float
>> between cores 8 and 11, but all the threads get pinned to lowest core
>> of the core-mask which is core 8.  Hope this clears the confusion.
>
>I'm sorry, I don't understand this behaviors.  I don't see that this is anything to
>do with ovs, specifically.  Meaning, when that option is specified, we pass it to
>the dpdk library directly, and take no actions as far as thread binding is
>concerned.  I have to admit, I've not delved that deeply into the dpdk side.

Completely agree Aaron. This isn't anything to do with OVS.  This is the current behavior of DPDK and more to do with how
it parses the coremask. For the given coremask, DPDK assumes the lowest core of the mask as 'mastercore/control core'
and pins the control threads to this core, leaving the remaining cores for actual packet forwarding. 

This behavior is reproducible with DPDK sample app.

Regards,
Bhanu Prakash. 

>
>Do you have the output from the EAL logs?  There are logs which state what
>dpdk thinks it should have set for an affinity.  Perhaps those shed some light?
>
>-Aaron