[ovs-dev] [PATCH v4 6/7] dpif-netdev: Allow pin rxq and non-isolate PMD.

Kevin Traynor ktraynor at redhat.com
Tue Jul 13 16:16:53 UTC 2021


Hi Jan,

On 13/07/2021 08:41, Jan Scheurich wrote:
> 
>> -----Original Message-----
>> From: dev <ovs-dev-bounces at openvswitch.org> On Behalf Of Kevin Traynor
>> Sent: Thursday, 8 July, 2021 15:54
>> To: dev at openvswitch.org
>> Cc: david.marchand at redhat.com
>> Subject: [ovs-dev] [PATCH v4 6/7] dpif-netdev: Allow pin rxq and non-isolate
>> PMD.
>>
>> Pinning an rxq to a PMD with pmd-rxq-affinity may be done for various reasons
>> such as reserving a full PMD for an rxq, or to ensure that multiple rxqs from a
>> port are handled on different PMDs.
>>
>> Previously pmd-rxq-affinity always isolated the PMD so no other rxqs could be
>> assigned to it by OVS. There may be cases where there is unused cycles on
>> those pmds and the user would like other rxqs to also be able to be assigned to
>> it by OVS.
>>
>> Add an option to pin the rxq and non-isolate the PMD. The default behaviour is
>> unchanged, which is pin and isolate the PMD.
>>
>> In order to pin and non-isolate:
>> ovs-vsctl set Open_vSwitch . other_config:pmd-rxq-isolate=false
>>
>> Note this is available only with group assignment type, as pinning conflicts with
>> the operation of the other rxq assignment algorithms.
>>
>> Signed-off-by: Kevin Traynor <ktraynor at redhat.com>
>> ---
>>  Documentation/topics/dpdk/pmd.rst |   9 ++-
>>  NEWS                              |   3 +
>>  lib/dpif-netdev.c                 |  34 ++++++++--
>>  tests/pmd.at                      | 105 ++++++++++++++++++++++++++++++
>>  vswitchd/vswitch.xml              |  19 ++++++
>>  5 files changed, 162 insertions(+), 8 deletions(-)
>>
>> diff --git a/Documentation/topics/dpdk/pmd.rst
>> b/Documentation/topics/dpdk/pmd.rst
>> index 29ba53954..30040d703 100644
>> --- a/Documentation/topics/dpdk/pmd.rst
>> +++ b/Documentation/topics/dpdk/pmd.rst
>> @@ -102,6 +102,11 @@ like so:
>>  - Queue #3 pinned to core 8
>>
>> -PMD threads on cores where Rx queues are *pinned* will become *isolated*.
>> This -means that this thread will only poll the *pinned* Rx queues.
>> +PMD threads on cores where Rx queues are *pinned* will become
>> +*isolated* by default. This means that this thread will only poll the *pinned*
>> Rx queues.
>> +
>> +If using ``pmd-rxq-assign=group`` PMD threads with *pinned* Rxqs can be
>> +*non-isolated* by setting::
>> +
>> +  $ ovs-vsctl set Open_vSwitch . other_config:pmd-rxq-isolate=false
> 
> Is there any specific reason why the new pmd-rxq-isolate option should be limited to the "group" scheduling algorithm? In my view it would make more sense and simplify documentation and code if these aspects of scheduling were kept orthogonal.
> 

David had a similar comment on an earlier version. I will add the fuller
reply below. In summary, pinning and the other algorithms (particularly
if there was multiple pinnings) conflict in how they operate because
they are based on RR pmd's to add equal number of Rxqs/PMD.

---
Yes, the main issue is that the other algorithms are based on a pmd
order on the assumption that they start from empty. For 'roundrobin' it
is to equally distribute num of rxqs on pmd RR - if we pin several rxqs
on one pmds, it is not clear what to do etc. Even worse for 'cycles' it
is based on placing rxqs in order of busy cycles on the pmds RR. If we
pin an rxq initially we are conflicting against how the algorithm operates.

Because 'group' algorithm is not tied to a defined pmd order, it is
better suited to be able to place a pinned rxq initially and be able to
consider that pmd on it's merits along with the others later.
---

Kevin.

> Regards, Jan
> 



More information about the dev mailing list