[ovs-dev] PMD auto load balance experimental tag
Yoga Ananth
sy.ananth at gmail.com
Mon Apr 5 13:01:20 UTC 2021
Hello Everyone
As a follow up to my previous mail and to add more context, would like to
share the observed behavior of OVS 2.13 and OVS 2.15 ALB with the traffic
generator[1].
[1] https://github.com/yogananth-subramanian/multiqueue_traffic_gen
*OVS 2.13PMD load balancing only triggered when processing cycles goes
above 95 %OVS 2.15(openvswitch2.15-2.15.0-2.el8fdp.x86_64)DPDK
20.11.0pmd-auto-lb-load-threshold="70"pmd-auto-lb-improvement-threshold="25"PMD
load balancing triggered based on pmd-auto-lb-improvement-threshold and
auto-lb-threshold values2 PMD threads and 2 rxq per nicsBefore rebalance
:pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port
queue-id: 0 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb
queue-id: 0 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb
queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd
usage: 24 % port: vhu29ba2330-e2 queue-id: 0 (enabled) pmd usage: 73
% port: vhu9b6ca64c-3c queue-id: 1 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port
queue-id: 1 (enabled) pmd usage: 0 % port: vhu29ba2330-e2
queue-id: 1 (enabled) pmd usage: 98 % port: vhu29ba2330-e2
queue-id: 2 (disabled) pmd usage: 0 % port: vhu9b6ca64c-3c
queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd
usage: 45 % port: vhu17dc0ce6-cb queue-id: 2 (disabled) pmd usage: 0
% port: vhu9b6ca64c-3c queue-id: 2 (enabled) pmd usage: 0 %After
rebalance :pmd thread numa_id 0 core_id 1: isolated : false port:
vhu29ba2330-e2 queue-id: 1 (enabled) pmd usage: 98 % port:
vhu9b6ca64c-3c queue-id: 1 (enabled) pmd usage: 0 % port:
vhu9b6ca64c-3c queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id:
0 (enabled) pmd usage: 46 % port: vhu17dc0ce6-cb queue-id: 1
(enabled) pmd usage: 0 % port: vhu9b6ca64c-3c queue-id: 0 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port:
dpdk-link1-port queue-id: 1 (enabled) pmd usage: 46 % port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port:
vhu17dc0ce6-cb queue-id: 2 (disabled) pmd usage: 0 % port:
vhu29ba2330-e2 queue-id: 2 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id:
1 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb queue-id: 0
(enabled) pmd usage: 0 % port: vhu29ba2330-e2 queue-id: 0 (enabled)
pmd usage: 98 %Before rebalance :pmd thread numa_id 0 core_id
1: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd
usage: 21 % port: vhu34d496aa-27 queue-id: 2 (enabled) pmd usage: 0
% port: vhu90528211-c5 queue-id: 2 (disabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port
queue-id: 1 (enabled) pmd usage: 22 % port: vhu34d496aa-27
queue-id: 0 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68
queue-id: 1 (enabled) pmd usage: 53 % port: vhu9a0e7eca-68
queue-id: 2 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id
13: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd
usage: 33 % port: vhu90528211-c5 queue-id: 0 (enabled) pmd usage: 0
% port: vhu90528211-c5 queue-id: 1 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port
queue-id: 1 (enabled) pmd usage: 32 % port: vhu34d496aa-27
queue-id: 1 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68
queue-id: 0 (enabled) pmd usage: 53 %After rebalance :pmd thread numa_id
0 core_id 1: isolated : false port: vhu34d496aa-27 queue-id: 0
(enabled) pmd usage: 0 % port: vhu90528211-c5 queue-id: 0 (enabled)
pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 1 (enabled) pmd
usage: 54 %pmd thread numa_id 0 core_id 2: isolated : false port:
dpdk-link1-port queue-id: 1 (enabled) pmd usage: 19 % port:
dpdk-link2-port queue-id: 1 (enabled) pmd usage: 29 % port:
vhu90528211-c5 queue-id: 2 (disabled) pmd usage: 0 % port:
vhu9a0e7eca-68 queue-id: 2 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 13: isolated : false port: vhu34d496aa-27 queue-id:
1 (enabled) pmd usage: 0 % port: vhu34d496aa-27 queue-id: 2
(enabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 0 (enabled)
pmd usage: 46 %pmd thread numa_id 0 core_id 14: isolated : false port:
dpdk-link1-port queue-id: 0 (enabled) pmd usage: 19 % port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 30 % port:
vhu90528211-c5 queue-id: 1 (enabled) pmd usage: 0 %3 PMD threads and
3 rxq per nicsBefore rebalance :pmd thread numa_id 0 core_id 1: isolated :
false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0
% port: vhub5eeedde-9e queue-id: 1 (enabled) pmd usage: 0 % port:
vhub5eeedde-9e queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id:
2 (enabled) pmd usage: 40 % port: vhuabdd63e5-1d queue-id: 0
(enabled) pmd usage: 57 % port: vhucb9e7d30-fa queue-id: 3 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port:
dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port:
vhuabdd63e5-1d queue-id: 3 (disabled) pmd usage: 0 % port:
vhucb9e7d30-fa queue-id: 0 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id:
1 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 2
(enabled) pmd usage: 47 % port: vhucb9e7d30-fa queue-id: 1 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port:
dpdk-link1-port queue-id: 0 (enabled) pmd usage: 69 % port:
vhub5eeedde-9e queue-id: 0 (enabled) pmd usage: 0 % port:
vhub5eeedde-9e queue-id: 3 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id:
1 (enabled) pmd usage: 77 % port: vhuabdd63e5-1d queue-id: 1
(enabled) pmd usage: 20 % port: vhucb9e7d30-fa queue-id: 2 (enabled)
pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated
: false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 73
% port: vhucb9e7d30-fa queue-id: 2 (enabled) pmd usage: 0 % port:
vhucb9e7d30-fa queue-id: 3 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id:
2 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 0
(enabled) pmd usage: 81 % port: vhub5eeedde-9e queue-id: 1 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port:
vhuabdd63e5-1d queue-id: 1 (enabled) pmd usage: 82 % port:
vhuabdd63e5-1d queue-id: 3 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id:
2 (enabled) pmd usage: 73 % port: vhub5eeedde-9e queue-id: 3
(disabled) pmd usage: 0 % port: vhucb9e7d30-fa queue-id: 0 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port:
dpdk-link1-port queue-id: 0 (enabled) pmd usage: 77 % port:
vhub5eeedde-9e queue-id: 0 (enabled) pmd usage: 0 % port:
vhucb9e7d30-fa queue-id: 1 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 15: isolated : false port: dpdk-link2-port queue-id:
1 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 2
(enabled) pmd usage: 91 % port: vhub5eeedde-9e queue-id: 2 (enabled)
pmd usage: 0 %Before rebalance :pmd thread numa_id 0 core_id
1: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd
usage: 21 % port: vhu30e312d0-d2 queue-id: 0 (enabled) pmd usage: 41
% port: vhua7d1a810-86 queue-id: 3 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port
queue-id: 1 (enabled) pmd usage: 0 % port: vhu30e312d0-d2
queue-id: 2 (enabled) pmd usage: 59 % port: vhua7d1a810-86
queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
3: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd
usage: 0 % port: vhu68c33115-4a queue-id: 1 (enabled) pmd usage: 0
% port: vhu68c33115-4a queue-id: 2 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port
queue-id: 0 (enabled) pmd usage: 24 % port: vhu68c33115-4a
queue-id: 0 (enabled) pmd usage: 0 % port: vhu68c33115-4a
queue-id: 3 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id
14: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd
usage: 22 % port: vhu30e312d0-d2 queue-id: 1 (enabled) pmd usage: 57
% port: vhua7d1a810-86 queue-id: 2 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 15: isolated : false port: dpdk-link2-port
queue-id: 2 (enabled) pmd usage: 0 % port: vhu30e312d0-d2
queue-id: 3 (disabled) pmd usage: 0 % port: vhua7d1a810-86
queue-id: 0 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id
0 core_id 1: isolated : false port: vhu30e312d0-d2 queue-id: 0
(enabled) pmd usage: 51 % port: vhu68c33115-4a queue-id: 1 (enabled)
pmd usage: 0 % port: vhua7d1a810-86 queue-id: 0 (enabled) pmd
usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port:
dpdk-link1-port queue-id: 1 (enabled) pmd usage: 25 % port:
dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port:
vhu68c33115-4a queue-id: 3 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 3: isolated : false port: vhu30e312d0-d2 queue-id:
2 (enabled) pmd usage: 59 % port: vhua7d1a810-86 queue-id: 2
(enabled) pmd usage: 0 % port: vhua7d1a810-86 queue-id: 3 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port:
vhu30e312d0-d2 queue-id: 1 (enabled) pmd usage: 64 % port:
vhu68c33115-4a queue-id: 0 (enabled) pmd usage: 0 % port:
vhua7d1a810-86 queue-id: 1 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id:
0 (enabled) pmd usage: 26 % port: dpdk-link2-port queue-id: 2
(enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 2 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port:
dpdk-link1-port queue-id: 2 (enabled) pmd usage: 29 % port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port:
vhu30e312d0-d2 queue-id: 3 (disabled) pmd usage: 0 %5 Physical
queues---5 PMDCores---5 vQueues (3 full throttling traffic streams) Before
rebalance :pmd thread numa_id 0 core_id 1: isolated : false port:
dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port:
vhu043b3d09-aa queue-id: 5 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 0 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id:
1 (enabled) pmd usage: 73 % port: vhu043b3d09-aa queue-id: 4
(enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 1 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port:
dpdk-link1-port queue-id: 0 (enabled) pmd usage: 69 % port:
vhu043b3d09-aa queue-id: 3 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 4: isolated : false port: dpdk-link1-port queue-id:
3 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 1
(enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 4 (enabled)
pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 5 (disabled) pmd
usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port:
dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port:
dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port:
vhu956b15b2-d8 queue-id: 1 (enabled) pmd usage: 80 % port:
vhu956b15b2-d8 queue-id: 2 (enabled) pmd usage: 17 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id:
0 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 0
(enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 5
(disabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 4 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port:
dpdk-link1-port queue-id: 2 (enabled) pmd usage: 15 % port:
dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port:
vhu956b15b2-d8 queue-id: 0 (enabled) pmd usage: 41 % port:
vhu956b15b2-d8 queue-id: 3 (enabled) pmd usage: 41 %pmd thread
numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id:
4 (enabled) pmd usage: 97 % port: vhu043b3d09-aa queue-id: 2
(enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 3 (enabled)
pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated
: false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 75
% port: vhu043b3d09-aa queue-id: 1 (enabled) pmd usage: 0 % port:
vhu043b3d09-aa queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id:
4 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 3
(enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 1 (enabled)
pmd usage: 86 %pmd thread numa_id 0 core_id 3: isolated : false port:
dpdk-link1-port queue-id: 1 (enabled) pmd usage: 73 % port:
dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port:
vhu043b3d09-aa queue-id: 4 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 4: isolated : false port: dpdk-link1-port queue-id:
3 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 0
(enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 0 (enabled)
pmd usage: 87 % port: vhu956b15b2-d8 queue-id: 5 (disabled) pmd
usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port:
dpdk-link1-port queue-id: 2 (enabled) pmd usage: 74 % port:
dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 3 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 4 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id:
0 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 1
(enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 3 (enabled)
pmd usage: 70 % port: vhu956b15b2-d8 queue-id: 4 (enabled) pmd
usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port:
vhu043b3d09-aa queue-id: 0 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 2 (enabled) pmd usage: 0 % port:
vhu0c6e626a-ff queue-id: 5 (disabled) pmd usage: 0 % port:
vhu956b15b2-d8 queue-id: 2 (enabled) pmd usage: 98 %pmd thread
numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id:
0 (enabled) pmd usage: 96 % port: dpdk-link2-port queue-id: 2
(enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 5 (enabled)
pmd usage: 0 %Before rebalance :pmd thread numa_id 0 core_id
1: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd
usage: 0 % port: vhu261f37a0-be queue-id: 5 (enabled) pmd usage: 0
% port: vhuc4013bd9-87 queue-id: 2 (enabled) pmd usage: 47 %pmd
thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port
queue-id: 1 (enabled) pmd usage: 0 % port: vhu597bfe26-ed
queue-id: 3 (enabled) pmd usage: 0 % port: vhu597bfe26-ed
queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
3: isolated : false port: dpdk-link2-port queue-id: 3 (enabled) pmd
usage: 0 % port: vhu261f37a0-be queue-id: 0 (enabled) pmd usage: 0
%pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link2-port
queue-id: 2 (enabled) pmd usage: 0 % port: vhu597bfe26-ed
queue-id: 0 (enabled) pmd usage: 0 % port: vhuc4013bd9-87
queue-id: 1 (enabled) pmd usage: 60 %pmd thread numa_id 0 core_id
5: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd
usage: 38 % port: vhu261f37a0-be queue-id: 1 (enabled) pmd usage: 0
%pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port
queue-id: 0 (enabled) pmd usage: 29 % port: vhu597bfe26-ed
queue-id: 1 (enabled) pmd usage: 0 % port: vhuc4013bd9-87
queue-id: 0 (enabled) pmd usage: 51 %pmd thread numa_id 0 core_id
14: isolated : false port: dpdk-link2-port queue-id: 4 (enabled) pmd
usage: 0 % port: vhu261f37a0-be queue-id: 3 (enabled) pmd usage: 0
% port: vhuc4013bd9-87 queue-id: 4 (enabled) pmd usage: 30 %pmd
thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port
queue-id: 1 (enabled) pmd usage: 47 % port: vhu597bfe26-ed
queue-id: 2 (enabled) pmd usage: 0 % port: vhu597bfe26-ed
queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id
16: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd
usage: 44 % port: vhu261f37a0-be queue-id: 2 (enabled) pmd usage: 0
% port: vhuc4013bd9-87 queue-id: 5 (disabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port
queue-id: 4 (enabled) pmd usage: 73 % port: vhu261f37a0-be
queue-id: 4 (enabled) pmd usage: 0 % port: vhuc4013bd9-87
queue-id: 3 (enabled) pmd usage: 23 %After rebalance :pmd thread numa_id
0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 1
(enabled) pmd usage: 31 % port: vhu261f37a0-be queue-id: 0 (enabled)
pmd usage: 0 % port: vhu597bfe26-ed queue-id: 3 (enabled) pmd
usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port:
dpdk-link1-port queue-id: 4 (enabled) pmd usage: 27 % port:
vhu261f37a0-be queue-id: 4 (enabled) pmd usage: 0 % port:
vhu261f37a0-be queue-id: 5 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id:
1 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 3
(enabled) pmd usage: 53 %pmd thread numa_id 0 core_id 4: isolated :
false port: vhu261f37a0-be queue-id: 1 (enabled) pmd usage: 0
% port: vhu597bfe26-ed queue-id: 2 (enabled) pmd usage: 0 % port:
vhuc4013bd9-87 queue-id: 2 (enabled) pmd usage: 69 %pmd thread
numa_id 0 core_id 5: isolated : false port: dpdk-link1-port queue-id:
0 (enabled) pmd usage: 44 % port: dpdk-link2-port queue-id: 0
(enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated :
false port: vhu261f37a0-be queue-id: 2 (enabled) pmd usage: 0
% port: vhu597bfe26-ed queue-id: 1 (enabled) pmd usage: 0 % port:
vhuc4013bd9-87 queue-id: 0 (enabled) pmd usage: 65 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id:
3 (enabled) pmd usage: 21 % port: dpdk-link2-port queue-id: 3
(enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 5
(disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated :
false port: vhu261f37a0-be queue-id: 3 (enabled) pmd usage: 0
% port: vhu597bfe26-ed queue-id: 0 (enabled) pmd usage: 0 % port:
vhuc4013bd9-87 queue-id: 1 (enabled) pmd usage: 31 %pmd thread
numa_id 0 core_id 16: isolated : false port: dpdk-link2-port queue-id:
2 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 4
(enabled) pmd usage: 71 % port: vhuc4013bd9-87 queue-id: 5
(disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated :
false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 39
% port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port:
vhu597bfe26-ed queue-id: 4 (enabled) pmd usage: 0 %7 PMD threads and
7 rxq per nicsBefore rebalance :pmd thread numa_id 0 core_id 1: isolated :
false port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0
% port: vhuc12aa03a-ae queue-id: 5 (enabled) pmd usage: 0 % port:
vhuc12aa03a-ae queue-id: 6 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id:
6 (enabled) pmd usage: 73 % port: vhuc12aa03a-ae queue-id: 4
(enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 7
(disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated :
false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 27
% port: vhuc12aa03a-ae queue-id: 3 (enabled) pmd usage: 0 % port:
vhuf8dcddb7-dd queue-id: 0 (enabled) pmd usage: 71 %pmd thread
numa_id 0 core_id 4: isolated : false port: dpdk-link2-port queue-id:
2 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 3
(enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated :
false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0
% port: vhuc59a8064-13 queue-id: 4 (enabled) pmd usage: 0 % port:
vhuf8dcddb7-dd queue-id: 7 (disabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 6: isolated : false port: dpdk-link1-port queue-id:
5 (enabled) pmd usage: 71 % port: vhuc59a8064-13 queue-id: 5
(enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 6 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port:
vhuc12aa03a-ae queue-id: 2 (enabled) pmd usage: 0 % port:
vhuf8dcddb7-dd queue-id: 1 (enabled) pmd usage: 92 %pmd thread
numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id:
5 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 0
(enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated :
false port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0
% port: vhuc59a8064-13 queue-id: 2 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 15: isolated : false port: dpdk-link2-port
queue-id: 4 (enabled) pmd usage: 0 % port: vhuc59a8064-13
queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
16: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd
usage: 47 % port: vhuc59a8064-13 queue-id: 6 (enabled) pmd usage: 0
% port: vhuf8dcddb7-dd queue-id: 5 (enabled) pmd usage: 51 %pmd
thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port
queue-id: 0 (enabled) pmd usage: 75 % port: vhuc12aa03a-ae
queue-id: 1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd
queue-id: 2 (enabled) pmd usage: 24 %pmd thread numa_id 0 core_id
18: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd
usage: 0 % port: vhuc59a8064-13 queue-id: 7 (enabled) pmd usage: 0
% port: vhuf8dcddb7-dd queue-id: 4 (enabled) pmd usage: 30 %pmd
thread numa_id 0 core_id 19: isolated : false port: dpdk-link1-port
queue-id: 2 (enabled) pmd usage: 43 % port: vhuc12aa03a-ae
queue-id: 0 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd
queue-id: 3 (enabled) pmd usage: 55 %After rebalance :pmd thread numa_id
0 core_id 1: isolated : false port: vhuc12aa03a-ae queue-id: 0
(enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 7 (enabled)
pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 1 (enabled) pmd
usage: 88 %pmd thread numa_id 0 core_id 2: isolated : false port:
dpdk-link1-port queue-id: 0 (enabled) pmd usage: 80 % port:
vhuc12aa03a-ae queue-id: 1 (enabled) pmd usage: 0 % port:
vhuc59a8064-13 queue-id: 6 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id:
6 (enabled) pmd usage: 78 % port: vhuc12aa03a-ae queue-id: 2
(enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 5 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port:
dpdk-link1-port queue-id: 1 (enabled) pmd usage: 97 % port:
dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 5: isolated : false port: dpdk-link2-port queue-id:
3 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 4
(enabled) pmd usage: 37 % port: vhuf8dcddb7-dd queue-id: 7
(disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 6: isolated :
false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 75
% port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port:
vhuf8dcddb7-dd queue-id: 6 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 7: isolated : false port: vhuc12aa03a-ae queue-id:
3 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 4
(enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 0 (enabled)
pmd usage: 59 %pmd thread numa_id 0 core_id 13: isolated : false port:
dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port:
vhuc59a8064-13 queue-id: 0 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id:
1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 2
(enabled) pmd usage: 88 %pmd thread numa_id 0 core_id 15: isolated :
false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0
% port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port
queue-id: 4 (enabled) pmd usage: 97 % port: dpdk-link2-port
queue-id: 5 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae
queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id
17: isolated : false port: dpdk-link1-port queue-id: 5 (enabled) pmd
usage: 97 % port: vhuc12aa03a-ae queue-id: 4 (enabled) pmd usage: 0
% port: vhuc59a8064-13 queue-id: 3 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 18: isolated : false port: vhuc12aa03a-ae
queue-id: 6 (enabled) pmd usage: 0 % port: vhuc59a8064-13
queue-id: 1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd
queue-id: 5 (enabled) pmd usage: 52 %pmd thread numa_id 0 core_id
19: isolated : false port: vhuc12aa03a-ae queue-id: 5 (enabled) pmd
usage: 0 % port: vhuc59a8064-13 queue-id: 2 (enabled) pmd usage: 0
% port: vhuf8dcddb7-dd queue-id: 3 (enabled) pmd usage: 97 %Before
rebalance :pmd thread numa_id 0 core_id 1: isolated : false port:
dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port:
vhu650a7bf5-41 queue-id: 3 (enabled) pmd usage: 36 % port:
vhu745c80e7-c3 queue-id: 0 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id:
6 (enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 5
(enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 6 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port:
dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port:
vhu853f185d-2f queue-id: 1 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 4: isolated : false port: dpdk-link2-port queue-id:
0 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 2
(enabled) pmd usage: 43 % port: vhu745c80e7-c3 queue-id: 1 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated : false port:
dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port:
vhu853f185d-2f queue-id: 3 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 6: isolated : false port: dpdk-link1-port queue-id:
1 (enabled) pmd usage: 16 % port: vhu650a7bf5-41 queue-id: 5
(enabled) pmd usage: 26 % port: vhu853f185d-2f queue-id: 6 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port:
dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port:
vhu853f185d-2f queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id:
6 (enabled) pmd usage: 16 % port: vhu650a7bf5-41 queue-id: 0
(enabled) pmd usage: 32 % port: vhu745c80e7-c3 queue-id: 3 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port:
dpdk-link1-port queue-id: 4 (enabled) pmd usage: 15 % port:
vhu650a7bf5-41 queue-id: 6 (enabled) pmd usage: 0 % port:
vhu853f185d-2f queue-id: 5 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id:
0 (enabled) pmd usage: 26 % port: vhu745c80e7-c3 queue-id: 4
(enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 7
(disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated :
false port: dpdk-link1-port queue-id: 5 (enabled) pmd usage: 23
% port: vhu650a7bf5-41 queue-id: 7 (disabled) pmd usage: 0 % port:
vhu853f185d-2f queue-id: 4 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id:
3 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 4
(enabled) pmd usage: 30 % port: vhu853f185d-2f queue-id: 7 (enabled)
pmd usage: 0 %pmd thread numa_id 0 core_id 18: isolated : false port:
dpdk-link1-port queue-id: 2 (enabled) pmd usage: 15 % port:
vhu650a7bf5-41 queue-id: 1 (enabled) pmd usage: 46 % port:
vhu745c80e7-c3 queue-id: 2 (enabled) pmd usage: 0 %pmd thread
numa_id 0 core_id 19: isolated : false port: dpdk-link2-port queue-id:
5 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 0
(enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id
1: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd
usage: 32 % port: vhu745c80e7-c3 queue-id: 5 (enabled) pmd usage: 0
% port: vhu853f185d-2f queue-id: 2 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 2: isolated : false port: vhu650a7bf5-41
queue-id: 2 (enabled) pmd usage: 69 % port: vhu745c80e7-c3
queue-id: 0 (enabled) pmd usage: 0 % port: vhu853f185d-2f
queue-id: 7 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
3: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd
usage: 0 % port: vhu853f185d-2f queue-id: 0 (enabled) pmd usage: 0
%pmd thread numa_id 0 core_id 4: isolated : false port: vhu650a7bf5-41
queue-id: 0 (enabled) pmd usage: 86 % port: vhu745c80e7-c3
queue-id: 4 (enabled) pmd usage: 0 % port: vhu853f185d-2f
queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
5: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd
usage: 32 % port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0
%pmd thread numa_id 0 core_id 6: isolated : false port: dpdk-link2-port
queue-id: 5 (enabled) pmd usage: 0 % port: vhu650a7bf5-41
queue-id: 5 (enabled) pmd usage: 44 % port: vhu745c80e7-c3
queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id
7: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd
usage: 44 % port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0
%pmd thread numa_id 0 core_id 13: isolated : false port: vhu650a7bf5-41
queue-id: 3 (enabled) pmd usage: 24 % port: vhu745c80e7-c3
queue-id: 2 (enabled) pmd usage: 0 % port: vhu853f185d-2f
queue-id: 5 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
14: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd
usage: 26 % port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0
% port: vhu650a7bf5-41 queue-id: 6 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 15: isolated : false port: vhu650a7bf5-41
queue-id: 1 (enabled) pmd usage: 32 % port: vhu745c80e7-c3
queue-id: 1 (enabled) pmd usage: 0 % port: vhu853f185d-2f
queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
16: isolated : false port: dpdk-link1-port queue-id: 6 (enabled) pmd
usage: 46 % port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0
% port: vhu650a7bf5-41 queue-id: 7 (disabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port
queue-id: 5 (enabled) pmd usage: 36 % port: vhu745c80e7-c3
queue-id: 6 (enabled) pmd usage: 0 % port: vhu853f185d-2f
queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id
18: isolated : false port: vhu650a7bf5-41 queue-id: 4 (enabled) pmd
usage: 22 % port: vhu745c80e7-c3 queue-id: 3 (enabled) pmd usage: 0
% port: vhu853f185d-2f queue-id: 4 (enabled) pmd usage: 0 %pmd
thread numa_id 0 core_id 19: isolated : false port: dpdk-link2-port
queue-id: 0 (enabled) pmd usage: 0 % port: dpdk-link2-port
queue-id: 6 (enabled) pmd usage: 0 % *
On Fri, Mar 19, 2021 at 2:34 AM Yoga Ananth <sy.ananth at gmail.com> wrote:
>
> Hello Kevin,
> On Fri, Mar 5, 2021 at 10:37 PM Kevin Traynor <ktraynor at redhat.com> wrote:
>
>> Hi All,
>>
>> PMD auto load balance (ALB) is an experimental feature since OVS 2.11.
>>
>> In short, RxQs are reassigned to PMDs based on load during a reconfig.
>>
>> However, there might not be a reconfig for a long time so ALB runs
>> periodically and checks PMD thread load and the load variance between
>> PMD threads. If load is high and it's estimated that it will be improved
>> by a certain amount after reassigning RxQs to PMDs, a reassignment is
>> triggered.
>>
>> ALB calls the reassignment code similar to if there was a reconfig or if
>> the rebalance command is used. There are some ideas for how to improve
>> RxQ to PMD assignments in general, but that's a slightly different topic
>> and can be a later optimization if needed.
>>
>> ALB is documented more here:
>>
>> https://docs.openvswitch.org/en/latest/topics/dpdk/pmd/?highlight=rebalance#automatic-assignment-of-port-rx-queue-to-pmd-threads-experimental
>>
>> I would like to progress it so the experimental tag on ALB can be
>> removed. There are a few things I think are needed:
>>
>> - Need to restructure the code so that ALB reassignment dry run uses the
>> same code path as the reassignment code it will call
>>
>> - ALB operation should be understandable through logs and stats
>> e.g. Should be able to confirm setting state/parameters in logs and see
>> if ALB is taking some action. Should be able to get more debug if needed
>> etc.
>>
>> - Unit tests
>> Some patches here:
>>
>> http://patchwork.ozlabs.org/project/openvswitch/patch/20210218104802.293740-2-ktraynor@redhat.com/
>>
>> - Iron out if the current ALB enable/disable user interface logging
>> based partly min resource requirements is right or we want to it to run
>> regardless
>> i.e.
>> https://mail.openvswitch.org/pipermail/ovs-dev/2021-February/380386.html
>>
>> - Review/Improve operation documentation
>>
>> - Further testing
>>
>> Thoughts?
>>
>> I was able to evaluate ALB with 2.15 release, by using a traffic
> generator[1] that is able to deterministically load only the desired rxqs
> associated with a PMD, hence I was able to deterministically evaluate ALB
> for different values of pmd-auto-lb-load-threshold and n_rxq.
> Currently working on enhancing traffic generator[1] to evaluate ALB with
> bidirectional traffic.
> If you have any scenario or traffic pattern in mind, happy to adapt the
> traffic generator and evaluate ALB with those too.
>
> [1] https://github.com/yogananth-subramanian/multiqueue_traffic_gen
>
>> thanks,
>> Kevin.
>>
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>
>
> --
> Kind Regards
> yogananth subramanian
>
--
Kind Regards
yogananth subramanian
More information about the dev
mailing list