[ovs-discuss] OVS control plane performance

[Alexey Elin]

Hi,

I'm using OVS with netdev datapath:

```
_uuid               : 914f8ab9-7b5a-4150-94b5-b8dcb633e97e
bridges             : [8139f44d-4411-41b9-85b3-7b6d403809dc]
cur_cfg             : 213
datapath_types      : [netdev, system]
db_version          : "7.15.0"
external_ids        : {hostname="rk-3", 
rundir="/usr/local/var/run/openvswitch", system-id=""}
iface_types         : [dpdk, dpdkr, dpdkvhostuser, dpdkvhostuserclient, 
geneve, gre, internal, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 213
other_config        : {dpdk-init="true", dpdk-lcore-mask="0xffff", 
dpdk-socket-mem="16384", n-handler-threads="2", 
n-revalidator-threads="1", pmd-cpu-mask="0xffff"}
ovs_version         : "2.8.1"
ssl                 : []
statistics          : {}
system_type         : Ubuntu
system_version      : "16.04-xenial"
```

I noticed that switch answers to controller (e.g. port stats) with 
increasing delay. Connected switch to another controller and sent 100 
port stats requests in 100 seconds. Switch was sending replies for 420 
seconds. Dump is attached. Meanwhile in logs:

```
2017-10-18T09:36:39.895Z|00050|ovs_rcu(urcu8)|WARN|blocked 1002 ms 
waiting for main to quiesce
2017-10-18T09:36:40.893Z|00051|ovs_rcu(urcu8)|WARN|blocked 2000 ms 
waiting for main to quiesce
2017-10-18T09:36:42.895Z|00052|ovs_rcu(urcu8)|WARN|blocked 4002 ms 
waiting for main to quiesce
2017-10-18T09:36:46.893Z|00053|ovs_rcu(urcu8)|WARN|blocked 8000 ms 
waiting for main to quiesce
2017-10-18T09:36:54.893Z|00054|ovs_rcu(urcu8)|WARN|blocked 16000 ms 
waiting for main to quiesce
2017-10-18T09:36:56.311Z|00493|timeval|WARN|Unreasonably long 17559ms 
poll interval (5832ms user, 0ms system)
2017-10-18T09:36:56.311Z|00494|timeval|WARN|context switches: 6 
voluntary, 16585 involuntary
2017-10-18T09:36:58.262Z|00055|ovs_rcu(urcu8)|WARN|blocked 1000 ms 
waiting for main to quiesce
2017-10-18T09:36:59.383Z|00056|ovs_rcu(urcu8)|WARN|blocked 2121 ms 
waiting for main to quiesce
2017-10-18T09:37:01.267Z|00057|ovs_rcu(urcu8)|WARN|blocked 4005 ms 
waiting for main to quiesce
2017-10-18T09:37:05.262Z|00058|ovs_rcu(urcu8)|WARN|blocked 8000 ms 
waiting for main to quiesce
2017-10-18T09:37:08.108Z|00495|timeval|WARN|Unreasonably long 11797ms 
poll interval (3576ms user, 0ms system)
2017-10-18T09:37:08.108Z|00496|timeval|WARN|context switches: 5 
voluntary, 10799 involuntary
```

What could I miss in configuration to have such performance problems?

Thank you.

-- 
Alexey Elin
Applied Research Center for Computer Networks
Phone:    +7(495)240-50-63 ext: 118
Mobile:   +7(963)775-35-22
           +7(977)831-35-22
web:      www.arccn.ru
email:    aelin at arccn.ru
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ovs_netdev_100_port_stats_1.pcapng
Type: application/x-pcapng
Size: 147824 bytes
Desc: not available
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20171019/dcc58ec9/attachment-0001.bin>