[ovs-discuss] poor ovs-dpdk performance vs ovs
Robert Brooks
robmbrooks at gmail.com
Thu Nov 15 22:17:17 UTC 2018
I have a pair of Centos 7 hosts on 10gig switch.
Using OVS without dpdk enabled I can get 9.4Gb/s with a simple iperf test.
After switching the receiving host to ovs-dpdk following guidance here:
http://docs.openvswitch.org/en/latest/intro/install/dpdk/
I get 1.02Gb/s with 1500 MTU and 5.75Gb/s with a 9000 MTU.
Hardware is a Dell R630, with two E5-2680 26 core @ 2.40GHz cpus, 256GB
RAM, Intel 92599ES NIC.
I have confirmed the doucmented kernel boot options are set and 1GB
hugepages are in use.
# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-862.3.2.el7.x86_64
root=UUID=7d4edcad-0fd5-4224-a55f-ff2db81aac27 ro crashkernel=auto
rd.lvm.lv=vg01/swap
rd.lvm.lv=vg01/usr console=ttyS0,115200 iommu=pt intel_iommu=on
default_hugepagesz=1G hugepagesz=1G hugepages=8
Software is openvswitch 2.10.1 built using the provided rpm spec file. I
have used both the Centos provided dpdk 17.11 and a re-compile using latest
LTS of 17.11.4.
I have tried various performance tweaks, including cpu pinning and isolated
cpus.
Switching back to regular ovs returns the performance to close to wire
speed.
Under load the following is logged:
2018-11-15T21:59:24.306Z|00170|poll_loop|INFO|wakeup due to [POLLIN] on fd
74 (character device /dev/net/tun) at lib/netdev-linux.c:1347 (98% CPU
usage)
Config dump:
# ovsdb-client dump Open_vSwitch
AutoAttach table
_uuid mappings system_description system_name
----- -------- ------------------ -----------
Bridge table
_uuid auto_attach controller datapath_id
datapath_type datapath_version external_ids fail_mode flood_vlans
flow_tables ipfix mcast_snooping_enable mirrors name netflow other_config
ports
------------------------------------ ----------- ----------
------------------ ------------- ---------------- ------------ ---------
----------- ----------- ----- --------------------- ------- ----- -------
------------
---------------------------------------------------------------------
006f021a-b14d-49fb-a11d-48df2fa2bca1 [] [] "0000001b21a6ddc4"
netdev "<built-in>" {} [] [] {}
[] false [] "br0" [] {}
[3bd79dba-777c-40d0-b573-bf9e027326f4,
60b0661f-2177-4283-a6cb-a80336
Controller table
_uuid connection_mode controller_burst_limit controller_rate_limit
enable_async_messages external_ids inactivity_probe is_connected
local_gateway local_ip local_netmask max_backoff other_config role status
target
----- --------------- ---------------------- ---------------------
--------------------- ------------ ---------------- ------------
------------- -------- ------------- ----------- ------------ ---- ------
------
Flow_Sample_Collector_Set table
_uuid bridge external_ids id ipfix
----- ------ ------------ -- -----
Flow_Table table
_uuid external_ids flow_limit groups name overflow_policy prefixes
----- ------------ ---------- ------ ---- --------------- --------
IPFIX table
_uuid cache_active_timeout cache_max_flows external_ids obs_domain_id
obs_point_id other_config sampling targets
----- -------------------- --------------- ------------ -------------
------------ ------------ -------- -------
Interface table
_uuid admin_state bfd bfd_status cfm_fault
cfm_fault_status cfm_flap_count cfm_health cfm_mpid cfm_remote_mpids
cfm_remote_opstate duplex error external_ids ifindex ingress_policing_burst
ingress_policing_rate lacp_current link_resets link_speed link_stat
------------------------------------ ----------- --- ---------- ---------
---------------- -------------- ---------- -------- ----------------
------------------ ------ ----- ------------ --------
---------------------- --------------------- ------------ -----------
----------- ----------
271077eb-97af-4c2d-a5ee-2c63c9367312 up {} {} [] []
[] [] [] [] []
full [] {} 13 0 0
[] 11 10000000 up
4c3f7cdb-fd16-44c7-bb82-4aa8cef0c136 up {} {} [] []
[] [] [] [] []
full [] {} 13032858 0 0
[] 0 10000000000 up s
Manager table
_uuid connection_mode external_ids inactivity_probe is_connected
max_backoff other_config status target
----- --------------- ------------ ---------------- ------------
----------- ------------ ------ ------
Mirror table
_uuid external_ids name output_port output_vlan select_all select_dst_port
select_src_port select_vlan snaplen statistics
----- ------------ ---- ----------- ----------- ---------- ---------------
--------------- ----------- ------- ----------
NetFlow table
_uuid active_timeout add_id_to_interface engine_id engine_type external_ids
targets
----- -------------- ------------------- --------- ----------- ------------
-------
Open_vSwitch table
_uuid bridges
cur_cfg
datapath_types db_version dpdk_initialized dpdk_version external_ids
iface_types
------------------------------------ --------------------------------------
------- ---------------- ---------- ---------------- --------------
--------------------------------------------------------------------------------------------------------
----------------------------------------
ea8e7bcc-df11-42bd-96c9-b7eaafdfa863 [006f021a-b14d-49fb-a11d-48df2fa2bca1]
45 [netdev, system] "7.16.1" true "DPDK 17.11.4"
{hostname="v41", rundir="/var/run/openvswitch",
system-id="7d5d1db0-87db-41bc-bb94-661dde831e9f"} [dpdk, dpdkr,
dpdkvhostuser, dpdkvhostum
Port table
_uuid bond_active_slave bond_downdelay
bond_fake_iface bond_mode bond_updelay cvlans external_ids fake_bridge
interfaces lacp mac name other_config
protected qos rstp_statistics rstp_status statistics status tag trunks
vlan_mode
------------------------------------ ----------------- --------------
--------------- --------- ------------ ------ ------------ -----------
-------------------------------------- ---- --- ------ ------------
--------- --- --------------- ----------- ---------- ------ --- ------
---------
60b0661f-2177-4283-a6cb-a80336a8c478 [] 0 false
[] 0 [] {} false
[271077eb-97af-4c2d-a5ee-2c63c9367312]
[] [] "br0" {} false [] {} {} {}
{} [] [] []
3bd79dba-777c-40d0-b573-bf9e027326f4 [] 0 false
[] 0 [] {} false
[4c3f7cdb-fd16-44c7-bb82-4aa8cef0c136]
[] [] "p2p1" {} false [] {} {} {}
{} [] [] []
QoS table
_uuid external_ids other_config queues type
----- ------------ ------------ ------ ----
Queue table
_uuid dscp external_ids other_config
----- ---- ------------ ------------
SSL table
_uuid bootstrap_ca_cert ca_cert certificate external_ids private_key
----- ----------------- ------- ----------- ------------ -----------
sFlow table
_uuid agent external_ids header polling sampling targets
----- ----- ------------ ------ ------- -------- -------
I see rx_dropped counters. Other than that I'm not sure what's going on, as
I've tried various tuning suggested by dpdk, Intel and Red Hat, but
performance remains unimproved.
Regards,
Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20181115/36b8c003/attachment-0001.html>
More information about the discuss
mailing list