[ovs-dev] 64Byte packet performance regression on 2.9 from 2.7

Ilya Maximets i.maximets at samsung.com
Fri Jun 15 07:44:12 UTC 2018


> Hi,
> I just upgraded from OvS 2.7 + DPDK 16.11 to OvS2.9 + DPDK 17.11 and
> running into performance issue with 64 Byte packet rate. One interesting
> thing that I notice that even at very light load from IXIA the processing
> cycles on all the PMD threads run close to 100% of the cpu cycle on 2.9
> OvS, but on OvS 2.7 even under full load the processing cycles remain at
> 75% of the cpu cycles.
> 
> Attaching the FlameGraphs of both the versions, the only thing that pops
> out to me is the new way invoking netdev_send() is on 2.9 is being invoked
> via  dp_netdev_pmd_flush_output_packets() which seems to be adding another
> ~11% to the whole rx to tx path.
> 
> I also did try the tx-flush-interval to 50 and more it does seem to help,
> but not significant enough to match the 2.7 performance.
> 
> 
> Any help or ideas would be really great. Thanks, Shahaji

Hello, Shahaji.
Could you, please, describe your testing scenario in more details? Also,
mail-list filters attachments, so they are not available. You need to
publish them somewhere else or write in text format inside the letter.

About the performance itself: Some performance degradation because of
output batching is expected for tests with low number of flows or simple
PHY-PHY tests. It was mainly targeted for cases with relatively large
number of flows, for amortizing of vhost-user penalties (PHY-VM-PHY,
VM-VM cases), OVS bonding cases.

If your test involves vhost-user ports, then you should also consider
vhost-user performance regression in stable DPDK 17.11 because of fixes
for CVE-2018-1059. Related bug:
	https://dpdk.org/tracker/show_bug.cgi?id=48

It'll be good if you'll be able to test OVS 2.8 + DPDK 17.05. There was
too many changes since 2.7. It'll be hard to track down the root cause.

Best regards, Ilya Maximets.


More information about the dev mailing list