[ovs-dev] 64Byte packet performance regression on 2.9 from 2.7

[Nitin Katiyar]

Hi,
We also experienced degradation from OVS2.6/2.7 to OVS2.8.2(with DPDK17.05.02). The drop is more for 64 bytes packet size (~8-10%) even with higher number of flows. I tried OVS 2.8 with DPDK17.11 and it improved for higher packet sizes but 64 bytes size is still the concern.

Regards,
Nitin

-----Original Message-----
From: Ilya Maximets [mailto:i.maximets at samsung.com] 
Sent: Monday, June 18, 2018 1:32 PM
To: ovs-dev at openvswitch.org; shahaji.bhosle at broadcom.com
Subject: Re: [ovs-dev] 64Byte packet performance regression on 2.9 from 2.7

CC: Shahaji Bhosle

Sorry, missed you in CC list.

Best regards, Ilya Maximets.

On 15.06.2018 10:44, Ilya Maximets wrote:
>> Hi,
>> I just upgraded from OvS 2.7 + DPDK 16.11 to OvS2.9 + DPDK 17.11 and 
>> running into performance issue with 64 Byte packet rate. One 
>> interesting thing that I notice that even at very light load from 
>> IXIA the processing cycles on all the PMD threads run close to 100% 
>> of the cpu cycle on 2.9 OvS, but on OvS 2.7 even under full load the 
>> processing cycles remain at 75% of the cpu cycles.
>>
>> Attaching the FlameGraphs of both the versions, the only thing that 
>> pops out to me is the new way invoking netdev_send() is on 2.9 is 
>> being invoked via  dp_netdev_pmd_flush_output_packets() which seems 
>> to be adding another ~11% to the whole rx to tx path.
>>
>> I also did try the tx-flush-interval to 50 and more it does seem to 
>> help, but not significant enough to match the 2.7 performance.
>>
>>
>> Any help or ideas would be really great. Thanks, Shahaji
> 
> Hello, Shahaji.
> Could you, please, describe your testing scenario in more details? 
> Also, mail-list filters attachments, so they are not available. You 
> need to publish them somewhere else or write in text format inside the letter.
> 
> About the performance itself: Some performance degradation because of 
> output batching is expected for tests with low number of flows or 
> simple PHY-PHY tests. It was mainly targeted for cases with relatively 
> large number of flows, for amortizing of vhost-user penalties 
> (PHY-VM-PHY, VM-VM cases), OVS bonding cases.
> 
> If your test involves vhost-user ports, then you should also consider 
> vhost-user performance regression in stable DPDK 17.11 because of 
> fixes for CVE-2018-1059. Related bug:
> 	https://dpdk.org/tracker/show_bug.cgi?id=48
> 
> It'll be good if you'll be able to test OVS 2.8 + DPDK 17.05. There 
> was too many changes since 2.7. It'll be hard to track down the root cause.
> 
> Best regards, Ilya Maximets.
>