[ovs-dev] 64Byte packet performance regression on 2.9 from 2.7

[Ilya Maximets]

CC: Shahaji Bhosle

Sorry, missed you in CC list.

Best regards, Ilya Maximets.

On 15.06.2018 10:44, Ilya Maximets wrote:
>> Hi,
>> I just upgraded from OvS 2.7 + DPDK 16.11 to OvS2.9 + DPDK 17.11 and
>> running into performance issue with 64 Byte packet rate. One interesting
>> thing that I notice that even at very light load from IXIA the processing
>> cycles on all the PMD threads run close to 100% of the cpu cycle on 2.9
>> OvS, but on OvS 2.7 even under full load the processing cycles remain at
>> 75% of the cpu cycles.
>>
>> Attaching the FlameGraphs of both the versions, the only thing that pops
>> out to me is the new way invoking netdev_send() is on 2.9 is being invoked
>> via  dp_netdev_pmd_flush_output_packets() which seems to be adding another
>> ~11% to the whole rx to tx path.
>>
>> I also did try the tx-flush-interval to 50 and more it does seem to help,
>> but not significant enough to match the 2.7 performance.
>>
>>
>> Any help or ideas would be really great. Thanks, Shahaji
> 
> Hello, Shahaji.
> Could you, please, describe your testing scenario in more details? Also,
> mail-list filters attachments, so they are not available. You need to
> publish them somewhere else or write in text format inside the letter.
> 
> About the performance itself: Some performance degradation because of
> output batching is expected for tests with low number of flows or simple
> PHY-PHY tests. It was mainly targeted for cases with relatively large
> number of flows, for amortizing of vhost-user penalties (PHY-VM-PHY,
> VM-VM cases), OVS bonding cases.
> 
> If your test involves vhost-user ports, then you should also consider
> vhost-user performance regression in stable DPDK 17.11 because of fixes
> for CVE-2018-1059. Related bug:
> 	https://dpdk.org/tracker/show_bug.cgi?id=48
> 
> It'll be good if you'll be able to test OVS 2.8 + DPDK 17.05. There was
> too many changes since 2.7. It'll be hard to track down the root cause.
> 
> Best regards, Ilya Maximets.
>