[ovs-dev] 10-25 packet drops every few (10-50) seconds TCP (iperf3)

Shahaji Bhosle shahaji.bhosle at broadcom.com
Mon Jun 1 23:27:09 UTC 2020


Hi Ben/Ilya,
Hope you guys are doing well and staying safe. I have been chasing a weird
problem with small drops and I think that is causing lots of TCP
retransmission.

Setup details
iPerf3(1k-5K
Servers)<--DPDK2:OvS+DPDK(VxLAN:BOND)[DPDK0+DPDK1)<====2x25G<====
[DPDK0+DPDK1)(VxLAN:BOND)OVS+DPDKDPDK2<---iPerf3(Clients)

All the Drops are ring drops on BONDed functions on the server side.  I
have 4 CPUs each with 3PMD threads, DPDK0, DPDK1 and DPDK2 all running with
4 Rx rings each.

What is interesting is when I give each Rx rings its own CPU the drops go
away. Or if I set cother_config:emc-insert-inv-prob=1 the drops go away.
But I need to scale up the number of flows so trying to run this with EMC
disabled.

I can tell that the rings are not getting serviced for 30-40usec because of
some kind context switch or interrupts on these cores. I have tried to do
the usual isolation, nohz_full rcu_nocbs etc. Move all the interrupts away
from these cores etc. But nothing helps. I mean it improves, but the drops
still happen.

I was hoping you have seen something like this and can point me in some
direction.
Thanks, Shahaji


More information about the dev mailing list