[ovs-dev] thundering herd wakeup of handler threads

Aaron Conole aconole at redhat.com
Thu Mar 12 18:43:15 UTC 2020


David Ahern <dsahern at gmail.com> writes:

> On 1/8/20 9:18 AM, Aaron Conole wrote:
>> David Ahern <dsahern at gmail.com> writes:
>> 
>>> On 12/16/19 2:42 PM, Aaron Conole wrote:
>>>> Can you try the following and see if your scalability issue is
>>>> addressed?  I think it could be better integrated, but this is a
>>>> different quick 'n dirty.
>>>
>>> your patch reduces the number of threads awakened, but it is still
>>> really high - 43 out of 71 handler threads in one test.
>> 
>> Thanks for getting back to me (sorry for the late response, catching
>> up).  I'll take a closer look.  I haven't thought about adding
>> EPOLLEXCLUSIVE to the poll block change, so maybe that will address it
>> completely, or maybe there's a different approach.
>> 
>
> This seems to have gone quiet for a few months now. In case my previous
> comments weren't clear, the change in question
> (69c51582ff786a68fc325c1c50624715482bc460) converted a control plane
> scaling problem into a datapath scaling problem. It manifests as an
> increase in squeezed counts and dropped packets (both at the enqueue
> point and at hand off to the vhost threads) because of the increased
> time spent on upcalls.
>
> I have reverted that commit and several on top of it for our purposes,
> but this really is something OVS devs should fix.

There is a patch set in the works that I think should help:

  https://mail.openvswitch.org/pipermail/ovs-dev/2020-February/368139.html
  https://mail.openvswitch.org/pipermail/ovs-dev/2020-February/368138.html

I will take a closer look at it.



More information about the dev mailing list