[ovs-dev] [PATCH v9] dpif-netlink: distribute polling to discreet handlers

[Aaron Conole]

Mark Gray <mark.d.gray at redhat.com> writes:

> From: Aaron Conole <aconole at redhat.com>
>
> Currently, the channel handlers are polled globally.  On some
> systems, this causes a thundering herd issue where multiple
> handler threads become active, only to do no work and immediately
> sleep.
>
> The approach here is to push the netlink socket channels to discreet
> handler threads to process, rather than polling on every thread.
> This will eliminate the need to wake multiple threads.
>
> To check:
>
>   ip netns add left
>   ip netns add right
>   ip link add center-left type veth peer name left0 netns left
>   ip link add center-right type veth peer name right0 netns right
>   ip link set center-left up
>   ip link set center-right up
>   ip -n left ip link set left0 up
>   ip -n left ip addr add 172.31.110.10/24 dev left0
>   ip -n right ip link set right0 up
>   ip -n right ip addr add 172.31.110.11/24 dev right0
>
>   ovs-vsctl add-br br0
>   ovs-vsctl add-port br0 center-right
>   ovs-vsctl add-port br0 center-left
>
>   # in one terminal
>   perf record -e sched:sched_wakeup,irq:softirq_entry -ag
>
>   # in a separate terminal
>   ip netns exec left arping -I left0 -c 1 172.31.110.11
>
>   # in the perf terminal after exiting
>   perf script
>
> Look for the number of 'handler' threads which were made active.
>
> Suggested-by: Ben Pfaff <blp at ovn.org>
> Suggested-by: Flavio Leitner <fbl at sysclose.org>
> Co-authored-by: Mark Gray <mark.d.gray at redhat.com>
> Reported-by: David Ahern <dsahern at gmail.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-dev/2019-December/365857.html
> Cc: Matteo Croce <technoboy85 at gmail.com>
> Fixes: 69c51582f ("dpif-netlink: don't allocate per thread netlink sockets")
> Signed-off-by: Aaron Conole <aconole at redhat.com>
> Signed-off-by: Mark Gray <mark.d.gray at redhat.com>
> ---
>
> v2: Oops - forgot to commit my whitespace cleanups.
> v3: one port latency results as per Matteo's comments
>
>     Stock:
>       min/avg/max/mdev = 21.5/36.5/96.5/1.0 us
>     With Patch:
>       min/avg/max/mdev = 5.3/9.7/98.4/0.5 us 
> v4: Oops - first email did not send
> v5:   min/avg/max/mdev = 9.3/10.4/33.6/2.2 us
> v6: Split out the AUTHORS patch and added a cover letter as
>     per Ilya's suggestion.
>     Fixed 0-day issues.
> v7: Merged patches as per Flavio's suggestion. This is
>     no longer a series. Fixed some other small issues.
> v9: Udated based on feedback from Ilya. Also implemented
>     another suggestion by Flavio
>       min/avg/max/mdev = 5.5/11.0/83.2/0.8 us

Ping?