[ovs-dev] [PATCH v9] dpif-netlink: distribute polling to discreet handlers
aconole at redhat.com
Fri Oct 23 20:09:39 UTC 2020
Mark Gray <mark.d.gray at redhat.com> writes:
> From: Aaron Conole <aconole at redhat.com>
> Currently, the channel handlers are polled globally. On some
> systems, this causes a thundering herd issue where multiple
> handler threads become active, only to do no work and immediately
> The approach here is to push the netlink socket channels to discreet
> handler threads to process, rather than polling on every thread.
> This will eliminate the need to wake multiple threads.
> To check:
> ip netns add left
> ip netns add right
> ip link add center-left type veth peer name left0 netns left
> ip link add center-right type veth peer name right0 netns right
> ip link set center-left up
> ip link set center-right up
> ip -n left ip link set left0 up
> ip -n left ip addr add 172.31.110.10/24 dev left0
> ip -n right ip link set right0 up
> ip -n right ip addr add 172.31.110.11/24 dev right0
> ovs-vsctl add-br br0
> ovs-vsctl add-port br0 center-right
> ovs-vsctl add-port br0 center-left
> # in one terminal
> perf record -e sched:sched_wakeup,irq:softirq_entry -ag
> # in a separate terminal
> ip netns exec left arping -I left0 -c 1 172.31.110.11
> # in the perf terminal after exiting
> perf script
> Look for the number of 'handler' threads which were made active.
> Suggested-by: Ben Pfaff <blp at ovn.org>
> Suggested-by: Flavio Leitner <fbl at sysclose.org>
> Co-authored-by: Mark Gray <mark.d.gray at redhat.com>
> Reported-by: David Ahern <dsahern at gmail.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-dev/2019-December/365857.html
> Cc: Matteo Croce <technoboy85 at gmail.com>
> Fixes: 69c51582f ("dpif-netlink: don't allocate per thread netlink sockets")
> Signed-off-by: Aaron Conole <aconole at redhat.com>
> Signed-off-by: Mark Gray <mark.d.gray at redhat.com>
> v2: Oops - forgot to commit my whitespace cleanups.
> v3: one port latency results as per Matteo's comments
> min/avg/max/mdev = 21.5/36.5/96.5/1.0 us
> With Patch:
> min/avg/max/mdev = 5.3/9.7/98.4/0.5 us
> v4: Oops - first email did not send
> v5: min/avg/max/mdev = 9.3/10.4/33.6/2.2 us
> v6: Split out the AUTHORS patch and added a cover letter as
> per Ilya's suggestion.
> Fixed 0-day issues.
> v7: Merged patches as per Flavio's suggestion. This is
> no longer a series. Fixed some other small issues.
> v9: Udated based on feedback from Ilya. Also implemented
> another suggestion by Flavio
> min/avg/max/mdev = 5.5/11.0/83.2/0.8 us
More information about the dev