[ovs-dev] [PATCH 2/2] netdev-afxdp: Add need_wakeup supprt.
William Tu
u9012063 at gmail.com
Wed Sep 4 13:50:15 UTC 2019
Hi Eelco,
Thanks for your testing and review.
On Wed, Sep 4, 2019 at 1:04 AM Eelco Chaudron <echaudro at redhat.com> wrote:
>
>
>
> On 27 Aug 2019, at 1:02, William Tu wrote:
>
> > The patch adds support for using need_wakeup flag in AF_XDP rings.
> > When this flag is used, it means that OVS has to explicitly wake
> > up the kernel RX, using poll() syscall and wake up TX, using sendto()
> > syscall. This feature improves the performance by avoiding unnecessary
> > syscalls, so keeping more CPU time in userspace to process packets.
> >
> > On Intel Xeon E5-2620 v3 2.4GHz system, performance of physical port
> > to physical port improves from 6.1Mpps to 7.3Mpps.
>
> Did some more testing and with PVP I see a performance decrease, with
> physical to physical I see an increase.
> Tests are performed with a port redirect open flow rule on an ixgbe
> (Xeon E5-2690 v4 2.60GHz):
>
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | PVP | Number of flows | 64 | 128 | 256 | 512 |
> 768 | 1024 | 1514 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | master | 1000 | 737896 | 700643 | 682915 | 648386 |
> 621792 | 582821 | 527899 |
> | Patch | 1000 | 734179 | 696515 | 676712 | 646536 |
> 619600 | 578546 | 519965 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
>
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | Port2Port | Number of flows | 64 | 128 | 256 | 512 |
> 768 | 1024 | 1514 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | master | 1000 | 3351114 | 3236581 | 3143710 | 2349598 |
> 1586276 | 1197304 | 814854 |
> | Patch | 1000 | 3571733 | 3488697 | 3448908 | 2349593 |
> 1586276 | 1197304 | 814854 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
>
> Did not research why PVP is slower, maybe related to the TAP interface
> with AF_XDP?
>
I haven't tried PVP with this feature.
Maybe for virtual device, we don't need "need_wakeup" feature.
Let me investigate more.
> See one small comment inline…
>
>
> > Suggested-by: Eelco Chaudron <echaudro at redhat.com>
> > Signed-off-by: William Tu <u9012063 at gmail.com>
> > ---
> > acinclude.m4 | 5 +++++
> > lib/netdev-afxdp.c | 51
> > +++++++++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 56 insertions(+)
> >
> > diff --git a/acinclude.m4 b/acinclude.m4
> > index 116ffcf9096d..8821b821ec3c 100644
> > --- a/acinclude.m4
> > +++ b/acinclude.m4
> > @@ -276,6 +276,11 @@ AC_DEFUN([OVS_CHECK_LINUX_AF_XDP], [
> > [Define to 1 if AF_XDP support is available and
> > enabled.])
> > LIBBPF_LDADD=" -lbpf -lelf"
> > AC_SUBST([LIBBPF_LDADD])
> > +
> > + AC_CHECK_DECL([xsk_ring_prod__needs_wakeup], [
> > + AC_DEFINE([HAVE_XDP_NEED_WAKEUP], [1],
> > + [XDP need wakeup support detected in xsk.h.])
> > + ], [], [#include <bpf/xsk.h>])
> > fi
> > AM_CONDITIONAL([HAVE_AF_XDP], test "$AF_XDP_ENABLE" = true)
> > ])
> > diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c
> > index e5b058d08a09..d14d100e8fa3 100644
> > --- a/lib/netdev-afxdp.c
> > +++ b/lib/netdev-afxdp.c
> > @@ -26,6 +26,7 @@
> > #include <linux/rtnetlink.h>
> > #include <linux/if_xdp.h>
> > #include <net/if.h>
> > +#include <poll.h>
> > #include <stdlib.h>
> > #include <sys/resource.h>
> > #include <sys/socket.h>
> > @@ -117,6 +118,48 @@ struct xsk_socket_info {
> > atomic_uint64_t tx_dropped;
> > };
> >
> > +#ifdef HAVE_XDP_NEED_WAKEUP
> > +static inline void
> > +xsk_rx_need_wakeup(struct xsk_umem_info *umem,
> > + struct netdev *netdev, int fd) {
> > + struct pollfd pfd;
> > + int ret;
> > +
> > + if (xsk_ring_prod__needs_wakeup(&umem->fq)) {
> > + pfd.fd = fd;
> > + pfd.events = POLLIN;
> > +
> > + ret = poll(&pfd, 1, 1000);
>
> Think we should call the poll with a 0 delay as the kernel should have
> something waiting. If for some reason it doesn’t we should not wait
> for a second.
>
> I ran the patch with a 0 timeout, and see no performance differences, or
> any of the below warning messages.
>
Agree,
Thanks!
William
More information about the dev
mailing list