[ovs-dev] [PATCH 2/2] netdev-afxdp: Add need_wakeup supprt.

William Tu u9012063 at gmail.com
Wed Sep 4 13:50:15 UTC 2019


Hi Eelco,

Thanks for your testing and review.

On Wed, Sep 4, 2019 at 1:04 AM Eelco Chaudron <echaudro at redhat.com> wrote:
>
>
>
> On 27 Aug 2019, at 1:02, William Tu wrote:
>
> > The patch adds support for using need_wakeup flag in AF_XDP rings.
> > When this flag is used, it means that OVS has to explicitly wake
> > up the kernel RX, using poll() syscall and wake up TX, using sendto()
> > syscall. This feature improves the performance by avoiding unnecessary
> > syscalls, so keeping more CPU time in userspace to process packets.
> >
> > On Intel Xeon E5-2620 v3 2.4GHz system, performance of physical port
> > to physical port improves from 6.1Mpps to 7.3Mpps.
>
> Did some more testing and with PVP I see a performance decrease, with
> physical to physical I see an increase.
> Tests are performed with a port redirect open flow rule on an ixgbe
> (Xeon E5-2690 v4 2.60GHz):
>
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> |  PVP      | Number of flows |   64    |   128   |   256   |   512   |
>   768   |   1024  |  1514  |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | master    |            1000 |  737896 |  700643 |  682915 |  648386 |
> 621792 |  582821 | 527899 |
> | Patch     |            1000 |  734179 |  696515 |  676712 |  646536 |
> 619600 |  578546 | 519965 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
>
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | Port2Port | Number of flows |   64    |   128   |   256   |   512   |
>   768   |  1024   |  1514  |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
> | master    |            1000 | 3351114 | 3236581 | 3143710 | 2349598 |
> 1586276 | 1197304 | 814854 |
> | Patch     |            1000 | 3571733 | 3488697 | 3448908 | 2349593 |
> 1586276 | 1197304 | 814854 |
> +-----------+-----------------+---------+---------+---------+---------+---------+---------+--------+
>
> Did not research why PVP is slower, maybe related to the TAP interface
> with AF_XDP?
>
I haven't tried PVP with this feature.
Maybe for virtual device, we don't need "need_wakeup" feature.
Let me investigate more.

> See one small comment inline…
>
>
> > Suggested-by: Eelco Chaudron <echaudro at redhat.com>
> > Signed-off-by: William Tu <u9012063 at gmail.com>
> > ---
> >  acinclude.m4       |  5 +++++
> >  lib/netdev-afxdp.c | 51
> > +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 56 insertions(+)
> >
> > diff --git a/acinclude.m4 b/acinclude.m4
> > index 116ffcf9096d..8821b821ec3c 100644
> > --- a/acinclude.m4
> > +++ b/acinclude.m4
> > @@ -276,6 +276,11 @@ AC_DEFUN([OVS_CHECK_LINUX_AF_XDP], [
> >                [Define to 1 if AF_XDP support is available and
> > enabled.])
> >      LIBBPF_LDADD=" -lbpf -lelf"
> >      AC_SUBST([LIBBPF_LDADD])
> > +
> > +    AC_CHECK_DECL([xsk_ring_prod__needs_wakeup], [
> > +      AC_DEFINE([HAVE_XDP_NEED_WAKEUP], [1],
> > +        [XDP need wakeup support detected in xsk.h.])
> > +    ], [], [#include <bpf/xsk.h>])
> >    fi
> >    AM_CONDITIONAL([HAVE_AF_XDP], test "$AF_XDP_ENABLE" = true)
> >  ])
> > diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c
> > index e5b058d08a09..d14d100e8fa3 100644
> > --- a/lib/netdev-afxdp.c
> > +++ b/lib/netdev-afxdp.c
> > @@ -26,6 +26,7 @@
> >  #include <linux/rtnetlink.h>
> >  #include <linux/if_xdp.h>
> >  #include <net/if.h>
> > +#include <poll.h>
> >  #include <stdlib.h>
> >  #include <sys/resource.h>
> >  #include <sys/socket.h>
> > @@ -117,6 +118,48 @@ struct xsk_socket_info {
> >      atomic_uint64_t tx_dropped;
> >  };
> >
> > +#ifdef HAVE_XDP_NEED_WAKEUP
> > +static inline void
> > +xsk_rx_need_wakeup(struct xsk_umem_info *umem,
> > +                   struct netdev *netdev, int fd) {
> > +    struct pollfd pfd;
> > +    int ret;
> > +
> > +    if (xsk_ring_prod__needs_wakeup(&umem->fq)) {
> > +        pfd.fd = fd;
> > +        pfd.events = POLLIN;
> > +
> > +        ret = poll(&pfd, 1, 1000);
>
> Think we should call the poll with a 0 delay as the kernel should have
> something waiting. If for some reason it doesn’t we should not wait
> for a second.
>
> I ran the patch with a 0 timeout, and see no performance differences, or
> any of the below warning messages.
>

Agree,
Thanks!
William


More information about the dev mailing list