[ovs-dev] [PATCH] Use TPACKET_V1/V2/V3 to accelerate veth for DPDK datapath

[William Tu]

On Tue, Jan 21, 2020 at 11:25 AM Ben Pfaff <blp at ovn.org> wrote:
>
> On Tue, Jan 21, 2020 at 02:49:47AM -0500, yang_y_yi at 163.com wrote:
> > From: Yi Yang <yangyi01 at inspur.com>
> >
> > We can avoid high system call overhead by using TPACKET_V1/V2/V3
> > and use DPDK-like poll to receive and send packets (Note: send
> > still needs to call sendto to trigger final packet transmission).
> >
> > I can see about 30% improvement compared to last recvmmsg
> > optimization if I use TPACKET_V3. TPACKET_V1/V2 is worse than
> > TPACKET_V3, but it still can improve about 20%.
> >
> > For veth, it is 1.47 Gbps before this patch, it is about 1.98
> > Gbps after applied this patch. But it is about 4.00 Gbps if we
> > use af_packet for veth, the bottle neck lies in ovs-vswitchd
> > thread, it will handle too many things for every loop (as below)
> > , so it can't work very efficintly as pmd_thread.
> >
> >         memory_run();
> >         bridge_run();
> >         unixctl_server_run(unixctl);
> >         netdev_run();
> >
> >         memory_wait();
> >         bridge_wait();
> >         unixctl_server_wait(unixctl);
> >         netdev_wait();
> >         poll_block();
> >
> > In the next step, it will be better if let pmd_thread to handle
> > tap and veth interface.
> >
> > Signed-off-by: Yi Yang <yangyi01 at inspur.com>
> > Co-authored-by: William Tu <u9012063 at gmail.com>
> > Signed-off-by: William Tu <u9012063 at gmail.com>
>
> Thanks for the patch!
>
> I am a bit concerned about version compatibility issues here.  There are
> two relevant kinds of versions.  The first is the version of the
> kernel/library headers.  This patch works pretty hard to adapt to the
> headers that are available at compile time, only dealing with the
> versions of the protocols that are available from the headers.  This
> approach is sometimes fine, but an approach can be better is to simply
> declare the structures or constants that the headers lack.  This is
> often pretty easy for Linux data structures.  OVS does this for some
> structures that it cares about with the headers in ovs/include/linux.
> This approach has two advantages: the OVS code (outside these special
> declarations) doesn't have to care whether particular structures are
> declared, because they are always declared, and the OVS build always
> supports a particular feature regardless of the headers of the system on
> which it was built.
>
> The second kind of version is the version of the system that OVS runs
> on.  Unless a given feature is one that is supported by every version
> that OVS cares about, OVS needs to test at runtime whether the feature
> is supported and, if not, fall back to the older feature.  I don't see
> that in this code.  Instead, it looks to me like it assumes that if the
> feature was available at build time, then it is available at runtime.
> This is not a good way to do things, since we want people to be able to
> get builds from distributors such as Red Hat or Debian and then run
> those builds on a diverse collection of kernels.
>
Hi Yiyang,

Can we just implement TPACKET v3, and drop v2 and v1?
V3 is supported since kernel 3.10,

commit f6fb8f100b807378fda19e83e5ac6828b638603a
Author: chetan loke <loke.chetan at gmail.com>
Date:   Fri Aug 19 10:18:16 2011 +0000

    af-packet: TPACKET_V3 flexible buffer implementation.

and based on OVS release
http://docs.openvswitch.org/en/latest/faq/releases/
after OVS 2.12, the minimum kernel requirement is 3.10.

Regards,
William