[ovs-dev] [ovs-dev, PATCHv6] netdev-afxdp: add new netdev type for AF_XDP.
Ilya Maximets
i.maximets at samsung.com
Tue Apr 30 13:14:47 UTC 2019
On 27.04.2019 16:28, William Tu wrote:
> > if WIN32
> > lib_libopenvswitch_la_SOURCES += \
> > diff --git a/lib/dp-packet.c b/lib/dp-packet.c
> > index 0976a35e758b..a61552f72988 100644
> > --- a/lib/dp-packet.c
> > +++ b/lib/dp-packet.c
> > @@ -22,6 +22,9 @@
> > #include "netdev-dpdk.h"
> > #include "openvswitch/dynamic-string.h"
> > #include "util.h"
> > +#ifdef HAVE_AF_XDP
> > +#include "xdpsock.h"
> > +#endif
> >
> > static void
> > dp_packet_init__(struct dp_packet *b, size_t allocated, enum dp_packet_source source)
> > @@ -122,6 +125,16 @@ dp_packet_uninit(struct dp_packet *b)
> > * created as a dp_packet */
> > free_dpdk_buf((struct dp_packet*) b);
> > #endif
> > + } else if (b->source == DPBUF_AFXDP) {
> > +#ifdef HAVE_AF_XDP
> > + struct dp_packet_afxdp *xpacket;
> > +
> > + xpacket = dp_packet_cast_afxdp(b);
> > + if (xpacket->mpool) {
> > + umem_elem_push(xpacket->mpool, dp_packet_base(b));
> > + }
> > +#endif
>
> Why not making the same trick as we have for DPDK few lines above?
> i.e. wrap this part in a function like 'free_afxdp_buf' and move it
> to the netdev-afxdp.c ? You will not need to expose so many internals
> to generic code. dp_packet_cast_afxdp() will also be moved there along
> with 'struct dp_packet_afxdp'.
>
> BTW, I hope, someday, I'll finally implement 'dp-packet-memory-provider'
> abstraction for OVS.
>
>
> Hi Ilya,
>
> Can you share more detail about this idea, dp-packet-memory-provider?
> Why do we need it?
Hi.
OVS uses way too different memory sources for 'dp-packet's. They are defined
in 'enum dp_packet_source'. Here is some list of issues (in my opinion) we
have:
* Some of dp-packet APIs are independent from the memory source, but all the
APIs that able to alloc/free/resize depends on the memory source and includes
all these ugly switches/ifs guarded by ifdefs to do completely different
things on packets from different sources.
* Some specific management of memory pools resides inside implementations of
netdevs, which is not good from the point of OVS design. This is moslty
about managing DPDK memory pools, but the same is applicable to your umem
memory pools. There was also attempt to implement netmap based netdev and
there was own memory pool implementation too.
* We even have DPDK specific code in the generic datapath code to prevent
having packets with different memory sources in a single batch.
* netdevs that requires specific memory sources (DPDK) has to reallocate and
copy all the packets that has different source. And this is done on netdev
level while handling send().
* 'dp_packet_new()' is able to allocate new packet only with DPBUF_MALLOC.
This triggers the previous issue with re-allocating on send. This is really
bad thing because 'dp_packet_new()' used by 'dp_packet_clone()'. i.e. to
clone dp-packet with DPDK source we're reallocating it on heap and again
re-allocating it to send to DPDK netdev.
* netdevs usually new packets with dp_packet_new() on malloced memory even
if they have no strict requirements about memory sources.
For example, netdev-linux is not able to receive packets to dp-packet
with DPBUF_DPDK/AFXDP source.
* Probably, there are more issues that I don't remember right now.
Solution is to have 'dp-packet-memory-provider' abstraction layer:
* Each memory-provider will implement specific alloc/free/etc functions.
* Each netdev might have preferred memory-provider that must be used while
sending packets to it.
* Datapath will be able to make decision about dp-packet reallocations in
case of memory-provider mismatch on a higher level decreasing the munber
of reallocations.
* Packet clone could take into account the memory-provider of the original
packet with ability to reallocate from the same provider.
* We might have globally preferred memory provider for all cases where
memory source is not important. For example, we could use dpdk memory
provider for packets received from netdev-linux to not re-allocate them
on send().
* Memory pools' management will be encapsulated inside specific providers.
* It might be possible even for AFXDP netdev to use DPDK mempools created
on top of memory shared with kernel. (just an idea)
Drawbacks:
* There are always performance concerns because this will have significant
impact on the hot path --> could produce performance issues. Extensive
testing required.
Best regards, Ilya Maximets.
More information about the dev
mailing list