[ovs-dev] [RFC v6 08/11] dp-packet: Handle multi-seg mbufs in resize__().

[Tiago Lam]

When enabled with DPDK OvS relies on mbufs allocated by mempools to
receive and output data on DPDK ports. Until now, each OvS dp_packet has
only one mbuf associated, which is allocated with the maximum possible
size, taking the MTU into account. This approach, however, doesn't allow
us to increase the allocated size in an mbuf, if needed, since an mbuf
is allocated and initialised upon mempool creation. Thus, in the current
implementatin this is dealt with by calling OVS_NOTEACHED() and
terminating OvS.

To avoid this, and allow the allocated size to be increased, multiple
mbufs can be linked together, in the multi-segment mbufs approach. Thus,
dp_packet_resize__() has been modified to handle the DPBUF_DPDK case and
allocate and link new mbufs together as needed.

Signed-off-by: Tiago Lam <tiago.lam at intel.com>
---
 lib/dp-packet.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 76 insertions(+), 1 deletion(-)

diff --git a/lib/dp-packet.c b/lib/dp-packet.c
index 1a666f2..8d2b313 100644
--- a/lib/dp-packet.c
+++ b/lib/dp-packet.c
@@ -238,8 +238,83 @@ dp_packet_resize__(struct dp_packet *b, size_t new_headroom, size_t new_tailroom
 
     switch (b->source) {
     case DPBUF_DPDK:
-        OVS_NOT_REACHED();
+    {
+        uint32_t miss_len;
+        uint16_t max_data_len, nb_segs;
+        int16_t head_diff = 0;
+        struct rte_mbuf *mbuf, *fmbuf;
+        struct rte_mempool *mp;
+
+        if (!netdev_dpdk_is_multi_segment_mbufs_enabled()) {
+            /* XXX: Handle single mbufs case better */
+            return;
+        }
+
+        /* Deal with headroom. Find out how much headroom we have, if any */
+        if (new_headroom != dp_packet_headroom(b)) {
+            head_diff = dp_packet_headroom(b) - new_headroom;
+        }
+
+        mbuf = CONST_CAST(struct rte_mbuf *, &b->mbuf);
+        /* Calculate missing length in need of allocation */
+        if (head_diff < 0) {
+            miss_len = new_allocated - head_diff - mbuf->pkt_len;
+        } else if (head_diff > 0){
+            miss_len = MIN(0, new_allocated - dp_packet_headroom(b)
+                - mbuf->pkt_len);
+        } else {
+            miss_len = new_allocated - mbuf->pkt_len;
+        }
+
+        /* All new allocated mbuf's max data len is the same */
+        max_data_len = mbuf->buf_len - mbuf->data_off;
+        /* Calculate # of needed mbufs to accomodate 'missing' */
+        nb_segs = miss_len / max_data_len;
+        if (miss_len % max_data_len) {
+            nb_segs += 1;
+        }
 
+        /* Proceed with the allocation of new mbufs */
+        mp = mbuf->pool;
+        fmbuf = mbuf;
+        mbuf = rte_pktmbuf_lastseg(mbuf);
+
+        for (int i = 0; i < nb_segs; i++) {
+            /* This takes care of initialising buf_len, data_len and other
+             * fields properly */
+            mbuf->next = rte_pktmbuf_alloc(mp);
+            if (!mbuf->next) {
+                /* XXX: Flag a WARN and return an error as this will likely
+                 * result in undefined behaviour */
+                rte_pktmbuf_free(fmbuf);
+                fmbuf = NULL;
+                break;
+            }
+
+            fmbuf->nb_segs += 1;
+        }
+
+        /* Deal with headroom. If 'new_headroom' is bigger than available mbuf
+         * RTE_PKTMBUF_HEADROOM (defaults to 128B), the 'head_diff' calculated
+         * above will be negative and be used for shifting the data right.
+         * Otherwise 'head_diff' will be positive and will be used for shifting
+         * the data left.
+         * XXX: Doesn't verify or take into account that new_headroom may fall
+         * outside of an mbuf limits */
+        if (head_diff > 0) {
+            new_headroom = new_headroom;
+
+            dp_packet_shift(b, -head_diff);
+        } else if (head_diff < 0) {
+            new_headroom = dp_packet_headroom(b) + -head_diff;
+
+            dp_packet_shift(b, -head_diff);
+        }
+
+        new_base = dp_packet_base(b);
+
+        break;
+    }
     case DPBUF_MALLOC:
         if (new_headroom == dp_packet_headroom(b)) {
             new_base = xrealloc(dp_packet_base(b), new_allocated);
-- 
2.7.4