[ovs-dev] [PATCH V3] netdev-dpdk: Add Jumbo Frame Support.

[Flavio Leitner]

On Mon, 8 Feb 2016 10:54:08 +0000
"Kavanagh, Mark B" <mark.b.kavanagh at intel.com> wrote:

> >
> >Hi,
> >
> >Some comments below.
> >  
> 
> Hi Flavio,
> 
> Thanks for your review!
> 
> I've responded to your comments inline - I've also addressed the
> cosmetic changes that you suggested in V3 of the patch.
> 
> Thanks again,
> Mark
> 
> >
> >On Mon,  1 Feb 2016 10:18:54 +0000
> >Mark Kavanagh <mark.b.kavanagh at intel.com> wrote:
> >  
> >> Add support for Jumbo Frames to DPDK-enabled port types,
> >> using single-segment-mbufs.
> >>
> >> Using this approach, the amount of memory allocated for each mbuf
> >> to store frame data is increased to a value greater than 1518B
> >> (typical Ethernet maximum frame length). The increased space
> >> available in the mbuf means that an entire Jumbo Frame can be
> >> carried in a single mbuf, as opposed to partitioning it across
> >> multiple mbuf segments.
> >>
> >> The amount of space allocated to each mbuf to hold frame data is
> >> defined dynamically by the user when adding a DPDK port to a
> >> bridge. If an MTU value is not supplied, or the user-supplied
> >> value is invalid, the MTU for the port defaults to standard
> >> Ethernet MTU (i.e. 1500B).
> >>
> >> Signed-off-by: Mark Kavanagh <mark.b.kavanagh at intel.com>
> >> ---
> >>  INSTALL.DPDK.md   |  59 +++++++++-
> >>  NEWS              |   2 +
> >>  lib/netdev-dpdk.c | 347
> >> +++++++++++++++++++++++++++++++++++++++++------------- 3 files
> >> changed, 328 insertions(+), 80 deletions(-)
[...]

> >> +When Jumbo Frames are enabled, the size of a DPDK port's mbuf
> >> segments are +increased, such that a full Jumbo Frame may be
> >> accommodated inside a single +mbuf segment. Once set, the MTU for a
> >> DPDK port is immutable. +  
> >
> >I think we need some protection here because if someone tries to
> >change that, it will segfault PMD threads.
> >  
> 
> Could you elaborate on this a bit more; do you mean that PMD threads
> would segfault if a user attempts to change the MTU of a running DPDK
> port? If so, how would that be possible (AFAIA, there's no way to
> programmatically modify the MTU of OVS ports, outside of standard
> Linux utilities such as ifconfig, which DPDK ports typically aren't
> subject to; there's also AFAIA no way to programmatically modify MTU
> of DPDK ports outside of modifying the source code).

I have a ovs bridge with two dpdk ports created with these commands:

# ovs-vsctl add-port ovsbr0 dpdk0 \
        -- set interface dpdk0 type=dpdk options:dpdk-mtu=9000 ofport_request=10
# ovs-vsctl add-port ovsbr0 dpdk1 \
        -- set interface dpdk1 type=dpdk options:dpdk-mtu=9000 ofport_request=11

Now I am pushing traffic (jumbo frames or small frames) and then
I change dpdk-mtu to 1500.

# ovs-vsctl set interface dpdk1 options:dpdk-mtu=1500

Then I see this:
2016-02-12T13:19:20.142Z|00003|daemon_unix(monitor)|ERR|1 crashes: 
pid 78009 died, killed (Segmentation fault), core dumped, restarting
...

This is the backtrace:
#0  0x00007f1909cf4423 in _recv_raw_pkts_vec () from /lib64/libdpdk.so
#1  0x00000000004fd595 in rte_eth_rx_burst (nb_pkts=32, rx_pkts=0x7f18c37fd970, queue_id=2, 
    port_id=1 '\001') at /root/dpdk/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:2510
#2  netdev_dpdk_rxq_recv (rxq_=<optimized out>, packets=0x7f18c37fd970, c=0x7f18c37fd96c)
    at ../lib/netdev-dpdk.c:1216
#3  0x000000000047c3c1 in netdev_rxq_recv (rx=<optimized out>, buffers=buffers at entry=0x7f18c37fd970, 
    cnt=cnt at entry=0x7f18c37fd96c) at ../lib/netdev.c:654
#4  0x000000000045e566 in dp_netdev_process_rxq_port (pmd=pmd at entry=0x1e1cd90, rxq=<optimized out>, 
    port=<optimized out>, port=<optimized out>) at ../lib/dpif-netdev.c:2555
#5  0x000000000045e9f9 in pmd_thread_main (f_=0x1e1cd90) at ../lib/dpif-netdev.c:2691
#6  0x00000000004bab64 in ovsthread_wrapper (aux_=<optimized out>) at ../lib/ovs-thread.c:340
#7  0x00007f1909583555 in start_thread (arg=0x7f18c37fe700) at pthread_create.c:333
#8  0x00007f1908daeb9d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109


Dump of assembler code for function _recv_raw_pkts_vec:
   0x00007fa8d51ccf30 <+0>:     push   %r15
   0x00007fa8d51ccf32 <+2>:     mov    %edx,%r15d
   0x00007fa8d51ccf35 <+5>:     push   %r14
   0x00007fa8d51ccf37 <+7>:     push   %r13
   0x00007fa8d51ccf39 <+9>:     push   %r12
   0x00007fa8d51ccf3b <+11>:    push   %rbp
   0x00007fa8d51ccf3c <+12>:    push   %rbx
   0x00007fa8d51ccf3d <+13>:    sub    $0x38,%rsp
=> 0x00007fa8d51ccf41 <+17>:    movzbl 0x69(%rdi),%eax

rdi            0x0      0

Looks like dev->data->rx_queues[queue_id] is NULL.

-- 
fbl