[ovs-discuss] ovs-vswitchd with crashes when adding a dpdk port

[Daniele Di Proietto]

On 9/26/14, 6:37 AM, "Daniel Badea" <daniel.badea at windriver.com> wrote:

>I finished writing the environment setup script following instructions
>from
>https://urldefense.proofpoint.com/v1/url?u=https://raw.githubusercontent.c
>om/openvswitch/ovs/master/INSTALL.DPDK&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A
>&r=MV9BdLjtFIdhBDBaw5z%2BU6SSA2gAfY4L%2F1HCy3VjlKU%3D%0A&m=hYxIZRKz2Tu6k7%
>2BbB88Xr821Qw1o9%2Fi9lioyfeoSSTM%3D%0A&s=521991b1c487f0cd9d612d478cf6247df
>14e5c19b49ff6c2b3973ec08f558443 and
>the results are:
>  - ovs still not building against dpdk head (version 1.7.1) because:
>const
>struct rte_mbuf' has no member named 'pkt'

Dpdk master is already version 1.8
(http://dpdk.org/browse/dpdk/tree/lib/librte_eal/common/include/rte_version
.h). I advise you to use 1.7.1, which is the last released version. We
will support dpdk 1.8 later, when it will be released.

>  - ovs builds against dpdk v1.7.0
>  - ovs-vswitchd starts and creates dpdk ports without crashing
>  - able to run iperf through the dpdk port between the ovs with dpdk and
>another Ubuntu machine where the NIC is handled by the kernel (ixgbe
>module). Results: 550 Mbits/sec on a 10G link

Few suggestions here:

- Make sure to compile OVS with optimizations
- We do not support DPDK KNI yet, so the internal port uses a tap
interface (which is not particularly fast). If you just want to measure
the throughput between two DPDK interfaces, you should just setup an
appropriate flow table (INSTALL.DPDK contains instruction about how to do
that). Otherwise each packet that hits the internal port will slow down
the entire system considerably.
- I've never used iperf, but, from my experience, software packets
generators might not be able to saturate a 10G link, especially with small
packets (notable exceptions: Pktgen DPDK, netmap pkt-gen).

>
>So I was not able to reproduce the crash after following the new set of
>instructions. I was probably repeatedly doing something wrong, but can't
>pinpoint the problem. I thought it was an issue caused by the number of
>cores or memory channels but it's not, I've tried with multiple
>combinations
>and everything is ok. Sorry for the false alarm.

No false alarm, the crash you've been experiencing should have been fixed
by this 
(https://github.com/openvswitch/ovs/commit/1b7a04e05b6fd15fb4caa7ec20d09ba7
aa73399f) commit. I'm glad that you can confirm it's fixed.

>
>What is the expected release date for OVS 2.4? (I'm interested in the dpdk
>vhost support)

OVS 2.4 will likely be released before the end of the year. Hopefully we
should have dpdk vhost support merged sooner.

>
>Thanks,
>Daniel

Thanks,

Daniele

>
>-----Original Message-----
>From: Daniele Di Proietto [mailto:ddiproietto at vmware.com]
>Sent: Thursday, September 25, 2014 11:14 PM
>To: Daniel Badea
>Cc: discuss at openvswitch.org; Alex Wang
>Subject: Re: [ovs-discuss] ovs-vswitchd with crashes when adding a dpdk
>port
>
>Hi Daniel,
>
>On 9/25/14, 10:34 AM, "Daniel Badea" <daniel.badea at windriver.com> wrote:
>
>>I encountered the following issue: ovs-vswitchd with crashes when
>>adding a dpdk port.
>> 
>>Building ovs with dpdk support is not straightforward because:
>>  - ovs doest not build with the latest dpdk version 1.7.1
>
>We're regularly building with dpdk 1.7.1 (dpdk 1.7.0 is also supported).
>Can you please tell us the error that you're experiencing, so that we can
>try to fix it?
>
>>  - the document here
>>https://urldefense.proofpoint.com/v1/url?u=http://git.openvswitch.org/cgi
>>-bin/gitweb.cgi?p%3Dopenvswitch%3Ba%3Dblob_plai&k=oIvRg1%2BdGAgOoM1BIlLLq
>>w%3D%3D%0A&r=MV9BdLjtFIdhBDBaw5z%2BU6SSA2gAfY4L%2F1HCy3VjlKU%3D%0A&m=hYxI
>>ZRKz2Tu6k7%2BbB88Xr821Qw1o9%2Fi9lioyfeoSSTM%3D%0A&s=c60dab2aafd9b2a16ab69
>>aea0575cfcbb680e0fd8de9f0a6ea757a13a4c6af1e
>>n;f =INSTALL.DPDK;hb=HEAD refers to dpdk 1.6.0r2 but that doesn't work
>>either.
>
>That's an old version of the file. Please read INSTALL.DPDK in the source
>tree (or
>https://urldefense.proofpoint.com/v1/url?u=https://raw.githubusercontent.c
>om/openvswitch/ovs/master/INSTALL.DPDK&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A
>&r=MV9BdLjtFIdhBDBaw5z%2BU6SSA2gAfY4L%2F1HCy3VjlKU%3D%0A&m=hYxIZRKz2Tu6k7%
>2BbB88Xr821Qw1o9%2Fi9lioyfeoSSTM%3D%0A&s=521991b1c487f0cd9d612d478cf6247df
>14e5c19b49ff6c2b3973ec08f558443)
>
>> 
>>So I checked out dpdk 1.7.0 did some minor changes (combine libs and
>>kcompat fix for kernel 3.13.0) Now ovs builds ok but:
>>  - if the database is empty then ovs-vswitchd starts but crashes on
>>ovs-vsctl add-port ... type=dpdk
>>  - if the database already contains dpdk port configuration then
>>ovs-vswitchd crashes at startup
>> 
>>Logs seem to indicate a numa socket = -1 problem.
>> 
>>Any ideea how to fix his?
>
>We've managed to reproduce the crash on a single node numa machine. Alex
>is
>about to post a fix to the mailing list.
>
>
>Thank you for the detailed report!
>
>Daniele
>