[ovs-discuss] Crash in openvswitch 2.0.2

Marco Kuendig marco at nuvula.ch
Tue Mar 31 19:52:00 UTC 2015


that brought us a step forward. thank Sab.

Important to know is:

I got 4 kvm servers, meshed with openvswitch. I use vxlan for tunnelling.

Sometimes when I restart a domain in kvm, 3 or 4 hosts crash at the same time.

I have STP enabled to avoid loops.


this is the output now:

root at nuv-vir-kvm-server-1 ~ # gdb /usr/sbin/ovs-vswitchd /var/crash/ovs/CoreDump
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/ovs-vswitchd...Reading symbols from /usr/lib/debug//usr/sbin/ovs-vswitchd...done.
done.
[New LWP 32725]
[New LWP 32732]
[New LWP 32726]
[New LWP 32730]
[New LWP 32727]
[New LWP 32728]
[New LWP 32729]
[New LWP 32731]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfi'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  nl_attr_get_size (nla=nla at entry=0x0) at ../lib/netlink.c:506
506	../lib/netlink.c: No such file or directory.


root at nuv-vir-kvm-server-1 ~ # crash /usr/sbin/ovs-vswitchd /var/crash/ovs/CoreDump

crash 7.0.3
Copyright (C) 2002-2013  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.


crash: /usr/sbin/ovs-vswitchd: no debugging data available

root at nuv-vir-kvm-server-1 ~ # ll /var/crash/ovs/
Architecture         Date                 ExecutableTimestamp  ProcCwd              ProcStatus           UserGroups
CoreDump             DistroRelease        ProblemType          ProcEnviron          Signal
CrashCounter         ExecutablePath       ProcCmdline          ProcMaps             Uname

 <http://www.nuvula.ch/>
Marco Kuendig / CEO / Founder 
marco at nuvula.ch <mailto:marco at nuvula.ch> / +41 78 751 99 71

Marco's Google Hangout <https://plus.google.com/hangouts/_/nuvula.ch/marco>
Nuvula AG - Hybrid Clouds 
Weierbachstrasse 7b 8193 Eglisau Switzerland 
http://www.nuvula.ch <http://www.nuvula.ch/>
> On 31 Mar 2015, at 21:45, Sabyasachi Sengupta <Sabyasachi.Sengupta at alcatel-lucent.com> wrote:
> 
> 
> Typically Ubuntu does not unpack the crashes. Can you try apport-unpack?
> # apport-unpack /var/crash/<name> <crash-dir>
> 
> On Tue, 31 Mar 2015, Marco Kuendig wrote:
> 
>> thanks Joe and Ben
>> have done:
>> 1. installed dgb symbols for kernel....doesn't help
>> 2. installed debug symbols for openvswitch
>> no change, gdb and crash still don't work for me. I'm not a dev, need
>> more help to get that backtrace done.
>> here some output:
>> root at nuv-vir-kvm-server-1 ~ # crash
>>  /usr/lib/debug/boot/vmlinux-3.13.0-48-generic
>> /var/crash/_usr_sbin_ovs-vswitchd.0.crash
>> crash 7.0.3
>> Copyright (C) 2002-2013  Red Hat, Inc.
>> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>> Copyright (C) 1999-2006  Hewlett-Packard Co
>> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>> Copyright (C) 2005, 2011  NEC Corporation
>> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>> This program is free software, covered by the GNU General Public License,
>> and you are welcome to change it and/or distribute copies of it under
>> certain conditions.  Enter "help copying" to see the conditions.
>> This program has absolutely no warranty.  Enter "help warranty" for
>> details.
>> crash: /var/crash/_usr_sbin_ovs-vswitchd.0.crash: not a supported file
>> format
>> Usage:
>> 
>>   crash [OPTION]... NAMELIST MEMORY-IMAGE  (dumpfile form)
>>   crash [OPTION]... [NAMELIST]             (live system form)
>> Enter "crash -h" for details.
>> root at nuv-vir-kvm-server-1 ~ # gdb /usr/sbin/ovs-vswitchd
>> /var/crash/_usr_sbin_ovs-vswitchd.0.crash
>> GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
>> Copyright (C) 2014 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show
>> copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-linux-gnu".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word"...
>> Reading symbols from /usr/sbin/ovs-vswitchd...Reading symbols from
>> /usr/lib/debug//usr/sbin/ovs-vswitchd...done.
>> done.
>> "/var/crash/_usr_sbin_ovs-vswitchd.0.crash" is not a core dump: File
>> format not recognized
>> (gdb) q
>> root at nuv-vir-kvm-server-1 ~ #
>> Nuvula AG
>> Marco Kuendig / CEO / Founder marco at nuvula.ch <mailto:marco at nuvula.ch> / +41 78 751 99 71
>> Marco's Google Hangout
>> Nuvula AG - Hybrid Clouds Weierbachstrasse 7b 8193 Eglisau Switzerland http://www.nuvula.ch
>> 
>>      On 31 Mar 2015, at 19:00, Joe Stringer
>>      <joestringer at nicira.com> wrote:
>> For the 'File format not recognized' problem, you might have better
>> luck with the 'crash' utility.
>> $ crash <binary> <crashdump>
>> On 31 March 2015 at 08:16, Marco Kuendig <marco at nuvula.ch> wrote:
>>      Have tried this:
>> http://openvswitch.org/pipermail/discuss/2015-February/016582.html
>> this is the output, so doesn't seem to be correct:
>> root at nuv-vir-kvm-server-2 ~ # gdb /usr/sbin/ovs-vswitchd
>> /var/crash/_usr_sbin_ovs-vswitchd.0.crash
>> GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
>> Copyright (C) 2014 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and
>> redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type
>> "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-linux-gnu".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online
>> at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to
>> "word"...
>> Reading symbols from /usr/sbin/ovs-vswitchd...(no debugging
>> symbols found)...done.
>> "/var/crash/_usr_sbin_ovs-vswitchd.0.crash" is not a core
>> dump: File format not recognized
>> (gdb) bt
>> No stack.
>> (gdb) quit
>> any more hints please ?
>> thanks
>> marco
>> Nuvula AG
>> Marco Kuendig / CEO / Founder marco at nuvula.ch <mailto:marco at nuvula.ch> / +41 78 751 99 71
>> Marco's Google Hangout
>> Nuvula AG - Hybrid Clouds Weierbachstrasse 7b 8193 Eglisau Switzerland http://www.nuvula.ch
>> 
>>      On 31 Mar 2015, at 17:00, Ben Pfaff
>>      <blp at nicira.com> wrote:
>> Can you get a backtrace for these?
>> On Tue, Mar 31, 2015 at 7:09 AM, Marco Kuendig
>> <marco at nuvula.ch> wrote:
>>      Folks,
>> any chance of having somebody look at these crash
>> files ?
>> I have several servers that are loosing network
>> connectivity because of this.
>> Downloads:
>> https://drive.google.com/file/d/0Bx_w1Tf2B5VSRU9yUmRpTDJLVEU/view?usp=sharing
>> Thanks for any hint or fix
>> marco
>> Nuvula AG
>> Marco Kuendig / CEO / Founder marco at nuvula.ch <mailto:marco at nuvula.ch> / +41 78 751 99 71
>> Marco's Google Hangout
>> Nuvula AG - Hybrid Clouds Weierbachstrasse 7b 8193 Eglisau Switzerland http://www.nuvula.ch
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> http://openvswitch.org/mailman/listinfo/discuss
>> --
>> "I don't normally do acked-by's.  I think it's my way
>> of avoiding
>> getting blamed when it all blows up." Andrew Morton
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> http://openvswitch.org/mailman/listinfo/discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20150331/2027045b/attachment-0002.html>


More information about the discuss mailing list