[ovs-discuss] ovs-vswitchd crash

[MAO Ruoyu]

Hi,
We install openstack using devstack with OVS-DPDK
We use host8 as compute node, use host16 as controller node. Now we create vms on both of them as bellow.
on host8, we create vms V7510_OVS_DPDK-SCM11 and V7510_OVS_DPDK-PIM20;
on host16, we create vms V7510_OVS_DPDK-SCM10 and V7510_OVS_DPDK-PIM19.
we create vms successfully.
But after reboot of  V7510_OVS_DPDK-SCM11 application, the ovs-vswitchd on host8 crashed.
We don't know whether the reboot leads the service crashed or the service crash leads the reboot.

We try QEMU 2.3.1, QEMU 2.2.1 and QEMU 2.1.3 on host8, but no effect.
Now we use DPDK2.0, OVS2.4.

[root at host8 ~]# coredumpctl gdb 5295
           PID: 5295 (ovs-vswitchd)
           UID: 0 (root)
           GID: 107 (qemu)
        Signal: 11 (SEGV)
     Timestamp: Thu 2015-12-10 16:49:17 CST (16h ago)
  Command Line: /usr/sbin/ovs-vswitchd --dpdk -vhost_sock_dir /var/run/openvswitch -c 2 -n 4 --proc-type primary --huge-dir /mnt/huge --socket-mem 2048 2048 -- unix:/var/run/openvswitch/db.sock
    Executable: /usr/sbin/ovs-vswitchd
Control Group: /user.slice/user-1000.slice/session-1.scope
          Unit: session-1.scope
         Slice: user-1000.slice
       Session: 1
     Owner UID: 1000 (diag)
       Boot ID: beb01a89b99f494f858c529df3cbf6a2
    Machine ID: 793afa475dbd4b84949ba36a4a0fcd13
      Hostname: host8
      Coredump: /var/lib/systemd/coredump/core.ovs-vswitchd.0.beb01a89b99f494f858c529df3cbf6a2.5295.1449737357000000.xz
       Message: Process 5295 (ovs-vswitchd) of user 0 dumped core.

                Stack trace of thread 5601:
                #0  0x0000000000530852 rte_log_add_in_history (ovs-vswitchd)
                #1  0x00000000004c9cbe console_log_write (ovs-vswitchd)
                #2  0x00007fcd9c9d9501 _IO_cookie_write (libc.so.6)
                #3  0x00007fcd9c9e51d9 _IO_do_write@@GLIBC_2.2.5 (libc.so.6)
                #4  0x00007fcd9c9e35d0 _IO_file_sync@@GLIBC_2.2.5 (libc.so.6)
                #5  0x00007fcd9c9d8f66 _IO_fflush (libc.so.6)
                #6  0x00000000005315f7 rte_vlog (ovs-vswitchd)
               #7  0x000000000040648d rte_log (ovs-vswitchd)
                #8  0x000000000046c930 rte_vhost_enqueue_burst (ovs-vswitchd)
                #9  0x0000000000676051 __netdev_dpdk_vhost_send (ovs-vswitchd)
                #10 0x0000000000676967 netdev_dpdk_vhost_send (ovs-vswitchd)
                #11 0x00000000005bcda0 netdev_send (ovs-vswitchd)
                #12 0x0000000000599d21 dp_execute_cb (ovs-vswitchd)
                #13 0x00000000005c60f4 odp_execute_actions (ovs-vswitchd)
                #14 0x0000000000598e6c dp_netdev_input (ovs-vswitchd)
                #15 0x0000000000599197 dp_netdev_process_rxq_port (ovs-vswitchd)
                #16 0x000000000059981c pmd_thread_main (ovs-vswitchd)
                #17 0x000000000060ba34 ovsthread_wrapper (ovs-vswitchd)
                #18 0x00007fcd9d47d52a start_thread (libpthread.so.0)
                #19 0x00007fcd9ca6b22d __clone (libc.so.6)

                Stack trace of thread 5600:
                #0  0x000000000046d675 rte_vhost_dequeue_burst (ovs-vswitchd)
                #1  0x0000000000672bbc netdev_dpdk_vhost_rxq_recv (ovs-vswitchd)
                #2  0x00000000005bcca1 netdev_rxq_recv (ovs-vswitchd)
                #3  0x00000000005990b4 dp_netdev_process_rxq_port (ovs-vswitchd)
                #4  0x000000000059981c pmd_thread_main (ovs-vswitchd)
                #5  0x000000000060ba34 ovsthread_wrapper (ovs-vswitchd)
                #6  0x00007fcd9d47d52a start_thread (libpthread.so.0)
                #7  0x00007fcd9ca6b22d __clone (libc.so.6)

                Stack trace of thread 6960:
                #0  0x00007fcd9ca5fc8d poll (libc.so.6)
                #1  0x0000000000637b9d time_poll (ovs-vswitchd)
                #2  0x000000000061e33c poll_block (ovs-vswitchd)
                #3  0x0000000000568c1f udpif_upcall_handler (ovs-vswitchd)
                #4  0x000000000060ba34 ovsthread_wrapper (ovs-vswitchd)
                #5  0x00007fcd9d47d52a start_thread (libpthread.so.0)
                #6  0x00007fcd9ca6b22d __clone (libc.so.6)

Our flavor like that:
[cid:image001.png at 01D133FE.2D6DB7D0]

Any try can we do? Thank you.


Best Regards
Mao Ruoyu
+86 21 38434078

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151211/0c52ebc9/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 33923 bytes
Desc: image001.png
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20151211/0c52ebc9/attachment-0002.png>