[ovs-discuss] [ovs-dpdk] Issues on booting VMs into OVS-DPDK instances

[Loftus, Ciara]

> Hi OVS-folks,
> 
> We have been trying to boot VMs via Openstack Nova (off Liberty release)
> into DPDK-enabled OVS instances.
> Our OVS is version 2.5  + DPDK 2.2 + Qemu 2.3 + Libvirt 1.2.16.
> 
> We followed all the guidelines here, except for booting VMs  we used
> Openstack Nova instead of directly
> using qemu command line:
> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
> 
> However, we were not able to boot the VMs successfully.   The VMs went
> into error state.
> The vhostdpdkuser ports were created in the OVS-DPDK instance, but Libvirt
> Driver invoked by Nova, could not successfully attach this port to the VM.
> 
> We got these same set of errors in libvirtd.log whenever we booted the
> VMs:
> 2016-03-26 13:57:20.535+0000: 7354: error : virSecurityDriverLookup:80 :
> unsupported configuration: Security driver apparmor not enabled
> 2016-03-26 13:57:51.007+0000: 7339: error : virNetSocketReadWire:1520 : End
> of file while reading data: Input/output error
> 2016-03-26 13:58:36.293+0000: 7341: error : qemuMonitorOpenUnix:358 :
> failed to connect to monitor socket: No such process
> 2016-03-26 13:58:36.293+0000: 7341: error :
> qemuProcessWaitForMonitor:2113 : internal error: process exited while
> connecting to monitor: 2016-03-26T13:58:36.235874Z qemu-system-x86_64: -
> chardev socket,id=charnet0,path=/var/lib/libvirt/qemu/vhu189ade8e-3b:
> Failed to connect socket: Permission denied
> 
> 
> On the QEMU log:
> 2016-03-26 13:58:35.479+0000: starting up libvirt version: 1.2.16, package:
> 1.2.16-2ubuntu11.15.10.4~cloud0, qemu version: 2.3.0 (Debian 1:2.3+dfsg-
> 5ubuntu9.2~cloud0)
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
> QEMU_AUDIO_DRV=none /usr/bin/kvm-spice -name instance-0000000c -S -
> machine pc-i440fx-vivid,accel=kvm,usb=off -m 512 -realtime mlock=off -smp
> 1,sockets=1,cores=1,threads=1 -uuid dbe34ace-960e-4915-884a-
> 5a30b8ec896d -smbios type=1,manufacturer=OpenStack
> Foundation,product=OpenStack Nova,version=12.0.3,serial=00000000-0000-
> 0000-0000-002590883f20,uuid=dbe34ace-960e-4915-884a-
> 5a30b8ec896d,family=Virtual Machine -no-user-config -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-
> 0000000c.monitor,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -
> global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot
> strict=on -kernel /opt/stack/data/nova/instances/dbe34ace-960e-4915-884a-
> 5a30b8ec896d/kernel -initrd /opt/stack/data/nova/instances/dbe34ace-
> 960e-4915-884a-5a30b8ec896d/ramdisk -append root=/dev/vda
> console=tty0 console=ttyS0 -device piix3-usb-
> uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> file=/opt/stack/data/nova/instances/dbe34ace-960e-4915-884a-
> 5a30b8ec896d/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none
> -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-
> disk0,id=virtio-disk0,bootindex=1 -drive
> file=/opt/stack/data/nova/instances/dbe34ace-960e-4915-884a-
> 5a30b8ec896d/disk.config,if=none,id=drive-ide0-1-
> 1,readonly=on,format=raw,cache=none -device ide-
> cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -chardev
> socket,id=charnet0,path=/var/lib/libvirt/qemu/vhu189ade8e-3b -netdev
> type=vhost-user,id=hostnet0,chardev=charnet0 -device virtio-net-
> pci,netdev=hostnet0,id=net0,mac=fa:16:3e:bd:cb:d6,bus=pci.0,addr=0x3 -
> chardev file,id=charserial0,path=/opt/stack/data/nova/instances/dbe34ace-
> 960e-4915-884a-5a30b8ec896d/console.log -device isa-
> serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-
> serial,chardev=charserial1,id=serial1 -vnc 10.183.254.64:0 -k en-us -device
> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-
> pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
> 2016-03-26T13:58:36.235874Z qemu-system-x86_64: -chardev
> socket,id=charnet0,path=/var/lib/libvirt/qemu/vhu189ade8e-3b: Failed to
> connect socket: Permission denied
> 
> We looked at this thread as well:
> http://openvswitch.org/pipermail/discuss/2016-February/020089.html
> 
> And we verified in our setup that we have given user/group as 'root/root' in
> etc/libvirt/qemu.conf and also
> affirmed that our ovs-vswitchd runs as 'root' with DPDK enabled.
> 
> The 'vhuxxx' ports are appearing in /var/lib/libvirt/qemu which is the
> vsock_host_dir folder that we have configured
> for ovs-vswitchd as well as in Nova.
> srwxr-xr-x 1 root         root    0 Mar 26 20:12 vhu98c8f690-41
> 
> But QEMU is not able to complete attaching the VM to the vhostdpdkuser
> port.
> We included "/var/lib/libvirt/qemu" which is the vhost_sock_dir into
> cgroup_device_acl as well.
> 
> And we also tried turning off 'security_driver' to 'None' in qemu.conf to see
> if that helps, but that didn't help either.
> 
> Appreciate any tips on this.

Hi could you try the following when launching ovs-vswitchd:

sudo sg $qemu_group -c "umask 002; ${OVS_INSTALL_DIR}/sbin/ovs-vswitchd -dpdk ."

Thanks,
Ciara

> 
> --
> Thanks,
> 
> Vivek
>