[ovs-discuss] [OVS-DPDK] vhost-user with multiple queues does not work
Felix Brucker
FBrucker at xantaro.net
Mon Apr 4 13:49:06 UTC 2016
Hi,
for my setup libvirts generated output found in /var/log/libvirt/qemu/vm1.log does not seem to include the mq params:
-chardev socket,id=charnet0,path=/usr/local/var/run/openvswitch/vhost-user-0 -netdev type=vhost-user,id=hostnet0,chardev=charnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:e3:eb:b2,bus=pci.0,addr=0x2
-chardev socket,id=charnet1,path=/usr/local/var/run/openvswitch/vhost-user-1 -netdev type=vhost-user,id=hostnet1,chardev=charnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:26:50:09,bus=pci.0,addr=0x3
||/ Name Version Architecture Description
+++-============================-===================-===================-==============================================================
ii libvirt-bin 1.2.16-2ubuntu11.15.10.3 amd64 programs for the libvirt library
ii libvirt0 1.2.16-2ubuntu11.15.10.3 amd64 library for interfacing with different virtualization systems
ii qemu-kvm 1:2.3+dfsg-5ubuntu9.2 amd64 QEMU Full virtualization
ii qemu-system-common 1:2.3+dfsg-5ubuntu9.2 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-x86 1:2.3+dfsg-5ubuntu9.2 amd64 QEMU full system emulation binaries (x86)
also it seems you have newer versions of libvirt and qemu, on which OS are you running this?
Best regards
Felix
Von: Christian Ehrhardt [mailto:christian.ehrhardt at canonical.com]
Gesendet: Montag, 4. April 2016 15:25
An: Felix Brucker <FBrucker at xantaro.net>; Daniele Di Proietto <diproiettod at vmware.com>
Cc: Loftus, Ciara <ciara.loftus at intel.com>; discuss at openvswitch.org
Betreff: Re: [ovs-discuss] [OVS-DPDK] vhost-user with multiple queues does not work
Hi,
I thought that is rather close to my setup so gave it a look as well.
And at least in the current setup I can confirm what Felix sees:
I added
<driver queues='4'/>
for the vhost_user device as in libvirt since https://www.redhat.com/archives/libvir-list/2015-June/msg00201.html
The section libvirt generates out of that is
-chardev socket,id=charnet1,path=
/var/run/openvswitch/vhost-user-2 -netdev type=vhost-user,id=hostnet1,chardev=charnet1,queues=4 -device virtio-net-pci,mq=on,vectors=10,netdev=hostnet1,id=net1,mac=52:54:00
:62:47:b6,bus=pci.0,addr=0x4
So the 4 queues get here by libvirt, and also the 2*queues+2 vectors get into the commandline.
I think it has all that was once summarized here http://wiki.qemu.org/Features/vhost-user-ovs-dpdk#Enabling_multi-queue
Yet the device in the guest looks like this (no queues able to be combined)
ethtool -l eth1
Channel parameters for eth1:
Pre-set maximums:
RX: 0
TX: 0
Other: 0
Combined: 1
Current hardware settings:
RX: 0
TX: 0
Other: 0
Combined: 1
Versions of involved programs:
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-===============================================================================
ii libvirt-bin 1.3.1-1ubuntu6 amd64 programs for the libvirt library
ii libvirt0:amd64 1.3.1-1ubuntu6 amd64 library for interfacing with different virtualization systems
ii qemu-kvm 1:2.5+dfsg-5ubuntu6 amd64 QEMU Full virtualization
ii dpdk 2.2.0-0ubuntu6 amd64 Data Plane Development Kit (runtime)
ii libdpdk0:amd64 2.2.0-0ubuntu6 amd64 Data Plane Development Kit (runtime libraries)
ii openvswitch-common 2.5.0-0ubuntu1 amd64 Open vSwitch common components
ii openvswitch-switch 2.5.0-0ubuntu1 amd64 Open vSwitch switch implementations
ii openvswitch-switch-dpdk 2.5.0-0ubuntu1 amd64 DPDK enabled Open vSwitch switch implementation
Full qemu commandline:
qemu-system-x86_64 -enable-kvm -name guest-dpdk-vhost-user-2 -S -machine pc-i440fx-wily,accel=kvm,usb=o
ff -cpu host -m 2048 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,share
=yes,size=2147483648 -numa node,nodeid=0,cpus=0,memdev=ram-node0 -uuid 380e03d4-ea96-410a-b50f-f7fdbb6facdd -nographic -no-user-config -nodefaults -chardev socket,id=charmo
nitor,path=/var/lib/libvirt/qemu/domain-guest-dpdk-vhost-user-2/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot
strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/uvtool/libvirt/images/guest-dpdk-vhost-user-2.qcow,format=qcow2,if=none,id=drive-virtio
-disk0,cache=unsafe -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/uvtool/libvirt/images/guest
-dpdk-vhost-user-2-ds.qcow,format=raw,if=none,id=drive-virtio-disk1,cache=unsafe -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1
-netdev tap,fd=31,id=hostnet0,vhost=on,vhostfd=33 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:10:f7:df,bus=pci.0,addr=0x3 -chardev socket,id=charnet1,path=
/var/run/openvswitch/vhost-user-2 -netdev type=vhost-user,id=hostnet1,chardev=charnet1,queues=4 -device virtio-net-pci,mq=on,vectors=10,netdev=hostnet1,id=net1,mac=52:54:00
:62:47:b6,bus=pci.0,addr=0x4 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-ba
lloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on
Kind Regards,
Christian
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
On Mon, Apr 4, 2016 at 12:17 PM, Felix Brucker <FBrucker at xantaro.net<mailto:FBrucker at xantaro.net>> wrote:
Hi,
i followed the Install.DPDK.md<http://Install.DPDK.md> initially, thats where i got
ovs-vsctl set Interface vhost-user-2 options:n_rxq=<requested queues>
to set the rx queues for vhost-user interfaces. Im using libvirt, so im not passing any cli arguments, rather my xml snippet looks like this:
<interface type='vhostuser'>
<mac address='52:54:00:e3:eb:b2'/>
<source type='unix' path='/usr/local/var/run/openvswitch/vhost-user-0' mode='client'/>
<model type='virtio'/>
<driver queues='2'/>
</interface>
<interface type='vhostuser'>
<mac address='52:54:00:26:50:09'/>
<source type='unix' path='/usr/local/var/run/openvswitch/vhost-user-1' mode='client'/>
<model type='virtio'/>
<driver queues='2'/>
</interface>
I also tried with an additional name parameter like so:
<driver name='vhost' queues='2'/>
as stated for "running ovs-vswitchd with DPDK backend inside a VM" (which im not doing right now), but this also did not change anything.
Best regards
Felix
-----Ursprüngliche Nachricht-----
Von: Loftus, Ciara [mailto:ciara.loftus at intel.com<mailto:ciara.loftus at intel.com>]
Gesendet: Montag, 4. April 2016 12:01
An: Felix Brucker <FBrucker at xantaro.net<mailto:FBrucker at xantaro.net>>
Cc: discuss at openvswitch.org<mailto:discuss at openvswitch.org>
Betreff: RE: [ovs-discuss] [OVS-DPDK] vhost-user with multiple queues does not work
>
> Hi,
>
> setting
>
> ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=2
>
> did not create two queues inside the guest, from the name of the
> command i guess it sets the number of rx queues for the pmd host
> interface to 2, but not the vhost-user interface for the vm.
> Im using branch-2.5 because a bug i encountered regarding vhost-thread
> was fixed 6 days ago on that branch.
>
> Best regards
> Felix
Can you share the command line you are using to start the VM? Do you have the necessary fields set correctly? eg.
qemu-system-x86_64 ...
-chardev socket,id=char0,path=/my/sock
-netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=2
-device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off,mq=on,vectors=6
There's more info in INSTALL.DPDK.md<http://INSTALL.DPDK.md> on configuring multiqueue.
Thanks,
Ciara
>
>
> -----Ursprüngliche Nachricht-----
> Von: Loftus, Ciara [mailto:ciara.loftus at intel.com<mailto:ciara.loftus at intel.com>]
> Gesendet: Montag, 4. April 2016 11:29
> An: Felix Brucker <FBrucker at xantaro.net<mailto:FBrucker at xantaro.net>>; discuss at openvswitch.org<mailto:discuss at openvswitch.org>
> Betreff: RE: [ovs-discuss] [OVS-DPDK] vhost-user with multiple queues
> does not work
>
> >
> > Hi,
> >
> > when configuring OVS vhost-user interfaces with multiple queues like so:
> >
> > ovs-vsctl set Interface vhost-user-0 options:n_rxq=2 ovs-vsctl set
> > Interface vhost-user-1 options:n_rxq=2
> >
> > i get the following warnings when starting ovs-vswitchd:
> >
> > 2016-04-04T08:53:28Z|00038|netdev|WARN|vhost-user-0: arguments
> > provided to device that is not configurable
> > 2016-04-04T08:53:28Z|00039|netdev|WARN|vhost-user-1: arguments
> > provided to device that is not configurable
> >
> > The guest vm also does not have multiple queues enabled as they are
> > not set in ovs.
> > Im using OVS 2.5 from git -> branch-2.5 -> commit
> > e099c86ca432d1daeaf8edb66225e7edf061c0b0
>
> Hi,
>
> If you are using branch-2.5 you cannot use the n_rxq field. Instead
> you need to set multiple queues like so:
> ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=4
>
> This sets the number of rxqs for all devices to 4.
>
> Thanks,
> Ciara
>
> >
> > Someone had a similar issue or knows a fix?
> >
> > Best regards
> > Felix
_______________________________________________
discuss mailing list
discuss at openvswitch.org<mailto:discuss at openvswitch.org>
http://openvswitch.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160404/73709a99/attachment-0002.html>
More information about the discuss
mailing list