[ovs-dev] Multiqueue in vhost using OVS-DPDK

Ravi Kerur rkerur at gmail.com
Thu Jul 12 16:42:50 UTC 2018


Hello Ian,

Document was very helpful. My testbed has OVS-DPDK vhost and virito in a
container, it had enough information for me to get it working inside the
container.

I have one question for you, I run 'iperf3' tcp and udp performance tests
for more than 5 minutes and packet traverses through vhost-virtio
interfaces, while tests are running I constantly run 'ovs-appctl
dpif-netdev/pmd-rxq-show' and 90% of the time pmd usage is less than 5%.
What is it measuring? pmd threads cpu usage? All PMD threads are in tight
loop taking 100% cpu time so wondering what it reports.

ovs-appctl dpif-netdev/pmd-rxq-show
pmd thread numa_id 0 core_id 6:
  isolated : false
  port: dpdk-eth1         queue-id:  1  pmd usage:  0 %
pmd thread numa_id 0 core_id 7:
  isolated : false
  port: vhost-user-0      queue-id:  0  pmd usage:  0 %
pmd thread numa_id 0 core_id 8:
  isolated : false
  port: vhost-user-0      queue-id:  1  pmd usage:  0 %
pmd thread numa_id 0 core_id 9:
  isolated : false
  port: vhost-user-1      queue-id:  1  pmd usage:  0 %
pmd thread numa_id 0 core_id 10:
  isolated : false
  port: vhost-user-1      queue-id:  0  pmd usage:  0 %
pmd thread numa_id 0 core_id 11:
  isolated : false
  port: dpdk-eth0         queue-id:  1  pmd usage:  3 %
pmd thread numa_id 0 core_id 12:
  isolated : false
  port: dpdk-eth1         queue-id:  0  pmd usage:  0 %
pmd thread numa_id 0 core_id 13:
  isolated : false
  port: dpdk-eth0         queue-id:  0  pmd usage:  6 %

Thanks.

On Wed, Jul 11, 2018 at 11:35 AM Ravi Kerur <rkerur at gmail.com> wrote:

>
>
> On Wed, Jul 11, 2018 at 11:32 AM Ian Stokes <ian.stokes at intel.com> wrote:
>
>> On 7/11/2018 7:23 PM, Ian Stokes wrote:
>> > On 7/11/2018 7:02 PM, Ravi Kerur wrote:
>> >> Hello OVS-DPDK team,
>> >>
>> >> I am using OVS 2.9.90 for my vhost-virtio testing and had some
>> >> questions on
>> >> vhost multiqueue support, it doesn't seem to be working correctly.
>> Kindly
>> >> let me know if multiqueue is supported? I am using following link as a
>> >> reference.
>> >>
>> >>
>> https://software.intel.com/en-us/articles/configure-vhost-user-multiqueue-for-ovs-with-dpdk
>> >>
>> >>
>> >
>> > Hi Ravi, that document was wrote for OVS 2.5 and DPDK 2.2. There have
>> > been a number of changes since then. I'll need to update it.
>> >
>> > I would suggest following the ovs docs
>> >
>> >
>> http://docs.openvswitch.org/en/latest/topics/dpdk/vhost-user/#adding-vhost-user-ports-to-the-guest-qemu
>> >
>> >
>> > specifically the section for multi queue as it describes the options
>> > required.
>> >
>> >> ovs-vswitchd --version
>> >> ovs-vswitchd (Open vSwitch) 2.9.90
>> >> DPDK 17.11.2
>> >> ovs-vsctl get Open_vSwitch . dpdk_version
>> >> "DPDK 17.11.2"
>> >> ovs-vsctl get Open_vSwitch . dpdk_initialized
>> >> true
>> >>
>> >> ovs-vsctl set Interface vhost-user-0 options:n_rxq=2
>> >> ovs-vsctl set Interface vhost-user-0 options:n_txq=2
>> >> ovs-vsctl set Interface vhost-user-1 options:n_rxq=2
>> >> ovs-vsctl set Interface vhost-user-1 options:n_txq=2
>> >
>> > You do not need to set the number of rxqs for vhost devices now in OVS
>> > DPDK. This is set automatically when you launch a guest vm.
>> >
>> > Txqs are autmatically set by ovs dpdk also (it's the numbers of PMDs in
>> > the core mask +1) so no need to set it here.
>> >
>> > You only need to set the number of rxqs if you were using multiple
>> ports
>> > on a physical dpdk port.
>>
>> Apologies, I meant multiple queues on a physical dpdk port.
>>
>
> Thank you Ian. Let me go through the document and get back to you if I
> have any questions.
>
>>
>> Ian
>> >
>> > Ian
>> >>
>> >> I see following messages in openvswitch logs
>> >> 2018-07-11T17:13:54.644Z|00210|netdev|WARN|vhost-user-1: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.514Z|00211|netdev|WARN|vhost-user-0: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.514Z|00212|netdev|WARN|vhost-user-1: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.516Z|00213|netdev|WARN|vhost-user-0: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.516Z|00214|netdev|WARN|vhost-user-1: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.517Z|00215|netdev|WARN|vhost-user-0: arguments
>> >> provided
>> >> to device that is not configurable
>> >> 2018-07-11T17:13:57.517Z|00216|netdev|WARN|vhost-user-1: arguments
>> >> provided
>> >> to device that is not configurable
>> >> ...
>> >>
>> >> ovs-appctl dpctl/show
>> >> netdev at ovs-netdev:
>> >>    lookups: hit:89395441 missed:216 lost:0
>> >>    flows: 0
>> >>    port 0: ovs-netdev (tap)
>> >>    port 1: br0 (tap)
>> >>    *port 2: vhost-user-0 (dpdkvhostuser: configured_rx_queues=1,
>> >> configured_tx_queues=1, mtu=9000, requested_rx_queues=1,
>> >> requested_tx_queues=1)*
>> >>    port 3: dpdk-eth0 (dpdk: configured_rx_queues=2,
>> >> configured_rxq_descriptors=1024, configured_tx_queues=5,
>> >> configured_txq_descriptors=1024, lsc_interrupt_mode=false, mtu=9000,
>> >> requested_rx_queues=2, requested_rxq_descriptors=1024,
>> >> requested_tx_queues=5, requested_txq_descriptors=1024,
>> >> rx_csum_offload=false)
>> >>    *port 4: vhost-user-1 (dpdkvhostuser: configured_rx_queues=1,
>> >> configured_tx_queues=1, mtu=9000, requested_rx_queues=1,
>> >> requested_tx_queues=1)*
>> >>    port 5: dpdk-eth1 (dpdk: configured_rx_queues=2,
>> >> configured_rxq_descriptors=1024, configured_tx_queues=5,
>> >> configured_txq_descriptors=1024, lsc_interrupt_mode=false, mtu=9000,
>> >> requested_rx_queues=2, requested_rxq_descriptors=1024,
>> >> requested_tx_queues=5, requested_txq_descriptors=1024,
>> >> rx_csum_offload=false)
>> >>    port 6: br1 (tap)
>> >>
>> >> Thanks.
>> >> _______________________________________________
>> >> dev mailing list
>> >> dev at openvswitch.org
>> >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>> >>
>> >
>> > _______________________________________________
>> > dev mailing list
>> > dev at openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>>


More information about the dev mailing list