[ovs-discuss] DPDK OVS and vhost-user: guest host cannot received packets from local host

Fri Sep 23 11:25:40 UTC 2016

Hi,
I am using ovs 2.5.0 + dpdk2.2.0 + libvirt1.2.18.1(qemu 2.3.1)

I have added 1 physical dpdk port and 1 vhostuser ports.
Following is my bridge:

    Bridge "br0"
        Port "dpdk0"
            Interface "dpdk0"
                type: dpdk
        Port "vhost-user1"
            Interface "vhost-user1"
                type: dpdkvhostuser
        Port "br0"
            Interface "br0"
                type: internal

the libvirt related setting as below

<domain type='kvm'>

...
<memory unit='KiB'>10485760</memory>
  <currentMemory unit='KiB'>10485760</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='1048576' unit='KiB' nodeset='0'/>
    </hugepages>
  </memoryBacking>

...
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='8' threads='2'/>
    <numa>
      <cell id='0' cpus='0-1' memory='10485760' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
...

   <interface type='vhostuser'>
      <mac address='52:00:00:00:00:01'/>
      <source type='unix' path='/usr/localvar/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
...

/domain>

However, after I started the guest host, I found the guest host can send packets to local host but cannot  received packets from local host.

#ovs-ofctl dump-ports br0
OFPST_PORT reply (xid=0x2): 3 ports
  port LOCAL: rx pkts=32, bytes=1536, drop=0, errs=0, frame=0, over=0, crc=0
           tx pkts=13974, bytes=6983918, drop=882, errs=0, coll=0
  port  1: rx pkts=15478, bytes=7120292, drop=0, errs=0, frame=?, over=?, crc=?
           tx pkts=44, bytes=2928, drop=0, errs=0, coll=?
  port  2: rx pkts=37, bytes=1866, drop=?, errs=0, frame=?, over=?, crc=?
           tx pkts=0, bytes=0, drop=13969, errs=?, coll=?

Observe the vhost-user1 port (port 2), we can see all the Tx packets has been dropped

The related syslog as below , it seems that the Tx queue mapping failed. Are there some wrong with my POC?

2016-09-23T18:48:14.693571+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: new virtio connection is 35
2016-09-23T18:48:14.693852+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: new device, handle is 0
2016-09-23T18:48:16.728354+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_OWNER
2016-09-23T18:48:16.728645+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
2016-09-23T18:48:16.728766+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
2016-09-23T18:48:16.728880+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring call idx:0 file:46
2016-09-23T18:48:16.728992+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
2016-09-23T18:48:16.729103+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring call idx:1 file:54
2016-09-23T18:48:24.112148+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
2016-09-23T18:48:24.113589+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring call idx:0 file:55
2016-09-23T18:48:24.114578+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
2016-09-23T18:48:24.114733+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring call idx:1 file:46
2016-09-23T18:48:24.114914+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_FEATURES
2016-09-23T18:48:24.115061+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE
2016-09-23T18:48:24.115176+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: mapped region 0 fd:54 to:0x2aaac0000000 sz:0x280000000 off:0xc0000000 align:0x40000000
2016-09-23T18:48:24.115289+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: mapped region 1 fd:56 to:0x2aad40000000 sz:0x40000000 off:0x0 align:0x40000000
2016-09-23T18:48:24.115402+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: mapped region 2 fd:57 to:0x2aad80000000 sz:0xc0000000 off:0xc0000 align:0x40000000
2016-09-23T18:48:24.115512+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
2016-09-23T18:48:24.115637+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
2016-09-23T18:48:24.115755+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
2016-09-23T18:48:24.115870+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
2016-09-23T18:48:24.115979+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring kick idx:0 file:58
2016-09-23T18:48:24.116104+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: virtio is not ready for processing.
2016-09-23T18:48:24.116214+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
2016-09-23T18:48:24.116324+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
2016-09-23T18:48:24.116431+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
2016-09-23T18:48:24.116537+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
2016-09-23T18:48:24.116647+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: vring kick idx:1 file:59
2016-09-23T18:48:24.116758+08:00 linux-yjaf ovs-vswitchd[10637]: VHOST_CONFIG: virtio is now ready for processing.
2016-09-23T18:48:24.116867+08:00 linux-yjaf ovs-vswitchd[10637]: ovs|00061|dpdk(vhost_thread1)|DBG|TX queue mapping for /usr/local/myOvs/var/run/openvswitch/vhost-user1
2016-09-23T18:48:24.116974+08:00 linux-yjaf ovs-vswitchd[10637]: ovs|00062|dpdk(vhost_thread1)|DBG| 0 --> -1
2016-09-23T18:48:24.117082+08:00 linux-yjaf ovs-vswitchd[10637]: ovs|00063|dpdk(vhost_thread1)|INFO|vHost Device '/usr/local/var/run/openvswitch/vhost-user1' 0 has been added

BR/Cheng

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160923/8cf0eaff/attachment-0002.html>