[ovs-discuss] OVS DPDK|DPDK|ERR|interface dpdk0 start error:operation not permitted.

Mon Jul 25 10:03:59 UTC 2016

Hello Mauricio,

On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez
<mauricio.vasquez at polito.it> wrote:
> Hi Sothy,
>
>
> On 07/21/2016 11:55 AM, sothy shan wrote:
>>
>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez
>> <mauricio.vasquez at polito.it> wrote:
>>>
>>>
>>> On 07/21/2016 10:51 AM, sothy shan wrote:
>>>>
>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan <sothy.e98 at gmail.com>
>>>> wrote:
>>>>>
>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez
>>>>> <mauricio.vasquez at polito.it> wrote:
>>>>>>
>>>>>> Hi Sothy,
>>>>>>
>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote:
>>>>>>>
>>>>>>> sothy shan <sothy.e98 at gmail.com> writes:
>>>>>>>
>>>>>>>> Hello all,
>>>>>>>
>>>>>>> Hi Sothy,
>>>>>>>
>>>>>>>> I followed the installion guide provided in the web site
>>>>>>>> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md#ovstc
>>>>>>>> to install dpdk OVS.
>>>>>>>>
>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running VMware.
>>>>>>>> One intereface is enabled by VMXNET3.
>>>>>>
>>>>>>
>>>>>> It is not clear for me, are you trying to install OvS within the VM?
>>>>>>
>>>>>>>> I used dpdk 16.04
>>>>>>>> Today I made clone of OVS (committ
>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383).
>>>>>>>>
>>>>>>>> The following are different configuration compared to above link.
>>>>>>>>
>>>>>>>> sysctl -w vm.nr_hugepages=2048
>>>>>>>> Binded interface via igb_uio.
>>>>>>>>
>>>>>>>> Then followed same configuration to start ovs-vsswitch with dpdk.
>>>>>>>
>>>>>>> Can you please post the steps exactly that you followed?  Which user
>>>>>>> you
>>>>>>> were running as, etc?  Can you also post how you expect packets to
>>>>>>> arrive into the vm?
>>>>>>>
>>>>>>>> I am able to creatre bridge. When I create a port. It gives error
>>>>>>>> message.
>>>>>>>
>>>>>>> Please show the steps for this, too.  The exact commands you used.
>>>>>>>
>>>>>>>> When I check ovs-vswitchd.log file,
>>>>>>>>
>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink family,
>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module may be
>>>>>>>> not
>>>>>>>> loaded.
>>>>>>>
>>>>>>> This is definitely a problem.  How did you start the ovs-vswitchd
>>>>>>> daemon?
>>>>>>
>>>>>>
>>>>>> This is not a problem Aaron, the ovs kernel module is not needed when
>>>>>> ovs is
>>>>>> compiled with DPDK.
>>>>>>>>
>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not permitted.
>>>>>>>>
>>>>>>>> The corresponding port is loaded by dpdk driver as shown in
>>>>>>>> dpdk_nic_bind --status.
>>>>>>>>
>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port details as
>>>>>>>> well.
>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt get
>>>>>>>> packets from the interface.
>>>>>>>>
>>>>>>>> Do you know why this error is popup? anyway to solve the problem?
>>>>>>>>
>>>>>>>> Thank you very much
>>>>>>>>
>>>>>>>> Best regars
>>>>>>>> Sothy
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> discuss mailing list
>>>>>>> discuss at openvswitch.org
>>>>>>> http://openvswitch.org/mailman/listinfo/discuss
>>>>>>
>>>>>> As Aaron wrote, having the exact commands is important to help you.
>>>>>
>>>>> My configuration and log info:
>>>>>
>>>>> sudo sysctl vm.nr_hugepages=2048
>>>>> sudo mkdir -p /dev/hugepages
>>>>> sudo  mount -t hugetlbfs hugetlbfs /dev/hugepages
>>>>> sudo modprobe uio
>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status
>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0
>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status
>>>>>
>>>>>
>>>>> Network devices using DPDK-compatible driver
>>>>> ============================================
>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>
>>>>> Network devices using kernel driver
>>>>> ===================================
>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
>>>>> unused=igb_uio *Active*
>>>>>
>>>>> Other network devices
>>>>> =====================
>>>>> <none>
>>>>>
>>>>> .........Then......................
>>>>>
>>>>>
>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
>>>
>>> These two commands are inverted.
>>> The bridge has to be created first and then ports have to be added to it.
>>>
>> It is my mistake when coping.
>>>>>
>>>>> Log file/
>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file
>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on
>>>>> NUMA node 0
>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes
>>>>> and 2 CPU cores
>>>>>
>>>>>
>>>>> 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>> connecting...
>>>>>
>>>>>
>>>>> 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>> connected
>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long 1076ms
>>>>> poll interval (80ms user, 875ms system)
>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, 0 major
>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3
>>>>> voluntary, 101 involuntary
>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, avg rate
>>>>> over last: 5 seconds, last minute, last hour,  hash=398361de:
>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 640
>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 35
>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 38
>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 11049
>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent
>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit
>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open vSwitch)
>>>>> 2.5.90
>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak resident set
>>>>> size after 75.3 seconds
>>>>> 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath supports recirculation
>>>>> 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> MPLS label stack length probed as 3
>>>>> 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath supports truncate action
>>>>> 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath supports unique flow ids
>>>>> 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath does not support ct_state
>>>>> 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath does not support ct_zone
>>>>> 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath does not support ct_mark
>>>>> 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath does not support ct_label
>>>>> 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev at ovs-netdev:
>>>>> Datapath does not support ct_state_nat
>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added interface
>>>>> br0 on port 65534
>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink
>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>>>> is probably not loaded.
>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using datapath
>>>>> ID 0000c664fbd86348
>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service
>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start error:
>>>>> Operation not permitted
>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev at ovs-netdev: failed to
>>>>> add dpdk0 as port: Operation not permitted
>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 flow_mods in
>>>>> the last 0 s (1 adds)
>>>
>>> Could you also provide the full terminal output when you run
>>> ovs-vswitchd?,
>>> I would like to see the DPDK output
>>
>> vrouter at vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd unix:$DB_SOCK
>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log
>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file
>> /var/log/openvswitch/ovs-vswitchd.log
>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>> node 0
>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU
>> cores
>>
>> 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>> connecting...
>>
>> 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>> connected
>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing
>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>> defaulting to /usr/local/var/run/openvswitch
>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>> --socket-mem 1024,0 -c 0x00000001
>> EAL: Detected lcore 0 as core 0 on socket 0
>> EAL: Detected lcore 1 as core 0 on socket 0
>> EAL: Support maximum 128 logical core(s) by configuration.
>> EAL: Detected 2 lcore(s)
>> EAL: Probing VFIO support...
>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>> directory)
>> EAL: VFIO modules not loaded, skipping VFIO support...
>> EAL: Setting up physically contiguous memory...
>> EAL: Ask a virtual area of 0x5e400000 bytes
>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000)
>> EAL: Ask a virtual area of 0xa1800000 bytes
>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000)
>> EAL: Requesting 512 pages of size 2MB from socket 0
>> EAL: TSC frequency is ~2799213 KHz
>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0])
>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   Not managed by a supported kernel driver, skipped
>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   PCI memory mapped at 0x7fa658400000
>> EAL:   PCI memory mapped at 0x7fa658401000
>> EAL:   PCI memory mapped at 0x7fa658402000
>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   PCI memory mapped at 0x7fa658404000
>> EAL:   PCI memory mapped at 0x7fa658405000
>> EAL:   PCI memory mapped at 0x7fa658406000
>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080,
>> virt:0x7fa6583fdec0, socket_id:0, flags:0
>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0,
>> virt:0x7fa658173d40, socket_id:0, flags:0
>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800,
>> virt:0x7fa658143200, socket_id:0, flags:0
>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath supports recirculation
>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev at ovs-netdev: MPLS
>> label stack length probed as 3
>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath supports truncate action
>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath supports unique flow ids
>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath does not support ct_state
>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath does not support ct_zone
>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath does not support ct_mark
>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath does not support ct_label
>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev at ovs-netdev:
>> Datapath does not support ct_state_nat
>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface br0
>> on port 65534
>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family
>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is
>> probably not loaded.
>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error:
>> Operation not permitted
>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath ID
>> 0000c664fbd86348
>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service controller
>> "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms poll
>> interval (233ms user, 880ms system)
>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major
>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10
>> voluntary, 214 involuntary
>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate over
>> last: 5 seconds, last minute, last hour,  hash=75cfd2fc:
>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand
>> 0.0/sec     0.000/sec        0.0000/sec   total: 22
>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink
>> 0.0/sec     0.000/sec        0.0000/sec   total: 7
>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add
>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush
>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get
>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put
>> 0.0/sec     0.000/sec        0.0000/sec   total: 10
>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del
>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute
>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract
>> 0.0/sec     0.000/sec        0.0000/sec   total: 4
>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc
>> 0.0/sec     0.000/sec        0.0000/sec   total: 16
>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand
>> 0.0/sec     0.000/sec        0.0000/sec   total: 653
>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node
>> 0.0/sec     0.000/sec        0.0000/sec   total: 15
>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change
>> 0.0/sec     0.000/sec        0.0000/sec   total: 575
>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open
>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc
>> 0.0/sec     0.000/sec        0.0000/sec   total: 11820
>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr
>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool
>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received
>> 0.0/sec     0.000/sec        0.0000/sec   total: 6
>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo
>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent
>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit
>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error:
>> Operation not permitted
>
> The PMD start function is failing, unfortunately that error does not tell
> that much information.
>
> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, set
> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base file of
> DPDK, then recompile DPDK and OvS.
>
> Please try once again and send the terminal output, I hope we can get a more
> information of what is going on with that log.

Based on your request, I reproduce the log with
CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock
--log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach
2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovs-vswitchd.log
2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA node 0
2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU cores
2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
connecting...
2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
connected
2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing
2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided -
defaulting to /usr/local/var/run/openvswitch
2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
--socket-mem 1024,0 -c 0x00000001
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Probing VFIO support...
EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory)
EAL: VFIO modules not loaded, skipping VFIO support...
EAL: Setting up physically contiguous memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000)
EAL: Ask a virtual area of 0x5e000000 bytes
EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fde47600000 (size = 0x200000)
EAL: Ask a virtual area of 0xa1800000 bytes
EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000)
EAL: Requesting 512 pages of size 2MB from socket 0
EAL: TSC frequency is ~2799209 KHz
EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0])
PMD: rte_vmxnet3_pmd_init():  >>
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   PCI memory mapped at 0x7fdea6200000
EAL:   PCI memory mapped at 0x7fdea6201000
EAL:   PCI memory mapped at 0x7fdea6202000
PMD: eth_vmxnet3_dev_init():  >>
PMD: eth_vmxnet3_dev_init(): Hardware version : 1
PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1
PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b
EAL: PCI device 0000:13:00.0 on NUMA socket -1
EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL:   Not managed by a supported kernel driver, skipped
Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080,
virt:0x7fdea61fdec0, socket_id:0, flags:0
Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0,
virt:0x7fde87975f00, socket_id:0, flags:0
Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800,
virt:0x7fdea61cd300, socket_id:0, flags:0

PMD: vmxnet3_dev_configure():  >>
PMD: vmxnet3_dev_tx_queue_setup():  >>
PMD: vmxnet3_dev_rx_queue_setup():  >>
PMD: vmxnet3_dev_start():  >>
PMD: vmxnet3_rss_configure():  >>
PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
PMD: vmxnet3_disable_intr():  >>
PMD: vmxnet3_dev_rxtx_init():  >>
2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
PMD: vmxnet3_dev_stop():  >>
PMD: vmxnet3_disable_intr():  >>
PMD: vmxnet3_dev_stop(): Device reset.
PMD: vmxnet3_dev_clear_queues():  >>
PMD: vmxnet3_dev_configure():  >>
PMD: vmxnet3_dev_tx_queue_setup():  >>
PMD: vmxnet3_dev_tx_queue_setup():  >>
PMD: vmxnet3_dev_tx_queue_setup():  >>
PMD: vmxnet3_dev_rx_queue_setup():  >>
PMD: vmxnet3_dev_start():  >>
PMD: vmxnet3_rss_configure():  >>
PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL

I deleted exisiting bridge and recreated::::::

Then I added bridge. After I try to create bridge. It gives error. The
erros is same as shown in log file.

>
> Another thing to do would be to bind the NICs to the uio_pci_generic and
> VFIO modules, just to be sure that the issue is also present there.

I tried in uio and uio_pci_genric also. Both are giving same error.
I suspect I cannt activate VFIO in VM since it is BIOS suport also. AM I right?

Thanks

>
>
>> in command line some information for your verification:
>>
>>   sudo ovs-vsctl show
>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>      Bridge "br0"
>>          Port "dpdk0"
>>              Interface "dpdk0"
>>                  type: dpdk
>>          Port "br0"
>>              Interface "br0"
>>                  type: internal
>> vrouter at vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0
>> vrouter at vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 --
>> set bridge br0 datapath_type=netdev
>> vrouter at vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0
>> dpdk0 -- set Interface dpdk0 type=dpdk
>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd
>> log for details.
>>
>>
>>>>>
>>>>>
>>>>>
>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> sudo ovs-vsctl show
>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>       Bridge "br0"
>>>>>           Port "dpdk0"
>>>>>               Interface "dpdk0"
>>>>>                   type: dpdk
>>>>>           Port "br0"
>>>>>               Interface "br0"
>>>>>                   type: internal
>>>>>
>>>>> sudo ovs-ofctl dump-flows br0
>>>>> NXST_FLOW reply (xid=0x4):
>>>>>    cookie=0x0, duration=487.638s, table=0, n_packets=0, n_bytes=0,
>>>>> idle_age=487, priority=0 actions=NORMAL
>>>>>
>>>>>
>>>>>> In addition I would recommend to use the test-pmd [1] application from
>>>>>> DPDK
>>>>>> to check if the issue is also present in that case.
>>>>
>>>> I run testpmd command.The result is
>>>>
>>>> Configuring Port 0 (socket 0)
>>>> Port 0: 00:0C:29:B5:9F:5B
>>>> Configuring Port 1 (socket 0)
>>>> Port 1: 00:0C:29:B5:9F:65
>>>> Checking link statuses...
>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex
>>>> Done
>>>> No commandline core given, start packet forwarding
>>>>     io packet forwarding - CRC stripping disabled - packets/burst=32
>>>>     nb forwarding cores=1 - nb forwarding ports=2
>>>>     RX queues=1 - RX desc=128 - RX free threshold=0
>>>>     RX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>     TX queues=1 - TX desc=512 - TX free threshold=0
>>>>     TX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>     TX RS bit threshold=0 - TXQ flags=0x200
>>>> Press enter to exit
>>>>
>>>> Telling cores to stop...
>>>> Waiting for lcores to finish...
>>>>
>>>>     ---------------------- Forward statistics for port 0
>>>> ----------------------
>>>>     RX-packets: 88             RX-dropped: 0             RX-total: 88
>>>>     TX-packets: 2              TX-dropped: 0             TX-total: 2
>>>>
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>>     ---------------------- Forward statistics for port 1
>>>> ----------------------
>>>>     RX-packets: 2              RX-dropped: 0             RX-total: 2
>>>>     TX-packets: 88             TX-dropped: 0             TX-total: 88
>>>>
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>>     +++++++++++++++ Accumulated forward statistics for all
>>>> ports+++++++++++++++
>>>>     RX-packets: 90             RX-dropped: 0             RX-total: 90
>>>>     TX-packets: 90             TX-dropped: 0             TX-total: 90
>>>>
>>>>
>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>> make sense?
>>>
>>>
>>> Yes, it makes sense. Ports are being recognized by DPDK and some packets
>>> were exchanged.
>>>
>>>
>>>>>> [1] http://dpdk.org/doc/guides-16.04/testpmd_app_ug/index.html
>>>>>>
>>>>>> Mauricio Vasquez,
>>>
>>>
> Mauricio V,