[ovs-dev] OVS with DPDK and Mellanox NICs

[gilad berman]

Hello,
This mail might belong to the discuss mailing list but since this is
regarding feature currently under dev i figure this will be the best place.

I am trying to install OVS with dpdk 1.8.0 support on a system with
Mellanox NIC (ConnectX-3 Pro, dpdk works with no issues).

Everything works as expected (i hope..  except minor change in the code i
had to do due to fact that ETH_RSS_TCP is not in dpdk 1.8.0) until the
stage i'm adding the dpdk0 port to br0.

I get the following error:
# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd log
for details.

And in the logs:
ovs-vswitchd[25530]: ovs|00021|bridge|WARN|could not open network device
dpdk0 (No such device)

It seems like the dpdk0 can't be created.
How does ovs assign the dpdkX to physical port? is there some parameter i
can pass? any chance it has some Intel specific code?
Any ideas what prevent it from being created?

There is a good chance i am missing something very basic so if this is the
case sorry in advance for that!   thanks in advance!

Some more info:
-----------------------
OVS start command:
 ovs-vswitchd --dpdk -c 0x1 -n 4 --pci-whitelist 1b:00.0 -d
/var/soft/pmd/mlx4_pmd_v2.7.4/librte_pmd_mlx4/librte_pmd_mlx4.so --
unix:/usr/local/var/run/openvswitch/db.sock --pidfile --detach

Output (with some errors, so might be relevant, although service is
running)
2015-04-30T18:21:43Z|00001|dpdk|INFO|No cuse_dev_name provided - defaulting
to /dev/vhost-net
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 6 on socket 0
EAL: Detected lcore 7 as core 7 on socket 0
EAL: Detected lcore 8 as core 0 on socket 1
EAL: Detected lcore 9 as core 1 on socket 1
EAL: Detected lcore 10 as core 2 on socket 1
EAL: Detected lcore 11 as core 3 on socket 1
EAL: Detected lcore 12 as core 4 on socket 1
EAL: Detected lcore 13 as core 5 on socket 1
EAL: Detected lcore 14 as core 6 on socket 1
EAL: Detected lcore 15 as core 7 on socket 1
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 16 lcore(s)
EAL:   unsupported IOMMU type!
EAL: VFIO support could not be initialized
EAL: Setting up memory...
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7f9a80000000 (size = 0x80000000)
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7f99c0000000 (size = 0x80000000)
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7f9900000000 (size = 0x80000000)
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x7f9840000000 (size = 0x80000000)
EAL: Requesting 4 pages of size 1024MB from socket 0
EAL: Requesting 4 pages of size 1024MB from socket 1
EAL: TSC frequency is ~2599998 KHz
EAL: open shared lib
/var/soft/pmd/mlx4_pmd_v2.7.4/librte_pmd_mlx4/librte_pmd_mlx4.so
EAL: /var/soft/pmd/mlx4_pmd_v2.7.4/librte_pmd_mlx4/librte_pmd_mlx4.so:
undefined symbol: per_lcore__lcore_id
EAL: Master core 0 is ready (tid=31ae6b80)
PMD: ENICPMD trace: rte_enic_pmd_init
Zone 0: name:<MALLOC_S0_HEAP_0>, phys:0x100000000, len:0xb00000,
virt:0x7f9a80000000, socket_id:0, flags:0
Zone 1: name:<RG_MP_log_history>, phys:0x100b00000, len:0x2080,
virt:0x7f9a80b00000, socket_id:0, flags:0
Zone 2: name:<MP_log_history>, phys:0x100b02080, len:0x28a0c0,
virt:0x7f9a80b02080, socket_id:0, flags:0
2015-04-30T18:21:48Z|00002|ovs_numa|INFO|Discovered 8 CPU cores on NUMA
node 0
2015-04-30T18:21:48Z|00003|ovs_numa|INFO|Discovered 8 CPU cores on NUMA
node 1
2015-04-30T18:21:48Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and 16 CPU
cores
2015-04-30T18:21:48Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
connecting...
VHOST_CONFIG: char device /dev/vhost-net already exists
2015-04-30T18:21:48Z|00006|dpdk|ERR|CUSE device setup failure.
2015-04-30T18:21:48Z|00007|netdev|ERR|failed to initialize dpdkvhost
network device class: Unknown error -1
2015-04-30T18:21:48Z|00008|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
connected
2015-04-30T18:21:48Z|00009|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath
supports recirculation
2015-04-30T18:21:48Z|00010|ofproto_dpif|INFO|netdev at ovs-netdev: MPLS label
stack length probed as 3
2015-04-30T18:21:48Z|00011|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath
supports unique flow ids
2015-04-30T18:21:48Z|00012|bridge|INFO|bridge br0: added interface br0 on
port 65534
2015-04-30T18:21:48Z|00013|dpif_netlink|ERR|Generic Netlink family
'ovs_datapath' does not exist. The Open vSwitch kernel module is probably
not loaded.
2015-04-30T18:21:48Z|00014|bridge|WARN|could not open network device dpdk0
(No such device)
2015-04-30T18:21:48Z|00015|bridge|INFO|bridge br0: using datapath ID
0000b2f5626b2d43
2015-04-30T18:21:48Z|00016|connmgr|INFO|br0: added service controller
"punix:/usr/local/var/run/openvswitch/br0.mgmt"