[ovs-discuss] ovs-vsctl add-port -- how to specify particular DPDK device port?

John Wei johntwei at gmail.com
Thu Mar 3 02:31:02 UTC 2016


Please ignore my question on starting ovs-vswitchd in 2nd container. I
solved that by giving each ovs-vswitchd its own hugepages directory.
Then I run into different issue.
On my machine, I have 126 SRIOV VFs. Each VF is bounded to DPDK. When
starting ovs-vswitchd, it was able to deal with port 0-31. I died when
dealing with 32nd port:
......
PMD: eth_ixgbevf_dev_init(): port 31 vendorID=0x8086 deviceID=0x10ed
mac.type=ixgbe_mac_82599_vf
EAL: PCI device 0000:42:14.0 on NUMA socket 1
EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
EAL:   PCI memory mapped at 0x7f10e8f00000
EAL:   PCI memory mapped at 0x7f10e8f04000
EAL: Error - exiting with code: 1
  Cause: Requested device 0000:42:14.0 cannot be used
Is there a parameter or code some where I can increase the number of ports
that vswitchd can deal with?

John




On Wed, Mar 2, 2016 at 4:32 PM, John Wei <johntwei at gmail.com> wrote:

> Hi Mark,
> Really appreciate your input. With your suggestion, I was able to pick the
> DPDK port be bounded with ovs-vswitchd.This helped me moved further. Now I
> realize the issue is that I can't run multiple copies of ovs-vswitchd in
> different containers. :)
> Since containers is supposed to give you virtual environment similar to
> VM, except shared kernel. I was trying to set  up OVS+DPDK in container,
> with multiple containers. I am using SRIOV VF. Each vswitchd in each
> container will be using two dpdk ports.
>
> With your suggestion, I can assign different dpdk ports to different
> vswtichd in different contains. As I experiment further, i found the issue
> is that I can't even bring up 2nd vswitchd in 2nd container.
>
> As soon I tried to run this command in 2nd container:
> ovs-vswitchd --dpdk -c 0x2 -n 4 -- unix:$DB_SOCK --pidfile --detach
> --log-file
>
> The ovs-vswitchd in 1st container exit without error message in the
> log-file.
>
> I wonder if there is way to work around this?
>
> Thanks for your help.
>
> John
>
>
>
> On Wed, Mar 2, 2016 at 7:07 AM, Kavanagh, Mark B <
> mark.b.kavanagh at intel.com> wrote:
>
>> Hi John,
>>
>> If I understand your question correctly, you have multiple NIC ports
>> bound to the DPDK driver, and of those, you want to bind a specific port to
>> ovs-vswitchd?
>>
>> It would help me to answer your query more effectively if I had some more
>> information on what your setup looks like, and what your desired OVS
>> end-state configuration is, but this answer should provide some general
>> guidance.
>>
>> To provide some context, consider how the DPDK port enumeration process
>> works, which is (roughly) as follows:
>> - as part of the DPDK init process, PCI devices are enumerated
>> - enumerated devices are typically ordered from low-to-high by PCI
>> Bus:Device:Function
>> - for each compatible PCI function detected, DPDK checks if that PCI
>> device is bound to a supported DPDK driver
>> - if so, that device is assigned a port ID, starting from 0 for the
>> lowest PCI ID, and increasing to 1 for the next PCI ID, and so on
>> - for more information, consult DPDK documentation and/or the DPDK
>> mailing list
>>
>> In your case, you have (presumably) two PCI functions/NIC ports bound to
>> a DPDK driver; on init, these devices are enumerated/indexed as 0 and 1,
>> respectively.
>>
>> Consequently, to add DPDK port 'N' to a bridge, simply run:
>>
>>         'ovs-vsctl add-port br0 dpdkN -- set Interface dpdkN type=dpdk'
>>
>>
>> Where N = the index of the DPDK device port, as enumerated by DPDK on
>> init.
>>
>> I hope this helps.
>>
>> Cheers,
>> Mark
>>
>>
>> >>URL added:
>> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
>> >
>> >On Tue, Mar 1, 2016 at 3:22 PM, John Wei <johntwei at gmail.com> wrote:
>> >I am following instructions in this URL to setup ovs to use with DPDK.
>> >With the instruction, the first two dpdk ports were always used bu ovs:
>> >ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>> >ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
>> >
>> >How to tell ovs-vswitchd to use other dpdk ports?
>> >
>> >John
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160302/d6a1328e/attachment-0002.html>


More information about the discuss mailing list