[ovs-discuss] OVS-DPDK

Advith Nagappa advith.nagappa at gmail.com
Wed Mar 22 09:47:49 UTC 2017


Hello Swetli,

Are you referring to:  *options:dpdk-devargs=0000:00:0a.0  *issue? This one
is clear now. It has to do with OVS version (support available in 2.7+)

If you are referring to:
*2017-03-21T03:14:01Z|00007|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem
1024,0 -c 0x00000001*
*EAL: Detected 5 lcore(s)*
*EAL: Probing VFIO support...*
*EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles *
*EAL: PCI device 0000:00:07.0 on NUMA socket -1*
*EAL:   probe driver: 8086:154c net_i40e_vf*
*EAL: PCI device 0000:00:09.0 on NUMA socket -1*
*EAL:   probe driver: 8086:1572 net_i40e*
*PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*
*EAL: PCI device 0000:00:0a.0 on NUMA socket -1*
*EAL:   probe driver: 8086:1572 net_i40e*
*PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*

Since I have done the binding 09.0 and 0a.0 to igb_uio driver, shouldn't we
see 'igb_uio' instead of 'net_i40e'?

I will try using dpdk-17.02 and see if we can observe the igb_uio driver
set up..

Best Regards
Advith

On Wed, Mar 22, 2017 at 9:49 AM, swetli <swetli at telco.com> wrote:

> Hey,
>
> I'm sure that there are much more knowledgeable guys here but just to
> answer your question: I had the same problem here with the versions you
> mention and it disappeared the moment I updated the dpdk to 17.02 and the
> ovs to master.
> ---
> BR,
>
> Swetli
>
>
> On 03/22/2017 03:48 AM, Darrell Ball wrote:
>
> Regarding the question:
>
>
>
> “Also, what is OVS-2.6.1's version on options:dpdk-devargs='....', by that
> I mean, how does the dpdk port know which interface should it bind to when
> its not specified.. “
>
>
>
> Here is a previous thread that answers this question:
>
> https://mail.openvswitch.org/pipermail/ovs-discuss/2016-March/040190.html
>
>
>
> We could document this in OVS DPDK as it is function impacting.
>
>
>
>
>
> *From: *Advith Nagappa <advith.nagappa at gmail.com>
> <advith.nagappa at gmail.com>
> *Date: *Tuesday, March 21, 2017 at 2:46 PM
> *To: *"Stokes, Ian" <ian.stokes at intel.com> <ian.stokes at intel.com>
> *Cc: *Darrell Ball <dball at vmware.com> <dball at vmware.com>,
> "discuss at openvswitch.org" <discuss at openvswitch.org>
> <discuss at openvswitch.org> <discuss at openvswitch.org>
> *Subject: *Re: [ovs-discuss] OVS-DPDK
>
>
>
> Hi Ian,
>
>
>
> Yes, I am using OVS-2.6.1 release.
>
>
>
> And yes, you are correct.
>
>
>
> *ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk*,
> infact fails in 2.6.1;
>
>
>
> whereas, *ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk, *
> succeeds.
>
>
>
>
>
> As regards the use of
>
>
>
> *options:dpdk-devargs=0000:00:0a.0*
>
>
>
> It is only needed if you are using arbitrary port naming/hotplug support
> which is included in OVS 2.7.0.
>
>
>
> This answers my query.
>
>
>
> But, If i may continue to ask a related query: from the dpdk log..
>
>
>
> *2017-03-21T03:14:01Z|00007|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem
> 1024,0 -c 0x00000001*
>
> *EAL: Detected 5 lcore(s)*
>
> *EAL: Probing VFIO support...*
>
> *EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !*
>
> *EAL: PCI device 0000:00:07.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:154c net_i40e_vf*
>
> *EAL: PCI device 0000:00:09.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:1572 net_i40e*
>
> *PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*
>
> *EAL: PCI device 0000:00:0a.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:1572 net_i40e*
>
> *PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*
>
>
>
>
>
> Since I have done the binding 09.0 and 0a.0 to igb_uio driver, shouldn't
> we see 'igb_uio' instead of 'net_i40e'?
>
>
>
> Also, what is OVS-2.6.1's version on options:dpdk-devargs='....', by that
> I mean, how does the dpdk port know which interface should it bind to when
> its not specified..
>
> Here is a previous thread that answers this question:
>
> https://mail.openvswitch.org/pipermail/ovs-discuss/2016-March/040190.html
>
>
>
> Best Regards
>
> Advith Nagappa
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Mar 21, 2017 12:00 PM, "Stokes, Ian" <ian.stokes at intel.com> wrote:
>
> Apologies for top posting,
>
>
>
> Hi Advith, there seems to be a mismatch of OVS versions and expected
> features from the details you have provided.
>
>
>
> To confirm you are using OVS 2.6.1? Is there a specific commit ID you are
> using? Or are you using the 2.6.1 tag or release package?
>
>
>
> From the commands you have provided it looks like your trying to add dpdk
> ports with arbitrary names and PCI addresses.
>
>
>
> This is not possible in OVS 2.6.1, this ability was only added as of OVS
> 2.7.0.
>
>
>
> If you wanted to add 2 physical dpdk ports with 2.6.1 can you try the
> following
>
>
>
> ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>
> ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
>
>
>
> Note the name of the dpdk port must start with dpdk followed by the number
> being added (starting at 0). This is a limitation that has since been
> removed in OVS 2.7.
>
>
>
> I’m surprised that you say you can use
>
>
>
> ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk
>
>
>
> Testing OVS 2.6.1 tag on master with that command returned the vsctl error
>
>
>
> ovs-vsctl: Error detected while setting up 'dpdk-p1'.  See ovs-vswitchd
> log for details.
>
>
>
> for myself which would be expected.
>
>
>
> As regards the use of
>
>
>
> *options:dpdk-devargs=0000:00:0a.0*
>
>
>
> It is only needed if you are using arbitrary port naming/hotplug support
> which is included in OVS 2.7.0.
>
>
>
> If you can give these suggestions a shot it will help narrow down the
> issue at hand.
>
>
>
> Regards
>
> Ian
>
>
>
> *From:* ovs-discuss-bounces at openvswitch.org [mailto:ovs-discuss-bounces@
> openvswitch.org] *On Behalf Of *Advith Nagappa
> *Sent:* Tuesday, March 21, 2017 8:12 AM
> *To:* Darrell Ball <dball at vmware.com>
> *Cc:* discuss at openvswitch.org
> *Subject:* Re: [ovs-discuss] OVS-DPDK
>
>
>
> I don’t see the pci memory mapping logs here.
>
> Can you also attach the full dpdk logs ?
>
>
>
> I have attached the dpdk-log.
>
>
>
> For PCI mapping, may below helps:
>
>
>
>
>
>
>
> *Bus info          Device      Class          Description*
>
> *========================================================*
>
> *pci at 0000:00:07.0 <pci at 0000:00:07.0>  ens7        network
>  XL710/X710 Virtual Function*
>
> *pci at 0000:00:09.0 <pci at 0000:00:09.0>              network        Ethernet
> Controller X710 for 10GbE
>                SFP+*
>
> *pci at 0000:00:0a.0 <pci at 0000:00:0a.0>              network        Ethernet
> Controller X710 for 10GbE
>               SFP+*
>
> *                                 br0         network        Ethernet
> interface*
>
> *                            ovs-netdev  network        Ethernet interface*
>
>
>
> additionally,
>
>
>
> *Network devices using DPDK-compatible driver*
>
> *============================================*
>
> *0000:00:09.0 'Ethernet Controller X710 for 10GbE SFP+' drv=igb_uio
> unused=i40e*
>
> *0000:00:0a.0 'Ethernet Controller X710 for 10GbE SFP+' drv=igb_uio
> unused=i40e*
>
>
>
> *Network devices using kernel driver*
>
> *===================================*
>
> *0000:00:07.0 'XL710/X710 Virtual Function' if=ens7 drv=i40evf
> unused=igb_uio *Active**
>
>
>
> *Other network devices*
>
> *=====================*
>
> *<none>*
>
>
>
> *Crypto devices using DPDK-compatible driver*
>
> *===========================================*
>
> *<none>*
>
>
>
> *Crypto devices using kernel driver*
>
> *==================================*
>
> *<none>*
>
>
>
> *Other crypto devices*
>
> *====================*
>
> *<none>*
>
>
>
>
>
>
>
> What is your kernel and dpdk versions ?
>
>
>
> Kernel - 4.4.0-66-generic
>
> DPDK - 16.11.1
>
>
>
>
>
> Can you share which portion of the following configuration you followed
>
> http://docs.openvswitch.org/en/latest/intro/install/dpdk/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_intro_install_dpdk_&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=yf52IHbp2qHhIRHw8qsm_y84UKQZ0Ggq-jzS_8sTR2s&s=0n1jprzoNqfZ_QavifoSZc8tCTS7ZG20K1GJPSglLHA&e=>
>
>
>
>
>
> Install *DPDK *- used 1 and 3 (avoided shared lib config)
>
>
>
> Install *OVS  *- As IS
>
>
>
> *Hugepages *- used 1 G huge page. Passed "default_hugepagesz=1G
> hugepagesz=1G hugepages=1" as boot time parameter.
>
>
>
> *grep -i huge /proc/meminfo*
>
> *AnonHugePages:     14336 kB*
>
> *HugePages_Total:       5*
>
> *HugePages_Free:        4*
>
> *HugePages_Rsvd:        0*
>
> *HugePages_Surp:        0*
>
> *Hugepagesize:    1048576 kB*
>
>
>
>
>
> *VFIO : *Did not use Vfio. Used Igb_uio.
>
>
> Setup OVS: As is. Did not specify pmd-cpu-mask.
>
> *Validating*: As is, with the 0000:00:09.0 and 0000:00:0a.0 as device
> address. Fails.
>
>
>
>
>
>
>
> Appreciate your inputs. Look forward to your response.
>
>
>
> Best Regards
>
> Advith
>
>
>
> On Tue, Mar 21, 2017 at 7:50 AM, Darrell Ball <dball at vmware.com> wrote:
>
>
>
>
>
> *From: *Darrell Ball <dball at vmware.com>
> *Date: *Monday, March 20, 2017 at 10:12 PM
> *To: *Advith Nagappa <advith.nagappa at gmail.com>, Ben Pfaff <blp at ovn.org>
> *Cc: *"discuss at openvswitch.org" <discuss at openvswitch.org>
> *Subject: *Re: [ovs-discuss] OVS-DPDK
>
>
>
>
>
>
>
> *From: *<ovs-discuss-bounces at openvswitch.org> on behalf of Advith Nagappa
> <advith.nagappa at gmail.com>
> *Date: *Monday, March 20, 2017 at 8:19 PM
> *To: *Ben Pfaff <blp at ovn.org>
> *Cc: *"discuss at openvswitch.org" <discuss at openvswitch.org>
> *Subject: *Re: [ovs-discuss] OVS-DPDK
>
>
>
> It did not tell me anything:
>
>
>
> *Mar 20 11:42:23  ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl
> add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
> options:dpdk-devargs=0000:00:0a.0*
>
> *Mar 20 11:42:23  ovs-vswitchd[23906]: ovs|00042|bridge|WARN|could not
> open network device dpdk-p0 (No such device)*
>
> *Mar 20 11:42:23  ovs-vswitchd[23906]: ovs|00044|bridge|WARN|could not
> open network device dpdk-p0 (No such device)*
>
>
>
>
>
> A ./dpdk-devbind.py --status shows the below results, confirming device
> binding
>
>
>
> *Network devices using DPDK-compatible driver*
>
> *============================================*
>
> *0000:00:09.0 'Ethernet Controller X710 for 10GbE SFP+' drv=igb_uio
> unused=i40e*
>
> *0000:00:0a.0 'Ethernet Controller X710 for 10GbE SFP+' drv=igb_uio
> unused=i40e*
>
>
>
>
>
>
>
> *Network devices using kernel driver*
>
> *===================================*
>
> *0000:00:07.0 'XL710/X710 Virtual Function' if=ens7 drv=i40evf
> unused=igb_uio *Active**
>
>
>
> *Other network devices*
>
> *=====================*
>
> *<none>*
>
>
>
> *Crypto devices using DPDK-compatible driver*
>
> *===========================================*
>
> *<none>*
>
>
>
> *Crypto devices using kernel driver*
>
> *==================================*
>
> *<none>*
>
>
>
> *Other crypto devices*
>
> *====================*
>
> *<none>*
>
>
>
> But, I did notice a curious thing:
>
>
>
> when I init OVS with dpdk, using:
>
>
>
> ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
>
> ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="
> 1024,0"
>
>
>
>
>
>
>
>
>
> *2017-03-21T03:14:01Z|00007|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem
> 1024,0 -c 0x00000001*
>
> *EAL: Detected 5 lcore(s)*
>
> *EAL: Probing VFIO support...*
>
> *EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !*
>
> *EAL: PCI device 0000:00:07.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:154c net_i40e_vf*
>
> *EAL: PCI device 0000:00:09.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:1572 net_i40e*
>
> *PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*
>
> *EAL: PCI device 0000:00:0a.0 on NUMA socket -1*
>
> *EAL:   probe driver: 8086:1572 net_i40e*
>
> *PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 80002284*
>
>
>
> I don’t see the pci memory mapping logs here.
>
> Can you also attach the full dpdk logs ?
>
> What is your kernel and dpdk versions ?
>
> Can you share which portion of the following configuration you followed
>
> http://docs.openvswitch.org/en/latest/intro/install/dpdk/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_intro_install_dpdk_&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=yf52IHbp2qHhIRHw8qsm_y84UKQZ0Ggq-jzS_8sTR2s&s=0n1jprzoNqfZ_QavifoSZc8tCTS7ZG20K1GJPSglLHA&e=>
>
>
>
>
>
>
>
> I was wondering if the probe driver for 09.0 and 0a.0 should have been
> something like 'igb_uio'.. I am not sure if this is how it should be?
>
>
>
> Appreciate your inputs.
>
>
>
> Best Regards
>
> Advith Nagappa
>
>
>
> On Tue, Mar 21, 2017 at 3:33 AM, Ben Pfaff <blp at ovn.org> wrote:
>
> On Tue, Mar 21, 2017 at 02:36:23AM +0100, Advith Nagappa wrote:
> > Hi,
> >
> > I have DPDK 16.11.1 and OVS 2.6.1, with 1 Gb Huge page/ and 2 NICs bound
> > with Igb_uio drivers.
> >
> > I run in to trouble while creating DPDK ports on the OVS bridge.
> >
> >
> > *ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk*
> >
> > works perfectly fine.
> >
> > But,
> >
> > *ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
> > options:dpdk-devargs=0000:00:0a.0*
> >
> > gives an error:
> > ovs-vsctl: Error detected while setting up 'dpdk-p0'.  See ovs-vswitchd
> log
> > for details.
>
> What's in the log?
>
>
>
>
>
>
>
> _______________________________________________
> discuss mailing listdiscuss at openvswitch.orghttps://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20170322/7923dd2a/attachment-0001.html>


More information about the discuss mailing list