[ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number if initialization fails.

Stokes, Ian ian.stokes at intel.com
Fri Jul 24 17:29:11 UTC 2015


That’s good news,

I actually had not tested on a X540-AT2 twinville NIC so it good to get
another NIC validated.

Thanks
Ian 

> -----Original Message-----
> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Luis E Pena
> Sent: Friday, July 24, 2015 6:22 PM
> To: Traynor, Kevin
> Cc: dev at openvswitch.org
> Subject: Re: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number
> if initialization fails.
> 
> After further investigation, it turned out to be a configuration problem
> in two different machines.
> I tried on a third machine, but none of the issues came up. I apologize
> for the false alarm.
> 
> Cheers,
> 
> Luis E. P.
> 
> > On Jul 24, 2015, at 09:40, Luis E Pena <lpena at vmware.com> wrote:
> >
> > The device was bound before adding the port.
> >
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=igb_uio
> unused=
> >
> > On Jul 24, 2015, at 09:38, Traynor, Kevin
> <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com>> wrote:
> >
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Luis E
> Pena
> > Sent: Friday, July 24, 2015 5:18 PM
> > To: Stokes, Ian
> > Cc: dev at openvswitch.org<mailto:dev at openvswitch.org>
> > Subject: Re: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number
> if
> > initialization fails.
> >
> > I am going to try to reproduce it on a different machine.
> >
> > On Jul 24, 2015, at 09:15, Stokes, Ian
> <ian.stokes at intel.com<mailto:ian.stokes at intel.com>> wrote:
> >
> > I have re-tested the patch on a number of our systems with the
> > latest commit on master and I cannot reproduce the error you
> > mention below.
> >
> > The only time I have seen that error occur would be if there was
> > no interface bound to igb_uio driver correctly before calling
> >
> > "ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk"
> >
> > I'm wondering if that’s the issue. Could you check this?
> >
> > If you go to your DPDK dir, you can use the script to bind and check
> the status.
> >
> > sudo ./tools/dpdk_nic_bind.py --status
> >
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
> drv=igb_uio unused=ixgbe
> > 0000:05:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
> drv=igb_uio unused=ixgbe
> >
> > Network devices using kernel driver
> > ===================================
> >
> >
> > Thanks
> > Ian
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Daniele Di
> > Proietto
> > Sent: Friday, July 24, 2015 1:33 PM
> > To: Luis E Pena
> > Cc: dev at openvswitch.org<mailto:dev at openvswitch.org>
> > Subject: Re: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number
> > if initialization fails.
> >
> > I'm not able to reproduce it on my systems.
> >
> > From the log, it looks like something might go wrong in
> > netdev_dpdk_construct(), but I can only guess from there.
> >
> > Could you understand where the error is coming from inside
> > netdev-dpdk.c?
> >
> > Thanks,
> >
> > Daniele
> >
> > On 24/07/2015 00:46, "Luis E Pena"
> <lpena at vmware.com<mailto:lpena at vmware.com>> wrote:
> >
> > Summary of issue:
> >
> > I ran the following commands:
> > 276  ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK --pidfile
> > --detach --log-file
> > 277  ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
> > 278  ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
> >
> > This is what the log file says:
> > 2015-07-22T23:38:40.792Z|00002|vlog|INFO|opened log file
> > /usr/local/var/log/openvswitch/ovs-vswitchd.log
> > 2015-07-22T23:38:40.794Z|00003|ovs_numa|INFO|Discovered 8 CPU cores on
> > NUMA node 0
> > 2015-07-22T23:38:40.794Z|00004|ovs_numa|INFO|Discovered 8 CPU cores on
> > NUMA node 1
> > 2015-07-22T23:38:40.794Z|00005|ovs_numa|INFO|Discovered 2 NUMA nodes
> > and
> > 16 CPU cores
> > 2015-07-
> > 22T23:38:40.794Z|00006|reconnect|INFO|unix:/usr/local/var/run/open
> > vswitch/db.sock: connecting...
> > 2015-07-
> > 22T23:38:40.794Z|00007|reconnect|INFO|unix:/usr/local/var/run/open
> > vswitch/db.sock: connected
> > 2015-07-22T23:38:40.797Z|00008|bridge|INFO|ovs-vswitchd (Open vSwitch)
> > 2.4.90
> > 2015-07-22T23:38:46.896Z|00009|ofproto_dpif|INFO|netdev at ovs-netdev:
> > Datapath supports recirculation
> > 2015-07-22T23:38:46.896Z|00010|ofproto_dpif|INFO|netdev at ovs-netdev:
> > MPLS
> > label stack length probed as 3
> > 2015-07-22T23:38:46.896Z|00011|ofproto_dpif|INFO|netdev at ovs-netdev:
> > Datapath supports unique flow ids
> > 2015-07-22T23:38:46.897Z|00012|bridge|INFO|bridge br0: added interface
> > br0 on port 65534
> > 2015-07-22T23:38:46.899Z|00013|dpif_netlink|ERR|Generic Netlink family
> > 'ovs_datapath' does not exist. The Open vSwitch kernel module is
> > probably
> > not loaded.
> > 2015-07-22T23:38:46.899Z|00014|bridge|INFO|bridge br0: using datapath
> > ID
> > 0000b603c14fb247
> > 2015-07-22T23:38:46.899Z|00015|connmgr|INFO|br0: added service
> > controller
> > "punix:/usr/local/var/run/openvswitch/br0.mgmt"
> > 2015-07-22T23:38:47.903Z|00016|memory|INFO|12400 kB peak resident set
> > size after 10.1 seconds
> > 2015-07-22T23:38:47.903Z|00017|memory|INFO|handlers:11 ports:1
> > revalidators:5 rules:5
> > 2015-07-22T23:38:51.118Z|00018|bridge|WARN|could not open network
> > device
> > dpdk0 (No such device)
> >
> > I can reproduce it if there are any more questions.
> >
> > Cheers,
> >
> > Luis E. P.
> >
> >
> > On Jul 23, 2015, at 16:33, Luis E Pena
> > <lpena at vmware.com<mailto:lpena at vmware.com><mailto:lpena at vmware.com>>
> wrote:
> >
> > Will do.
> >
> > Luis E. P.
> >
> > Sent from my iPhone
> >
> > On Jul 23, 2015, at 16:31, Ethan Jackson
> > <ethan at nicira.com<mailto:ethan at nicira.com><mailto:ethan at nicira.com>>
> wrote:
> >
> > Would you please summarize the errors on list?  Daniele should
> > probably have a look at it since he wrote the patch originally.
> >
> > Ethan
> >
> > On Thu, Jul 23, 2015 at 4:27 PM, Luis E Pena
> > <lpena at vmware.com<mailto:lpena at vmware.com><mailto:lpena at vmware.com>>
> wrote:
> > I ran into errors when using master when adding a dpdk port. I worked
> > with Pravin yesterday and we think that the cause is this patch.
> > When we tried branch-2.4, we ran into no errors.
> > Today I am working on confirming that this is the patch.
> >
> > Luis E. P.
> >
> > Sent from my iPhone
> >
> > On Jul 23, 2015, at 15:38, Ethan Jackson
> > <ethan at nicira.com<mailto:ethan at nicira.com><mailto:ethan at nicira.com>>
> wrote:
> >
> > Ben, Justin, should this be backported?  I'm not up on the policy at
> > the
> > moment.
> >
> > Etha
> >
> > On Thu, Jul 23, 2015 at 3:42 AM, Traynor, Kevin
> >
> <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com><mailto:kevin.tr
> aynor at intel.com>> wrote:
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Daniele Di
> > Proietto
> > Sent: Thursday, July 16, 2015 7:48 PM
> > To:
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.o
> rg>
> > Subject: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number if
> > initialization fails.
> >
> > netdev_dpdk_set_multiq() should not set the number of configured rxq
> > and txq if the driver initialization fails (meaning that the driver
> > failed to setup the queues).  Otherwise, on a subsequent call to
> > netdev_dpdk_set_multiq(), the code may believe that the queues have
> > already been setup and there's no work to be done.
> >
> > This commit fixes the problem by restoring the old values if
> > dpdk_eth_dev_init() fails.
> >
> > Reported-by: Ian Stokes
> >
> <ian.stokes at intel.com<mailto:ian.stokes at intel.com><mailto:ian.stokes at int
> el.com>>
> > Signed-off-by: Daniele Di Proietto
> >
> <diproiettod at vmware.com<mailto:diproiettod at vmware.com><mailto:diproietto
> d at vmware.com>>
> > ---
> > lib/netdev-dpdk.c | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> > index 8b843db..5ae805e 100644
> > --- a/lib/netdev-dpdk.c
> > +++ b/lib/netdev-dpdk.c
> > @@ -743,6 +743,7 @@ netdev_dpdk_set_multiq(struct netdev *netdev_,
> > unsigned
> > int n_txq,
> > {
> > struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
> > int err = 0;
> > +    int old_rxq, old_txq;
> >
> > if (netdev->up.n_txq == n_txq && netdev->up.n_rxq == n_rxq) {
> >   return err;
> > @@ -753,12 +754,20 @@ netdev_dpdk_set_multiq(struct netdev *netdev_,
> > unsigned
> > int n_txq,
> >
> > rte_eth_dev_stop(netdev->port_id);
> >
> > +    old_txq = netdev->up.n_txq;
> > +    old_rxq = netdev->up.n_rxq;
> > netdev->up.n_txq = n_txq;
> > netdev->up.n_rxq = n_rxq;
> >
> > rte_free(netdev->tx_q);
> > err = dpdk_eth_dev_init(netdev);
> > netdev_dpdk_alloc_txq(netdev, netdev->real_n_txq);
> > +    if (err) {
> > +        /* If there has been an error, it means that the requested
> > queues
> > +         * have not been created.  Restore the old numbers. */
> > +        netdev->up.n_txq = old_txq;
> > +        netdev->up.n_rxq = old_rxq;
> >
> > I had thought that we should restore the previous netdev->tx_q but at
> > present
> > txq's are fixed, so I think it is fine. If txq's become configurable
> we
> > can change.
> >
> > It would be good to get these patches into OVS2.4 branch if there is
> > still time?
> >
> > Acked-by: Kevin Traynor
> >
> <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com><mailto:kevin.tr
> aynor at intel.com>>
> >
> > +    }
> >
> > netdev->txq_needs_locking = netdev->real_n_txq != netdev->up.n_txq;
> >
> > --
> > 2.1.4
> >
> > _______________________________________________
> > dev mailing list
> >
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.o
> rg>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__openvswitch.org_mailma
> > n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> > uEs&r=vL
> >
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> > Qho
> > T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> > _______________________________________________
> > dev mailing list
> >
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.o
> rg>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__openvswitch.org_mailma
> > n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> > uEs&r=vL
> >
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> > Qho
> > T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> > _______________________________________________
> > dev mailing list
> >
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.o
> rg>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__openvswitch.org_mailma
> > n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> > uEs&r=vL
> >
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> > Qho
> > T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> > _______________________________________________
> > dev mailing list
> >
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.o
> rg>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__openvswitch.org_mailma
> > n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> > uEs&r=vL
> > BOU1QC09fNCgzE9c0HcQ&m=aqpj8SxwF4NW4mmDwGIC_eclkrgYAxoupPbG-
> > yCZWZs&s=WqVqA
> > Hhg7Rl9oDWOF0Gz-R5JPuCyBN0ZRwWqsHeu15M&e=
> >
> > _______________________________________________
> > dev mailing list
> > dev at openvswitch.org<mailto:dev at openvswitch.org>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__openvswitch.org_mailma
> > n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> > uEs&r=Sm
> >
> B5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=6n4k_DZvKfX4HEvrJv_QEiLameQ
> > UB7
> > tXVjBHHSClfc8&s=ge40W3hqGNXmLb8hQXXTC8r06dbBc9pmBFErxhziWfI&e=
> >
> > _______________________________________________
> > dev mailing list
> > dev at openvswitch.org<mailto:dev at openvswitch.org>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> >
> 3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKI
> iDJAX
> > VeAw-YihVMNtXt-uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=BpOU3TjN5KsJnn9sRdBo-
> > mDQJ2X7M1Mgt8Us6-
> WeRcw&s=_mvayNnuwio4u8ARE5e0TfKNNRDN12YMW6BzCSLh13w&e=
> >
> > _______________________________________________
> > dev mailing list
> > dev at openvswitch.org<mailto:dev at openvswitch.org>
> > https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKI
> iDJAXVeAw-YihVMNtXt-
> uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=CUAiXD7D3mymXQmd9EJpbfNeDrN33TqghH_um_j7U
> PI&s=SwVf-_GQyFofPCMdzfADCSkHe8Oqrwu4laNWST_J_eA&e=
> >
> > _______________________________________________
> > dev mailing list
> > dev at openvswitch.org
> > https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKI
> iDJAXVeAw-YihVMNtXt-
> uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=nGLVy_jtE_EFNnuWVT2AgnMpz9IpTrjTysRljZY3F
> A4&s=zTaIi-3ZsvzRWj7uj0N653tk97Sd0GaLJC6reeDzl2k&e=
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev


More information about the dev mailing list