[ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number if initialization fails.

Luis E Pena lpena at vmware.com
Fri Jul 24 17:22:08 UTC 2015


After further investigation, it turned out to be a configuration problem in two different machines.
I tried on a third machine, but none of the issues came up. I apologize for the false alarm.

Cheers,

Luis E. P.

> On Jul 24, 2015, at 09:40, Luis E Pena <lpena at vmware.com> wrote:
> 
> The device was bound before adding the port.
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=igb_uio unused=
> 
> On Jul 24, 2015, at 09:38, Traynor, Kevin <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com>> wrote:
> 
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Luis E Pena
> Sent: Friday, July 24, 2015 5:18 PM
> To: Stokes, Ian
> Cc: dev at openvswitch.org<mailto:dev at openvswitch.org>
> Subject: Re: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number if
> initialization fails.
> 
> I am going to try to reproduce it on a different machine.
> 
> On Jul 24, 2015, at 09:15, Stokes, Ian <ian.stokes at intel.com<mailto:ian.stokes at intel.com>> wrote:
> 
> I have re-tested the patch on a number of our systems with the
> latest commit on master and I cannot reproduce the error you
> mention below.
> 
> The only time I have seen that error occur would be if there was
> no interface bound to igb_uio driver correctly before calling
> 
> "ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk"
> 
> I'm wondering if that’s the issue. Could you check this?
> 
> If you go to your DPDK dir, you can use the script to bind and check the status.
> 
> sudo ./tools/dpdk_nic_bind.py --status
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe
> 0000:05:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe
> 
> Network devices using kernel driver
> ===================================
> 
> 
> Thanks
> Ian
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Daniele Di
> Proietto
> Sent: Friday, July 24, 2015 1:33 PM
> To: Luis E Pena
> Cc: dev at openvswitch.org<mailto:dev at openvswitch.org>
> Subject: Re: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number
> if initialization fails.
> 
> I'm not able to reproduce it on my systems.
> 
> From the log, it looks like something might go wrong in
> netdev_dpdk_construct(), but I can only guess from there.
> 
> Could you understand where the error is coming from inside
> netdev-dpdk.c?
> 
> Thanks,
> 
> Daniele
> 
> On 24/07/2015 00:46, "Luis E Pena" <lpena at vmware.com<mailto:lpena at vmware.com>> wrote:
> 
> Summary of issue:
> 
> I ran the following commands:
> 276  ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK --pidfile
> --detach --log-file
> 277  ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
> 278  ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
> 
> This is what the log file says:
> 2015-07-22T23:38:40.792Z|00002|vlog|INFO|opened log file
> /usr/local/var/log/openvswitch/ovs-vswitchd.log
> 2015-07-22T23:38:40.794Z|00003|ovs_numa|INFO|Discovered 8 CPU cores on
> NUMA node 0
> 2015-07-22T23:38:40.794Z|00004|ovs_numa|INFO|Discovered 8 CPU cores on
> NUMA node 1
> 2015-07-22T23:38:40.794Z|00005|ovs_numa|INFO|Discovered 2 NUMA nodes
> and
> 16 CPU cores
> 2015-07-
> 22T23:38:40.794Z|00006|reconnect|INFO|unix:/usr/local/var/run/open
> vswitch/db.sock: connecting...
> 2015-07-
> 22T23:38:40.794Z|00007|reconnect|INFO|unix:/usr/local/var/run/open
> vswitch/db.sock: connected
> 2015-07-22T23:38:40.797Z|00008|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.4.90
> 2015-07-22T23:38:46.896Z|00009|ofproto_dpif|INFO|netdev at ovs-netdev:
> Datapath supports recirculation
> 2015-07-22T23:38:46.896Z|00010|ofproto_dpif|INFO|netdev at ovs-netdev:
> MPLS
> label stack length probed as 3
> 2015-07-22T23:38:46.896Z|00011|ofproto_dpif|INFO|netdev at ovs-netdev:
> Datapath supports unique flow ids
> 2015-07-22T23:38:46.897Z|00012|bridge|INFO|bridge br0: added interface
> br0 on port 65534
> 2015-07-22T23:38:46.899Z|00013|dpif_netlink|ERR|Generic Netlink family
> 'ovs_datapath' does not exist. The Open vSwitch kernel module is
> probably
> not loaded.
> 2015-07-22T23:38:46.899Z|00014|bridge|INFO|bridge br0: using datapath
> ID
> 0000b603c14fb247
> 2015-07-22T23:38:46.899Z|00015|connmgr|INFO|br0: added service
> controller
> "punix:/usr/local/var/run/openvswitch/br0.mgmt"
> 2015-07-22T23:38:47.903Z|00016|memory|INFO|12400 kB peak resident set
> size after 10.1 seconds
> 2015-07-22T23:38:47.903Z|00017|memory|INFO|handlers:11 ports:1
> revalidators:5 rules:5
> 2015-07-22T23:38:51.118Z|00018|bridge|WARN|could not open network
> device
> dpdk0 (No such device)
> 
> I can reproduce it if there are any more questions.
> 
> Cheers,
> 
> Luis E. P.
> 
> 
> On Jul 23, 2015, at 16:33, Luis E Pena
> <lpena at vmware.com<mailto:lpena at vmware.com><mailto:lpena at vmware.com>> wrote:
> 
> Will do.
> 
> Luis E. P.
> 
> Sent from my iPhone
> 
> On Jul 23, 2015, at 16:31, Ethan Jackson
> <ethan at nicira.com<mailto:ethan at nicira.com><mailto:ethan at nicira.com>> wrote:
> 
> Would you please summarize the errors on list?  Daniele should
> probably have a look at it since he wrote the patch originally.
> 
> Ethan
> 
> On Thu, Jul 23, 2015 at 4:27 PM, Luis E Pena
> <lpena at vmware.com<mailto:lpena at vmware.com><mailto:lpena at vmware.com>> wrote:
> I ran into errors when using master when adding a dpdk port. I worked
> with Pravin yesterday and we think that the cause is this patch.
> When we tried branch-2.4, we ran into no errors.
> Today I am working on confirming that this is the patch.
> 
> Luis E. P.
> 
> Sent from my iPhone
> 
> On Jul 23, 2015, at 15:38, Ethan Jackson
> <ethan at nicira.com<mailto:ethan at nicira.com><mailto:ethan at nicira.com>> wrote:
> 
> Ben, Justin, should this be backported?  I'm not up on the policy at
> the
> moment.
> 
> Etha
> 
> On Thu, Jul 23, 2015 at 3:42 AM, Traynor, Kevin
> <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com><mailto:kevin.traynor at intel.com>> wrote:
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of Daniele Di
> Proietto
> Sent: Thursday, July 16, 2015 7:48 PM
> To: dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>
> Subject: [ovs-dev] [PATCH 1/2] netdev-dpdk: Restore txq/rxq number if
> initialization fails.
> 
> netdev_dpdk_set_multiq() should not set the number of configured rxq
> and txq if the driver initialization fails (meaning that the driver
> failed to setup the queues).  Otherwise, on a subsequent call to
> netdev_dpdk_set_multiq(), the code may believe that the queues have
> already been setup and there's no work to be done.
> 
> This commit fixes the problem by restoring the old values if
> dpdk_eth_dev_init() fails.
> 
> Reported-by: Ian Stokes
> <ian.stokes at intel.com<mailto:ian.stokes at intel.com><mailto:ian.stokes at intel.com>>
> Signed-off-by: Daniele Di Proietto
> <diproiettod at vmware.com<mailto:diproiettod at vmware.com><mailto:diproiettod at vmware.com>>
> ---
> lib/netdev-dpdk.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
> 
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index 8b843db..5ae805e 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -743,6 +743,7 @@ netdev_dpdk_set_multiq(struct netdev *netdev_,
> unsigned
> int n_txq,
> {
> struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
> int err = 0;
> +    int old_rxq, old_txq;
> 
> if (netdev->up.n_txq == n_txq && netdev->up.n_rxq == n_rxq) {
>   return err;
> @@ -753,12 +754,20 @@ netdev_dpdk_set_multiq(struct netdev *netdev_,
> unsigned
> int n_txq,
> 
> rte_eth_dev_stop(netdev->port_id);
> 
> +    old_txq = netdev->up.n_txq;
> +    old_rxq = netdev->up.n_rxq;
> netdev->up.n_txq = n_txq;
> netdev->up.n_rxq = n_rxq;
> 
> rte_free(netdev->tx_q);
> err = dpdk_eth_dev_init(netdev);
> netdev_dpdk_alloc_txq(netdev, netdev->real_n_txq);
> +    if (err) {
> +        /* If there has been an error, it means that the requested
> queues
> +         * have not been created.  Restore the old numbers. */
> +        netdev->up.n_txq = old_txq;
> +        netdev->up.n_rxq = old_rxq;
> 
> I had thought that we should restore the previous netdev->tx_q but at
> present
> txq's are fixed, so I think it is fine. If txq's become configurable we
> can change.
> 
> It would be good to get these patches into OVS2.4 branch if there is
> still time?
> 
> Acked-by: Kevin Traynor
> <kevin.traynor at intel.com<mailto:kevin.traynor at intel.com><mailto:kevin.traynor at intel.com>>
> 
> +    }
> 
> netdev->txq_needs_locking = netdev->real_n_txq != netdev->up.n_txq;
> 
> --
> 2.1.4
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailma
> n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> uEs&r=vL
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> Qho
> T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailma
> n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> uEs&r=vL
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> Qho
> T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailma
> n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> uEs&r=vL
> BOU1QC09fNCgzE9c0HcQ&m=BYGI54GsZwT1uHiylyB910omvTxfD_EkW_A8R8jN_5M&s=3C
> Qho
> T6BwcZjI5qd5549pOjn6pDX3E3MsUbJc8QKufg&e=
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org><mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailma
> n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> uEs&r=vL
> BOU1QC09fNCgzE9c0HcQ&m=aqpj8SxwF4NW4mmDwGIC_eclkrgYAxoupPbG-
> yCZWZs&s=WqVqA
> Hhg7Rl9oDWOF0Gz-R5JPuCyBN0ZRwWqsHeu15M&e=
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailma
> n_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
> uEs&r=Sm
> B5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=6n4k_DZvKfX4HEvrJv_QEiLameQ
> UB7
> tXVjBHHSClfc8&s=ge40W3hqGNXmLb8hQXXTC8r06dbBc9pmBFErxhziWfI&e=
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAX
> VeAw-YihVMNtXt-uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=BpOU3TjN5KsJnn9sRdBo-
> mDQJ2X7M1Mgt8Us6-WeRcw&s=_mvayNnuwio4u8ARE5e0TfKNNRDN12YMW6BzCSLh13w&e=
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org<mailto:dev at openvswitch.org>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=CUAiXD7D3mymXQmd9EJpbfNeDrN33TqghH_um_j7UPI&s=SwVf-_GQyFofPCMdzfADCSkHe8Oqrwu4laNWST_J_eA&e=
> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_dev&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=vLBOU1QC09fNCgzE9c0HcQ&m=nGLVy_jtE_EFNnuWVT2AgnMpz9IpTrjTysRljZY3FA4&s=zTaIi-3ZsvzRWj7uj0N653tk97Sd0GaLJC6reeDzl2k&e= 



More information about the dev mailing list