[ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue setup until we don't get any failure.

Thu Aug 6 16:00:42 UTC 2015

No problem,

I've sent a patch to the mailing list with these changes for INSTALL.DPDK.md.

Thanks
Ian

> -----Original Message-----
> From: Ethan Jackson [mailto:ethan at nicira.com]
> Sent: Tuesday, August 04, 2015 10:46 PM
> To: Stokes, Ian
> Cc: Ben Pfaff; Justin Pettit; Pravin Shelar; Traynor, Kevin; Daniele Di
> Proietto; dev at openvswitch.org
> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue setup
> until we don't get any failure.
> 
> Sure let's do that.  If you write an INSTALL.DPDK note, and post it
> here.  I'll incorporate it into the patch and merge.  Thanks for your
> patience.
> 
> Ethan
> 
> On Tue, Aug 4, 2015 at 4:48 AM, Stokes, Ian <ian.stokes at intel.com>
> wrote:
> > This issue currently affects Fortville cards on systems with a logical
> core count greater than 64.
> >
> > Just to call out however that the number of cores setup by default to
> forward traffic to a NIC using OVS with DPDK will be the total number of
> cores detected on the system. This is the default behavior, thus there
> is no way user can "devote" less cores to forwarding on the NIC.
> >
> > This means they will always hit this issue when using a system with 72
> logical cores and a Fortville card. There is no way they can work around
> the issue with tx configuration changes.
> >
> > If this preferable then I can write up a note for theINSTALL.DPDK.md
> explaining the issue.
> >
> > Thanks
> > Ian
> >
> >> -----Original Message-----
> >> From: Ben Pfaff [mailto:blp at nicira.com]
> >> Sent: Monday, August 03, 2015 6:11 PM
> >> To: Ethan Jackson
> >> Cc: Stokes, Ian; Justin Pettit; Pravin Shelar; Traynor, Kevin;
> Daniele
> >> Di Proietto; dev at openvswitch.org
> >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue
> setup
> >> until we don't get any failure.
> >>
> >> If that's an accurate description of the problem then that seems fine
> to
> >> me.  We could add a note to INSTALL.DPDK.md describing the issue I
> >> suppose, if someone wants to write one up.
> >>
> >> On Sat, Aug 01, 2015 at 02:40:16PM -0700, Ethan Jackson wrote:
> >> > I personally am fine with waiting on this being fixed until
> December.
> >> > My reading is, it only happens on one specific NIC, and even then
> only
> >> > if you devote a huge number of cores to forwarding on that NIC.
> >> >
> >> > That said, I won't block this if another committer disagrees with
> me.
> >> > Ben Justin Pravin?  What do you think?
> >> >
> >> > Ethan
> >> >
> >> > On Thu, Jul 30, 2015 at 1:13 AM, Stokes, Ian <ian.stokes at intel.com>
> >> wrote:
> >> > > So ideally this will be fixed in a future release of DPDK. We
> have
> >> flagged this. However that solution will not be in place until the
> DPDK
> >> 2.2 release in December at the earliest (DPDK 2.1 is currently in
> >> release candidate mode at the moment so it won't make it to that).
> When
> >> this has been changed in DPDK we can revisit the OVS code.
> >> > >
> >> > > Technically DPDK is doing what it is supposed to with the current
> >> implementation i.e. it is returning the max number of queues it
> >> supports. From the OVS side I think we need to understand that this
> has
> >> a different connotation to what it had with previously with NICS in
> >> terms of how many of those queues are usable.
> >> > >
> >> > > Unfortunately I don’t see another way to negotiate the tx queue
> >> initialization without something like the patch below.
> >> > > Not until we have more explicit configuration details available
> for
> >> the HW device from DPDK.
> >> > >
> >> > > Thanks
> >> > > Ian
> >> > >
> >> > >> -----Original Message-----
> >> > >> From: Ethan Jackson [mailto:ethan at nicira.com]
> >> > >> Sent: Wednesday, July 29, 2015 10:13 PM
> >> > >> To: Stokes, Ian
> >> > >> Cc: Traynor, Kevin; Daniele Di Proietto; dev at openvswitch.org
> >> > >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
> queue
> >> setup
> >> > >> until we don't get any failure.
> >> > >>
> >> > >> Sorry for taking so long to get to this.  The one question I
> have
> >> is:
> >> > >> Is OVS the right layer to be fixing this?  Isn't this really an
> >> issue
> >> > >> of DPDK reporting a number of available queues that for
> practical
> >> > >> purposes is wrong?  I.E. Shouldn't this be fixed by the DPDK
> driver
> >> of
> >> > >> this system?  This patch feels like a hack to me . . .
> >> > >>
> >> > >> Ethan
> >> > >>
> >> > >> On Tue, Jul 28, 2015 at 2:36 AM, Stokes, Ian
> <ian.stokes at intel.com>
> >> > >> wrote:
> >> > >> > Hi all,
> >> > >> >
> >> > >> > Just wondering what the status of this patch is? Is there any
> >> feedback
> >> > >> > or queries we can answer to help?
> >> > >> >
> >> > >> > Thanks
> >> > >> > Ian
> >> > >> >
> >> > >> >> -----Original Message-----
> >> > >> >> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of
> >> Traynor,
> >> > >> >> Kevin
> >> > >> >> Sent: Thursday, July 23, 2015 11:28 AM
> >> > >> >> To: Daniele Di Proietto; dev at openvswitch.org
> >> > >> >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
> >> queue
> >> > >> setup
> >> > >> >> until we don't get any failure.
> >> > >> >>
> >> > >> >>
> >> > >> >> > -----Original Message-----
> >> > >> >> > From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of
> >> Daniele
> >> > >> Di
> >> > >> >> > Proietto
> >> > >> >> > Sent: Thursday, July 16, 2015 7:48 PM
> >> > >> >> > To: dev at openvswitch.org
> >> > >> >> > Subject: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
> queue
> >> setup
> >> > >> >> until we
> >> > >> >> > don't get any failure.
> >> > >> >> >
> >> > >> >> > It has been observed that some DPDK device (e.g intel
> xl710)
> >> report
> >> > >> an
> >> > >> >> > high number of queues but make some of them available only
> for
> >> > >> special
> >> > >> >> > functions (SRIOV).  Therefore the queues will be counted in
> >> > >> >> > rte_eth_dev_info_get(), but rte_eth_tx_queue_setup() will
> >> fail.
> >> > >> >> >
> >> > >> >> > This commit works around the issue by retrying the device
> >> > >> >> initialization
> >> > >> >> > with a smaller number of queues, if a queue fails to setup.
> >> > >> >> >
> >> > >> >> > Reported-by: Ian Stokes <ian.stokes at intel.com>
> >> > >> >> > Signed-off-by: Daniele Di Proietto <diproiettod at vmware.com>
> >> > >> >> > ---
> >> > >> >> >  lib/netdev-dpdk.c | 100
> >> +++++++++++++++++++++++++++++++++++++++---
> >> > >> ---
> >> > >> >> -------
> >> > >> >> > --
> >> > >> >> >  1 file changed, 73 insertions(+), 27 deletions(-)
> >> > >> >>
> >> > >> >>
> >> > >> >> Acked-by: Kevin Traynor <kevin.traynor at intel.com>
> >> > >> >> _______________________________________________
> >> > >> >> dev mailing list
> >> > >> >> dev at openvswitch.org
> >> > >> >> http://openvswitch.org/mailman/listinfo/dev
> >> > >> > _______________________________________________
> >> > >> > dev mailing list
> >> > >> > dev at openvswitch.org
> >> > >> > http://openvswitch.org/mailman/listinfo/dev