[ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue setup until we don't get any failure.

Ethan Jackson ethan at nicira.com
Thu Aug 6 18:23:33 UTC 2015


Great, I'll merge both today.

Thanks a lot,
Ethan

On Thu, Aug 6, 2015 at 9:00 AM, Stokes, Ian <ian.stokes at intel.com> wrote:
> No problem,
>
> I've sent a patch to the mailing list with these changes for INSTALL.DPDK.md.
>
> Thanks
> Ian
>
>> -----Original Message-----
>> From: Ethan Jackson [mailto:ethan at nicira.com]
>> Sent: Tuesday, August 04, 2015 10:46 PM
>> To: Stokes, Ian
>> Cc: Ben Pfaff; Justin Pettit; Pravin Shelar; Traynor, Kevin; Daniele Di
>> Proietto; dev at openvswitch.org
>> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue setup
>> until we don't get any failure.
>>
>> Sure let's do that.  If you write an INSTALL.DPDK note, and post it
>> here.  I'll incorporate it into the patch and merge.  Thanks for your
>> patience.
>>
>> Ethan
>>
>> On Tue, Aug 4, 2015 at 4:48 AM, Stokes, Ian <ian.stokes at intel.com>
>> wrote:
>> > This issue currently affects Fortville cards on systems with a logical
>> core count greater than 64.
>> >
>> > Just to call out however that the number of cores setup by default to
>> forward traffic to a NIC using OVS with DPDK will be the total number of
>> cores detected on the system. This is the default behavior, thus there
>> is no way user can "devote" less cores to forwarding on the NIC.
>> >
>> > This means they will always hit this issue when using a system with 72
>> logical cores and a Fortville card. There is no way they can work around
>> the issue with tx configuration changes.
>> >
>> > If this preferable then I can write up a note for theINSTALL.DPDK.md
>> explaining the issue.
>> >
>> > Thanks
>> > Ian
>> >
>> >> -----Original Message-----
>> >> From: Ben Pfaff [mailto:blp at nicira.com]
>> >> Sent: Monday, August 03, 2015 6:11 PM
>> >> To: Ethan Jackson
>> >> Cc: Stokes, Ian; Justin Pettit; Pravin Shelar; Traynor, Kevin;
>> Daniele
>> >> Di Proietto; dev at openvswitch.org
>> >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx queue
>> setup
>> >> until we don't get any failure.
>> >>
>> >> If that's an accurate description of the problem then that seems fine
>> to
>> >> me.  We could add a note to INSTALL.DPDK.md describing the issue I
>> >> suppose, if someone wants to write one up.
>> >>
>> >> On Sat, Aug 01, 2015 at 02:40:16PM -0700, Ethan Jackson wrote:
>> >> > I personally am fine with waiting on this being fixed until
>> December.
>> >> > My reading is, it only happens on one specific NIC, and even then
>> only
>> >> > if you devote a huge number of cores to forwarding on that NIC.
>> >> >
>> >> > That said, I won't block this if another committer disagrees with
>> me.
>> >> > Ben Justin Pravin?  What do you think?
>> >> >
>> >> > Ethan
>> >> >
>> >> > On Thu, Jul 30, 2015 at 1:13 AM, Stokes, Ian <ian.stokes at intel.com>
>> >> wrote:
>> >> > > So ideally this will be fixed in a future release of DPDK. We
>> have
>> >> flagged this. However that solution will not be in place until the
>> DPDK
>> >> 2.2 release in December at the earliest (DPDK 2.1 is currently in
>> >> release candidate mode at the moment so it won't make it to that).
>> When
>> >> this has been changed in DPDK we can revisit the OVS code.
>> >> > >
>> >> > > Technically DPDK is doing what it is supposed to with the current
>> >> implementation i.e. it is returning the max number of queues it
>> >> supports. From the OVS side I think we need to understand that this
>> has
>> >> a different connotation to what it had with previously with NICS in
>> >> terms of how many of those queues are usable.
>> >> > >
>> >> > > Unfortunately I don’t see another way to negotiate the tx queue
>> >> initialization without something like the patch below.
>> >> > > Not until we have more explicit configuration details available
>> for
>> >> the HW device from DPDK.
>> >> > >
>> >> > > Thanks
>> >> > > Ian
>> >> > >
>> >> > >> -----Original Message-----
>> >> > >> From: Ethan Jackson [mailto:ethan at nicira.com]
>> >> > >> Sent: Wednesday, July 29, 2015 10:13 PM
>> >> > >> To: Stokes, Ian
>> >> > >> Cc: Traynor, Kevin; Daniele Di Proietto; dev at openvswitch.org
>> >> > >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
>> queue
>> >> setup
>> >> > >> until we don't get any failure.
>> >> > >>
>> >> > >> Sorry for taking so long to get to this.  The one question I
>> have
>> >> is:
>> >> > >> Is OVS the right layer to be fixing this?  Isn't this really an
>> >> issue
>> >> > >> of DPDK reporting a number of available queues that for
>> practical
>> >> > >> purposes is wrong?  I.E. Shouldn't this be fixed by the DPDK
>> driver
>> >> of
>> >> > >> this system?  This patch feels like a hack to me . . .
>> >> > >>
>> >> > >> Ethan
>> >> > >>
>> >> > >> On Tue, Jul 28, 2015 at 2:36 AM, Stokes, Ian
>> <ian.stokes at intel.com>
>> >> > >> wrote:
>> >> > >> > Hi all,
>> >> > >> >
>> >> > >> > Just wondering what the status of this patch is? Is there any
>> >> feedback
>> >> > >> > or queries we can answer to help?
>> >> > >> >
>> >> > >> > Thanks
>> >> > >> > Ian
>> >> > >> >
>> >> > >> >> -----Original Message-----
>> >> > >> >> From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of
>> >> Traynor,
>> >> > >> >> Kevin
>> >> > >> >> Sent: Thursday, July 23, 2015 11:28 AM
>> >> > >> >> To: Daniele Di Proietto; dev at openvswitch.org
>> >> > >> >> Subject: Re: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
>> >> queue
>> >> > >> setup
>> >> > >> >> until we don't get any failure.
>> >> > >> >>
>> >> > >> >>
>> >> > >> >> > -----Original Message-----
>> >> > >> >> > From: dev [mailto:dev-bounces at openvswitch.org] On Behalf Of
>> >> Daniele
>> >> > >> Di
>> >> > >> >> > Proietto
>> >> > >> >> > Sent: Thursday, July 16, 2015 7:48 PM
>> >> > >> >> > To: dev at openvswitch.org
>> >> > >> >> > Subject: [ovs-dev] [PATCH 2/2] netdev-dpdk: Retry tx/rx
>> queue
>> >> setup
>> >> > >> >> until we
>> >> > >> >> > don't get any failure.
>> >> > >> >> >
>> >> > >> >> > It has been observed that some DPDK device (e.g intel
>> xl710)
>> >> report
>> >> > >> an
>> >> > >> >> > high number of queues but make some of them available only
>> for
>> >> > >> special
>> >> > >> >> > functions (SRIOV).  Therefore the queues will be counted in
>> >> > >> >> > rte_eth_dev_info_get(), but rte_eth_tx_queue_setup() will
>> >> fail.
>> >> > >> >> >
>> >> > >> >> > This commit works around the issue by retrying the device
>> >> > >> >> initialization
>> >> > >> >> > with a smaller number of queues, if a queue fails to setup.
>> >> > >> >> >
>> >> > >> >> > Reported-by: Ian Stokes <ian.stokes at intel.com>
>> >> > >> >> > Signed-off-by: Daniele Di Proietto <diproiettod at vmware.com>
>> >> > >> >> > ---
>> >> > >> >> >  lib/netdev-dpdk.c | 100
>> >> +++++++++++++++++++++++++++++++++++++++---
>> >> > >> ---
>> >> > >> >> -------
>> >> > >> >> > --
>> >> > >> >> >  1 file changed, 73 insertions(+), 27 deletions(-)
>> >> > >> >>
>> >> > >> >>
>> >> > >> >> Acked-by: Kevin Traynor <kevin.traynor at intel.com>
>> >> > >> >> _______________________________________________
>> >> > >> >> dev mailing list
>> >> > >> >> dev at openvswitch.org
>> >> > >> >> http://openvswitch.org/mailman/listinfo/dev
>> >> > >> > _______________________________________________
>> >> > >> > dev mailing list
>> >> > >> > dev at openvswitch.org
>> >> > >> > http://openvswitch.org/mailman/listinfo/dev



More information about the dev mailing list