[ovs-dev] Is this an issue for DPDK vhost rss?

[Traynor, Kevin]

> -----Original Message-----
> From: Flavio Leitner [mailto:fbl at sysclose.org]
> Sent: Wednesday, June 17, 2015 2:31 PM
> To: Traynor, Kevin
> Cc: Daniele Di Proietto; dev at openvswitch.org; Gray, Mark D
> Subject: Re: [ovs-dev] Is this an issue for DPDK vhost rss?
> 
> On Wed, Jun 17, 2015 at 01:18:29PM +0000, Traynor, Kevin wrote:
> >
> > > -----Original Message-----
> > > From: Traynor, Kevin
> > > Sent: Wednesday, June 17, 2015 10:12 AM
> > > To: Flavio Leitner; Daniele Di Proietto
> > > Cc: dev at openvswitch.org; Gray, Mark D
> > > Subject: RE: [ovs-dev] Is this an issue for DPDK vhost rss?
> > >
> > >
> > > > -----Original Message-----
> > > > From: Flavio Leitner [mailto:fbl at sysclose.org]
> > > > Sent: Tuesday, June 16, 2015 6:28 PM
> > > > To: Daniele Di Proietto
> > > > Cc: Traynor, Kevin; dev at openvswitch.org; Gray, Mark D
> > > > Subject: Re: [ovs-dev] Is this an issue for DPDK vhost rss?
> > > >
> > > > On Mon, Jun 15, 2015 at 05:55:13PM +0000, Daniele Di Proietto wrote:
> > > > > On 15/06/2015 12:16, "Traynor, Kevin" <kevin.traynor at intel.com>
> wrote:
> > > > > >There is a dpdk patchset that contains a potential fix for this and
> lots
> > > > > >of
> > > > > >other changes, but I haven't tested yet.
> > > > > >https://urldefense.proofpoint.com/v2/url?u=http-
> > > 3A__dpdk.org_ml_archives_d
> > > > > >ev_2015-2DJune_018436.html&d=BQIFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-
> > > YihVMN
> > > > > >tXt-
> > > uEs&r=SmB5nZacmXNq0gKCC1s_Cw5yUNjxgD4v5kJqZ2uWLlE&m=FDVPKa2SqwpyYOTmA2
> > > > >
> > >
> >zGdscCPa1FVdQG3Zbr4tHrp38&s=fjg7wArWvYLJlgEGKijK6W6ECAxGk660UrPF3rAr4Rs&e=
> > > >
> > > > I skimmed over the patchset and it is an ABI breaker, so I
> > > > think the policy demands to announce it on 2.1 and merge only
> > > > in 2.2 release.
> > >
> > > I tested with a subset of the patches and the ol_flags.rss bit is being
> set
> > > correctly.
> > >
> > > It could be merged and available in DPDK 2.1 with a config parameter.
> > > However,
> > > even with this we'd need to assess the rest of the changes and
> compatibility
> > > with OVS.
> > >
> > > >
> > > > Maybe it is possible to separate the ol_flags fix into a
> > > > smaller and simple patch to be accepted as bugfix yet in 2.1.
> > >
> > > That would be ideal.
> > >
> > > >
> > > >
> > > > > >> > Otherwise, should we avoid using the vectorized version?
> > > > > >>
> > > > > >> that's debatable - from a performance view it may be better to
> leave
> > > it
> > > > > >>in
> > > > > >> and take the hit elsewhere for the time being if there's a
> possibility
> > > > > >>that
> > > > > >> it will be changed in DPDK later.
> > > > > >
> > > > > >With a loop to reset the rss after the rte_vhost_dequeue_burst()
> call
> > > I'm
> > > > > >seeing a drop of ~100kpps in vhost performance. Rx vectoristion
> gives a
> > > > > >gain
> > > > > >of about ~1 mpps on my system for the phy2phy cases.
> > > > > >
> > > > > >Using the ol_flags check is the right option when DPDK supports
> setting
> > > it
> > > > > >correctly with rx vectorisation. In the meantime there's choice of
> using
> > > > > >the
> > > > > >reset loop or removing rx vectorisation - what do you think?
> > > > >
> > > > > Thanks for sharing these results.  I've observed that if OVS can't
> use
> > > the
> > > > > RSS
> > > > > hash and has to compute we lose ~2Mpps on a single flow phy2phy test.
> > > > >
> > > > > Despite this, I still think we should consider the ol_flags because:
> > > > >
> > > > > * DPDK drivers (other than ixgbe) should use ol_flags as well to mark
> the
> > > > >   RSS hash as valid
> > > > > * ixgbe_recv_pkts_vec() will report PKT_RX_RSS_HASH in future
> releases
> > > (the
> > > > >   patch you sent will be effective since DPDK 2.2, right?)
> > > >
> > > > I agree with the above.
> >
> > I'm also seeing a ~3 mpps drop in phy2phy when not using the hardware hash.
> > It's a ~25% drop on phy2phy vs. a ~2.5% drop on the vhost interface with
> the
> > hash reset workaround. There may not be a DPDK fix that we can incorporate
> > until towards the end of the year (DPDK 2.2?) so IMHO, with this size of
> > performance drop it would be better to use the workaround until there's a
> > DPDK fix.
> 
> What happens if you just set of_flags in DPDK when there is a
> valid hash with the proposed patch from Daniele?

I'm assuming performance would be fine with that. I haven't gone through the
code to see if there's a simple DPDK patch to enable just that yet.

> 
> fbl
>