[ovs-dev] [PATCH v3 1/4] dpif-netdev: Skip EMC lookup/insert for recirc packets

Fischetti, Antonio antonio.fischetti at intel.com
Wed Aug 23 14:35:40 UTC 2017


> -----Original Message-----
> From: Jan Scheurich [mailto:jan.scheurich at ericsson.com]
> Sent: Thursday, August 17, 2017 1:42 PM
> To: Fischetti, Antonio <antonio.fischetti at intel.com>; Darrell Ball
> <dball at vmware.com>; dev at openvswitch.org
> Subject: RE: [ovs-dev] [PATCH v3 1/4] dpif-netdev: Skip EMC lookup/insert for
> recirc packets
> 
> Hi Antonio,
> 
> > > Is there a reason to assume that a deterministic selection on some non-
> > random
> > > criteria like the recirculation count will on average (over deployments
> > and
> > > applications) give a better performance than a random selection?
> >
> > [Antonio]
> > If we consider latency and jitter a deterministic solution should be
> > more preferable than a solution which behaves differently depending
> > on the particular values of the packet fields, eg the IP addresses.
> 
> Do you have measurements showing that latency is significantly affected
> by EMC hit vs DPCLS hit?  I wouldn't think so.  Only throughput should vary.
> 

[Antonio]
Agree. 
What I meant to say is that - broadly speaking - it should be
preferable to adopt solutions that seem to be more deterministic, 
especially in a Telco deployment.
This approach - at least at a first glance - seems to be more deterministic
than other approaches like the "RSS hash threshold method" because
the latter can treat the packet differently depending on their header.

IMPO it could be good to have this approach in parallel with some other 
strategies - like the "RSS hash threshold method" - because they operate 
on two different causes/levels of the same problem.
 

> Probabilistic EMC lookup should only apply in situations where EMC is
> overloaded, meaning we have thousands of packet flows. In this case we
> maximize the aggregate throughput of the statistical flow mix. But it is not
> that a flow using EMC would see higher throughput than analogous flows that
> don't.
> 
> > > I don't believe so. For example, the number of "EMC flows" in each pass
> > through
> > > the datapath can differ hugely: 1 GRE tunnel flow in first pass (from phy
> > > port), 100K tenant flows after tunnel decapsulation. Or 100K tenant flows
> > in
> > > first pass (from VM) but 1 flow after NSH encapsulation in second pass.
> >
> > [Antonio]
> > Maybe I'm wrong but shouldn't the different flows encapped in a GRE
> > tunnel hit the EMC in different locations? Because even if they all have the
> > same outer IP addresses, they differ in the L4 ports so the 5-tuple hash
> > - and the emc locations - should vary. Same thing for NSH encapsulation?
> 
> Neither GRE nor NSH packets have L4 ports for RSS hashing. GRE is a separate
> IP protocol (not UDP). All packets of a GRE tunnel share the same pair of IP
> addresses. NSH is even a non-IP protocol.
> 
> > > I believe a random selection with dynamically adapted probability is the
> > best
> > > we can do without a priori knowledge about the traffic patterns and
> > pipeline
> > > organization.
> >
> > [Antonio]
> > This proposal is orthogonal to other approaches that look at the usage
> > of the single locations, eg policies not to overwrite active locations or to
> > reduce in general the emc usage.
> > I think we should consider both the two strategies to tackle two different
> > aspects of the thrashing and use emc more efficiently:
> >  1. skip emc lookup/insert for recirc packets (which is only activated when
> >    emc entries exceeds EMC_RECIRCT_NO_INSERT_THRESHOLD);
> >  2. any other strategy that limits emc usage or offers a better entries
> > eviction.
> >
> > So - being agnostic of what's the traffic type - if we have 100k flows
> > that could potentially be recirculated:
> >  1. allows to tackle the thrashing due to recirculation, which is activated
> >     when the emc entries exceeds a threshold.
> >  2. allows to limit the emc usage to fewer flows because we don't want
> >     100k flows to hit emc.
> 
> First of all: we only discuss limiting EMC lookups in the case of EMC overload.
> I still don't think that it is a good idea to general skip EMC lookup for
> recirculated  flows in that situation. It may be the right thing to do in some
> scenarios (e.g. GRE -> VM), but exactly the wrong in others (e.g. VM -> GRE).
> 
> If we go for a probabilistic reduction of EMC lookups we'd statistically have a
> balanced improvement in all (known and unknown) scenarios.
> 
> BR, Jan


More information about the dev mailing list