[ovs-discuss] packet reorderoing

Gray, Mark D mark.d.gray at intel.com
Wed Mar 4 19:08:03 UTC 2015


> From: Ben Pfaff [mailto:blp at nicira.com]
> Sent: Wednesday, March 4, 2015 6:12 PM
> On Wed, Mar 04, 2015 at 05:05:33PM +0000, Gray, Mark D wrote:
> > > From: Ben Pfaff [mailto:blp at nicira.com]
> > > Sent: Wednesday, March 4, 2015 4:59 PM
> > >
> > > On Wed, Mar 04, 2015 at 04:53:45PM +0000, Gray, Mark D wrote:
> > > > > From: Ben Pfaff [mailto:blp at nicira.com]
> > > > > Sent: Wednesday, March 4, 2015 4:51 PM On Wed, Mar 04, 2015 at
> > > > > 04:42:36PM +0000, Gray, Mark D wrote:
> > > > > > >
> > > > > > > On Wed, Feb 25, 2015 at 11:28:56AM -0600, swair shah wrote:
> > > > > > > > When packets which does not match any rule in ovs, are
> > > > > > > > buffered at the ingress switch, say first n packets are
> > > > > > > > buffered. Switch sends a flow_mod and message and
> > > > > > > > packet_out for the buffered packets, meanwhile if some
> > > > > > > > more packets of the same flow arrive (say n+1 to m),
> > > > > > > > they'll also get buffered (as
> > > flow_mod) hasn't arrived yet.
> > > > > > > >
> > > > > > > > Once flow_mod message arrives at the switch then
> > > > > > > > subsequent packets of the flow will be forwarded but
> > > > > > > > packets (n+1 to m) will still be buffered waiting for
> > > > > > > > their packet_outs. This results in reordering of
> > > > > > > packets.
> > > > > > > >
> > > > > > > > Is there a way around this, besides from proactively
> > > > > > > > setting up a
> > > flow?
> > > > > > > >
> > > > > > > > Is there a way to match flow_mod to existing buffered packets?
> > > > > > >
> > > > > > > Reordering can happen.  There isn't a way to avoid it
> > > > > > > entirely (besides proactive flow setup).  It isn't usually a
> > > > > > > problem, though, because most protocols only send one packet
> > > > > > > before they receive a reply from the opposite end.
> > > > > >
> > > > > > The only way that I can think to nail up a flow is via dpctl.
> > > > > > Is this how you suggest to do this?
> > > > >
> > > > > Can you be more specific about what you want to do?  I can think
> > > > > of a few possibilities.
> > > >
> > > > I don't know what Swair wants but I would like to be certain to
> > > > avoid reordering of packets traversing the vswitch. I suspect the
> > > > way to do this is to add a flow into the datapath and make it
> > > > persistent so that upcalls are never triggered.
> > >
> > > I don't see how that can solve the problem that he is reporting.
> > > Did you read his whole scenario?  If we knew how to handle the
> > > packet in the datapath at the beginning of the scenario then we
> > > would not be sending it to the controller.
> >
> > I see the confusion, he is referring to using a controller and it not
> > matching the switch, I am thinking of missing the datapath. My bad ..
> > >
> > > I doubt that this is a real problem that needs to be solved, because
> > > it's been in the back of my mind since about 2008 as a possible
> > > concern but this is the
> >
> > It has been a concern of mine for a while and I also expected it to
> > come up at some stage but it never did. I don't think it is an issue
> > in most cloud deployments but could see it be an issue in NFV
> > deployments as usually there would be an expectation that a router or
> > another network element wouldn't reorder.
> >
> > However, hypothetically, how would you resolve this?
> 
> One way would be to send only the first packet in a given microflow to the
> controller and buffer the rest.  OpenFlow provides three ways for controllers
> to handle packet-ins:

I was talking about the similar problem across the dpif interface. 

* Packet group A arrives at NIC and get sent to ofproto as an upcall. 

* Upcall processes some packets from A and calls operate() from dpif.
operate() will execute ops, a FLOW_PUT followed by EXECUTEs.
However, some packets may not have been executed are still
buffered in the vswitchd.
* At this point, packet group B arrives at the NIC but matches on the fast path
as the flow has already been "put" and are sent directly without having an 
upcall. 

The remaining packets from group A that are buffered in the
vswitchd will arrive out of order. How could we resolve this?

> 
>         * The controller can send a flow_mod that says how to handle it
>           and all similar packets.  In this case, the switch would treat
>           the packet and then all the buffered packets in order in the
>           way the controller requested, then set up a datapath flow.
> 
>         * The controller can send a packet_out that says how to handle
>           just that single packet.  In this case, the switch would
>           handle the packet in the specified way, then send the next
>           buffered packet to the controller.  (This is not ideal, I
>           guess, because it inserts at least a RTT delay between each
>           packet to the controller.)
> 
>         * The controller never says what to do with the packet and the
>           switch expires it from its buffers a few seconds later.  I
>           guess at that point the switch might as well just drop the
>           other buffered packets since they're seconds old at that point
>           and may even have already been retransmitted.



More information about the discuss mailing list