[ovs-dev] Hitless resynchronisation of forwarding state

[Jarno Rajahalme]

OVS already has the notion of “generation ID” on each flow table entry, in the form of a version range.

Overall, I’d rather see the bundle mechanism extended to cover meters and groups (i.e., extend the group and meter tables to support the same version range mechanism). More comments below.

> On Jan 25, 2016, at 2:40 AM, Jan Scheurich <jan.scheurich at ericsson.com> wrote:
> 
>> 
>> I don't see how the proposed feature is better than what we already have.  It saves a little memory temporarily, but memory isn't
>> usually a big deal in typical OVS environments.  Is there some other advantage?
> 
> A couple of points that come into my mind:
> 
> 1. OpenFlow bundle depends on OF 1.4. Not many controllers support OF 1.4 and its bundle mechanism yet. Our proposal has no such dependency.
> 

Any new extension needs new controller support. Also, it should not be hard to make bundles available on earlier OpenFlow versions as an extension, if that is desirable. At least there would be a path for getting rid of the extension by (eventually) moving to OpenFlow 1.4.

> 2. The OVS bundle implementation does not cover groups and meters (yet). Our proposal covers all flow state.
> 

As said, I’d rather see a contribution that adds the groups and meters support to OpenFlow bundles, rather than adding another non-standard extension for doing essentially the same thing.

> 3. Memory to store millions of bundled OFP messages may well be significant even on typical compute nodes running OVS.
> 

You don’t actually have to start from scratch. Bundles can be used to change only the flows (and hopefully meters and groups) that need to be modified. A simple “ovs-ofctl —bundle replace-flows” command does that already (but not for meters and groups yet).

> 4. How quickly can OVS install a million of bundled flow_mod messages after commit?  It appears that the main ofproto_mutex is locked all the time. In our proposal all fresh flow state is installed immediately at reception. The cleanup of stale flow state after a resync_fisnih command can be handled by a background walker thread. No locking required.
> 

See above for the “millions”. I recall that we already postpone cleanup of stale state to be done outside of the commit locking. Maybe the current code could be improved in releasing memory for the bundled messages as soon as the new flow entry has been added, so that at most there would be two copies of each rule (the old flow table entry and the new bundle message, or the old and the new flow table entries).

Also, maybe there are ways of decoupling the version number bump from the bundle commit, locking wise. Any parallel table changes should be made to take an effect in future versions after the bundle has been committed. We’d have to decide how to treat conflicting flow mods, though, e.g., if a flow entry is added in the tail end of a bundle (version n + 1), and a concurrent flow mod deletes the same flow (version n + 2), but the flow did not exist at the time of the delete. Then in the version n + 2 the flow added by the bundle would still exist. On the other hand, if concurrent changes are not an issue, then why would holding ofproto_mutex be an issue?

> 5. During transmission of a "resync bundle" (delete all, download all state) a controller would have to either buffer new flow/group/meter mods or insert them into the bundle. Both requires additional controller logic. In our proposal insertion/modification/removal of flows/groups/meters can happen asynchronously and almost fully uncoordinated with the resync download, allowing a cleaner decoupling in the controller between applications generating flow state and the generic switch handler function (e.g. FRM in ODL).
> 

I don’t know enough of FRM or ODL to have a comment here. Are you saying that ODL would have some fundamental difficulty in implementing the controller side of OpenFlow bundles? Adding an OpenFlow message to a bundle is fundamentally not hard, as the whole OpenFlow message is encapsulated. Yes, some additional logic is needed, but it should not be that hard to implement. I know that this may not be a fair comparison, but adding bundle support to ovs-ofctl was not that difficult (see commit db5076eee).

  Jarno

> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev