[ovs-discuss] [OVN] Incremental processing patches

Han Zhou zhouhan at gmail.com
Tue May 7 21:16:16 UTC 2019

On Tue, May 7, 2019 at 9:56 AM Justin Pettit <jpettit at ovn.org> wrote:
> Hi, Daniel.  I don't think this is a bad approach.  However, at the
moment, Han's work is with ovn-controller and ddlog is with ovn-northd.
We've talked about using ddlog in ovn-controller, but there's been no work
in that direction yet.  At this point, I think Han's patches should be
evaluated on their own merits and not held back for ddlog.  (Unless someone
is looking to run with it.  :-) )
> --Justin
> > On May 7, 2019, at 9:04 AM, Daniel Alvarez Sanchez <dalvarez at redhat.com>
> >
> > Hi folks,
> >
> > After some conversations with Han (thanks for your time and great
> > talk!) at the Open Infrastructure Summit in Denver last week, here I
> > go with this - somehow crazy - idea.
> >
> > Since DDlog approach for incremental processing is not going to happen
> > soon and Han's reported his patches to be working quite well and seem
> > to be production ready (please correct me if I'm wrong Han), would it
> > be possible to somehow enable those and then drop it after DDlog is in
> > a good shape?
> >
> > Han keeps rebasing them [0] and I know we could use them but I think
> > that the whole OVN project would get better adoption if we could have
> > them in place in the main repo. The main downside is its complexity
> > but I wonder if we can live with it until DDlog becomes a reality.
> >
> > Apologies in advance if this is just a terrible idea since I'm not
> > fully aware of what this exactly involves in terms of technical
> > complexity and feasibility. I believe that it'll make DDlog harder as
> > it'll have to deal with both approaches at the same time but looks
> > like the performance benefits are huge so worth to at least consider
> > it?
> >
> > Thanks a lot!
> > Daniel
> >

Thanks Daniel and Justin!

Daniel, I can't guarantee anything, but from my perspective the patches are
production ready, at least it has been used in eBay for months without any
issues. Without it we would have been suffering from the high CPU load on
all OVN HVs. For your concerns with DDlog, it should not make DDlog harder.
Instead, it should help, since the dependencies will be much more clear,
which is needed for rewriting with DDlog, and makes it more straightforward.

For the complexity, it surely is more complex than just simply recomputing,
and supposed to be more complex than DDlog, too. However, I don't think it
is really too complex to maintain - at least I was able to rebase it from
version to version (although rebasing such big change is always a pain for
me). What's special for this approach to stay maintainable is that the
change_handler implementation is optional, so that we can choose to
implement the simple but effective ones first, and leave the less effective
but complex ones in the future, or never - at least we don't lose anything.
The current version, with most frequent changes handled incrementally, is
effective and not very complex, from my perspective.

I think Justin had a good point that the patches should be evaluated on
their own, without considering DDlog. Although I support DDlog in the long
run, I didn't get time to start any serious work on it yet (my apology). I
want to work on that, but now that's not the highest priority for the OVN
scalability since my incremental patches solved most problems for me w.r.t
ovn-controller. I think maybe we should wait and see how DDlog works for
northd first, so that we get more experience with DDlog and avoid pitfalls.
I guess even if we start full speed on rewriting ovn-controller with DDlog,
it may take us at least one more release cycle to achieve same stability
and performance as the current incremental processing patches, if not too
optimistic. So I'd support upstreaming the incremental processing patches
first, regardless of the DDlog schedule. And I admit there is a strong
personal reason: maintaining branches and keep rebasing is not fun at all.

I'd like to hear more from Ben and Mark, who both worked on this with me
and helped a lot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20190507/b27c3e42/attachment.html>

More information about the discuss mailing list