[ovs-dev] OVN: proposal of introducing nlog engine to ovn-northd

Mon Feb 1 19:52:47 UTC 2016

How about reusing Openstack Congress which is already out there and also
implements something like nlog?

--
Saurabh (सौरभ)

On Mon, Feb 1, 2016 at 11:12 AM, Ben Pfaff <blp at ovn.org> wrote:

> On Mon, Feb 01, 2016 at 10:12:49AM -0500, Russell Bryant wrote:
> > On 02/01/2016 05:03 AM, Yusheng Wang wrote:
> > > We are trying to propose a new computation model for the ovn-northd
> > > application. Currently, ovn-northd computes all logical data path
> > > flows for every change. This full computation model is simple and
> > > helped verifying the concept of logical data path design, but it also
> > > suffers performance drawback.
> > >
> > > In order to move from full computation to incremental computation,
> > > more sophisticated tool is needed to manage the complexity associated
> > > with both generating more and more complex logical data path flows
> > > and dealing with irrelevant sequence of incremental changes.
> > >
> > > Now we think it is good time to introduce the new computation model -
> > > nlog. Nlog is a specialized declarative programming language based on
> > > datalog.
> > >
> > > The nlog computation model has been proved to work well for quite a
> > > few systems. We have done a POC in which a brand new nlog compiler
> > > and nlog run-time engine are implemented. Nlog program has been
> > > written to implement basic ovn features to verify its capabilities.
> > >
> > > We see a good match of the ovn data schema with nlog computation
> > > model. Both ovn-northd and ovn-controller use ovsdb and the data to
> > > be processed are tuples. This is also true for nlog programs and
> > > minimum data transformation is needed to adapt to the new computation
> > > model.
> > >
> > > This move will take multiple phases and current ovn-northd
> > > functionality will not be impacted. The north and south side database
> > > schema will not be changed during and after this move.
> >
> > Thanks for sharing your ideas!
> >
> > I haven't heard of nlog before.  You mentioned that it has worked well
> > for quite a few systems.  Can you provide some references for where I
> > can go to learn more?
>
> I can provide some information here, because I've been talking to
> Yusheng and encouraging him to start a conversation here.  Yusheng, by
> the way, is a VMware developer working from the Beijing office, who has
> been interested in getting involved with OVN for a while and sees an
> opportunity here.  Hi Yusheng!
>
> First, nlog has nothing to do with any of at least the first page of
> hits on Google, which appear to all be about some kind of logging system
> for .NET.
>
> Instead, this "nlog" is a variant of the Datalog database language.
> Datalog is a language with a pretty long pedigree dating back to the
> 1980s, at least, but there aren't very many good resources for it
> online.  Wikipedia has a very basic introduction.  A paper I've found
> that gives some insight on how to implement it is "What You Always
> Wanted to Know About Datalog (And Never Dared to Ask)" here:
>
> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.384.5801&rep=rep1&type=pdf
>
> Personally, I think of nlog as analogous to Makefiles, in that you
> specify dependencies and formulas and there's an engine that does all
> the smart bits of figuring out what's the minimum amount of computation
> to get the outputs you want.
>
> Yusheng made a prototype implementation of nlog in Java and did a demo
> of it for me.  I found this pretty impressive.  It only took a few
> thousand lines of code and yielded something that produces incremental
> results for incremental changes to the input data.  I think we'll want
> something like that for scale at ovn-northd, at least eventually.
> (ovn-controller will also want incremental computation but I don't know
> whether it should be the same solution.)
>
> I don't know whether there are public implementations of nlog.  The best
> reference I know for use in a product is the 2014 NSDI paper on NVP,
> especially section 4.3: http://benpfaff.org/papers/net-virt.pdf
>
> > In general, I definitely agree that this problem is something we need to
> > work on.  I've been wanting to look into adding a way for users of the
> > IDL API to receive callbacks on row events (create, update, delete) so
> > that it knows which rows it should apply updates for.  That's about as
> > far as I've gone though.  I haven't written any code yet because I keep
> > staying busy with other things.
>
> That's a necessary feature to implement nlog in ovn-northd.  It's there
> now, by the way, contributed by HP.
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
>