[ovs-dev] Update on DDlog port of ovn-northd
Mark Michelson
mmichels at redhat.com
Thu Jul 18 17:18:12 UTC 2019
Hi, just wanted to send a follow-up and a test result.
I got everything up and running (yay!). So I set up a server and ran a
sandbox test using ovn with traditional C-based ovn-northd and then ran
the same test using ovn with DDlog-based ovn-northd.
The test script can be found here:
https://gist.github.com/putnopvut/3df7156b6d44b81d75a598262c5e959b
The test sets up a network with 159 logical switches all connected to a
single logical router. Each logical switch has 92 logical switch ports.
For every 2 logical switch ports we add, we create a new port group, and
ACLs are created for each port group. The important part of this test is
that the final call to ovn-nbctl in each iteration of the loop uses
--wait=sb. In other words, each new logical switch port we add has to
wait for ovn-northd to process it completely before we can start adding
the next logical switch port.
With traditional C-based ovn-northd:
[root at wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh
real 128m59.280s
user 1m27.601s
sys 2m0.986s
With DDlog-based ovn-northd:
[root at wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh
real 33m40.586s
user 1m26.976s
sys 1m59.784s
As you can see, the test takes about a quarter of the time to complete
when running with DDlog. So far, it seems like a good improvement.
As a point of reference, if I remove the --wait=sb from the test, then
the result is:
[root at wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh
real 3m59.696s
user 1m18.937s
sys 1m44.669s
This test was run with DDlog-based ovn-northd, but the result is nearly
identical with C-based ovn-northd since the test no longer is bound by
ovn-northd's performance.
I'm a bit surprised that with DDlog, adding --wait=sb to each iteration
causes such a large difference in the overall test time (~4 minutes up
to ~34 minutes).
My next step is going to be to plot the time of each individual
iteration during the test. I'll share those results when I have them
ready. I don't want to speculate on anything until I have that data to
share.
Thanks,
Mark Michelson
On 7/15/19 12:34 PM, Leonid Ryzhyk wrote:
> Hi Mark,
>
> Thanks for the feedback. We will add the `--version` flag.
>
> Yes, `stack install` merely copies the two DDlog executable to
> `~/.local/bin/` (or a custom path specified using `--local-bin-path
> <custom_path>`).
>
> Also, just want to mention that we maintain up-to-date binary releases
> of DDlog at `https://github.com/vmware/differential-datalog/releases`,
> so most users should not need to install the Haskell tool stack and
> compile DDlog. When using binary releases, all you have to do is add the
> `ddlog/bin` directory to `$PATH`.
>
> Leonid
>
> On Mon, Jul 15, 2019 at 6:30 AM Mark Michelson <mmichels at redhat.com
> <mailto:mmichels at redhat.com>> wrote:
>
> I did install DDlog originally back in early December last year. So
> there are probably remnants of that still present. However, I did
> perform a `stack install` using an updated pull of master on Friday. So
> I guess the `stack install` didn't get rid of the old installation?
>
> Also, there's no `ddlog --version` or anything similar to see what
> version of DDlog is installed. That could be a nice feature to have in
> the near future.
>
> On 7/12/19 12:40 PM, Leonid Ryzhyk wrote:
> > Thanks for trying it out! Sounds like you have a very old
> version of
> > ddlog. If you install from source, please make sure that you run
> `stack
> > install` and that there is no other version off ddlog in your
> path other
> > than the one created by `stack install`.
> >
> > Leonid
> >
> > On Fri, Jul 12, 2019, 9:05 AM Mark Michelson <mmichels at redhat.com
> <mailto:mmichels at redhat.com>
> > <mailto:mmichels at redhat.com <mailto:mmichels at redhat.com>>> wrote:
> >
> > On 7/12/19 3:29 AM, Leonid Ryzhyk wrote:
> > > Dear OVN developers,
> > >
> > > This is a brief update on the state of the DDlog port of
> ovn-northd.
> > >
> > > We completed the initial implementation of ovn-northd in DDlog
> > few months
> > > ago. Justin kindly
> > > helped to integrate it with OVN, so that it can be used as
> a drop-in
> > > replacement for the C
> > > version (and passes all the tests in the OVN test suite).
> The DDlog
> > > implementation does
> > > not have any of the new features/improvements added in
> April 2019
> > or later.
> > >
> > > ## Repository
> > >
> > > The code is in the `ddlog-dev` branch of the `ovn-org/ovn`
> > repository:
> > > https://github.com/ovn-org/ovn/tree/ddlog-dev
> >
> > Hi Leonid,
> >
> > I ran into an issue when attempting to build ovn-northd. I
> successfully
> > installed DDLog, but then I encountered this issue when
> building OVN:
> >
> > ddlog -i ovn/northd/ovn_northd.dl -L
> > /home/putnopvut/differential-datalog/lib
> > ddlog: Failed to parse input file: "ovn/northd/ovn_northd.dl"
> (line 94,
> > column 5):
> > unexpected "&"
> > expecting "not", variable name, relation name, "var",
> expression or "."
> >
> > The line in question looks like this:
> >
> > &SwitchPort(.lsp = lsp, .sw = &sw),
> >
> > Any idea what's gone wrong here?
> >
> > Thanks,
> >
> > Mark Michelson
> >
> >
> >
> > >
> > > ## Documentation
> > >
> > > Building and using ovn-northd-ddlog:
> > >
> >
> https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/design.md
> > >
> > > Debugging ovn-northd-ddlog:
> > >
> >
> https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/debugging.md
> > >
> > > ## Preliminary performance results
> > >
> > > Han Zhou kindly tested ovn-northd-ddlog with his OVN scale
> test
> > and even
> > > found a nasty
> > > performance bug in the process (thanks, Han!). He reports
> that DDlog
> > > speeds up the test
> > > by almost a factor of 10:
> > >
> > > - ddlog version: 7:39min
> > > - C version: 67:47min
> > >
> > > This is great, and in fact profiling shows that there is still
> > plenty of
> > > space for
> > > improvement. He also reports a 10+ times increase in memory
> > footprint:
> > >
> > > - ddlog version: 1944696KB
> > > - C version: 147984KB
> > >
> > > Again, we are working on a number of optimizations, which
> should
> > reduce
> > > this overhead; although it will never be as low as C,
> since DDlog
> > > fundamentally
> > > needs to cache more state to enable fast incremental
> computation.
> > >
> > > Han also used DDlog's record&replay feature to capture all
> northd
> > > transactions
> > > performed by the scale test in a format that can be replayed
> > against the
> > > standalone DDlog executable without having to reproduce Han's
> > OpenStack
> > > setup.
> > > The replay file is here: http://ryzhyk.net/replay.tgz
> > >
> > > Instructions for replaying this script:
> > >
> >
> https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/debugging.md#record-and-replay-ddlog-execution
> > > The script will run for a few
> > > minutes and finally print some profiling information,
> including the
> > > breakdown of
> > > DDlog's CPU and memory usage.
> > >
> > > ## Next steps
> > >
> > > We seek help from the OVN community in maintaining
> > ovn-northd-ddlog. The
> > > first
> > > step is to start porting new OVN features introduced in
> the last
> > few months
> > > to
> > > DDlog.
> > >
> > > Leonid
> >
> > Hi Leonid,
> >
> > I attempted
> >
> > > _______________________________________________
> > > dev mailing list
> > > dev at openvswitch.org <mailto:dev at openvswitch.org>
> <mailto:dev at openvswitch.org <mailto:dev at openvswitch.org>>
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > >
> >
>
More information about the dev
mailing list