[ovs-discuss] ovn-northd-ddlog and rust crate build process

Ben Pfaff blp at ovn.org
Sun Mar 7 18:30:52 UTC 2021


On Sun, Mar 07, 2021 at 12:40:56PM +0100, Frode Nordahl wrote:
> On Fri, Mar 5, 2021 at 11:17 PM Ben Pfaff <blp at ovn.org> wrote:
> >
> > On Fri, Mar 05, 2021 at 11:13:09PM +0100, Frode Nordahl wrote:
> > > However, when I got to the building of the rust crates part I found
> > > that it would both rebuild stuff when not needed and not do any
> > > parallelization for individual crate builds.
> > >
> > > This will probably itch enough for me to eventually look at it, but I
> > > thought it would be worthwhile to send this e-mail to the list and ask
> > > if anyone has ideas for how to improve re-use of already built objects
> > > and parallelization of individual crate builds for a rust build
> > > process.
> >
> > I don't think that building an individual crate is something that can be
> > parallelized.  (I'd be happy to be shown to be wrong.)
> 
> Unfortunately you're right, this fairly recent update [0] in the Rustc
> development guide suggests there is a fair amount of groundwork that
> needs to be done before any parallelizing work can come to fruition.
> 
> Some people suggest splitting code into Cargo Workspaces, and I see we
> already do that to some extent, and I kind of doubt it would be worth
> splitting any further just for the sake of faster compilation. Having
> said that, some of the generated rust code files are pretty huge, so
> maybe there could be something there.

Leonid has started work on dividing the ddlog-generated code into
multiple crates.  Some ddlog programs benefit from it already, but OVN
is the largest ddlog program out there and it doesn't divide up very
easily automatically.

Trust me, I'd like to parallelize it better myself, I have a 64-core
Threadripper 3990X here.

One thing I haven't asked Leonid yet is, would it help if we divided the
.dl files up into multiple files more granularly?


More information about the discuss mailing list