[ovs-dev] [RFC] cirrus: Add Cirrus CI integration for FreeBSD build

[aserdean at ovn.org]

> -----Mesaj original-----
> De la: ovs-dev-bounces at openvswitch.org <ovs-dev-
> bounces at openvswitch.org> În numele Ilya Maximets
> Trimis: Wednesday, December 12, 2018 6:59 PM
> Către: aserdean at ovn.org; 'Ben Pfaff' <blp at ovn.org>
> Cc: ovs-dev at openvswitch.org
> Subiect: Re: [ovs-dev] [RFC] cirrus: Add Cirrus CI integration for FreeBSD
build
> 
> On 12.12.2018 19:50, aserdean at ovn.org wrote:
> >> -----Mesaj original-----
> >> De la: ovs-dev-bounces at openvswitch.org <ovs-dev-
> >> bounces at openvswitch.org> În numele Ilya Maximets
> >> Trimis: Wednesday, December 12, 2018 6:36 PM
> >> Către: Ben Pfaff <blp at ovn.org>
> >> Cc: ovs-dev at openvswitch.org; Alin Gabriel Serdean
> <aserdean at ovn.org>
> >> Subiect: Re: [ovs-dev] [RFC] cirrus: Add Cirrus CI integration for
> >> FreeBSD
> > build
> >>
> >> On 12.12.2018 19:24, Ben Pfaff wrote:
> >>> On Wed, Dec 12, 2018 at 07:00:28PM +0300, Ilya Maximets wrote:
> >>>> On 12.12.2018 0:06, Ben Pfaff wrote:
> >>>>> On Tue, Dec 11, 2018 at 05:34:17PM +0300, Ilya Maximets wrote:
> >>>>>> CirrusCI [1] is free for open-sorce projects and provides similar
> >>>>>> to TravisCI interfaces. One significant difference is ability to
> >>>>>> run tasks on FreeBSD instances.
> >>>>>>
> >>>>>> This patch adds simple configuration file to test OVS build on
> >>>>>> two FreeBSD releases with gcc and clang.
> >>>>>> Unit tests are commented out because they are broken for now.
> >>>>>>
> >>>>>> To enable the automated checks Cirrus CI application from GitHub
> >>>>>> Marketplace should be installed. See details in Quick Start guide
> > [2].
> >>>>>>
> >>>>>> [1] https://cirrus-ci.org
> >>>>>> [2] https://cirrus-ci.org/guide/quick-start/
> >>>>>>
> >>>>>> Signed-off-by: Ilya Maximets <i.maximets at samsung.com>
> >>>>>> ---
> >>>>>>
> >>>>>> This could be useful to test FreeBSD build automatically to early
> >>>>>> detect issues. CirrusCI looks suitable for that purpose and it's
> >>>>>> the only public CI system that I managed to find that supports
> >>>>>> FreeBSD images. Sending as RFC to start discussion.
> >>>>>>
> >>>>>> Few more differentiation points with Travis. Cirrus allows to
> >>>>>> configure number of CPUs and the amount of required memory. It
> >>>>>> also supports Linux, OSX and Windows containers. So, maybe, if
> >>>>>> we'll like it, we could drop Travis and AppVeyor and use single
> >>>>>> CI system for all the tests.
> >>>>>>
> >>>>>> I'm not advertising and not trying to convince to replace our
> >>>>>> current CI systems. I found CirrusCI just few days ago by a mail
> >>>>>> [3] in qemu-devel list. I just think that it's the only option
> >>>>>> for now to test BSD builds with public CI. And it worked fine for
> >>>>>> me with below configuration.
> >>>>>>
> >>>>>> Thoughts ?
> >>>>>>
> >>>>>> As a note: I just spotted that TravisCI adds windows support.
> >>>>>> Maybe we could use it in the future for windows builds.
> >>>>>>
> >>>>>> [3]
> >>>>>> https://lists.nongnu.org/archive/html/qemu-devel/2018-
> >> 12/msg00935.h
> >>>>>> tml
> >>>>>
> >>>>> Good idea.  It's hard for me to see what harm this would cause, so
> >>>>> I enabled cirrus-ci at github and applied this to master.
> >>>>>
> >>>>> The big weakness with appveyor is that its limits are too low to
> >>>>> run the whole OVS testsuite.  I wonder whether cirrus-ci has
> >>>>> better limits for this purpose.
> >>>>
> >>>> As far as I know, the upper limit for resources is 8CPU + 24GB RAM.
> >>>> Default timeout for tasks is 60 minutes and it, probably, could be
> >>>> extended using special config in .yml file.
> >>>
> >>> I think that the issue with appveyor is the time limit of 60 minutes.
> >>> If cirrus-ci can allow for a longer build time, it would probably
> >>> make a big difference.
> >>
> >> There is a field 'timeout_in' in .cirrus.yml that could be used to
> >> extend
> > the
> >> time limit:
> >>     https://cirrus-ci.org/faq/#instance-timed-out
> >>
> >> We could try it out.
> >>
> >> It'll be better if someone with "OVS on Windows" experience could
> >> look at this.
> >
> > Sorry for the delay.
> >
> > I'll play a bit with it tonight and send out updates.
> 
> Cool. Thanks.
> 
> I just wanted to mention that even without extending the timeout we can
> just split the test suite into separate tasks like this:
> 
>   env:
>     matrix:
>       TESTSUITEFLAGS: -1000
>       TESTSUITEFLAGS: 1001-2000
>       TESTSUITEFLAGS: 2001-
> 
>   check_script:
>     - make -j8 check TESTSUITEFLAGS="-j8 $TESTSUITEFLAGS"
> 
> And each of three tasks will have its own 60 minutes limit.
> This could help if the testsuite is the main time consuming issue.

Thanks a lot for the pointers!