[ovs-dev] [PATCH] rhel, xenserver: Run unit tests while creating rpms.

Mon Jun 30 22:55:22 UTC 2014

On Mon, Jun 30, 2014 at 02:43:35PM -0700, Gurucharan Shetty wrote:
> On Mon, Jun 30, 2014 at 2:29 PM, Flavio Leitner <fbl at redhat.com> wrote:
> >
> > Hi Gurucharan,
> >
> > I've tried to do the same with Fedora's spec but the time to run
> > "make check" when doing frequent builds is considerable.
> >
> > I had quite a few false-positives during parallel execution of
> > "make check" and that breaks the entire rpm construction.
> I have used the "make check TESTSUITEFLAGS='--recheck' " option. This
> re-runs the failed unit tests once more serially. Hopefully that
> should not cause any breakage.

I haven't tried to do that yet, good to know though.

> > I believe most of developers are running "make check".  At least
> > I know Ben does :-) So, having this during package building process
> > doesn't seem to buy us anything.
> It turned out that there were a few unit test failures on Xenserver
> for quite a while and nobody knew about it. So, the reason to do this
> is to prevent such a thing from happening.

I agree.

> > Perhaps invert the default to not do that, but if someone wants to
> > do it, then pass rpmbuild -D "make_check true"
> I will wait to hear what Ben has to say on what should be the default
> option. I am fine eitherways.

My concern now is if the Xenserver breakage would have been found with
the default to not run make check.

> > Anyway, if you still want to go ahead with this, I suggest to use
> > rpmbuild --with/without options instead.  In the snipped below, the
> > default is to not do it. But you can force to run make check by running:
> > # rpmbuild -bb --with make_check <specfile>
> > or force to not run by running:
> > # rpmbuild -bb --without make_check <specfile>
> I did look at this, but got a feeling that they were mostly used for
> conditional configure options. But now that you say it, I will defer
> to your expertise and I will go ahead and use your suggestion.

The correct place to put it is in %check section of the spec file.
It's not common to see specs using %check section, but it could be
because most of the projects don't have a testsuite.

Anyway, this is the package guide lines for Fedora:
http://fedoraproject.org/wiki/Packaging:Guidelines#Writing_a_package_from_scratch
Look for: "Test Suites" or at rpmbuild man-page look for "--nocheck"

The %check section was introduced in rpm 4.2 (2003) so I believe it is old
enough to support old distros, but the "--nocheck" option was introduced
in Aug/2012.

binutils does what I had suggested in %build section.
# Do not use %%check as it is run after %%install where libbfd.so is
# rebuild with -fvisibility=hidden no longer being usable in its shared form.
...
    # rpmbuild parameters:
    # --define "binutils_target arm-linux-gnu" to create arm-linux-gnu-binutils.
    # --with debug: Build without optimizations and without splitting the debuginfo.
    # --without testsuite: Do not run the testsuite.  Default is to run it.
    # --with testsuite: Run the testsuite.  Default --with debug is not to run it.
...    

The autoconf does better. It mixes %check and the --without parameter
to work around the fact that "--nocheck" is too recent yet:
...
# run "make check" by default
%bcond_without check
...
%check
%if %{with check}
make check # TESTSUITEFLAGS='1-198 200-' # will disable nr. 199.
%endif

The %with/%without/%bcond_* macros dates back to 2004, so it's seems
better to go with autoconf solution. (Enabled by default, using
bconf_without macro and the condition in %check section).

fbl

> > This would be the rpm spec part:
> >
> > %define _default_make_check 0
> > %if %{?_with_make_check: 1}%{!?_with_make_check: 0}
> > %define with_make_check 1
> > %else
> > %define with_make_check %{?_without_make_check: 0}%{!?_without_make_check: %{_default_make_check}}
> > %endif
> > ...
> >
> > %build
> > ...
> > %if %{with_make_check}
> >     if make check TESTSUITEFLAGS='%{_smp_mflags}' ||
> >         make check TESTSUITEFLAGS='--recheck'; then :;
> >     else
> >         cat tests/testsuite.log
> >         exit 1
> >     fi
> > %endif
> >
> >
> > More info:
> > http://www.rpm.org/wiki/PackagerDocs/ConditionalBuilds
> > http://rpm5.org/docs/api/conditionalbuilds.html
> >
> > Thanks!
> > fbl
> >
> > On Mon, Jun 30, 2014 at 01:10:44PM -0700, Gurucharan Shetty wrote:
> >> For RHEL, Fedora and Xenserver, run unit tests while
> >> building rpms.  This may catch some cross-platform bugs.
> >>
> >> The commit also allows the users to optionally skip unit tests.
> >> (On debian, the default is to run unit tests. For consistency,
> >> do the same for rpms.)
> >>
> >> VMware-BZ: 1267127
> >>
> >> CC: Flavio Leitner <fbl at redhat.com>
> >> CC: Ben Pfaff <blp at nicira.com>
> >> Signed-off-by: Gurucharan Shetty <gshetty at nicira.com>
> >> ---
> >>  INSTALL.Fedora                    |    5 +++++
> >>  INSTALL.RHEL                      |    5 +++++
> >>  INSTALL.XenServer                 |    5 +++++
> >>  rhel/openvswitch-fedora.spec.in   |   19 +++++++++++++++++++
> >>  rhel/openvswitch.spec.in          |   19 +++++++++++++++++++
> >>  xenserver/openvswitch-xen.spec.in |   20 ++++++++++++++++++++
> >>  6 files changed, 73 insertions(+)
> >>
> >> diff --git a/INSTALL.Fedora b/INSTALL.Fedora
> >> index d711e24..bc4c5d8 100644
> >> --- a/INSTALL.Fedora
> >> +++ b/INSTALL.Fedora
> >> @@ -45,6 +45,11 @@ $HOME/rpmbuild/SOURCES.
> >>
> >>     This produces one RPM: "openvswitch".
> >>
> >> +   The above command automatically runs the Open vSwitch unit tests.
> >> +   To disable the unit tests, run:
> >> +
> >> +       rpmbuild -D "make_check false" -bb rhel/openvswitch-fedora.spec
> >> +
> >>  5. On Fedora 17, to build the Open vSwitch kernel module, run:
> >>
> >>       rpmbuild -bb rhel/openvswitch-kmod-fedora.spec
> >> diff --git a/INSTALL.RHEL b/INSTALL.RHEL
> >> index de85199..685b535 100644
> >> --- a/INSTALL.RHEL
> >> +++ b/INSTALL.RHEL
> >> @@ -94,6 +94,11 @@ $HOME/rpmbuild/SOURCES.
> >>
> >>     This produces two RPMs: "openvswitch" and "openvswitch-debuginfo".
> >>
> >> +   The above command automatically runs the Open vSwitch unit tests.
> >> +   To disable the unit tests, run:
> >> +
> >> +       rpmbuild -D "make_check false" -bb rhel/openvswitch.spec
> >> +
> >>     If the build fails with "configure: error: source dir
> >>     /lib/modules/2.6.32-279.el6.x86_64/build doesn't exist" or similar,
> >>     then the kernel-devel package is missing or buggy.  Go back to step
> >> diff --git a/INSTALL.XenServer b/INSTALL.XenServer
> >> index ba25e43..5177ef8 100644
> >> --- a/INSTALL.XenServer
> >> +++ b/INSTALL.XenServer
> >> @@ -36,6 +36,11 @@ RPMs for Citrix XenServer is the DDK VM available from Citrix.
> >>     "openvswitch", "openvswitch-modules-xen", and
> >>     "openvswitch-debuginfo".
> >>
> >> +   The above command automatically runs the Open vSwitch unit tests.
> >> +   To disable the unit tests, run:
> >> +
> >> +       rpmbuild -D "make_check false" -bb rhel/openvswitch-xen.spec
> >> +
> >>  Build Parameters
> >>  ----------------
> >>
> >> diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in
> >> index 44cd7b9..3fb8dd2 100644
> >> --- a/rhel/openvswitch-fedora.spec.in
> >> +++ b/rhel/openvswitch-fedora.spec.in
> >> @@ -6,6 +6,10 @@
> >>  # are permitted in any medium without royalty provided the copyright
> >>  # notice and this notice are preserved.  This file is offered as-is,
> >>  # without warranty of any kind.
> >> +#
> >> +# When building, if tests have to be skipped, define the variable 'make_check'.
> >> +# For example:
> >> +# rpmbuild -D "make_check false" -bb rhel/openvswitch.spec
> >>
> >>  #%define kernel 2.6.40.4-5.fc15.x86_64
> >>
> >> @@ -26,6 +30,11 @@ Requires(post):  systemd-units
> >>  Requires(preun): systemd-units
> >>  Requires(postun): systemd-units
> >>
> >> +%if %{?make_check:0}%{!?make_check:1}
> >> +# %{make_check} is not defined
> >> +%define make_check true
> >> +%endif
> >> +
> >>  %description
> >>  Open vSwitch provides standard network bridging functions augmented with
> >>  support for the OpenFlow protocol for remote per-flow control of
> >> @@ -74,6 +83,16 @@ install -d -m 755 $RPM_BUILD_ROOT/var/lib/openvswitch
> >>  # Get rid of stuff we don't want to make RPM happy.
> >>  (cd "$RPM_BUILD_ROOT" && rm -f usr/lib/lib*)
> >>
> >> +%check
> >> +%if "%{make_check}" != "false"
> >> +    if make check TESTSUITEFLAGS='%{_smp_mflags}' ||
> >> +       make check TESTSUITEFLAGS='--recheck'; then :;
> >> +    else
> >> +        cat tests/testsuite.log
> >> +        exit 1
> >> +    fi
> >> +%endif
> >> +
> >>  %clean
> >>  rm -rf $RPM_BUILD_ROOT
> >>
> >> diff --git a/rhel/openvswitch.spec.in b/rhel/openvswitch.spec.in
> >> index 18bc10c..da526ac 100644
> >> --- a/rhel/openvswitch.spec.in
> >> +++ b/rhel/openvswitch.spec.in
> >> @@ -6,6 +6,10 @@
> >>  # are permitted in any medium without royalty provided the copyright
> >>  # notice and this notice are preserved.  This file is offered as-is,
> >>  # without warranty of any kind.
> >> +#
> >> +# When building, if tests have to be skipped, define the variable 'make_check'.
> >> +# For example:
> >> +# rpmbuild -D "make_check false" -bb rhel/openvswitch.spec
> >>
> >>  Name: openvswitch
> >>  Summary: Open vSwitch daemon/database/utilities
> >> @@ -21,6 +25,11 @@ Buildroot: /tmp/openvswitch-rpm
> >>  Requires: openvswitch-kmod, logrotate, python
> >>  BuildRequires: openssl-devel
> >>
> >> +%if %{?make_check:0}%{!?make_check:1}
> >> +# %{make_check} is not defined
> >> +%define make_check true
> >> +%endif
> >> +
> >>  %description
> >>  Open vSwitch provides standard network bridging functions and
> >>  support for the OpenFlow protocol for remote per-flow control of
> >> @@ -67,6 +76,16 @@ rm \
> >>
> >>  install -d -m 755 $RPM_BUILD_ROOT/var/lib/openvswitch
> >>
> >> +%check
> >> +%if "%{make_check}" != "false"
> >> +    if make check TESTSUITEFLAGS='%{_smp_mflags}' ||
> >> +       make check TESTSUITEFLAGS='--recheck'; then :;
> >> +    else
> >> +        cat tests/testsuite.log
> >> +        exit 1
> >> +    fi
> >> +%endif
> >> +
> >>  %clean
> >>  rm -rf $RPM_BUILD_ROOT
> >>
> >> diff --git a/xenserver/openvswitch-xen.spec.in b/xenserver/openvswitch-xen.spec.in
> >> index ae29649..06c8742 100644
> >> --- a/xenserver/openvswitch-xen.spec.in
> >> +++ b/xenserver/openvswitch-xen.spec.in
> >> @@ -17,6 +17,11 @@
> >>  #      -D "kernel_version 2.6.32.12-0.7.1.xs5.6.100.323.170596"
> >>  #      -D "kernel_flavor xen"
> >>  #      -bb /usr/src/redhat/SPECS/openvswitch-xen.spec
> >> +#
> >> +# To disable unit test run, define "make_check" as "false".
> >> +# for example:
> >> +#
> >> +#   rpmbuild -D "make_check false" -bb xenserver/openvswitch-xen.spec
> >>
> >>  %if %{?openvswitch_version:0}%{!?openvswitch_version:1}
> >>  %define openvswitch_version @VERSION@
> >> @@ -37,6 +42,11 @@
> >>  # build-supplemental-pack.sh requires this naming for kernel module packages
> >>  %define module_package modules-%{kernel_flavor}-%{kernel_version}
> >>
> >> +%if %{?make_check:0}%{!?make_check:1}
> >> +# %{make_check} is not defined
> >> +%define make_check true
> >> +%endif
> >> +
> >>  Name: openvswitch
> >>  Summary: Open vSwitch daemon/database/utilities
> >>  Group: System Environment/Daemons
> >> @@ -134,6 +144,16 @@ rm \
> >>
> >>  install -d -m 755 $RPM_BUILD_ROOT/var/lib/openvswitch
> >>
> >> +%check
> >> +%if "%{make_check}" != "false"
> >> +    if make check TESTSUITEFLAGS='%{_smp_mflags}' ||
> >> +       make check TESTSUITEFLAGS='--recheck'; then :;
> >> +    else
> >> +        cat tests/testsuite.log
> >> +        exit 1
> >> +    fi
> >> +%endif
> >> +
> >>  %clean
> >>  rm -rf $RPM_BUILD_ROOT
> >>
> >> --
> >> 1.7.9.5
> >>
> 