[ovs-dev] [PATCHv2] poc: Introduce Proof of Concepts (Package building)
Gregory Rose
gvrose8192 at gmail.com
Mon Feb 12 18:41:50 UTC 2018
On 2/4/2018 6:48 PM, Ansis Atteka wrote:
> From: Ansis Atteka <ansisatteka at gmail.com>
>
> This patch sets up foundations for Proof of Concepts that
> simply materialize documentation into Ansible instructions
> executed in virtualized Vagrant environment.
>
> This Proof of Concept allows to easily build:
> 1. *.deb packages on Ubuntu 16.04; AND
> 2. *.rpm packages on CentOS 7.4.
> It also sets up DEB and RPM repository over HTTP that can
> be used to pull these openvswitch packages with apt-get
> or yum from another host.
>
> This particular Proof of Concept is intended to address
> following use-cases:
> 1. for new OVS users to see how debian and rpm packages are
> built;
> 2. for developers to easily check for packaging build
> regressions;
> 3. for developers to easily share their sandbox builds
> into QE setups (opposed to manually copying binaries);
> 4. for developers to add other Proof of Concepts
> that possibly may require full end-to-end integration
> with other thirdparty projects (e.g. DPI, libvirt, IPsec)
> and need Open vSwitch packages.
>
> Tested-by: Greg Rose <gvrose8192 at gmail.com>
> Reviewed-by: Greg Rose <gvrose8192 at gmail.com>
> Signed-off-by: Ansis Atteka <aatteka at ovn.org>
Rev 2 looks good. I did a quick sanity check test and LGTM.
Reviewed-by: Greg Rose <gvrose8192 at gmail.com>
> ---
> .gitignore | 2 +
> Documentation/topics/testing.rst | 56 ++++++++++++++++++++
> Makefile.am | 3 ++
> poc/builders/Vagrantfile | 33 ++++++++++++
> poc/playbook-centos-builder.yml | 108 +++++++++++++++++++++++++++++++++++++++
> poc/playbook-ubuntu-builder.yml | 70 +++++++++++++++++++++++++
> 6 files changed, 272 insertions(+)
> create mode 100644 poc/builders/Vagrantfile
> create mode 100644 poc/playbook-centos-builder.yml
> create mode 100644 poc/playbook-ubuntu-builder.yml
>
> diff --git a/.gitignore b/.gitignore
> index 8019bee41..81faf270d 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -16,6 +16,7 @@
> *.lib
> *.pdb
> *.pyc
> +*.retry
> *.so
> *.suo
> **/*.sym
> @@ -29,6 +30,7 @@
> .dirstamp
> .libs
> .tmp_versions
> +.vagrant
> .gitattributes
> /Makefile
> /Makefile.in
> diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst
> index a49336b79..253857171 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -389,3 +389,59 @@ validate the suitability of different vSwitch implementations in a telco
> deployment environment. More information can be found on the `OPNFV wiki`_.
>
> .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home
> +
> +Proof of Concepts
> +~~~~~~~~~~~~~~~~~
> +
> +Proof of Concepts are documentation materialized into Ansible recipes
> +executed in VirtualBox or Libvirt environments orchastrated by Vagrant.
> +Proof of Concepts allow developers to create small virtualized setups that
> +demonstrate how certain Open vSwitch features are intended to work avoiding
> +user introduced errors by overlooking instructions. Proof of Concepts
> +are also helpful when integrating with thirdparty software, because standard
> +unit tests with make check are limited.
> +
> +Vagrant by default uses VirtualBox provider. However, if Libvirt is your
> +choice of virtualization technology, then you can use it by installing Libvirt
> +plugin:
> +
> + $ vagrant plugin install vagrant-libvirt
> +
> +And then appending --provider=libvirt flag to vagrant commands.
> +
> +The host where Vagrant runs does not need to have any special software
> +installed besides vagrant, virtualbox (or libvirt and libvirt-dev) and
> +ansible.
> +
> +The following Proof of Concepts are supported:
> +
> +Builders
> +++++++++
> +
> +This particular Proof of Concept demonsrtates integration with Debian and RPM
> +packaging tools:
> +
> + $ cd ./poc/builders
> +
> + $ vagrant up
> +
> +Once that command finished you can get packages from /var/www/html
> +directory. Since those hosts are also configured as repositories then
> +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d
> +configuration files on another host to retrieve packages with yum or
> +apt-get.
> +
> +hen you have made changes to OVS source code and want to rebuild packages run:
> + $ git commit -a
> + $ vagrant rsync && vagrant provision
> +
> +Whenever packages are rebuilt the Open vSwitch release number increases
> +by one and you can simply upgrade Open vSwitch by running yum or apt-get
> +update commands.
> +
> +Once you are done with experimenting you can tear down setup with:
> +
> + $ vagrant destroy
> +
> +Sometimes deployment of Proof of Concept may fail, if, for example, VMs
> +don't have network reachability to the Internet.
> diff --git a/Makefile.am b/Makefile.am
> index ed4b7fd7c..174a421e6 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -90,6 +90,9 @@ EXTRA_DIST = \
> build-aux/sodepends.py \
> build-aux/soexpand.py \
> build-aux/xml2nroff \
> + poc/builders/Vagrantfile \
> + poc/playbook-centos-builder.yml \
> + poc/playbook-ubuntu-builder.yml \
> $(MAN_FRAGMENTS) \
> $(MAN_ROOTS) \
> Vagrantfile \
> diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile
> new file mode 100644
> index 000000000..af2986789
> --- /dev/null
> +++ b/poc/builders/Vagrantfile
> @@ -0,0 +1,33 @@
> +# -*- mode: ruby -*-
> +# vi: set ft=ruby :
> +
> +VAGRANTFILE_API_VERSION = "2"
> +
> +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
> + config.ssh.insert_key = false
> +
> + # Centos-7.4 builder host
> + config.vm.define "centosbuilder" do |builder|
> + builder.vm.hostname = "centosbuilder.dev"
> + builder.vm.box = "centos/7"
> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
> + rsync__args: ["--archive", "--delete", "-z"]
> + builder.vm.provision "builder", type: "ansible" do |ansible|
> + ansible.playbook = "../playbook-centos-builder.yml"
> + ansible.sudo = true
> + end
> + end
> +
> + # Ubuntu-16.04 builder host
> + config.vm.define "ubuntubuilder" do |builder|
> + builder.vm.hostname = "ubuntubuilder.dev"
> + builder.vm.box = "generic/ubuntu1604"
> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
> + rsync__args: ["--archive", "--delete", "-z"]
> + builder.vm.provision "builder", type: "ansible" do |ansible|
> + ansible.playbook = "../playbook-ubuntu-builder.yml"
> + ansible.sudo = true
> + end
> + end
> +
> +end
> diff --git a/poc/playbook-centos-builder.yml b/poc/playbook-centos-builder.yml
> new file mode 100644
> index 000000000..71f104010
> --- /dev/null
> +++ b/poc/playbook-centos-builder.yml
> @@ -0,0 +1,108 @@
> +---
> +- hosts: all
> + become: true
> + name: builder
> + tasks:
> +
> + - name: Create Ansible Local Facts Directory
> + file: path=/etc/ansible/facts.d state=directory
> +
> + - name: Initiate Build Numbering
> + copy:
> + content: '{ "release":"1" }'
> + dest: "/etc/ansible/facts.d/builder.fact"
> + force: no
> +
> + - name: Set source directory for building
> + set_fact:
> + SOURCE: "/root/rpmbuild/SOURCES"
> +
> + - name: Reload Ansible Local Facts
> + setup: filter=ansible_local
> +
> + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd", "git"
> + yum: update_cache=yes name={{item}} state=present
> + with_items:
> + - yum-utils
> + - rpmdevtools
> + - createrepo
> + - httpd
> + - git
> +
> + - name: Remove untracked files from Open vSwitch GIT repository
> + command: chdir=/git/ovs/ git clean -xdf
> +
> + - name: Reset Open vSwitch GIT repository to last comitted state
> + command: chdir=/git/ovs/ git reset --hard
> +
> + - name: Generate spec files for easy build dependency retrieval
> + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}}
> + args:
> + chdir: /git/ovs/rhel
> + with_items:
> + - openvswitch.spec
> + - openvswitch-kmod-rhel6.spec
> +
> + - name: Install build dependencies specified from spec files
> + shell: echo "y" | yum-builddep /tmp/{{item}}
> + with_items:
> + - openvswitch.spec
> + - openvswitch-kmod-rhel6.spec
> +
> + - name: Create rpm dev tree
> + command: rpmdev-setuptree
> +
> + - name: Run "./boot.sh"
> + command: chdir=/git/ovs/ ./boot.sh
> +
> + - name: Run "./configure"
> + command: chdir=/git/ovs/ ./configure
> +
> + - name: Run "make dist"
> + command: chdir=/git/ovs/ make dist
> +
> + - name: Parse out Open vSwitch version from "configure.ac"
> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
> + register: version
> +
> + - name: Copy source tarball to rpm dev tree
> + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz {{SOURCE}}
> +
> + - name: Unarchive openvswitch source tarball
> + unarchive:
> + src: "{{SOURCE}}/openvswitch-{{version.stdout}}.tar.gz"
> + dest: "{{SOURCE}}"
> + remote_src: yes
> +
> + - name: Update release number in spec files
> + lineinfile:
> + path: "{{SOURCE}}/openvswitch-{{version.stdout}}/rhel/{{item}}"
> + regexp: '^Release:'
> + line: "Release: {{ ansible_local.builder.release }}"
> + with_items:
> + - openvswitch.spec
> + - openvswitch-kmod-rhel6.spec
> +
> + - name: Build Open vSwitch user space rpms
> + command: rpmbuild -bb --without check rhel/openvswitch.spec
> + args:
> + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}"
> +
> + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel)
> + command: rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec
> + args:
> + chdir: "{{SOURCE}}/openvswitch-{{version.stdout}}"
> +
> + - name: Copy RPM packages to /var/www/html
> + command: cp -r /root/rpmbuild/RPMS/ /var/www/html
> +
> + - name: Create RPM Package index file for repository
> + command: chdir=/var/www/html createrepo /var/www/html
> +
> + - name: Make sure Apache is running
> + systemd: state=started name=httpd
> +
> + - name: Bump up Build Number
> + copy:
> + content: '{ "release":"{{ansible_local.builder.release|int+1}}" }'
> + dest: "/etc/ansible/facts.d/builder.fact"
> diff --git a/poc/playbook-ubuntu-builder.yml b/poc/playbook-ubuntu-builder.yml
> new file mode 100644
> index 000000000..5a13c6069
> --- /dev/null
> +++ b/poc/playbook-ubuntu-builder.yml
> @@ -0,0 +1,70 @@
> +---
> +- hosts: all
> + become: true
> + name: builder
> + gather_facts: no
> + pre_tasks:
> + - name: 'install python2'
> + raw: sudo apt-get -y install python-simplejson
> + tasks:
> +
> + - name: Create Ansible Local Facts Directory
> + file: path=/etc/ansible/facts.d state=directory
> +
> + - name: Initiate Build Numbering
> + copy:
> + content: '{ "release":"1" }'
> + dest: "/etc/ansible/facts.d/builder.fact"
> + force: no
> +
> + - name: Reload Ansible Local Facts
> + setup: filter=ansible_local
> +
> + - name: Install "devscripts", "equivs", "apache2", "autoconf"
> + apt: update_cache=yes name={{item}} state=present
> + with_items:
> + - devscripts
> + - equivs
> + - apache2
> + - autoconf
> +
> + - name: Remove untracked files from Open vSwitch GIT repository
> + command: chdir=/git/ovs/ git clean -xdf
> +
> + - name: Reset Open vSwitch GIT repository to last comitted state
> + command: chdir=/git/ovs/ git reset --hard
> +
> + - name: Parse out Open vSwitch version from "configure.ac"
> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
> + register: version
> +
> + - name: Concatenate full version
> + set_fact:
> + full_version: "{{version.stdout}}-{{ansible_local.builder.release}}"
> +
> + - name: Update Open vSwitch version to {{full_version}}
> + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build
> +
> + - name: Build debian package with Open vSwitch build dependencies
> + command: chdir=/git/ovs/ mk-build-deps -B debian/control
> +
> + - name: Install Open vSwitch {{full_version}} build dependencies
> + apt: deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb
> +
> + - name: Build Open vSwitch {{full_version}} debian packages
> + shell: DEB_BUILD_OPTIONS='nocheck' fakeroot debian/rules binary
> + args:
> + chdir: /git/ovs/
> +
> + - name: Move debian packages to /var/www/html
> + shell: mv /git/*.deb /var/www/html/
> +
> + - name: Create Debian Package index file for repository
> + shell: dpkg-scanpackages . | gzip -9c > Packages.gz
> + args:
> + chdir: /var/www/html
> +
> + - name: Bump up Build Number
> + copy:
> + content: '{ "release":"{{ansible_local.builder.release|int+1}}" }'
> + dest: "/etc/ansible/facts.d/builder.fact"
More information about the dev
mailing list