[ovs-dev] [PATCH] poc: Introduce Proof of Concepts (Package building)
Ansis Atteka
ansisatteka at gmail.com
Mon Jan 29 21:24:37 UTC 2018
On 26 January 2018 at 09:13, Gregory Rose <gvrose8192 at gmail.com> wrote:
> On 1/19/2018 7:55 PM, Ansis Atteka wrote:
>>
>> From: Ansis Atteka <ansisatteka at gmail.com>
>>
>> This patch sets up foundations for Proof of Concepts that
>> simply materialize documentation into Ansible instructions
>> executed in virtualized Vagrant environment.
>>
>> This Proof of Concept allows to easily build:
>> 1. *.deb packages on Ubuntu 16.04; AND
>> 2. *.rpm packages on CentOS 7.4.
>> It also sets up DEB and RPM repository over HTTP that can
>> be used to pull these openvswitch packages with apt-get
>> or yum from another host.
>>
>> This particular Proof of Concept is intended to address
>> following use-cases:
>> 1. for new OVS users to see how debian and rpm packages are
>> built;
>> 2. for developers to easily check for packaging build
>> regressions;
>> 3. for developers to easily share their sandbox builds
>> into QE setups (opposed to manually copying binaries);
>> 4. for developers to add other Proof of Concepts
>> that possibly may require full end-to-end integration
>> with other thirdparty projects (e.g. DPI, libvirt, IPsec)
>> and need Open vSwitch packages.
>>
>> Signed-off-by: Ansis Atteka <aatteka at ovn.org>
>> ---
>> .gitignore | 2 +
>> Documentation/topics/testing.rst | 46 ++++++++++++++++++
>> Makefile.am | 3 ++
>> poc/builders/Vagrantfile | 35 ++++++++++++++
>> poc/playbook-centos-builder.yml | 100
>> +++++++++++++++++++++++++++++++++++++++
>> poc/playbook-ubuntu-builder.yml | 66 ++++++++++++++++++++++++++
>> 6 files changed, 252 insertions(+)
>> create mode 100644 poc/builders/Vagrantfile
>> create mode 100644 poc/playbook-centos-builder.yml
>> create mode 100644 poc/playbook-ubuntu-builder.yml
>>
>> diff --git a/.gitignore b/.gitignore
>> index 8019bee41..81faf270d 100644
>> --- a/.gitignore
>> +++ b/.gitignore
>> @@ -16,6 +16,7 @@
>> *.lib
>> *.pdb
>> *.pyc
>> +*.retry
>> *.so
>> *.suo
>> **/*.sym
>> @@ -29,6 +30,7 @@
>> .dirstamp
>> .libs
>> .tmp_versions
>> +.vagrant
>> .gitattributes
>> /Makefile
>> /Makefile.in
>> diff --git a/Documentation/topics/testing.rst
>> b/Documentation/topics/testing.rst
>> index a49336b79..4d93944c7 100644
>> --- a/Documentation/topics/testing.rst
>> +++ b/Documentation/topics/testing.rst
>> @@ -389,3 +389,49 @@ validate the suitability of different vSwitch
>> implementations in a telco
>> deployment environment. More information can be found on the `OPNFV
>> wiki`_.
>> .. _OPNFV wiki: https://wiki.opnfv.org/display/vsperf/VSperf+Home
>> +
>> +Proof of Concepts
>> +~~~~~~~~~~~~~~~~~
>> +
>> +Proof of Concepts are documentation materialized into Ansible recipes
>> +executed in Virtualbox environment orchastrated by Vagrant. Proof of
>> +Concepts allow developers to create small virtualized setups that
>> +demonstrate how certain Open vSwitch features are intended to work,
>> +especially when integration with thirdparty software is involved.
>
> s/thirdparty/third party
>
>> +
>> +The host where Vagrant runs does not need to have any special software
>> +installed besides vagrant, virtualbox and ansible.
>
>
> Seem it also requires Apache?
Since the Ansible recipes are executed only on the guest (and not
host) then only guests need Apache/httpd installed. Let me know if I
am missing something and you still had to install Apache on host?
Apache is required only to have the packages distributed via repository.
>
>> +
>> +The following Proof of Concepts are supported:
>> +
>> +Builders
>> +++++++++
>> +
>> +This Proof of Concept demonsrtates integration with Debian and RPM
>> +packaging tools:
>> +
>> + $ cd ./poc/builders
>> +
>> + # When setting up Proof of Concept for the first time run:
>> + $ vagrant up
>> +
>> +Once that command finished you can get packages from /var/www/html
>> +directory. Since those hosts are also configured as repositories then
>> +you can add them to /etc/apt/sources.list.d or /etc/yum.repos.d
>> +configuration files.
>> +
>> + # When you have made changes to OVS and want to rebuild packages run:
>> + $ git commit -a
>> + $ vagrant rsync && vagrant provision
>> +
>> +Each packages are rebuilt the Open vSwitch release number increases
>
>
> Maybe "When packages are rebuilt" ?
You are right. Will send V2.
>
> There are some checkpatch warnings about lines too long but I'm not worried
> about those. There
> are a few other checkpatch warnings about lines with trailing whitespace.
> Those should probably
> be cleaned up.
Agree about whitespace errors, not sure how I missed them...
Will look into truncating lines to 80 characters. There are some lines that
require file-system paths so it may require a little more creativity to truncate
them without sacrificing readability.
>
> The rest of the patch seems fine. I applied it and ran the simple test case
> you outline
> and it worked as advertised. I have a bit of a learning curve to get over
> for me to make
> further use of this but looks promising!
>
> Thanks Ansis!
Great. I am somewhat curious if we should switch from Virtualbox to KVM/libvirt,
because it appears that is what most developers are using here. Will let others
chime in as well.
>
> Tested-by: Greg Rose <gvrose8192 at gmail.com>
> Reviewed-by: Greg Rose <gvrose8192 at gmail.com>
>
>
>> +by one and you can simply upgrade them by running yum or apt-get.
>> +
>> +Once you are done with Particular Proof of Concept run
>> +
>> + $ vagrant destroy
>> +
>> +Or simply go to VirtualBox management console and delete those VMs.
>> +
>> +Sometime deployment of Proof of Concept may fail if for example VMs
>> +don't have network reachability to the Internet.
>> diff --git a/Makefile.am b/Makefile.am
>> index ed4b7fd7c..174a421e6 100644
>> --- a/Makefile.am
>> +++ b/Makefile.am
>> @@ -90,6 +90,9 @@ EXTRA_DIST = \
>> build-aux/sodepends.py \
>> build-aux/soexpand.py \
>> build-aux/xml2nroff \
>> + poc/builders/Vagrantfile \
>> + poc/playbook-centos-builder.yml \
>> + poc/playbook-ubuntu-builder.yml \
>> $(MAN_FRAGMENTS) \
>> $(MAN_ROOTS) \
>> Vagrantfile \
>> diff --git a/poc/builders/Vagrantfile b/poc/builders/Vagrantfile
>> new file mode 100644
>> index 000000000..00b114475
>> --- /dev/null
>> +++ b/poc/builders/Vagrantfile
>> @@ -0,0 +1,35 @@
>> +# -*- mode: ruby -*-
>> +# vi: set ft=ruby :
>> +
>> +VAGRANTFILE_API_VERSION = "2"
>> +
>> +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
>> + config.ssh.insert_key = false
>> + config.vm.provider :virtualbox do |vb|
>> + vb.customize ["modifyvm", :id, "--memory", "1024"]
>> + vb.customize ["modifyvm", :id, "--cpus", "4"]
>> + end
>> +
>> + # Centos-7.4 builder host
>> + config.vm.define "centosbuilder" do |builder|
>> + builder.vm.hostname = "centosbuilder.dev"
>> + builder.vm.box = "bento/centos-7.4"
>> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
>> rsync__args: ["--verbose", "--archive", "--delete", "-z"]
>> + builder.vm.provision "builder", type: "ansible" do |ansible|
>> + ansible.playbook = "../playbook-centos-builder.yml"
>> + ansible.sudo = true
>> + end
>> + end
>> +
>> + # Ubuntu-16.04 builder host
>> + config.vm.define "ubuntubuilder" do |builder|
>> + builder.vm.hostname = "ubuntubuilder.dev"
>> + builder.vm.box = "bento/ubuntu-16.04"
>> + builder.vm.synced_folder "../../", "/git/ovs", type: "rsync",
>> rsync__args: ["--verbose", "--archive", "--delete", "-z"]
>> + builder.vm.provision "builder", type: "ansible" do |ansible|
>> + ansible.playbook = "../playbook-ubuntu-builder.yml"
>> + ansible.sudo = true
>> + end
>> + end
>> +
>> +end
>> diff --git a/poc/playbook-centos-builder.yml
>> b/poc/playbook-centos-builder.yml
>> new file mode 100644
>> index 000000000..b9b6e0776
>> --- /dev/null
>> +++ b/poc/playbook-centos-builder.yml
>> @@ -0,0 +1,100 @@
>> +---
>> +- hosts: all
>> + become: true
>> + name: builder
>> + tasks:
>> +
>> + - name: Create Ansible Local Facts Directory
>> + file: path=/etc/ansible/facts.d state=directory
>> +
>> + - name: Initiate Build Numbering
>> + copy:
>> + content: '{ "build_counter":"1" }'
>> + dest: "/etc/ansible/facts.d/builder.fact"
>> + force: no
>> +
>> + - name: Reload Ansible Local Facts
>> + setup: filter=ansible_local
>> +
>> + - name: Install "yum-utils", "rpmdevtools", "createrepo", "httpd",
>> "git"
>> + yum: update_cache=yes name={{item}} state=present
>> + with_items:
>> + - yum-utils
>> + - rpmdevtools
>> + - createrepo
>> + - httpd
>> + - git
>> +
>> + - name: Remove untracked files from Open vSwitch GIT repository
>> + command: chdir=/git/ovs/ git clean -xdf
>> +
>> + - name: Reset Open vSwitch GIT repository to last comitted state
>> + command: chdir=/git/ovs/ git reset --hard
>> +
>> + - name: Generate "openvswitch.spec" and "openvswitch-kmod-rhel6.spec"
>> files for easy build dependency retrieval
>> + shell: sed -e 's/@VERSION@/0.0.1/' {{item}}.in > /tmp/{{item}}
>> + args:
>> + chdir: /git/ovs/rhel
>> + with_items:
>> + - openvswitch.spec
>> + - openvswitch-kmod-rhel6.spec
>> +
>> + - name: Install build dependencies specified in "openvswitch.spec" and
>> "openvswitch-kmod-rhel6.spec"
>> + shell: echo "y" | yum-builddep /tmp/{{item}}
>> + with_items:
>> + - openvswitch.spec
>> + - openvswitch-kmod-rhel6.spec
>> +
>> + - name: Create rpm dev tree
>> + command: rpmdev-setuptree
>> +
>> + - name: Run "./boot.sh"
>> + command: chdir=/git/ovs/ ./boot.sh
>> +
>> + - name: Run "./configure"
>> + command: chdir=/git/ovs/ ./configure
>> +
>> + - name: Run "make dist"
>> + command: chdir=/git/ovs/ make dist
>> +
>> + - name: Parse out Open vSwitch version from "configure.ac"
>> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
>> + register: version
>> +
>> + - name: Copy source tarball to rpm dev tree
>> + command: cp /git/ovs/openvswitch-{{version.stdout}}.tar.gz
>> /root/rpmbuild/SOURCES/
>> +
>> + - name: Unarchive openvswitch source tarball
>> + unarchive:
>> + src: /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}.tar.gz
>> + dest: /root/rpmbuild/SOURCES
>> + remote_src: yes
>> +
>> + - name: Update release number in "openvswitch.spec" and
>> "openvswitch-kmod-rhel6.spec"
>> + lineinfile:
>> + path:
>> /root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}/rhel/{{item}}
>> + regexp: '^Release:'
>> + line: "Release: {{ ansible_local.builder.build_counter }}"
>> + with_items:
>> + - openvswitch.spec
>> + - openvswitch-kmod-rhel6.spec
>> +
>> + - name: Build Open vSwitch user space rpms
>> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}
>> rpmbuild -bb --without check rhel/openvswitch.spec
>> +
>> + - name: Build Open vSwitch kmod rpms (only for currently loaded kernel)
>> + command: chdir=/root/rpmbuild/SOURCES/openvswitch-{{version.stdout}}
>> rpmbuild -bb --without check rhel/openvswitch-kmod-rhel6.spec
>> +
>> + - name: Copy RPM packages to /var/www/html
>> + command: cp -r /root/rpmbuild/RPMS/ /var/www/html
>> +
>> + - name: Create RPM Package index file for repository
>> + command: chdir=/var/www/html createrepo /var/www/html
>> +
>> + - name: Make sure Apache is running
>> + systemd: state=started name=httpd
>> +
>> + - name: Bump up Build Number
>> + copy:
>> + content: '{
>> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }'
>> + dest: "/etc/ansible/facts.d/builder.fact"
>> diff --git a/poc/playbook-ubuntu-builder.yml
>> b/poc/playbook-ubuntu-builder.yml
>> new file mode 100644
>> index 000000000..c6f5b6eb0
>> --- /dev/null
>> +++ b/poc/playbook-ubuntu-builder.yml
>> @@ -0,0 +1,66 @@
>> +---
>> +- hosts: all
>> + become: true
>> + name: builder
>> + tasks:
>> +
>> + - name: Create Ansible Local Facts Directory
>> + file: path=/etc/ansible/facts.d state=directory
>> +
>> + - name: Initiate Build Numbering
>> + copy:
>> + content: '{ "build_counter":"1" }'
>> + dest: "/etc/ansible/facts.d/builder.fact"
>> + force: no
>> +
>> + - name: Reload Ansible Local Facts
>> + setup: filter=ansible_local
>> +
>> + - name: Install "devscripts", "equivs", "apache2", "autoconf"
>> + apt: update_cache=yes name={{item}} state=present
>> + with_items:
>> + - devscripts
>> + - equivs
>> + - apache2
>> + - autoconf
>> +
>> + - name: Remove untracked files from Open vSwitch GIT repository
>> + command: chdir=/git/ovs/ git clean -xdf
>> +
>> + - name: Reset Open vSwitch GIT repository to last comitted state
>> + command: chdir=/git/ovs/ git reset --hard
>> +
>> + - name: Parse out Open vSwitch version from "configure.ac"
>> + command: chdir=/git/ovs autoconf -t AC_INIT:'$2'
>> + register: version
>> +
>> + - name: Concatenate full version
>> + set_fact:
>> + full_version:
>> "{{version.stdout}}-{{ansible_local.builder.build_counter}}"
>> +
>> + - name: Update Open vSwitch version to {{full_version}}
>> + command: chdir=/git/ovs/ dch -b -v {{full_version}} Vagrant Build
>> +
>> + - name: Build debian package with Open vSwitch build dependencies
>> + command: chdir=/git/ovs/ mk-build-deps -B debian/control
>> +
>> + - name: Install Open vSwitch {{full_version}} build dependencies
>> + apt:
>> deb=/git/ovs/openvswitch-build-deps-depends_{{full_version}}_all.deb
>> +
>> + - name: Build Open vSwitch {{full_version}} debian packages
>> + shell: DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules
>> binary
>> + args:
>> + chdir: /git/ovs/
>> +
>> + - name: Move debian packages to /var/www/html
>> + shell: mv /git/*.deb /var/www/html/
>> +
>> + - name: Create Debian Package index file for repository
>> + shell: dpkg-scanpackages . | gzip -9c > Packages.gz
>> + args:
>> + chdir: /var/www/html
>> +
>> + - name: Bump up Build Number
>> + copy:
>> + content: '{
>> "build_counter":"{{ansible_local.builder.build_counter|int+1}}" }'
>> + dest: "/etc/ansible/facts.d/builder.fact"
>
>
More information about the dev
mailing list