[ovs-dev] [PATCH v1 ovn] Containerize components

aginwala aginwala at asu.edu
Sat Aug 17 07:27:12 UTC 2019


On Fri, Aug 16, 2019 at 10:33 AM aginwala <aginwala at asu.edu> wrote:

>
>
> On Thu, Aug 15, 2019 at 11:32 PM Numan Siddique <nusiddiq at redhat.com>
> wrote:
>
>> On Thu, Aug 1, 2019 at 3:01 AM Aliasgar Ginwala <amginwal at gmail.com>
>> wrote:
>>
>> > 1. Containerize ovn central components
>> > 2. Containerize ovn host
>> > 3. Update documentation about building/running ovn in containers.
>> >
>> >
>>
>> Hi Aliasgar,
>>
>> Thanks for the patch. This is interesting. I tried out both the OVS patch
>> and this one.
>>
>> When applying this patch (and the OVS one too), I found some white space
>> warnings.
>> Please run ovs/utilities/checkpatch.py -1
>>
>> Thanks for the review and testing.Appreciate that.  Ack, will do!
>
Done.  Sent v2 https://patchwork.ozlabs.org/patch/1148576/ and
https://patchwork.ozlabs.org/patch/1148575/ for both OVS and OVN.

> I was able to successfully build the docker images on my Fedora 30.
>> ovn-nb, ovn-sb and northd containers
>> work fine without any issues.
>>
>> I have trouble running ovn-controller container as it requires that OVS
>> containers are run first.
>>
>> With the OVS container patch, I had issues starting ovs-vswitchd
>> container.
>> Looks like it tries
>> to load ovs kernel module. I think it fails since I am running fedora and
>> the image is ubuntu.
>>
>> Correct. You need to use host OS kernel version and distro in the
> container as vswitchd
>  is started as privileged container to load ovs modules on host.
>
>> I am not sure if you want to build ovs kernel module from sources. Why
>> can't we rely on the host kernel module
>> from the distro ?
>>
>> Ohk trying to understand the cons of not using ovs source. May be do you
> have suggestions how to do it from container perspective for levering host
> module? We can add support for both options for sure. At-least to support
> dynamic kernel upgrades on computes, it helps to certify if old ovs will
> work new kernel and old kernel by releasing a single container image from
> source for each kernel upgrade. This makes it easy to load on host and skip
> any extra installation on host accordingly.
>
For this comment, if we can address that in upcoming patches as
enhancement, it can be ok too. Thoughts?

>
>> Is it possible to delink OVN from OVS. I mean we should be able to start
>> ovn-conroller container even if ovs is running
>> as a service. Mounting proper host directories like /var/run/openvswitch
>> when starting the container should be good enough
>> for ovn-controller to talk to ovsdb-server and ovs-vswitchd.
>>
> Sure. I actually didn't try ovn controller container running with
> different OS than that of OVS containers. I will give it a try and update
> usage details in doc in v2.
>
I tried running ovn-controller container when ovs is running either as
service or in container and it works fine. I have updated both cases in doc
accordingly. Feel free to try that too.

>
>> For ovn db services, it using tcp. Can't we add support to use unix
>> sockets
>> as well ? I am fine adding this support as a follow up
>> patch.
>>
>> We can for sure as a follow up patch.
>
>
>> Thanks
>> Numan
>>
>>
>>
>>
>> > Signed-off-by: aginwala <aginwala at ebay.com>
>> > ---
>> >  Documentation/intro/install/general.rst  | 68 ++++++++++++++++++++++++
>> >  utilities/automake.mk                    | 10 +++-
>> >  utilities/docker/Makefile                | 22 ++++++++
>> >  utilities/docker/create_ovn_dbs.sh       | 16 ++++++
>> >  utilities/docker/debian/Dockerfile       | 21 ++++++++
>> >  utilities/docker/debian/build.sh         | 43 +++++++++++++++
>> >  utilities/docker/ovn_default_nb_port     |  1 +
>> >  utilities/docker/ovn_default_northd_host |  1 +
>> >  utilities/docker/ovn_default_sb_port     |  1 +
>> >  utilities/docker/start-ovn               | 40 ++++++++++++++
>> >  10 files changed, 222 insertions(+), 1 deletion(-)
>> >  create mode 100644 utilities/docker/Makefile
>> >  create mode 100755 utilities/docker/create_ovn_dbs.sh
>> >  create mode 100644 utilities/docker/debian/Dockerfile
>> >  create mode 100755 utilities/docker/debian/build.sh
>> >  create mode 100644 utilities/docker/ovn_default_nb_port
>> >  create mode 100644 utilities/docker/ovn_default_northd_host
>> >  create mode 100644 utilities/docker/ovn_default_sb_port
>> >  create mode 100755 utilities/docker/start-ovn
>> >
>> > diff --git a/Documentation/intro/install/general.rst
>> > b/Documentation/intro/install/general.rst
>> > index 99d8fec04..4dc7e8d72 100644
>> > --- a/Documentation/intro/install/general.rst
>> > +++ b/Documentation/intro/install/general.rst
>> > @@ -380,6 +380,60 @@ domain socket::
>> >
>> >      $ ovn-northd --pidfile --detach --log-file
>> >
>> > +
>> > +Starting OVN Central services in containers
>> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > +
>> > +For OVN central node, we dont need to load ovs kernel modules on host.
>> > +Hence, OVN central containers OS need not depend on host OS.
>> > +
>> > +Also we can leverage deploying entire OVN control plane in a pod spec
>> for
>> > use
>> > +cases like OVN-kubernetes
>> > +
>> > +Export following variables in .env  and place it under
>> > +project root::
>> > +
>> > +    $ OVN_BRANCH=<BRANCH>
>> > +    $ OVN_VERSION=<VERSION>
>> > +    $ DISTRO=<LINUX_DISTRO>
>> > +    $ KERNEL_VERSION=<LINUX_KERNEL_VERSION>
>> > +    $ GITHUB_SRC=<GITHUB_URL>
>> > +    $ DOCKER_REPO=<REPO_TO_PUSH_IMAGE>
>> > +
>> > +To build ovn modules::
>> > +
>> > +    $ cd utilities/docker
>> > +    $ make build
>> > +
>> > +Compiled Modules will be tagged with docker image
>> > +
>> > +To Push ovn modules::
>> > +
>> > +    $ make push
>> > +
>> > +OVN docker image will be pushed to specified docker repo.
>> > +
>> > +Start OVN containers using below command::
>> > +
>> > +    $ docker run -itd --net=host --name=ovn-nb \
>> > +      <docker_repo>:<tag> ovn-nb-tcp
>> > +
>> > +    $ docker run -itd --net=host --name=ovn-sb \
>> > +      <docker_repo>:<tag> ovn-sb-tcp
>> > +
>> > +    $ docker run -itd --net=host --name=ovn-northd \
>> > +      <docker_repo>:<tag> ovn-northd-tcp
>> > +
>> > +.. note::
>> > +    Current ovn central components comes up in docker image in a
>> > standalone
>> > +    mode with protocol tcp.
>> > +
>> > +    The debian docker file use ubuntu 16.04 as a base image for
>> reference.
>> > +
>> > +    User can use any other base image for debian, e.g. u14.04, etc.
>> > +
>> > +    RHEL based docker build support needs to be added.
>> > +
>> >  Starting OVN host service
>> >  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >
>> > @@ -406,6 +460,17 @@ domain socket::
>> >
>> >      $ ovn-controller --pidfile --detach --log-file
>> >
>> > +Starting OVN host service in containers
>> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > +Start ovsdb-server and ovs-vswitchd components as per
>> > +http://docs.openvswitch.org/en/latest/intro/install/general/
>> > +
>> > +start local ovn-controller::
>> > +
>> > +    $ docker run -itd --net=host --name=ovn-controller \
>> > +      --volumes-from=ovsdb-server \
>> > +      <docker_repo>:<tag> ovn-controller
>> > +
>> >  Validating
>> >  ----------
>> >
>> > @@ -419,6 +484,9 @@ logical switch ``sw0`` and add logical port
>> ``sw0-p1``
>> > ::
>> >
>> >  Refer to ovn-nbctl(8) and ovn-sbctl (8) for more details.
>> >
>> > +When using ovn in container, exec to container to run above commands::
>> > +
>> > +    $ docker exec -it <ovn-nb/ovn-sb/ovn-northd/ovn-controller>
>> /bin/bash
>> >
>> >  Reporting Bugs
>> >  --------------
>> > diff --git a/utilities/automake.mk b/utilities/automake.mk
>> > index d666b9661..4d86f082b 100644
>> > --- a/utilities/automake.mk
>> > +++ b/utilities/automake.mk
>> > @@ -27,7 +27,15 @@ EXTRA_DIST += \
>> >      utilities/ovn-nbctl.8.xml \
>> >      utilities/ovn-trace.8.xml \
>> >      utilities/ovn-detrace.in \
>> > -    utilities/ovndb-servers.ocf
>> > +    utilities/ovndb-servers.ocf \
>> > +    utilities/docker/Makefile \
>> > +    utilities/docker/start-ovn \
>> > +    utilities/docker/create_ovn_dbs.sh \
>> > +    utilities/docker/ovn_default_nb_port \
>> > +    utilities/docker/ovn_default_sb_port \
>> > +    utilities/docker/ovn_default_northd_host \
>> > +    utilities/docker/debian/Dockerfile \
>> > +    utilities/docker/debian/build.sh
>> >
>> >  CLEANFILES += \
>> >      utilities/ovn-ctl.8 \
>> > diff --git a/utilities/docker/Makefile b/utilities/docker/Makefile
>> > new file mode 100644
>> > index 000000000..e2b2c2a17
>> > --- /dev/null
>> > +++ b/utilities/docker/Makefile
>> > @@ -0,0 +1,22 @@
>> > +#export OVN_BRANCH=master
>> > +#export OVN_VERSION=2.12
>> > +#export KERNEL_VERSION=4.15.0-54-generic
>> > +#export DISTRO=debian
>> > +#export GITHUB_SRC=https://github.com/ovn-org/ovn.git
>> > +#export DOCKER_REPO=ovn-org/ovn
>> > +
>> > +# Example:
>> > +#   make build
>> > +#   make push
>> > +
>> > +REPO = ${DOCKER_REPO}
>> > +tag = ${OVN_VERSION}_${KERNEL_VERSION}
>> > +
>> > +build: ;docker build -t ${REPO}:${tag} --build-arg DISTRO=${DISTRO} \
>> > +--build-arg OVN_BRANCH=${OVN_BRANCH} \
>> > +--build-arg KERNEL_VERSION=${KERNEL_VERSION} \
>> > +--build-arg GITHUB_SRC=${GITHUB_SRC} -f ${DISTRO}/Dockerfile .
>> > +
>> > +.PHONY: build
>> > +
>> > +push: ;docker push ${REPO}:${tag}
>> > diff --git a/utilities/docker/create_ovn_dbs.sh
>> > b/utilities/docker/create_ovn_dbs.sh
>> > new file mode 100755
>> > index 000000000..33dba2ae9
>> > --- /dev/null
>> > +++ b/utilities/docker/create_ovn_dbs.sh
>> > @@ -0,0 +1,16 @@
>> > +#!/bin/sh
>> > +#
>> > +# Licensed under the Apache License, Version 2.0 (the "License");
>> > +# you may not use this file except in compliance with the License.
>> > +# You may obtain a copy of the License at:
>> > +#
>> > +#     http://www.apache.org/licenses/LICENSE-2.0
>> > +#
>> > +# Unless required by applicable law or agreed to in writing, software
>> > +# distributed under the License is distributed on an "AS IS" BASIS,
>> > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> implied.
>> > +# See the License for the specific language governing permissions and
>> > +# limitations under the License.
>> > +
>> > +ovsdb-tool create /etc/openvswitch/ovnnb_db.db
>> > /usr/share/openvswitch/ovn-nb.ovsschema
>> > +ovsdb-tool create /etc/openvswitch/ovnsb_db.db
>> > /usr/share/openvswitch/ovn-sb.ovsschema
>> > diff --git a/utilities/docker/debian/Dockerfile
>> > b/utilities/docker/debian/Dockerfile
>> > new file mode 100644
>> > index 000000000..6642dc70c
>> > --- /dev/null
>> > +++ b/utilities/docker/debian/Dockerfile
>> > @@ -0,0 +1,21 @@
>> > +FROM ubuntu:16.04
>> > +MAINTAINER "Aliasgar Ginwala" <aginwala at ebay.com>
>> > +
>> > +ARG OVN_BRANCH
>> > +ARG KERNEL_VERSION
>> > +ARG GITHUB_SRC
>> > +ARG DISTRO
>> > +
>> > +copy $DISTRO/build.sh /build.sh
>> > +RUN /build.sh $KERNEL_VERSION $OVN_BRANCH $GITHUB_SRC
>> > +
>> > +COPY create_ovn_dbs.sh /etc/openvswitch/create_ovn_dbs.sh
>> > +RUN /etc/openvswitch/create_ovn_dbs.sh
>> > +
>> > +COPY ovn_default_nb_port /etc/openvswitch/ovn_default_nb_port
>> > +COPY ovn_default_sb_port /etc/openvswitch/ovn_default_sb_port
>> > +COPY ovn_default_northd_host /etc/openvswitch/ovn_default_northd_host
>> > +
>> > +COPY start-ovn /bin/start-ovn
>> > +VOLUME ["/var/log/openvswitch", "/var/lib/openvswitch",
>> > "/var/run/openvswitch", "/etc/openvswitch"]
>> > +ENTRYPOINT ["start-ovn"]
>> > diff --git a/utilities/docker/debian/build.sh
>> > b/utilities/docker/debian/build.sh
>> > new file mode 100755
>> > index 000000000..1461f19e7
>> > --- /dev/null
>> > +++ b/utilities/docker/debian/build.sh
>> > @@ -0,0 +1,43 @@
>> > +#!/bin/sh
>> > +#
>> > +# Licensed under the Apache License, Version 2.0 (the "License");
>> > +# you may not use this file except in compliance with the License.
>> > +# You may obtain a copy of the License at:
>> > +#
>> > +#     http://www.apache.org/licenses/LICENSE-2.0
>> > +#
>> > +# Unless required by applicable law or agreed to in writing, software
>> > +# distributed under the License is distributed on an "AS IS" BASIS,
>> > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> implied.
>> > +# See the License for the specific language governing permissions and
>> > +# limitations under the License.
>> > +
>> > +KERNEL_VERSION=$1
>> > +OVN_BRANCH=$2
>> > +GITHUB_SRC=$3
>> > +
>> > +# Install deps
>> > +linux="linux-image-$KERNEL_VERSION linux-headers-$KERNEL_VERSION"
>> > +build_deps="apt-utils libelf-dev build-essential libssl-dev python
>> > python-six wget vim \
>> > +gdb autoconf libtool git automake bzip2 debhelper dh-autoreconf
>> openssl"
>> > +
>> > +apt-get update
>> > +apt-get install -y ${linux} ${build_deps}
>> > +
>> > +# get the source
>> > +mkdir /build; cd /build
>> > +git clone --depth 1 -b $OVN_BRANCH $GITHUB_SRC
>> > +cd ovn
>> > +
>> > +# build and install
>> > +./boot.sh
>> > +./configure --localstatedir="/var" --sysconfdir="/etc" --prefix="/usr"
>> \
>> > +--with-linux=/lib/modules/$KERNEL_VERSION/build --enable-ssl
>> > +make -j8; make install
>> > +
>> > +# remove deps to make the container light weight.
>> > +apt-get remove --purge -y ${build_deps}
>> > +apt-get autoremove -y --purge
>> > +cd ..; rm -rf ovn
>> > +basic_utils="vim kmod net-tools uuid-runtime iproute2"
>> > +apt-get install -y ${basic_utils}
>> > diff --git a/utilities/docker/ovn_default_nb_port
>> > b/utilities/docker/ovn_default_nb_port
>> > new file mode 100644
>> > index 000000000..d83211678
>> > --- /dev/null
>> > +++ b/utilities/docker/ovn_default_nb_port
>> > @@ -0,0 +1 @@
>> > +nb_db_port=6641
>> > diff --git a/utilities/docker/ovn_default_northd_host
>> > b/utilities/docker/ovn_default_northd_host
>> > new file mode 100644
>> > index 000000000..55d4ab7aa
>> > --- /dev/null
>> > +++ b/utilities/docker/ovn_default_northd_host
>> > @@ -0,0 +1 @@
>> > +northd_host=127.0.0.1
>> > diff --git a/utilities/docker/ovn_default_sb_port
>> > b/utilities/docker/ovn_default_sb_port
>> > new file mode 100644
>> > index 000000000..4c9e3f585
>> > --- /dev/null
>> > +++ b/utilities/docker/ovn_default_sb_port
>> > @@ -0,0 +1 @@
>> > +sb_db_port=6642
>> > diff --git a/utilities/docker/start-ovn b/utilities/docker/start-ovn
>> > new file mode 100755
>> > index 000000000..7f87e65a3
>> > --- /dev/null
>> > +++ b/utilities/docker/start-ovn
>> > @@ -0,0 +1,40 @@
>> > +#!/bin/bash
>> > +#
>> > +# Licensed under the Apache License, Version 2.0 (the "License");
>> > +# you may not use this file except in compliance with the License.
>> > +# You may obtain a copy of the License at:
>> > +#
>> > +#     http://www.apache.org/licenses/LICENSE-2.0
>> > +#
>> > +# Unless required by applicable law or agreed to in writing, software
>> > +# distributed under the License is distributed on an "AS IS" BASIS,
>> > +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>> implied.
>> > +# See the License for the specific language governing permissions and
>> > +# limitations under the License.
>> > +
>> > +case $1 in
>> > +        "ovn-nb-tcp") source /etc/openvswitch/ovn_default_nb_port
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl
>> start_ovsdb
>> > +                      ovn-nbctl set-connection ptcp:$nb_db_port
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl stop_ovsdb
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl
>> run_nb_ovsdb
>> > +        ;;
>> > +        "ovn-sb-tcp") source /etc/openvswitch/ovn_default_sb_port
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl
>> start_ovsdb
>> > +                      ovn-sbctl set-connection ptcp:$sb_db_port
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl stop_ovsdb
>> > +                      /usr/share/openvswitch/scripts/ovn-ctl
>> run_sb_ovsdb
>> > +        ;;
>> > +        "ovn-northd-tcp") source
>> /etc/openvswitch/ovn_default_northd_host
>> > +                          source /etc/openvswitch/ovn_default_nb_port
>> > +                          source /etc/openvswitch/ovn_default_sb_port
>> > +                          ovn-northd --pidfile \
>> > +                          --ovnnb-db="tcp:$northd_host:$nb_db_port" \
>> > +                          --ovnsb-db="tcp:$northd_host:$sb_db_port" \
>> > +
>> --log-file=/var/log/openvswitch/ovn-northd.log
>> > +        ;;
>> > +        "ovn-controller") ovn-controller --pidfile \
>> > +
>> > --log-file=/var/log/openvswitch/ovn-controller.log
>> > +        ;;
>> > +        *) echo "$0
>> [ovn-nb-tcp|ovn-sb-tcp|ovn-northd-tcp|ovn-controller]"
>> > +esac
>> > --
>> > 2.20.1 (Apple Git-117)
>> >
>> > _______________________________________________
>> > dev mailing list
>> > dev at openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>> >
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>


More information about the dev mailing list