[ovs-dev] [PATCH ovn 10/19] ovn-ic: Interconnection controller with AZ registeration.

Han Zhou hzhou at ovn.org
Wed Oct 30 21:17:46 UTC 2019


On Tue, Oct 29, 2019 at 5:50 PM aginwala <aginwala at asu.edu> wrote:
>
> Thanks Han for the patches.
>
> On Sun, Oct 20, 2019 at 5:54 PM Han Zhou <hzhou at ovn.org> wrote:
>>
>> This patch introduces interconnection controller, ovn-ic, and
>> implements the basic AZ registration feature: taking the AZ
>> name from NB DB and create an Availability_Zone entry in
>> IC-SB DB.
>>
>> Signed-off-by: Han Zhou <hzhou at ovn.org>
>> ---
>>  Makefile.am          |   1 +
>>  ic/.gitignore        |   2 +
>>  ic/automake.mk       |  10 ++
>>  ic/ovn-ic.8.xml      | 111 ++++++++++++++
>>  ic/ovn-ic.c          | 413
+++++++++++++++++++++++++++++++++++++++++++++++++++
>>  ovn-nb.ovsschema     |   5 +-
>>  ovn-nb.xml           |   7 +
>>  tests/automake.mk    |   4 +-
>>  tests/ovn-ic.at      |  29 ++++
>>  tests/ovn-macros.at  | 161 ++++++++++++++++----
>>  tests/testsuite.at   |   1 +
>>  tutorial/ovs-sandbox |  78 +++++++++-
>>  12 files changed, 792 insertions(+), 30 deletions(-)
>>  create mode 100644 ic/.gitignore
>>  create mode 100644 ic/automake.mk
>>  create mode 100644 ic/ovn-ic.8.xml
>>  create mode 100644 ic/ovn-ic.c
>>  create mode 100644 tests/ovn-ic.at
>>
>> diff --git a/Makefile.am b/Makefile.am
>> index 33c18c5..d22a220 100644
>> --- a/Makefile.am
>> +++ b/Makefile.am
>> @@ -500,4 +500,5 @@ include selinux/automake.mk
>>  include controller/automake.mk
>>  include controller-vtep/automake.mk
>>  include northd/automake.mk
>> +include ic/automake.mk
>>  include build-aux/automake.mk
>> diff --git a/ic/.gitignore b/ic/.gitignore
>> new file mode 100644
>> index 0000000..1b73eb4
>> --- /dev/null
>> +++ b/ic/.gitignore
>> @@ -0,0 +1,2 @@
>> +/ovn-ic
>> +/ovn-ic.8
>> diff --git a/ic/automake.mk b/ic/automake.mk
>> new file mode 100644
>> index 0000000..8e71bc3
>> --- /dev/null
>> +++ b/ic/automake.mk
>> @@ -0,0 +1,10 @@
>> +# ovn-ic
>> +bin_PROGRAMS += ic/ovn-ic
>> +ic_ovn_ic_SOURCES = ic/ovn-ic.c
>> +ic_ovn_ic_LDADD = \
>> +       lib/libovn.la \
>> +       $(OVSDB_LIBDIR)/libovsdb.la \
>> +       $(OVS_LIBDIR)/libopenvswitch.la
>> +man_MANS += ic/ovn-ic.8
>> +EXTRA_DIST += ic/ovn-ic.8.xml
>> +CLEANFILES += ic/ovn-ic.8
>> diff --git a/ic/ovn-ic.8.xml b/ic/ovn-ic.8.xml
>> new file mode 100644
>> index 0000000..00f33aa
>> --- /dev/null
>> +++ b/ic/ovn-ic.8.xml
>> @@ -0,0 +1,111 @@
>> +<?xml version="1.0" encoding="utf-8"?>
>> +<manpage program="ovn-ic" section="8" title="ovn-ic">
>> +    <h1>Name</h1>
>> +    <p>ovn-ic -- Open Virtual Network interconnection controller</p>
>> +
>> +    <h1>Synopsis</h1>
>> +    <p><code>ovn-ic</code> [<var>options</var>]</p>
>> +
>> +    <h1>Description</h1>
>> +    <p>
>> +      <code>ovn-ic</code>, OVN interconnection controller, is a
centralized
>> +      daemon which communicates with global interconnection databases
INB/ISB
>> +      to configure and exchange data with local NB/SB for
interconnecting
>> +      with other OVN deployments.
>> +    </p>
>> +
>> +    <h1>Options</h1>
>> +    <dl>
>> +      <dt><code>--ovnnb-db=<var>database</var></code></dt>
>> +      <dd>
>> +        The OVSDB database containing the OVN Northbound Database.  If
the
>> +        <env>OVN_NB_DB</env> environment variable is set, its value is
used
>> +        as the default.  Otherwise, the default is
>> +        <code>unix:@RUNDIR@/ovnnb_db.sock</code>.
>> +      </dd>
>> +      <dt><code>--ovnsb-db=<var>database</var></code></dt>
>> +      <dd>
>> +        The OVSDB database containing the OVN Southbound Database.  If
the
>> +        <env>OVN_SB_DB</env> environment variable is set, its value is
used
>> +        as the default.  Otherwise, the default is
>> +        <code>unix:@RUNDIR@/ovnsb_db.sock</code>.
>> +      </dd>
>> +      <dt><code>--ovninb-db=<var>database</var></code></dt>
>> +      <dd>
>> +        The OVSDB database containing the OVN Interconnection Northbound
>> +        Database.  If the <env>OVN_INB_DB</env> environment variable is
set,
>> +        its value is used as the default.  Otherwise, the default is
>> +        <code>unix:@RUNDIR@/ovninb_db.sock</code>.
>> +      </dd>
>> +      <dt><code>--ovnisb-db=<var>database</var></code></dt>
>> +      <dd>
>> +        The OVSDB database containing the OVN Interconnection Southbound
>> +        Database.  If the <env>OVN_ISB_DB</env> environment variable is
set,
>> +        its value is used as the default.  Otherwise, the default is
>> +        <code>unix:@RUNDIR@/ovnisb_db.sock</code>.
>> +      </dd>
>> +    </dl>
>> +    <p>
>> +      <var>database</var> in the above options must be an OVSDB active
or
>> +      passive connection method, as described in <code>ovsdb</code>(7).
>> +    </p>
>> +
>> +    <h2>Daemon Options</h2>
>> +    <xi:include href="lib/daemon.xml" xmlns:xi="
http://www.w3.org/2003/XInclude"/>
>> +
>> +    <h2>Logging Options</h2>
>> +    <xi:include href="lib/vlog.xml" xmlns:xi="
http://www.w3.org/2003/XInclude"/>
>> +
>> +    <h2>PKI Options</h2>
>> +    <p>
>> +      PKI configuration is required in order to use SSL for the
connections to
>> +      the Northbound and Southbound databases.
>> +    </p>
>> +    <xi:include href="lib/ssl.xml" xmlns:xi="
http://www.w3.org/2003/XInclude"/>
>> +
>> +    <h2>Other Options</h2>
>> +    <xi:include href="lib/unixctl.xml"
>> +     xmlns:xi="http://www.w3.org/2003/XInclude"/>
>> +    <h3></h3>
>> +    <xi:include href="lib/common.xml"
>> +     xmlns:xi="http://www.w3.org/2003/XInclude"/>
>> +
>> +    <h1>Runtime Management Commands</h1>
>> +    <p>
>> +      <code>ovs-appctl</code> can send commands to a running
>> +      <code>ovn-ic</code> process.  The currently supported commands
>> +      are described below.
>> +      <dl>
>> +      <dt><code>exit</code></dt>
>> +      <dd>
>> +        Causes <code>ovn-ic</code> to gracefully terminate.
>> +      </dd>
>> +
>> +      <dt><code>pause</code></dt>
>> +      <dd>
>> +        Pauses the ovn-ic operation from processing any Northbound and
>> +        Southbound database changes.
>> +      </dd>
>> +
>> +      <dt><code>resume</code></dt>
>> +      <dd>
>> +        Resumes the ovn-ic operation to process Northbound and
>> +        Southbound database contents and generate logical flows.
>> +      </dd>
>> +
>> +      <dt><code>is-paused</code></dt>
>> +      <dd>
>> +        Returns "true" if ovn-ic is currently paused, "false" otherwise.
>> +      </dd>
>> +      </dl>
>> +    </p>
>> +
>> +    <h1>Active-Standby for High Availability</h1>
>> +    <p>
>> +      You may run <code>ovn-ic</code> more than once in an OVN
deployment.
>> +      OVN will automatically ensure that only one of them is active at
a time.
>> +      If multiple instances of <code>ovn-ic</code> are running and the
>> +      active <code>ovn-ic</code> fails, one of the hot standby instances
>> +      of <code>ovn-ic</code> will automatically take over.
>> +    </p>
>> +</manpage>
>> diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c
>> new file mode 100644
>> index 0000000..7cc80bc
>> --- /dev/null
>> +++ b/ic/ovn-ic.c
>> @@ -0,0 +1,413 @@
>> +/*
>> + * Licensed under the Apache License, Version 2.0 (the "License");
>> + * you may not use this file except in compliance with the License.
>> + * You may obtain a copy of the License at:
>> + *
>> + *     http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +
>> +#include <config.h>
>> +
>> +#include <getopt.h>
>> +#include <stdlib.h>
>> +#include <stdio.h>
>> +
>> +#include "bitmap.h"
>> +#include "command-line.h"
>> +#include "daemon.h"
>> +#include "dirs.h"
>> +#include "openvswitch/dynamic-string.h"
>> +#include "fatal-signal.h"
>> +#include "hash.h"
>> +#include "openvswitch/hmap.h"
>> +#include "lib/ovn-inb-idl.h"
>> +#include "lib/ovn-isb-idl.h"
>> +#include "lib/ovn-nb-idl.h"
>> +#include "lib/ovn-sb-idl.h"
>> +#include "lib/ovn-util.h"
>> +#include "openvswitch/poll-loop.h"
>> +#include "smap.h"
>> +#include "sset.h"
>> +#include "stream.h"
>> +#include "stream-ssl.h"
>> +#include "unixctl.h"
>> +#include "util.h"
>> +#include "uuid.h"
>> +#include "openvswitch/vlog.h"
>> +
>> +VLOG_DEFINE_THIS_MODULE(ovn_ic);
>> +
>> +static unixctl_cb_func ovn_ic_exit;
>> +static unixctl_cb_func ovn_ic_pause;
>> +static unixctl_cb_func ovn_ic_resume;
>> +static unixctl_cb_func ovn_ic_is_paused;
>> +
>> +struct ic_context {
>> +    struct ovsdb_idl *ovnnb_idl;
>> +    struct ovsdb_idl *ovnsb_idl;
>> +    struct ovsdb_idl *ovninb_idl;
>> +    struct ovsdb_idl *ovnisb_idl;
>> +    struct ovsdb_idl_txn *ovnnb_txn;
>> +    struct ovsdb_idl_txn *ovnsb_txn;
>> +    struct ovsdb_idl_txn *ovninb_txn;
>> +    struct ovsdb_idl_txn *ovnisb_txn;
>> +};
>> +
>> +static const char *ovnnb_db;
>> +static const char *ovnsb_db;
>> +static const char *ovninb_db;
>> +static const char *ovnisb_db;
>> +static const char *unixctl_path;
>> +
>> +
>> +static void
>> +usage(void)
>> +{
>> +    printf("\
>> +%s: OVN interconnection management daemon\n\
>> +usage: %s [OPTIONS]\n\
>> +\n\
>> +Options:\n\
>> +  --ovnnb-db=DATABASE       connect to ovn-nb database at DATABASE\n\
>> +                            (default: %s)\n\
>> +  --ovnsb-db=DATABASE       connect to ovn-sb database at DATABASE\n\
>> +                            (default: %s)\n\
>> +  --unixctl=SOCKET          override default control socket name\n\
>> +  -h, --help                display this help message\n\
>> +  -o, --options             list available options\n\
>> +  -V, --version             display version information\n\
>> +", program_name, program_name, default_nb_db(), default_sb_db());
>> +    daemon_usage();
>> +    vlog_usage();
>> +    stream_usage("database", true, true, false);
>> +}
>> +
>> +static const struct isbrec_availability_zone *
>> +az_run(struct ic_context *ctx)
>> +{
>> +    const struct nbrec_nb_global *nb_global =
>> +        nbrec_nb_global_first(ctx->ovnnb_idl);
>> +
>> +    if (!nb_global) {
>> +        VLOG_INFO("NB Global not exist.");
>> +        return NULL;
>> +    }
>> +
>> +    /* Delete old AZ if name changes.  Note: if name changed when ovn-ic
>> +     * is not running, one has to manually delete the old AZ with:
>> +     * "ovn-isbctl destroy avail <az>". */
>> +    static char *az_name;
>> +    const struct isbrec_availability_zone *az;
>> +    if (az_name && strcmp(az_name, nb_global->name)) {
>> +        ISBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_idl) {
>> +            if (!strcmp(az->name, az_name)) {
>> +                isbrec_availability_zone_delete(az);
>> +                break;
>> +            }
>> +        }
>> +        free(az_name);
>> +        az_name = NULL;
>> +    }
>> +
>> +    if (!nb_global->name[0]) {
>> +        return NULL;
>> +    }
>> +
>> +    if (!az_name) {
>> +        az_name = xstrdup(nb_global->name);
>> +    }
>> +
>> +    ISBREC_AVAILABILITY_ZONE_FOR_EACH (az, ctx->ovnisb_idl) {
>> +        if (!strcmp(az->name, az_name)) {
>> +            return az;
>> +        }
>> +    }
>> +
>> +    /* Create AZ in ISB */
>> +    if (ctx->ovnisb_txn) {
>> +        VLOG_INFO("Register AZ %s to interconnection DB.", az_name);
>> +        az = isbrec_availability_zone_insert(ctx->ovnisb_txn);
>> +        isbrec_availability_zone_set_name(az, az_name);
>> +        return az;
>> +    }
>> +    return NULL;
>> +}
>> +
>> +static void
>> +ovn_db_run(struct ic_context *ctx)
>> +{
>> +    const struct isbrec_availability_zone *az = az_run(ctx);
>> +    VLOG_DBG("Availability zone: %s", az ? az->name : "not created
yet.");
>> +}
>> +
>> +static void
>> +parse_options(int argc OVS_UNUSED, char *argv[] OVS_UNUSED)
>> +{
>> +    enum {
>> +        DAEMON_OPTION_ENUMS,
>> +        VLOG_OPTION_ENUMS,
>> +        SSL_OPTION_ENUMS,
>> +    };
>> +    static const struct option long_options[] = {
>> +        {"ovnsb-db", required_argument, NULL, 'd'},
>> +        {"ovnnb-db", required_argument, NULL, 'D'},
>> +        {"ovnisb-db", required_argument, NULL, 'i'},
>> +        {"ovninb-db", required_argument, NULL, 'I'},
>> +        {"unixctl", required_argument, NULL, 'u'},
>> +        {"help", no_argument, NULL, 'h'},
>> +        {"options", no_argument, NULL, 'o'},
>> +        {"version", no_argument, NULL, 'V'},
>> +        DAEMON_LONG_OPTIONS,
>> +        VLOG_LONG_OPTIONS,
>> +        STREAM_SSL_LONG_OPTIONS,
>> +        {NULL, 0, NULL, 0},
>> +    };
>> +    char *short_options =
ovs_cmdl_long_options_to_short_options(long_options);
>> +
>> +    for (;;) {
>> +        int c;
>> +
>> +        c = getopt_long(argc, argv, short_options, long_options, NULL);
>> +        if (c == -1) {
>> +            break;
>> +        }
>> +
>> +        switch (c) {
>> +        DAEMON_OPTION_HANDLERS;
>> +        VLOG_OPTION_HANDLERS;
>> +        STREAM_SSL_OPTION_HANDLERS;
>> +
>> +        case 'd':
>> +            ovnsb_db = optarg;
>> +            break;
>> +
>> +        case 'D':
>> +            ovnnb_db = optarg;
>> +            break;
>> +
>> +        case 'u':
>> +            unixctl_path = optarg;
>> +            break;
>> +
>> +        case 'h':
>> +            usage();
>> +            exit(EXIT_SUCCESS);
>> +
>> +        case 'o':
>> +            ovs_cmdl_print_options(long_options);
>> +            exit(EXIT_SUCCESS);
>> +
>> +        case 'V':
>> +            ovs_print_version(0, 0);
>> +            exit(EXIT_SUCCESS);
>> +
>> +        default:
>> +            break;
>> +        }
>> +    }
>> +
>> +    if (!ovnsb_db) {
>> +        ovnsb_db = default_sb_db();
>> +    }
>> +
>> +    if (!ovnnb_db) {
>> +        ovnnb_db = default_nb_db();
>> +    }
>> +
>> +    if (!ovnisb_db) {
>> +        ovnisb_db = default_isb_db();
>
> This still did not work if we are starting ic controller on different azs
with remote="tcp:10.xxx:6645/46" after patching ovn-ctl. As per code it
checks env vars  OVN_INB/SB_DB without passing values in command line. So
rectification is needed here.

Thanks Ali for reporting the problem. I fixed it in v2:
https://patchwork.ozlabs.org/project/openvswitch/list/?series=139731


More information about the dev mailing list