[ovs-dev] [PATCH ovn v4 6/9] lib: Add infrastructure for plugging providers.
Numan Siddique
numans at ovn.org
Tue Sep 14 20:48:27 UTC 2021
On Fri, Sep 3, 2021 at 3:29 PM Frode Nordahl
<frode.nordahl at canonical.com> wrote:
>
> New module contains the infrastructure for registering plugging
> classes which may be hosted inside or outside the core OVN
> repository. The data structures and functions for interacting
> with these plugging classes also live there.
>
> Extend build system to allow enabling building of built-in plugging
> providers and linking an externally built plugging provider.
>
> Signed-off-by: Frode Nordahl <frode.nordahl at canonical.com>
Hi Frode,
Looks like this patch is having some compilation issues. Please take
a look at - https://github.com/ovsrobot/ovn/runs/3509456438
Thanks
Numan
> ---
> Documentation/automake.mk | 1 +
> Documentation/topics/index.rst | 1 +
> Documentation/topics/plug_providers/index.rst | 32 +++
> .../topics/plug_providers/plug-providers.rst | 199 ++++++++++++++
> acinclude.m4 | 49 ++++
> configure.ac | 2 +
> lib/automake.mk | 12 +-
> lib/plug-dummy.c | 123 +++++++++
> lib/plug-dummy.h | 33 +++
> lib/plug-provider.h | 98 +++++++
> lib/plug.c | 255 ++++++++++++++++++
> lib/plug.h | 107 ++++++++
> lib/test-plug.c | 72 +++++
> ovn-architecture.7.xml | 35 ++-
> tests/automake.mk | 4 +-
> tests/ovn-plug.at | 8 +
> 16 files changed, 1016 insertions(+), 15 deletions(-)
> create mode 100644 Documentation/topics/plug_providers/index.rst
> create mode 100644 Documentation/topics/plug_providers/plug-providers.rst
> create mode 100644 lib/plug-dummy.c
> create mode 100644 lib/plug-dummy.h
> create mode 100644 lib/plug-provider.h
> create mode 100644 lib/plug.c
> create mode 100644 lib/plug.h
> create mode 100644 lib/test-plug.c
> create mode 100644 tests/ovn-plug.at
>
> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> index b3fd3d62b..92a843d76 100644
> --- a/Documentation/automake.mk
> +++ b/Documentation/automake.mk
> @@ -28,6 +28,7 @@ DOC_SOURCE = \
> Documentation/topics/ovn-news-2.8.rst \
> Documentation/topics/role-based-access-control.rst \
> Documentation/topics/debugging-ddlog.rst \
> + Documentation/topics/plug_providers/plug-providers.rst \
> Documentation/howto/index.rst \
> Documentation/howto/docker.rst \
> Documentation/howto/firewalld.rst \
> diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst
> index d58d5618b..12bd113b7 100644
> --- a/Documentation/topics/index.rst
> +++ b/Documentation/topics/index.rst
> @@ -41,6 +41,7 @@ OVN
> high-availability
> role-based-access-control
> ovn-news-2.8
> + plug_providers/index
> testing
>
> .. list-table::
> diff --git a/Documentation/topics/plug_providers/index.rst b/Documentation/topics/plug_providers/index.rst
> new file mode 100644
> index 000000000..837eeae15
> --- /dev/null
> +++ b/Documentation/topics/plug_providers/index.rst
> @@ -0,0 +1,32 @@
> +..
> + Licensed under the Apache License, Version 2.0 (the "License"); you may
> + not use this file except in compliance with the License. You may obtain
> + a copy of the License at
> +
> + http://www.apache.org/licenses/LICENSE-2.0
> +
> + Unless required by applicable law or agreed to in writing, software
> + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
> + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
> + License for the specific language governing permissions and limitations
> + under the License.
> +
> + Convention for heading levels in OVN documentation:
> +
> + ======= Heading 0 (reserved for the title in a document)
> + ------- Heading 1
> + ~~~~~~~ Heading 2
> + +++++++ Heading 3
> + ''''''' Heading 4
> +
> + Avoid deeper levels because they do not render well.
> +
> +==============
> +Plug Providers
> +==============
> +
> +
> +.. toctree::
> + :maxdepth: 2
> +
> + plug-providers
> diff --git a/Documentation/topics/plug_providers/plug-providers.rst b/Documentation/topics/plug_providers/plug-providers.rst
> new file mode 100644
> index 000000000..7b891156c
> --- /dev/null
> +++ b/Documentation/topics/plug_providers/plug-providers.rst
> @@ -0,0 +1,199 @@
> +..
> + Licensed under the Apache License, Version 2.0 (the "License"); you may
> + not use this file except in compliance with the License. You may obtain
> + a copy of the License at
> +
> + http://www.apache.org/licenses/LICENSE-2.0
> +
> + Unless required by applicable law or agreed to in writing, software
> + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
> + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
> + License for the specific language governing permissions and limitations
> + under the License.
> +
> + Convention for heading levels in OVN documentation:
> +
> + ======= Heading 0 (reserved for the title in a document)
> + ------- Heading 1
> + ~~~~~~~ Heading 2
> + +++++++ Heading 3
> + ''''''' Heading 4
> +
> + Avoid deeper levels because they do not render well.
> +
> +==================
> +Plugging Providers
> +==================
> +
> +Traditionally it has been the CMSes responsibility to create VIFs as part of
> +instance life cycle, and subsequently manage plug/unplug operations on the
> +integration bridge following the conventions described in the
> +`Open vSwitch Integration Guide`_ for mapping of VIFs to OVN logical port.
> +
> +With the advent of NICs connected to multiple distinct CPUs we can have a
> +topology where the instance runs on one host and Open vSwitch and OVN runs on
> +a different host, the smartnic control plane CPU. The host facing interfaces
> +will be visible to Open vSwitch and OVN as representor ports.
> +
> +The actions necessary for plugging and unplugging the representor port in
> +Open vSwitch running on the smartnic control plane CPU would be the same for
> +every CMS.
> +
> +Instead of every CMS having to develop their own version of an agent to do
> +the plugging, we provide a pluggable infrastructure in OVN that allows the
> +`ovn-controller` to perform the plugging on CMS direction.
> +
> +Hardware or platform specific details for initialization and lookup of
> +representor ports is provided by an plugging provider library hosted inside or
> +outside the core OVN repository, and linked at OVN build time.
> +
> +Life Cycle of an OVN plugged VIF
> +--------------------------------
> +
> +1. CMS creates a record in the OVN Northbound Logical_Switch_Port table with
> + the options column containing the `plug-type` key with a value corresponding
> + to the `const char *type` provided by the plugging provider implementation
> + as well as a `requested-chassis` key with a value pointing at the name or
> + hostname of the chassis it wants the VIF plugged on. Additional plugging
> + provider specific key/value pairs must be provided for successful lookup.
> +
> +2. `ovn-northd` looks up the name or hostname provided in the
> + `requested-chassis` option and fills the OVN Southbound Port_Binding
> + requested_chassis column, it also copies relevant options over to the
> + Port_Binding record.
> +
> +3. `ovn-controller` monitors Southbound Port_Binding entries wth a
> + requested_chassis column pointing at its chassis UUID and when it encounters
> + a entry with option `plug-type` and it has registered a plug provider
> + matching that type it will act on it even if no local binding exists yet.
> +
> +4. It will fill the `struct plug_port_ctx_in` as defined in `lib/plug.h` with
> + `op_type` set to 'PLUG_OP_CREATE' and make a call to the plug providers
> + `plug_port_prepare` callback function. Plug provider performs lookup and
> + fills the `struct plug_port_ctx_out` as defined in `lib/plug.h`.
> +
> +5. `ovn-controller` creates a port and interface record in the local OVSDB
> + using the details provided by the plug provider and also adds
> + `external-ids:iface-id` with value matching the logical port name and
> + `external-ids:ovn-plugged` with value matching the logical port `plug-type`.
> + When the port creation is done a call will first be made to the plug
> + providers `plug_port_finish` function and then to the
> + `plug_port_ctx_destroy` function to free any memory allocated by the plug
> + implementation.
> +
> +6. The Open vSwitch vswitchd will assign a ofport to the newly created
> + interface and on the next `ovn-controller` main loop iteration flows will be
> + installed.
> +
> +7. On any change to the Southbound Port_Binding record or full recomputation
> + the `ovn-controller` will in addition to normal flow processing make a call
> + to the plug provider again similar to the first creation in case anything
> + needs updating for the interface record.
> +
> +8. The port will be unplugged when an event occurs which would make the
> + `ovn-controller` release a logical port, for example the Logical_Switch_Port
> + and Port_Binding entry disappearing from the database or its
> + `requested_chassis` column being pointed to a different chassis.
> +
> +
> +The plug provider interface
> +---------------------------
> +
> +The interface between internals of OVN and a plugging provider is a set of
> +callbacks as defined by the `struct plug_class` in `lib/plug-provider.h`.
> +
> +It is important to note that these callbacks will be called in the critical
> +path of the `ovn-controller` processing loop, so care must be taken to make the
> +implementation as efficient as possible, and under no circumstance can any of
> +the callback functions make calls that block.
> +
> +On `ovn-controller` startup, plug providers made available at build time will
> +be registered by the identifier provided in the `const char *type` pointer, at
> +this time the `init` function pointer will be called if it is non-NULL.
> +
> +> **Note**: apart from the `const char *type` pointer, no attempt will be made
> +to access plug provider data or functions before the call to the `init` has
> +been made.
> +
> +On `ovn-controller` exit, the plug providers registered in the above mentioned
> +procedure will have their `destroy` function pointer called if it is non-NULL.
> +
> +If the plug provider has internal lookup tables that need to be maintained they
> +can define a `run` function which will be called as part of the
> +`ovn-controller` incremental processing engine loop. If the changes
> +encountered can be safely processed incrementally the function should return
> +'true', if the changes cannot be processed incrementally, for example because
> +the change would impact already handled interfaces that we might not process in
> +a while, the function should return 'false' to force the `ovn-controller` to
> +perform a full recomputation.
> +
> +On update of Interface records the `ovn-controller` will pass on a `sset`
> +to the `ovsport_update_iface` function containing options the plug
> +implementation finds pertinent to maintain for successful operation. This
> +`sset` is retrieved by making a call to the plug implementation
> +`plug_get_maintained_iface_options` function pointer if it is non-NULL. This
> +allows presence of other users of the OVSDB maintaining a different set of
> +options on the same set of Interface records without wiping out their changes.
> +
> +Before creating or updating an existing interface record the plug provider
> +`plug_port_prepare` function pointer will be called with valid pointers to
> +`struct plug_port_ctx_in` and `struct plug_port_ctx_out` data structures. If
> +the plug provider implementation is able to perform lookup it should fill the
> +`struct plug_port_ctx_out` data structure and return 'true'. The
> +`ovn-controller` will then create or update the port/interface records and
> +then call `plug_port_finish` and `plug_port_ctx_destroy`. If the plug provider
> +implementation is unable to perform lookup or prepare the desired resource it
> +should return 'false' which will tell the `ovn-controller` to not create or
> +update the port, in this case it will also not call `plug_port_finish`, it will
> +however make a call to `plug_port_ctx_destroy`.
> +
> +Before removing port and interface records previously plugged by the
> +`ovn-controller` as identified by presence of the Interface
> +`external-ids:ovn-plugged` key, the `ovn-controller` will look up the
> +`plug-type` from `external-ids:ovn-plugged`, fill `struct plug_port_ctx_in`
> +with `op_type` set to 'PLUG_OP_REMOVE' and make a call to `plug_port_prepare`.
> +After the port and interface has been removed a call will be made to
> +`plug_port_finish`. Both calls will be made with the pointer to
> +`plug_port_ctx_out` set to 'NULL', and no call will be made to
> +`plug_port_ctx_destroy`.
> +
> +Building with in-tree plugging providers
> +----------------------------------------
> +
> +Plugging providers hosted in the OVN repository living under
> +`lib/plug_providers`:
> +
> +To enable them, provide the `--enable-plug-providers` command line option to
> +the configure script when building OVN.
> +
> +Building with an externally provided plugging provider
> +------------------------------------------------------
> +
> +There is also infrastructure in place to support linking OVN with an externally
> +built plugging provider.
> +
> +This external plugging provider must define a NULL-terminated array of pointers
> +to `struct plug_class` data structures named `plug_provider_classes`. Example:
> +
> +.. code-block:: C
> +
> + const struct plug_class *plug_provider_classes[] = {
> + &plug_foo,
> + NULL,
> + };
> +
> +The name of the repostiroy for the external plugging provider should be the
> +same as the name of the library it produces, and the built library artifact
> +should be placed in lib/.libs. Example:
> +
> +.. code-block:: none
> +
> + ovn-vif-foo/
> + ovn-vif-foo/lib/.libs/libovn-vif-foo.la
> +
> +To enable such a plugging provider provide the
> +`--with-plug-provider=/path/to/ovn-vif-foo` command line option to the
> +configure script when building OVN.
> +
> +.. LINKS
> +.. _Open vSwitch Integration Guide: https://docs.openvswitch.org/en/latest/topics/integration/
> diff --git a/acinclude.m4 b/acinclude.m4
> index e7f829520..793a073d1 100644
> --- a/acinclude.m4
> +++ b/acinclude.m4
> @@ -441,3 +441,52 @@ AC_DEFUN([OVN_CHECK_OVS], [
> AC_MSG_CHECKING([OVS version])
> AC_MSG_RESULT([$OVSVERSION])
> ])
> +
> +dnl OVN_CHECK_PLUG_PROVIDER
> +dnl
> +dnl Check for external plug provider
> +AC_DEFUN([OVN_CHECK_PLUG_PROVIDER], [
> + AC_ARG_VAR([PLUG_PROVIDER])
> + AC_ARG_WITH(
> + [plug-provider],
> + [AC_HELP_STRING([--with-plug-provider=/path/to/provider/repository],
> + [Specify path to a configured and built plug provider repository])],
> + [if test "$withval" = yes; then
> + if test -z "$PLUG_PROVIDER"; then
> + AC_MSG_ERROR([To build with external plug provider, specify the path to a configured and built plug provider repository --with-plug-provider or in \$PLUG_PROVIDER]),
> + fi
> + PLUG_PROVIDER="$(realpath $PLUG_PROVIDER)"
> + else
> + PLUG_PROVIDER="$(realpath $withval)"
> + fi
> + _plug_provider_name="$(basename $PLUG_PROVIDER)"
> + if test ! -f "$PLUG_PROVIDER/lib/.libs/lib${_plug_provider_name}.la"; then
> + AC_MSG_ERROR([$withval is not a configured and built plug provider library repository])
> + fi
> + PLUG_PROVIDER_LDFLAGS="-L$PLUG_PROVIDER/lib/.libs -l$_plug_provider_name"
> + ],
> + [PLUG_PROVIDER=no])
> + AC_MSG_CHECKING([for plug provider])
> + AC_MSG_RESULT([$PLUG_PROVIDER])
> + AC_SUBST([PLUG_PROVIDER_LDFLAGS])
> + AM_CONDITIONAL([HAVE_PLUG_PROVIDER], [test "$PLUG_PROVIDER" != no])
> + if test "$PLUG_PROVIDER" != no; then
> + AC_DEFINE([HAVE_PLUG_PROVIDER], [1],
> + [Build and link with external plug provider])
> + fi
> +])
> +
> +dnl OVN_ENABLE_PLUG
> +dnl
> +dnl Enable built-in plug providers
> +AC_DEFUN([OVN_ENABLE_PLUG], [
> + AC_ARG_ENABLE(
> + [plug-providers],
> + [AC_HELP_STRING([--enable-plug-providers], [Enable building of built-in plug providers])],
> + [], [enable_plug=no])
> + AM_CONDITIONAL([ENABLE_PLUG], [test "$enable_plug" != no])
> + if test "$enable_plug" != no; then
> + AC_DEFINE([ENABLE_PLUG], [1],
> + [Build built-in plug providers])
> + fi
> +])
> diff --git a/configure.ac b/configure.ac
> index df0b98295..7f3274e59 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -172,6 +172,8 @@ OVS_ENABLE_SPARSE
> OVS_CHECK_DDLOG([0.38])
> OVS_CHECK_PRAGMA_MESSAGE
> OVN_CHECK_OVS
> +OVN_CHECK_PLUG_PROVIDER
> +OVN_ENABLE_PLUG
> OVS_CTAGS_IDENTIFIERS
> AC_SUBST([OVS_CFLAGS])
> AC_SUBST([OVS_LDFLAGS])
> diff --git a/lib/automake.mk b/lib/automake.mk
> index ddfe33948..086fbd62d 100644
> --- a/lib/automake.mk
> +++ b/lib/automake.mk
> @@ -3,6 +3,11 @@ lib_libovn_la_LDFLAGS = \
> $(OVS_LTINFO) \
> -Wl,--version-script=$(top_builddir)/lib/libovn.sym \
> $(AM_LDFLAGS)
> +
> +if HAVE_PLUG_PROVIDER
> +lib_libovn_la_LDFLAGS += $(PLUG_PROVIDER_LDFLAGS)
> +endif
> +
> lib_libovn_la_SOURCES = \
> lib/acl-log.c \
> lib/acl-log.h \
> @@ -32,7 +37,12 @@ lib_libovn_la_SOURCES = \
> lib/inc-proc-eng.h \
> lib/lb.c \
> lib/lb.h \
> - lib/stopwatch-names.h
> + lib/stopwatch-names.h \
> + lib/plug-provider.h \
> + lib/plug.h \
> + lib/plug.c \
> + lib/plug-dummy.h \
> + lib/plug-dummy.c
> nodist_lib_libovn_la_SOURCES = \
> lib/ovn-dirs.c \
> lib/ovn-nb-idl.c \
> diff --git a/lib/plug-dummy.c b/lib/plug-dummy.c
> new file mode 100644
> index 000000000..505c37c9c
> --- /dev/null
> +++ b/lib/plug-dummy.c
> @@ -0,0 +1,123 @@
> +/*
> + * Copyright (c) 2021 Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#include <config.h>
> +#include "plug-dummy.h"
> +#include "plug-provider.h"
> +#include "plug.h"
> +
> +#include <stdint.h>
> +
> +#include "openvswitch/vlog.h"
> +#include "smap.h"
> +#include "sset.h"
> +
> +#ifndef IFNAMSIZ
> +#define IFNAMSIZ 16
> +#endif
> +
> +VLOG_DEFINE_THIS_MODULE(plug_dummy);
> +
> +static struct sset plug_dummy_maintained_iface_options;
> +
> +static int
> +plug_dummy_init(void)
> +{
> + sset_init(&plug_dummy_maintained_iface_options);
> + sset_add(&plug_dummy_maintained_iface_options, "plug-dummy-option");
> +
> + return 0;
> +}
> +
> +static int
> +plug_dummy_destroy(void)
> +{
> + sset_destroy(&plug_dummy_maintained_iface_options);
> +
> + return 0;
> +}
> +
> +static const struct sset*
> +plug_dummy_get_maintained_iface_options(void)
> +{
> + return &plug_dummy_maintained_iface_options;
> +}
> +
> +static bool
> +plug_dummy_run(struct plug_class *plug)
> +{
> + VLOG_DBG("plug_dummy_run(%p)", plug);
> +
> + return true;
> +}
> +
> +static bool
> +plug_dummy_port_prepare(const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out)
> +{
> + VLOG_DBG("plug_dummy_port_prepare: %s", ctx_in->lport_name);
> +
> + if (ctx_in->op_type == PLUG_OP_CREATE) {
> + size_t lport_name_len = strlen(ctx_in->lport_name);
> + ctx_out->name = xzalloc(IFNAMSIZ);
> + memcpy(ctx_out->name, ctx_in->lport_name,
> + (lport_name_len < IFNAMSIZ) ? lport_name_len : IFNAMSIZ - 1);
> + ctx_out->type = xstrdup("internal");
> + ctx_out->iface_options = xmalloc(sizeof *ctx_out->iface_options);
> + smap_init(ctx_out->iface_options);
> + smap_add(ctx_out->iface_options, "plug-dummy-option", "value");
> + }
> +
> + return true;
> +}
> +
> +static void
> +plug_dummy_port_finish(const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out OVS_UNUSED)
> +{
> + VLOG_DBG("plug_dummy_port_finish: %s", ctx_in->lport_name);
> +}
> +
> +static void
> +plug_dummy_port_ctx_destroy(const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out)
> +{
> + VLOG_DBG("plug_dummy_port_ctx_destroy: %s", ctx_in->lport_name);
> + ovs_assert(ctx_in->op_type == PLUG_OP_CREATE);
> + free(ctx_out->name);
> + free(ctx_out->type);
> + smap_destroy(ctx_out->iface_options);
> + free(ctx_out->iface_options);
> +}
> +
> +const struct plug_class plug_dummy_class = {
> + .type = "dummy",
> + .init = plug_dummy_init,
> + .destroy = plug_dummy_destroy,
> + .plug_get_maintained_iface_options =
> + plug_dummy_get_maintained_iface_options,
> + .run = plug_dummy_run,
> + .plug_port_prepare = plug_dummy_port_prepare,
> + .plug_port_finish = plug_dummy_port_finish,
> + .plug_port_ctx_destroy = plug_dummy_port_ctx_destroy,
> +};
> +
> +void
> +plug_dummy_enable(void)
> +{
> + plug_register_provider(&plug_dummy_class);
> +}
> +
> diff --git a/lib/plug-dummy.h b/lib/plug-dummy.h
> new file mode 100644
> index 000000000..6ea33671e
> --- /dev/null
> +++ b/lib/plug-dummy.h
> @@ -0,0 +1,33 @@
> +/*
> + * Copyright (c) 2021 Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#ifndef PLUG_DUMMY_H
> +#define PLUG_DUMMY_H 1
> +
> +/*
> + * The dummy plugger, allows for experimenting with plugging in a sandbox */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +void plug_dummy_enable(void);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* plug-dummy.h */
> diff --git a/lib/plug-provider.h b/lib/plug-provider.h
> new file mode 100644
> index 000000000..487534ee5
> --- /dev/null
> +++ b/lib/plug-provider.h
> @@ -0,0 +1,98 @@
> +/*
> + * Copyright (c) 2021 Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#ifndef PLUG_PROVIDER_H
> +#define PLUG_PROVIDER_H 1
> +
> +/* Provider interface to pluggers. A plugger implementation performs lookup
> + * and/or initialization of ports, typically representor ports, using generic
> + * non-blocking hardware interfaces. This allows the ovn-controller to, upon
> + * the CMS's request, create ports and interfaces in the chassis's Open vSwitch
> + * instances (also known as vif plugging).
> + */
> +
> +#include <stdbool.h>
> +
> +#include "plug.h"
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +struct plug_class {
> + /* Type of plugger in this class. */
> + const char *type;
> +
> + /* Called when the plug provider is registered, typically at program
> + * startup.
> + *
> + * This function may be set to null if a plug class needs no
> + * initialization at registration time. */
> + int (*init)(void);
> +
> + /* Called when the plug provider is unregistered, typically at program
> + * exit.
> + *
> + * This function may be set to null if a plug class needs no
> + * de-initialization at unregister time.*/
> + int (*destroy)(void);
> +
> + /* Performs periodic work needed by plugger, if any is necessary. Returns
> + * 'true; if the changes encountered could be handled incrementally,
> + * 'false' otherwise.
> + *
> + * Note that returning 'false' will instruct the incremental processing
> + * engine to perform a full recomputation. */
> + bool (*run)(struct plug_class *);
> +
> + /* Retrieve Interface options this plugger will maintain. This set is used
> + * to know which items to remove when maintaining the database record. */
> + const struct sset * (*plug_get_maintained_iface_options)(void);
> +
> + /* Pass plug_port_ctx_in to plug implementation to prepare for port
> + * creation/update.
> + *
> + * The plug implemantation can perform lookup or any per port
> + * initialization and should fill plug_port_ctx_out with data required for
> + * port/interface creation. The plug implementation should return true if
> + * it wants the caller to create/update a port/interface, false otherwise.
> + *
> + * Data in the plug_port_ctx_out struct is owned by the plugging library,
> + * and a call must be made to the plug_port_ctx_destroy callback to free
> + * up any allocations when done with port creation/update.
> + */
> + bool (*plug_port_prepare)(const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +
> + /* Notify plugging library that port update is done. */
> + void (*plug_port_finish)(const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +
> + /* Free any allocations made by the plug_port_prepare callback. */
> + void (*plug_port_ctx_destroy)(const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +};
> +
> +extern const struct plug_class plug_dummy_class;
> +#ifdef HAVE_PLUG_PROVIDER
> +extern const struct plug_class *plug_provider_classes[];
> +#endif
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* plug-provider.h */
> diff --git a/lib/plug.c b/lib/plug.c
> new file mode 100644
> index 000000000..c0c34214e
> --- /dev/null
> +++ b/lib/plug.c
> @@ -0,0 +1,255 @@
> +/*
> + * Copyright (c) 2021 Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#include <config.h>
> +#include "plug-provider.h"
> +#include "plug.h"
> +
> +#include <errno.h>
> +#include <stdint.h>
> +#include <string.h>
> +
> +#include "openvswitch/vlog.h"
> +#include "openvswitch/shash.h"
> +#include "smap.h"
> +#include "sset.h"
> +#include "lib/inc-proc-eng.h"
> +
> +VLOG_DEFINE_THIS_MODULE(plug);
> +
> +#ifdef ENABLE_PLUG
> +static const struct plug_class *base_plug_classes[] = {
> +};
> +#endif
> +
> +static struct shash plug_classes = SHASH_INITIALIZER(&plug_classes);
> +
> +/* Protects the 'plug_classes' shash. */
> +static struct ovs_mutex plug_classes_mutex = OVS_MUTEX_INITIALIZER;
> +
> +/* Initialize the the plug infrastructure by registering known plug classes */
> +void
> +plug_initialize(void)
> +{
> + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER;
> +
> + if (ovsthread_once_start(&once)) {
> +#ifdef ENABLE_PLUG
> + /* Register built-in plug provider classes */
> + for (int i = 0; i < ARRAY_SIZE(base_plug_classes); i++) {
> + plug_register_provider(base_plug_classes[i]);
> + }
> +#endif
> +#ifdef HAVE_PLUG_PROVIDER
> + /* Register external plug provider classes.
> + *
> + * Note that we cannot use the ARRAY_SIZE macro here as
> + * plug_provider_classes is defined in external code which is not
> + * available at compile time. The convention is to use a
> + * NULL-terminated array instead. */
> + for (const struct plug_class **pp = plug_provider_classes;
> + pp && *pp;
> + pp++)
> + {
> + plug_register_provider(*pp);
> + }
> +#endif
> + ovsthread_once_done(&once);
> + }
> +}
> +
> +static int
> +plug_register_provider__(const struct plug_class *new_class)
> +{
> + struct plug_class *plug_class;
> + int error;
> +
> + if (shash_find(&plug_classes, new_class->type)) {
> + VLOG_WARN("attempted to register duplicate plug provider: %s",
> + new_class->type);
> + return EEXIST;
> + }
> +
> + error = new_class->init ? new_class->init() : 0;
> + if (error) {
> + VLOG_WARN("failed to initialize %s plug class: %s",
> + new_class->type, ovs_strerror(error));
> + return error;
> + }
> +
> + plug_class = xmalloc(sizeof *plug_class);
> + memcpy(plug_class, new_class, sizeof *plug_class);
> +
> + shash_add(&plug_classes, new_class->type, plug_class);
> +
> + return 0;
> +}
> +
> +/* Register the new plug provider referred to in 'new_class' and perform any
> + * class level initialization as specified in its plug_class. */
> +int
> +plug_register_provider(const struct plug_class *new_class)
> +{
> + int error;
> +
> + ovs_mutex_lock(&plug_classes_mutex);
> + error = plug_register_provider__(new_class);
> + ovs_mutex_unlock(&plug_classes_mutex);
> +
> + return error;
> +}
> +
> +static int
> +plug_unregister_provider__(const char *type)
> +{
> + int error;
> + struct shash_node *node;
> + struct plug_class *plug_class;
> +
> + node = shash_find(&plug_classes, type);
> + if (!node) {
> + return EINVAL;
> + }
> +
> + plug_class = node->data;
> + error = plug_class->destroy ? plug_class->destroy() : 0;
> + if (error) {
> + VLOG_WARN("failed to destroy %s plug class: %s",
> + plug_class->type, ovs_strerror(error));
> + return error;
> + }
> +
> + shash_delete(&plug_classes, node);
> + free(plug_class);
> +
> + return 0;
> +}
> +
> +/* Unregister the plug provider identified by 'type' and perform any class
> + * level de-initialization as specified in its plug_class. */
> +int
> +plug_unregister_provider(const char *type)
> +{
> + int error;
> +
> + ovs_mutex_lock(&plug_classes_mutex);
> + error = plug_unregister_provider__(type);
> + ovs_mutex_unlock(&plug_classes_mutex);
> +
> + return error;
> +}
> +
> +const struct plug_class *
> +plug_get_provider(const char *type)
> +{
> + struct plug_class *plug_class;
> +
> + ovs_mutex_lock(&plug_classes_mutex);
> + plug_class = shash_find_data(&plug_classes, type);
> + ovs_mutex_unlock(&plug_classes_mutex);
> +
> + return plug_class;
> +}
> +
> +/* De-initialize and unregister the plug provider classes. */
> +void
> +plug_destroy_all(void)
> +{
> + struct shash_node *node, *next;
> +
> + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) {
> + struct plug_class *plug_class = node->data;
> + plug_unregister_provider(plug_class->type);
> + }
> +}
> +
> +/* Get the class level 'maintained_iface_options' set. */
> +const struct sset *
> +plug_get_maintained_iface_options(const struct plug_class *plug_class)
> +{
> + return plug_class->plug_get_maintained_iface_options();
> +}
> +
> +/* Prepare the logical port as identified by 'ctx_in' for port creation, update
> + * or removal as specified by 'ctx_in->op_type'.
> + *
> + * When 'ctx_in->op_type' is PLUG_OP_CREATE the plug implementation must fill
> + * 'ctx_out' with data to apply to the interface record maintained by OVN on
> + * its behalf.
> + *
> + * When 'ctx_in_op_type' is PLUG_OP_REMOVE 'ctx_out' should be set to NULL and
> + * the plug implementation must not attempt to use 'ctx_out'.
> + *
> + * The data in 'ctx_out' is owned by the plug implementation, and a call must
> + * be made to plug_port_ctx_destroy when done with it. */
> +bool
> +plug_port_prepare(const struct plug_class *plug_class,
> + const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out)
> +{
> + if (ctx_out) {
> + memset(ctx_out, 0, sizeof(*ctx_out));
> + }
> + return plug_class->plug_port_prepare(ctx_in, ctx_out);
> +}
> +
> +/* Notify the plug implementation that a port creation, update or removal has
> + * been completed */
> +void
> +plug_port_finish(const struct plug_class *plug_class,
> + const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out)
> +{
> + plug_class->plug_port_finish(ctx_in, ctx_out);
> +}
> +
> +/* Free any data allocated to 'ctx_out' in a prevous call to
> + * plug_port_prepare. */
> +void
> +plug_port_ctx_destroy(const struct plug_class *plug_class,
> + const struct plug_port_ctx_in *ctx_in,
> + struct plug_port_ctx_out *ctx_out)
> +{
> + plug_class->plug_port_ctx_destroy(ctx_in, ctx_out);
> +}
> +
> +/* Iterate over registered plug provider classes and call their 'run'
> + * function if defined.
> + *
> + * If any of the classes report that something has changed we will trigger a
> + * recompute. */
> +void
> +en_plug_provider_run(struct engine_node *inc_eng_node,
> + void *inc_eng_data OVS_UNUSED)
> +{
> + struct shash_node *node, *next;
> + bool handled = true;
> +
> + SHASH_FOR_EACH_SAFE (node, next, &plug_classes) {
> + struct plug_class *plug_class = node->data;
> + if (plug_class->run && !plug_class->run(plug_class)) {
> + handled = false;
> + }
> + }
> +
> + if (!handled) {
> + /* as we do not have a change handler registered in the incremental
> + * processing engine this will trigger a recompute. */
> + engine_set_node_state(inc_eng_node, EN_UPDATED);
> + } else {
> + engine_set_node_state(inc_eng_node, EN_UNCHANGED);
> + }
> +}
> diff --git a/lib/plug.h b/lib/plug.h
> new file mode 100644
> index 000000000..fef4c7f64
> --- /dev/null
> +++ b/lib/plug.h
> @@ -0,0 +1,107 @@
> +/*
> + * Copyright (c) 2021 Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#ifndef PLUG_H
> +#define PLUG_H 1
> +
> +/*
> + * Plug, the plugging interface. This module contains the infrastructure for
> + * registering and instantiating plugging classes which may be hosted inside
> + * or outside the core OVN repository. The data structures and functions for
> + * interacting with these plugging classes also live here.
> + */
> +
> +#include "smap.h"
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +struct plug_class;
> +struct ovsdb_idl_txn;
> +struct ovsrec_bridge;
> +
> +enum plug_op_type {
> + PLUG_OP_CREATE = 1, /* Port is created or updated */
> + PLUG_OP_REMOVE, /* Port is removed from this chassis */
> +};
> +
> +struct plug_port_ctx_in {
> + /* Operation being performed */
> + enum plug_op_type op_type;
> +
> + /* These are provided so that the plug implementation may make decisions
> + * based on environmental factors such as settings in the open-vswitch
> + * table and datapath type settings on the integration bridge. */
> + const struct ovsrec_open_vswitch_table *ovs_table;
> + const struct ovsrec_bridge *br_int;
> +
> + /* Name of logical port, can be useful for plugging library to track any
> + * per port resource initialization. */
> + const char *lport_name;
> +
> + /* Logical port options, while OVN will forward the contents verbatim from
> + * the Southbound database, the convention is for the plugging library to
> + * only make decisions based on the plug-* options. */
> + const struct smap *lport_options;
> +
> + /* When OVN knows about an existing interface record associated with this
> + * lport, these will be filled in with information about it. */
> + const char *iface_name;
> + const char *iface_type;
> + const struct smap *iface_options;
> +};
> +
> +struct plug_port_ctx_out {
> + /* The name to use for port and interface record. */
> + char *name;
> +
> + /* Type of interface to create. */
> + char *type;
> +
> + /* Options to set on the interface record. */
> + struct smap *iface_options;
> +};
> +
> +
> +void plug_initialize(void);
> +int plug_register_provider(const struct plug_class *);
> +int plug_unregister_provider(const char *type);
> +void plug_destroy_all(void);
> +const struct plug_class * plug_get_provider(const char *);
> +
> +const struct sset * plug_get_maintained_iface_options(
> + const struct plug_class *plug_class);
> +
> +bool plug_port_prepare(const struct plug_class *,
> + const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +void plug_port_finish(const struct plug_class *,
> + const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +void plug_port_ctx_destroy(const struct plug_class *,
> + const struct plug_port_ctx_in *,
> + struct plug_port_ctx_out *);
> +
> +struct engine_node;
> +
> +void en_plug_provider_run(struct engine_node *, void *);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* plug.h */
> diff --git a/lib/test-plug.c b/lib/test-plug.c
> new file mode 100644
> index 000000000..bddce23c4
> --- /dev/null
> +++ b/lib/test-plug.c
> @@ -0,0 +1,72 @@
> +/* Copyright (c) 2021, Canonical
> + *
> + * Licensed under the Apache License, Version 2.0 (the "License");
> + * you may not use this file except in compliance with the License.
> + * You may obtain a copy of the License at:
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +#include <config.h>
> +#include <errno.h>
> +
> +#include "plug.h"
> +#include "plug-dummy.h"
> +#include "plug-provider.h"
> +#include "smap.h"
> +#include "sset.h"
> +#include "tests/ovstest.h"
> +
> +static void
> +test_plug(struct ovs_cmdl_context *ctx OVS_UNUSED)
> +{
> + const struct plug_class *plug_class;
> +
> + ovs_assert(plug_unregister_provider("dummy") == EINVAL);
> +
> + ovs_assert(!plug_register_provider(&plug_dummy_class));
> + plug_class = plug_get_provider("dummy");
> + ovs_assert(plug_register_provider(&plug_dummy_class) == EEXIST);
> +
> + ovs_assert(sset_contains(plug_get_maintained_iface_options(plug_class),
> + "plug-dummy-option"));
> +
> + struct smap fake_lport_options = SMAP_INITIALIZER(&fake_lport_options);
> + struct plug_port_ctx_in ctx_in = {
> + .op_type = PLUG_OP_CREATE,
> + .lport_name = "lsp1",
> + .lport_options = &fake_lport_options,
> + };
> + struct plug_port_ctx_out ctx_out;
> + plug_port_prepare(plug_class, &ctx_in, &ctx_out);
> + ovs_assert(!strcmp(ctx_out.name, "lsp1"));
> + ovs_assert(!strcmp(ctx_out.type, "internal"));
> + ovs_assert(!strcmp(smap_get(
> + ctx_out.iface_options, "plug-dummy-option"), "value"));
> +
> + plug_port_finish(plug_class, &ctx_in, &ctx_out);
> + plug_port_ctx_destroy(plug_class, &ctx_in, &ctx_out);
> + plug_destroy_all();
> +}
> +
> +static void
> +test_plug_main(int argc, char *argv[])
> +{
> + set_program_name(argv[0]);
> + static const struct ovs_cmdl_command commands[] = {
> + {"run", NULL, 0, 0, test_plug, OVS_RO},
> + {NULL, NULL, 0, 0, NULL, OVS_RO},
> + };
> + struct ovs_cmdl_context ctx;
> + ctx.argc = argc - 1;
> + ctx.argv = argv + 1;
> + ovs_cmdl_run_command(&ctx, commands);
> +}
> +
> +OVSTEST_REGISTER("test-plug", test_plug_main);
> diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml
> index 3d2910358..152367a89 100644
> --- a/ovn-architecture.7.xml
> +++ b/ovn-architecture.7.xml
> @@ -67,8 +67,9 @@
> <li>
> One or more (usually many) <dfn>hypervisors</dfn>. Hypervisors must run
> Open vSwitch and implement the interface described in
> - <code>Documentation/topics/integration.rst</code> in the OVN source tree.
> - Any hypervisor platform supported by Open vSwitch is acceptable.
> + <code>Documentation/topics/integration.rst</code> in the Open vSwitch
> + source tree. Any hypervisor platform supported by Open vSwitch is
> + acceptable.
> </li>
>
> <li>
> @@ -318,11 +319,19 @@
>
> <li>
> On a hypervisor, any VIFs that are to be attached to logical networks.
> - The hypervisor itself, or the integration between Open vSwitch and the
> - hypervisor (described in
> - <code>Documentation/topics/integration.rst</code>) takes care of this.
> - (This is not part of OVN or new to OVN; this is pre-existing integration
> - work that has already been done on hypervisors that support OVS.)
> + For instances connected through software emulated ports such as TUN/TAP
> + or VETH pairs, the hypervisor itself will normally create ports and plug
> + them into the integration bridge. For instances connected through
> + representor ports, typically used with hardware offload, the
> + <code>ovn-controller</code> may on CMS direction consult a plugging
> + provider library for representor port lookup and plug them into the
> + integration bridge (please refer to
> + <code>Documentation/topics/plugging-providers.rst</code> for more
> + information). In both cases the conventions described in
> + <code>Documentation/topics/integration.rst</code> in the Open vSwitch
> + source tree is followed to ensure mapping between OVN logical port and
> + VIF. (This is pre-existing integration work that has already been done
> + on hypervisors that support OVS.)
> </li>
>
> <li>
> @@ -921,12 +930,12 @@
> Eventually, a user powers on the VM that owns the VIF. On the hypervisor
> where the VM is powered on, the integration between the hypervisor and
> Open vSwitch (described in
> - <code>Documentation/topics/integration.rst</code>) adds the VIF to the OVN
> - integration bridge and stores <var>vif-id</var> in
> - <code>external_ids</code>:<code>iface-id</code> to indicate that the
> - interface is an instantiation of the new VIF. (None of this code is new
> - in OVN; this is pre-existing integration work that has already been done
> - on hypervisors that support OVS.)
> + <code>Documentation/topics/integration.rst</code> in the Open vSwitch
> + source tree) adds the VIF to the OVN integration bridge and stores
> + <var>vif-id</var> in <code>external_ids</code>:<code>iface-id</code> to
> + indicate that the interface is an instantiation of the new VIF. (None of
> + this code is new in OVN; this is pre-existing integration work that has
> + already been done on hypervisors that support OVS.)
> </li>
>
> <li>
> diff --git a/tests/automake.mk b/tests/automake.mk
> index 5b890d644..ad8978541 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -38,7 +38,8 @@ TESTSUITE_AT = \
> tests/ovn-ipam.at \
> tests/ovn-features.at \
> tests/ovn-lflow-cache.at \
> - tests/ovn-ipsec.at
> + tests/ovn-ipsec.at \
> + tests/ovn-plug.at
>
> SYSTEM_KMOD_TESTSUITE_AT = \
> tests/system-common-macros.at \
> @@ -248,6 +249,7 @@ tests_ovstest_SOURCES = \
> controller/ofctrl-seqno.c \
> controller/ofctrl-seqno.h \
> lib/test-ovn-features.c \
> + lib/test-plug.c \
> northd/test-ipam.c \
> northd/ipam.c \
> northd/ipam.h
> diff --git a/tests/ovn-plug.at b/tests/ovn-plug.at
> new file mode 100644
> index 000000000..d5c6a1b6d
> --- /dev/null
> +++ b/tests/ovn-plug.at
> @@ -0,0 +1,8 @@
> +#
> +# Unit tests for the lib/plug.c module.
> +#
> +AT_BANNER([OVN unit tests - plug])
> +
> +AT_SETUP([unit test -- plugging infrastructure tests])
> +AT_CHECK([ovstest test-plug run], [0], [])
> +AT_CLEANUP
> --
> 2.32.0
>
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
More information about the dev
mailing list