[ovs-dev] [PATCH v4] Remove OVN.

Numan Siddique nusiddiq at redhat.com
Tue Aug 6 14:33:16 UTC 2019


Tested-by: Numan Siddique <nusiddiq at redhat.com>
Acked-by: Numan Siddique <nusiddiq at redhat.com>

Numan


On Tue, Jul 30, 2019 at 1:13 AM Mark Michelson <mmichels at redhat.com> wrote:

> OVN is separated into its own repo. This commit removes the OVN source,
> OVN tests, and OVN documentation. It also removes mentions of OVN from
> most documentation. The only place where OVN has been left is in
> changelogs/NEWS, since we shouldn't mess with the history of the
> project.
>
> Signed-off-by: Mark Michelson <mmichels at redhat.com>
> ---
> v3 -> v4 Fixed 0-day Robot problems with NEWS and documentation
> v2 -> v3 Added NEWS entry for removal of OVN
> v1 -> v2 Rebase
> ---
>  Documentation/automake.mk                          |    12 -
>  Documentation/faq/index.rst                        |     1 -
>  Documentation/faq/ovn.rst                          |    90 -
>  Documentation/howto/docker.rst                     |   326 -
>  Documentation/howto/firewalld.rst                  |   107 -
>  Documentation/howto/index.rst                      |     9 -
>  Documentation/howto/openstack-containers.rst       |   135 -
>  Documentation/index.rst                            |    20 +-
>  Documentation/intro/install/fedora.rst             |    12 -
>  Documentation/intro/install/index.rst              |     8 -
>  Documentation/intro/install/ovn-upgrades.rst       |   115 -
>  Documentation/ref/ovs-sim.1.rst                    |   100 -
>  Documentation/topics/high-availability.rst         |   440 -
>  Documentation/topics/index.rst                     |    18 +-
>  Documentation/topics/ovn-news-2.8.rst              |   278 -
>  Documentation/topics/role-based-access-control.rst |   101 -
>  Documentation/tutorials/index.rst                  |     7 +-
>  Documentation/tutorials/ovn-ipsec.rst              |   146 -
>  Documentation/tutorials/ovn-openstack.rst          |  1922 ---
>  Documentation/tutorials/ovn-rbac.rst               |   134 -
>  Documentation/tutorials/ovn-sandbox.rst            |   177 -
>  Makefile.am                                        |     1 -
>  NEWS                                               |     5 +-
>  configure.ac                                       |     3 -
>  debian/.gitignore                                  |     5 -
>  debian/automake.mk                                 |    22 -
>  debian/control                                     |    78 -
>  debian/ovn-central.dirs                            |     1 -
>  debian/ovn-central.init                            |    60 -
>  debian/ovn-central.install                         |     3 -
>  debian/ovn-central.manpages                        |     1 -
>  debian/ovn-central.postinst                        |    49 -
>  debian/ovn-central.postrm                          |    48 -
>  debian/ovn-central.template                        |     5 -
>  debian/ovn-common.install                          |     7 -
>  debian/ovn-common.manpages                         |     8 -
>  debian/ovn-common.postinst                         |    24 -
>  debian/ovn-common.postrm                           |    23 -
>  debian/ovn-controller-vtep.init                    |    54 -
>  debian/ovn-controller-vtep.install                 |     1 -
>  debian/ovn-controller-vtep.manpages                |     1 -
>  debian/ovn-docker.install                          |     2 -
>  debian/ovn-host.dirs                               |     1 -
>  debian/ovn-host.init                               |    54 -
>  debian/ovn-host.install                            |     1 -
>  debian/ovn-host.manpages                           |     1 -
>  debian/ovn-host.postinst                           |    49 -
>  debian/ovn-host.postrm                             |    44 -
>  debian/ovn-host.template                           |     5 -
>  debian/rules                                       |     6 -
>  include/automake.mk                                |     1 -
>  include/ovn/actions.h                              |   622 -
>  include/ovn/automake.mk                            |     6 -
>  include/ovn/expr.h                                 |   518 -
>  include/ovn/lex.h                                  |   152 -
>  include/ovn/logical-fields.h                       |   130 -
>  lib/db-ctl-base.xml                                |    12 +-
>  manpages.mk                                        |    28 -
>  ovn/.gitignore                                     |     8 -
>  ovn/TODO.rst                                       |   147 -
>  ovn/automake.mk                                    |    92 -
>  ovn/controller-vtep/.gitignore                     |     2 -
>  ovn/controller-vtep/automake.mk                    |    14 -
>  ovn/controller-vtep/binding.c                      |   274 -
>  ovn/controller-vtep/binding.h                      |    27 -
>  ovn/controller-vtep/gateway.c                      |   230 -
>  ovn/controller-vtep/gateway.h                      |    26 -
>  ovn/controller-vtep/ovn-controller-vtep.8.xml      |    80 -
>  ovn/controller-vtep/ovn-controller-vtep.c          |   272 -
>  ovn/controller-vtep/ovn-controller-vtep.h          |    51 -
>  ovn/controller-vtep/vtep.c                         |   600 -
>  ovn/controller-vtep/vtep.h                         |    27 -
>  ovn/controller/.gitignore                          |     2 -
>  ovn/controller/automake.mk                         |    32 -
>  ovn/controller/bfd.c                               |   268 -
>  ovn/controller/bfd.h                               |    41 -
>  ovn/controller/binding.c                           |   764 -
>  ovn/controller/binding.h                           |    57 -
>  ovn/controller/chassis.c                           |   671 -
>  ovn/controller/chassis.h                           |    46 -
>  ovn/controller/encaps.c                            |   409 -
>  ovn/controller/encaps.h                            |    48 -
>  ovn/controller/ha-chassis.c                        |   203 -
>  ovn/controller/ha-chassis.h                        |    50 -
>  ovn/controller/ip-mcast.c                          |   164 -
>  ovn/controller/ip-mcast.h                          |    52 -
>  ovn/controller/lflow.c                             |   898 --
>  ovn/controller/lflow.h                             |   184 -
>  ovn/controller/lport.c                             |   102 -
>  ovn/controller/lport.h                             |    52 -
>  ovn/controller/ofctrl.c                            |  1393 --
>  ovn/controller/ofctrl.h                            |    87 -
>  ovn/controller/ovn-controller.8.xml                |   456 -
>  ovn/controller/ovn-controller.c                    |  2366 ---
>  ovn/controller/ovn-controller.h                    |    85 -
>  ovn/controller/patch.c                             |   273 -
>  ovn/controller/patch.h                             |    42 -
>  ovn/controller/physical.c                          |  1459 --
>  ovn/controller/physical.h                          |    74 -
>  ovn/controller/pinctrl.c                           |  4343 ------
>  ovn/controller/pinctrl.h                           |    51 -
>  ovn/lib/.gitignore                                 |     7 -
>  ovn/lib/acl-log.c                                  |   105 -
>  ovn/lib/acl-log.h                                  |    54 -
>  ovn/lib/actions.c                                  |  2902 ----
>  ovn/lib/automake.mk                                |    57 -
>  ovn/lib/chassis-index.c                            |    67 -
>  ovn/lib/chassis-index.h                            |    30 -
>  ovn/lib/expr.c                                     |  3450 -----
>  ovn/lib/extend-table.c                             |   208 -
>  ovn/lib/extend-table.h                             |    82 -
>  ovn/lib/inc-proc-eng.c                             |   201 -
>  ovn/lib/inc-proc-eng.h                             |   234 -
>  ovn/lib/ip-mcast-index.c                           |    40 -
>  ovn/lib/ip-mcast-index.h                           |    36 -
>  ovn/lib/lex.c                                      |  1023 --
>  ovn/lib/libovn.sym.in                              |     4 -
>  ovn/lib/logical-fields.c                           |   261 -
>  ovn/lib/mcast-group-index.c                        |    43 -
>  ovn/lib/mcast-group-index.h                        |    32 -
>  ovn/lib/ovn-l7.h                                   |   322 -
>  ovn/lib/ovn-nb-idl.ann                             |     9 -
>  ovn/lib/ovn-sb-idl.ann                             |    29 -
>  ovn/lib/ovn-util.c                                 |   373 -
>  ovn/lib/ovn-util.h                                 |    84 -
>  ovn/northd/.gitignore                              |     2 -
>  ovn/northd/automake.mk                             |    10 -
>  ovn/northd/ovn-northd.8.xml                        |  2544 ----
>  ovn/northd/ovn-northd.c                            |  9447 ------------
>  ovn/ovn-architecture.7.xml                         |  2074 ---
>  ovn/ovn-nb.ovsschema                               |   449 -
>  ovn/ovn-nb.xml                                     |  2917 ----
>  ovn/ovn-sb.ovsschema                               |   404 -
>  ovn/ovn-sb.xml                                     |  3638 -----
>  ovn/utilities/.gitignore                           |    11 -
>  ovn/utilities/automake.mk                          |    57 -
>  ovn/utilities/bugtool/automake.mk                  |     9 -
>  ovn/utilities/bugtool/ovn-bugtool-nbctl-show       |    19 -
>  ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list |    19 -
>  ovn/utilities/bugtool/ovn-bugtool-sbctl-show       |    19 -
>  .../bugtool/plugins/network-status/ovn.xml         |    23 -
>  ovn/utilities/ovn-ctl                              |   822 --
>  ovn/utilities/ovn-ctl.8.xml                        |   215 -
>  ovn/utilities/ovn-detrace.1.in                     |    38 -
>  ovn/utilities/ovn-detrace.in                       |   215 -
>  ovn/utilities/ovn-docker-overlay-driver.in         |   442 -
>  ovn/utilities/ovn-docker-underlay-driver.in        |   677 -
>  ovn/utilities/ovn-nbctl.8.xml                      |  1228 --
>  ovn/utilities/ovn-nbctl.c                          |  6061 --------
>  ovn/utilities/ovn-sbctl.8.in                       |   303 -
>  ovn/utilities/ovn-sbctl.c                          |  1541 --
>  ovn/utilities/ovn-trace.8.xml                      |   485 -
>  ovn/utilities/ovn-trace.c                          |  2373 ---
>  ovn/utilities/ovndb-servers.ocf                    |   642 -
>  ovsdb/ovsdb-tool.1.in                              |    23 +-
>  rhel/automake.mk                                   |    16 +-
>  rhel/ovn-fedora.spec.in                            |   432 -
>  ...walld_services_ovn-central-firewall-service.xml |     7 -
>  ...irewalld_services_ovn-host-firewall-service.xml |     6 -
>  ..._lib_systemd_system_ovn-controller-vtep.service |    50 -
>  rhel/usr_lib_systemd_system_ovn-controller.service |    34 -
>  rhel/usr_lib_systemd_system_ovn-northd.service     |    35 -
>  tests/atlocal.in                                   |     4 -
>  tests/automake.mk                                  |    46 +-
>  tests/ofproto-macros.at                            |     2 +-
>  tests/oss-fuzz/automake.mk                         |    10 -
>  tests/oss-fuzz/config/expr.dict                    |   120 -
>  tests/oss-fuzz/config/expr_parse_target.options    |     3 -
>  tests/oss-fuzz/expr_parse_target.c                 |   464 -
>  tests/ovn-controller-vtep.at                       |   467 -
>  tests/ovn-controller.at                            |   294 -
>  tests/ovn-macros.at                                |   180 -
>  tests/ovn-nbctl.at                                 |  1660 ---
>  tests/ovn-northd.at                                |   900 --
>  tests/ovn-performance.at                           |   424 -
>  tests/ovn-sbctl.at                                 |   150 -
>  tests/ovn.at                                       | 14702
> -------------------
>  tests/ovsdb-cluster-testsuite.at                   |    10 -
>  tests/ovsdb-cluster.at                             |   450 -
>  tests/system-kmod-testsuite.at                     |     2 -
>  tests/system-ovn.at                                |  1663 ---
>  tests/system-userspace-testsuite.at                |     2 -
>  tests/test-ovn.c                                   |  1584 --
>  tests/testsuite.at                                 |     8 -
>  tutorial/automake.mk                               |     3 +-
>  tutorial/ovn-setup.sh                              |    37 -
>  tutorial/ovs-sandbox                               |   261 -
>  utilities/bugtool/automake.mk                      |     9 +-
>  utilities/ovs-sim.in                               |   237 +-
>  xenserver/openvswitch-xen.spec.in                  |     7 -
>  190 files changed, 43 insertions(+), 93307 deletions(-)
>  delete mode 100644 Documentation/faq/ovn.rst
>  delete mode 100644 Documentation/howto/docker.rst
>  delete mode 100644 Documentation/howto/firewalld.rst
>  delete mode 100644 Documentation/howto/openstack-containers.rst
>  delete mode 100644 Documentation/intro/install/ovn-upgrades.rst
>  delete mode 100644 Documentation/topics/high-availability.rst
>  delete mode 100644 Documentation/topics/ovn-news-2.8.rst
>  delete mode 100644 Documentation/topics/role-based-access-control.rst
>  delete mode 100644 Documentation/tutorials/ovn-ipsec.rst
>  delete mode 100644 Documentation/tutorials/ovn-openstack.rst
>  delete mode 100644 Documentation/tutorials/ovn-rbac.rst
>  delete mode 100644 Documentation/tutorials/ovn-sandbox.rst
>  delete mode 100644 debian/ovn-central.dirs
>  delete mode 100755 debian/ovn-central.init
>  delete mode 100644 debian/ovn-central.install
>  delete mode 100644 debian/ovn-central.manpages
>  delete mode 100755 debian/ovn-central.postinst
>  delete mode 100755 debian/ovn-central.postrm
>  delete mode 100644 debian/ovn-central.template
>  delete mode 100644 debian/ovn-common.install
>  delete mode 100644 debian/ovn-common.manpages
>  delete mode 100644 debian/ovn-common.postinst
>  delete mode 100644 debian/ovn-common.postrm
>  delete mode 100755 debian/ovn-controller-vtep.init
>  delete mode 100644 debian/ovn-controller-vtep.install
>  delete mode 100644 debian/ovn-controller-vtep.manpages
>  delete mode 100644 debian/ovn-docker.install
>  delete mode 100644 debian/ovn-host.dirs
>  delete mode 100755 debian/ovn-host.init
>  delete mode 100644 debian/ovn-host.install
>  delete mode 100644 debian/ovn-host.manpages
>  delete mode 100755 debian/ovn-host.postinst
>  delete mode 100755 debian/ovn-host.postrm
>  delete mode 100644 debian/ovn-host.template
>  delete mode 100644 include/ovn/actions.h
>  delete mode 100644 include/ovn/automake.mk
>  delete mode 100644 include/ovn/expr.h
>  delete mode 100644 include/ovn/lex.h
>  delete mode 100644 include/ovn/logical-fields.h
>  delete mode 100644 ovn/.gitignore
>  delete mode 100644 ovn/TODO.rst
>  delete mode 100644 ovn/automake.mk
>  delete mode 100644 ovn/controller-vtep/.gitignore
>  delete mode 100644 ovn/controller-vtep/automake.mk
>  delete mode 100644 ovn/controller-vtep/binding.c
>  delete mode 100644 ovn/controller-vtep/binding.h
>  delete mode 100644 ovn/controller-vtep/gateway.c
>  delete mode 100644 ovn/controller-vtep/gateway.h
>  delete mode 100644 ovn/controller-vtep/ovn-controller-vtep.8.xml
>  delete mode 100644 ovn/controller-vtep/ovn-controller-vtep.c
>  delete mode 100644 ovn/controller-vtep/ovn-controller-vtep.h
>  delete mode 100644 ovn/controller-vtep/vtep.c
>  delete mode 100644 ovn/controller-vtep/vtep.h
>  delete mode 100644 ovn/controller/.gitignore
>  delete mode 100644 ovn/controller/automake.mk
>  delete mode 100644 ovn/controller/bfd.c
>  delete mode 100644 ovn/controller/bfd.h
>  delete mode 100644 ovn/controller/binding.c
>  delete mode 100644 ovn/controller/binding.h
>  delete mode 100644 ovn/controller/chassis.c
>  delete mode 100644 ovn/controller/chassis.h
>  delete mode 100644 ovn/controller/encaps.c
>  delete mode 100644 ovn/controller/encaps.h
>  delete mode 100644 ovn/controller/ha-chassis.c
>  delete mode 100644 ovn/controller/ha-chassis.h
>  delete mode 100644 ovn/controller/ip-mcast.c
>  delete mode 100644 ovn/controller/ip-mcast.h
>  delete mode 100644 ovn/controller/lflow.c
>  delete mode 100644 ovn/controller/lflow.h
>  delete mode 100644 ovn/controller/lport.c
>  delete mode 100644 ovn/controller/lport.h
>  delete mode 100644 ovn/controller/ofctrl.c
>  delete mode 100644 ovn/controller/ofctrl.h
>  delete mode 100644 ovn/controller/ovn-controller.8.xml
>  delete mode 100644 ovn/controller/ovn-controller.c
>  delete mode 100644 ovn/controller/ovn-controller.h
>  delete mode 100644 ovn/controller/patch.c
>  delete mode 100644 ovn/controller/patch.h
>  delete mode 100644 ovn/controller/physical.c
>  delete mode 100644 ovn/controller/physical.h
>  delete mode 100644 ovn/controller/pinctrl.c
>  delete mode 100644 ovn/controller/pinctrl.h
>  delete mode 100644 ovn/lib/.gitignore
>  delete mode 100644 ovn/lib/acl-log.c
>  delete mode 100644 ovn/lib/acl-log.h
>  delete mode 100644 ovn/lib/actions.c
>  delete mode 100644 ovn/lib/automake.mk
>  delete mode 100644 ovn/lib/chassis-index.c
>  delete mode 100644 ovn/lib/chassis-index.h
>  delete mode 100644 ovn/lib/expr.c
>  delete mode 100644 ovn/lib/extend-table.c
>  delete mode 100644 ovn/lib/extend-table.h
>  delete mode 100644 ovn/lib/inc-proc-eng.c
>  delete mode 100644 ovn/lib/inc-proc-eng.h
>  delete mode 100644 ovn/lib/ip-mcast-index.c
>  delete mode 100644 ovn/lib/ip-mcast-index.h
>  delete mode 100644 ovn/lib/lex.c
>  delete mode 100644 ovn/lib/libovn.sym.in
>  delete mode 100644 ovn/lib/logical-fields.c
>  delete mode 100644 ovn/lib/mcast-group-index.c
>  delete mode 100644 ovn/lib/mcast-group-index.h
>  delete mode 100644 ovn/lib/ovn-l7.h
>  delete mode 100644 ovn/lib/ovn-nb-idl.ann
>  delete mode 100644 ovn/lib/ovn-sb-idl.ann
>  delete mode 100644 ovn/lib/ovn-util.c
>  delete mode 100644 ovn/lib/ovn-util.h
>  delete mode 100644 ovn/northd/.gitignore
>  delete mode 100644 ovn/northd/automake.mk
>  delete mode 100644 ovn/northd/ovn-northd.8.xml
>  delete mode 100644 ovn/northd/ovn-northd.c
>  delete mode 100644 ovn/ovn-architecture.7.xml
>  delete mode 100644 ovn/ovn-nb.ovsschema
>  delete mode 100644 ovn/ovn-nb.xml
>  delete mode 100644 ovn/ovn-sb.ovsschema
>  delete mode 100644 ovn/ovn-sb.xml
>  delete mode 100644 ovn/utilities/.gitignore
>  delete mode 100644 ovn/utilities/automake.mk
>  delete mode 100644 ovn/utilities/bugtool/automake.mk
>  delete mode 100644 ovn/utilities/bugtool/ovn-bugtool-nbctl-show
>  delete mode 100644 ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list
>  delete mode 100644 ovn/utilities/bugtool/ovn-bugtool-sbctl-show
>  delete mode 100644 ovn/utilities/bugtool/plugins/network-status/ovn.xml
>  delete mode 100755 ovn/utilities/ovn-ctl
>  delete mode 100644 ovn/utilities/ovn-ctl.8.xml
>  delete mode 100644 ovn/utilities/ovn-detrace.1.in
>  delete mode 100755 ovn/utilities/ovn-detrace.in
>  delete mode 100755 ovn/utilities/ovn-docker-overlay-driver.in
>  delete mode 100755 ovn/utilities/ovn-docker-underlay-driver.in
>  delete mode 100644 ovn/utilities/ovn-nbctl.8.xml
>  delete mode 100644 ovn/utilities/ovn-nbctl.c
>  delete mode 100644 ovn/utilities/ovn-sbctl.8.in
>  delete mode 100644 ovn/utilities/ovn-sbctl.c
>  delete mode 100644 ovn/utilities/ovn-trace.8.xml
>  delete mode 100644 ovn/utilities/ovn-trace.c
>  delete mode 100755 ovn/utilities/ovndb-servers.ocf
>  delete mode 100644 rhel/ovn-fedora.spec.in
>  delete mode 100644
> rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml
>  delete mode 100644
> rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml
>  delete mode 100644 rhel/usr_lib_systemd_system_ovn-controller-vtep.service
>  delete mode 100644 rhel/usr_lib_systemd_system_ovn-controller.service
>  delete mode 100644 rhel/usr_lib_systemd_system_ovn-northd.service
>  delete mode 100644 tests/oss-fuzz/config/expr.dict
>  delete mode 100644 tests/oss-fuzz/config/expr_parse_target.options
>  delete mode 100644 tests/oss-fuzz/expr_parse_target.c
>  delete mode 100644 tests/ovn-controller-vtep.at
>  delete mode 100644 tests/ovn-controller.at
>  delete mode 100644 tests/ovn-macros.at
>  delete mode 100644 tests/ovn-nbctl.at
>  delete mode 100644 tests/ovn-northd.at
>  delete mode 100644 tests/ovn-performance.at
>  delete mode 100644 tests/ovn-sbctl.at
>  delete mode 100644 tests/ovn.at
>  delete mode 100644 tests/ovsdb-cluster-testsuite.at
>  delete mode 100644 tests/ovsdb-cluster.at
>  delete mode 100644 tests/system-ovn.at
>  delete mode 100644 tests/test-ovn.c
>  delete mode 100755 tutorial/ovn-setup.sh
>
> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> index 2a3214a3c..cd68f3b15 100644
> --- a/Documentation/automake.mk
> +++ b/Documentation/automake.mk
> @@ -18,7 +18,6 @@ DOC_SOURCE = \
>         Documentation/intro/install/fedora.rst \
>         Documentation/intro/install/general.rst \
>         Documentation/intro/install/netbsd.rst \
> -       Documentation/intro/install/ovn-upgrades.rst \
>         Documentation/intro/install/rhel.rst \
>         Documentation/intro/install/userspace.rst \
>         Documentation/intro/install/windows.rst \
> @@ -26,12 +25,8 @@ DOC_SOURCE = \
>         Documentation/tutorials/index.rst \
>         Documentation/tutorials/faucet.rst \
>         Documentation/tutorials/ovs-advanced.rst \
> -       Documentation/tutorials/ovn-openstack.rst \
> -       Documentation/tutorials/ovn-sandbox.rst \
>         Documentation/tutorials/ovs-conntrack.rst \
>         Documentation/tutorials/ipsec.rst \
> -       Documentation/tutorials/ovn-ipsec.rst \
> -       Documentation/tutorials/ovn-rbac.rst \
>         Documentation/topics/index.rst \
>         Documentation/topics/bonding.rst \
>         Documentation/topics/idl-compound-indexes.rst \
> @@ -54,28 +49,22 @@ DOC_SOURCE = \
>         Documentation/topics/fuzzing/ovs-fuzzers.rst \
>         Documentation/topics/fuzzing/security-analysis-of-ovs-fuzzers.rst \
>         Documentation/topics/testing.rst \
> -       Documentation/topics/high-availability.rst \
>         Documentation/topics/integration.rst \
>         Documentation/topics/language-bindings.rst \
>         Documentation/topics/networking-namespaces.rst \
>         Documentation/topics/openflow.rst \
> -       Documentation/topics/ovn-news-2.8.rst \
>         Documentation/topics/ovsdb-replication.rst \
>         Documentation/topics/porting.rst \
> -       Documentation/topics/role-based-access-control.rst \
>         Documentation/topics/tracing.rst \
>         Documentation/topics/windows.rst \
>         Documentation/howto/index.rst \
> -       Documentation/howto/docker.rst \
>         Documentation/howto/dpdk.rst \
> -       Documentation/howto/firewalld.rst \
>         Documentation/howto/ipsec.rst \
>         Documentation/howto/kvm.rst \
>         Documentation/howto/libvirt.rst \
>         Documentation/howto/selinux.rst \
>         Documentation/howto/ssl.rst \
>         Documentation/howto/lisp.rst \
> -       Documentation/howto/openstack-containers.rst \
>         Documentation/howto/qos.png \
>         Documentation/howto/qos.rst \
>         Documentation/howto/sflow.png \
> @@ -94,7 +83,6 @@ DOC_SOURCE = \
>         Documentation/faq/general.rst \
>         Documentation/faq/issues.rst \
>         Documentation/faq/openflow.rst \
> -       Documentation/faq/ovn.rst \
>         Documentation/faq/qos.rst \
>         Documentation/faq/releases.rst \
>         Documentation/faq/terminology.rst \
> diff --git a/Documentation/faq/index.rst b/Documentation/faq/index.rst
> index ad3cc2b6f..334b828b2 100644
> --- a/Documentation/faq/index.rst
> +++ b/Documentation/faq/index.rst
> @@ -41,4 +41,3 @@ Open vSwitch FAQ
>     terminology
>     vlan
>     vxlan
> -   ovn
> diff --git a/Documentation/faq/ovn.rst b/Documentation/faq/ovn.rst
> deleted file mode 100644
> index 4d96b4aa5..000000000
> --- a/Documentation/faq/ovn.rst
> +++ /dev/null
> @@ -1,90 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -===
> -OVN
> -===
> -
> -Q: Why does OVN use STT and Geneve instead of VLANs or VXLAN (or GRE)?
> -
> -    A: OVN implements a fairly sophisticated packet processing pipeline in
> -    "logical datapaths" that can implement switching or routing
> functionality.
> -    A logical datapath has an ingress pipeline and an egress pipeline,
> and each
> -    of these pipelines can include logic based on packet fields as well as
> -    packet metadata such as the logical ingress and egress ports (the
> latter
> -    only in the egress pipeline).
> -
> -    The processing for a logical datapath can be split across
> hypervisors.  In
> -    particular, when a logical ingress pipeline executes an "output"
> action,
> -    OVN passes the packet to the egress pipeline on the hypervisor (or,
> in the
> -    case of output to a logical multicast group, hypervisors) on which the
> -    logical egress port is located.  If this hypervisor is not the same
> as the
> -    ingress hypervisor, then the packet has to be transmitted across a
> physical
> -    network.
> -
> -    This situation is where tunneling comes in.  To send the packet to
> another
> -    hypervisor, OVN encapsulates it with a tunnel protocol and sends the
> -    encapsulated packet across the physical network.  When the remote
> -    hypervisor receives the tunnel packet, it decapsulates it and passes
> it
> -    through the logical egress pipeline.  To do so, it also needs the
> metadata,
> -    that is, the logical ingress and egress ports.
> -
> -    Thus, to implement OVN logical packet processing, at least the
> following
> -    metadata must pass across the physical network:
> -
> -    * Logical datapath ID, a 24-bit identifier.  In Geneve, OVN uses the
> VNI to
> -      hold the logical datapath ID; in STT, OVN uses 24 bits of STT's
> 64-bit
> -      context ID.
> -
> -    * Logical ingress port, a 15-bit identifier.  In Geneve, OVN uses an
> option
> -      to hold the logical ingress port; in STT, 15 bits of the context ID.
> -
> -    * Logical egress port, a 16-bit identifier.  In Geneve, OVN uses an
> option
> -      to hold the logical egress port; in STT, 16 bits of the context ID.
> -
> -    See ``ovn-architecture(7)``, under "Tunnel Encapsulations", for
> details.
> -
> -    Together, these metadata require 24 + 15 + 16 = 55 bits.  GRE
> provides 32
> -    bits, VXLAN provides 24, and VLAN only provides 12.  Most notably, if
> -    logical egress pipelines do not match on the logical ingress port,
> thereby
> -    restricting the class of ACLs available to users, then this
> eliminates 15
> -    bits, bringing the requirement down to 40 bits.  At this point, one
> can
> -    choose to limit the size of the OVN logical network in various ways,
> e.g.:
> -
> -    * 16 bits of logical datapaths + 16 bits of logical egress ports.
> This
> -      combination fits within a 32-bit GRE tunnel key.
> -
> -    * 12 bits of logical datapaths + 12 bits of logical egress ports.
> This
> -      combination fits within a 24-bit VXLAN VNI.
> -
> -    * It's difficult to identify an acceptable compromise for a VLAN-based
> -      deployment.
> -
> -    These compromises wouldn't suit every site, since some deployments
> -    may need to allocate more bits to the datapath or egress port
> -    identifiers.
> -
> -    As a side note, OVN does support VXLAN for use with ASIC-based top of
> rack
> -    switches, using ``ovn-controller-vtep(8)`` and the OVSDB VTEP schema
> -    described in ``vtep(5)``, but this limits the features available from
> OVN
> -    to the subset available from the VTEP schema.
> diff --git a/Documentation/howto/docker.rst
> b/Documentation/howto/docker.rst
> deleted file mode 100644
> index a68b02fdb..000000000
> --- a/Documentation/howto/docker.rst
> +++ /dev/null
> @@ -1,326 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -===================================
> -Open Virtual Networking With Docker
> -===================================
> -
> -This document describes how to use Open Virtual Networking with Docker
> 1.9.0
> -or later.
> -
> -.. important::
> -
> -  Requires Docker version 1.9.0 or later. Only Docker 1.9.0+ comes with
> support
> -  for multi-host networking. Consult www.docker.com for instructions on
> how to
> -  install Docker.
> -
> -.. note::
> -
> -  You must build and install Open vSwitch before proceeding with the below
> -  guide. Refer to :doc:`/intro/install/index` for more information.
> -
> -Setup
> ------
> -
> -For multi-host networking with OVN and Docker, Docker has to be started
> with a
> -distributed key-value store. For example, if you decide to use consul as
> your
> -distributed key-value store and your host IP address is ``$HOST_IP``,
> start
> -your Docker daemon with::
> -
> -    $ docker daemon --cluster-store=consul://127.0.0.1:8500 \
> -        --cluster-advertise=$HOST_IP:0
> -
> -OVN provides network virtualization to containers. OVN's integration with
> -Docker currently works in two modes - the "underlay" mode or the "overlay"
> -mode.
> -
> -In the "underlay" mode, OVN requires a OpenStack setup to provide
> container
> -networking. In this mode, one can create logical networks and can have
> -containers running inside VMs, standalone VMs (without having any
> containers
> -running inside them) and physical machines connected to the same logical
> -network. This is a multi-tenant, multi-host solution.
> -
> -In the "overlay" mode, OVN can create a logical network amongst containers
> -running on multiple hosts. This is a single-tenant (extendable to
> multi-tenants
> -depending on the security characteristics of the workloads), multi-host
> -solution. In this mode, you do not need a pre-created OpenStack setup.
> -
> -For both the modes to work, a user has to install and start Open vSwitch
> in
> -each VM/host that they plan to run their containers on.
> -
> -.. _docker-overlay:
> -
> -The "overlay" mode
> -------------------
> -
> -.. note::
> -
> -  OVN in "overlay" mode needs a minimum Open vSwitch version of 2.5.
> -
> -1. Start the central components.
> -
> -  OVN architecture has a central component which stores your networking
> intent
> -  in a database. On one of your machines, with an IP Address of
> -  ``$CENTRAL_IP``, where you have installed and started Open vSwitch, you
> will
> -  need to start some central components.
> -
> -  Start ovn-northd daemon. This daemon translates networking intent from
> Docker
> -  stored in the OVN\_Northbound database to logical flows in
> ``OVN_Southbound``
> -  database. For example::
> -
> -      $ /usr/share/openvswitch/scripts/ovn-ctl start_northd
> -
> -  With Open vSwitch version of 2.7 or greater, you need to run the
> following
> -  additional commands (Please read the manpages of ovn-nb for more control
> -  on the types of connection allowed.) ::
> -
> -      $ ovn-nbctl set-connection ptcp:6641
> -      $ ovn-sbctl set-connection ptcp:6642
> -
> -2. One time setup
> -
> -   On each host, where you plan to spawn your containers, you will need
> to run
> -   the below command once. You may need to run it again if your OVS
> database
> -   gets cleared. It is harmless to run it again in any case::
> -
> -       $ ovs-vsctl set Open_vSwitch . \
> -           external_ids:ovn-remote="tcp:$CENTRAL_IP:6642" \
> -           external_ids:ovn-nb="tcp:$CENTRAL_IP:6641" \
> -           external_ids:ovn-encap-ip=$LOCAL_IP \
> -           external_ids:ovn-encap-type="$ENCAP_TYPE"
> -
> -   where:
> -
> -   ``$LOCAL_IP``
> -     is the IP address via which other hosts can reach this host.  This
> acts as
> -     your local tunnel endpoint.
> -
> -   ``$ENCAP_TYPE``
> -     is the type of tunnel that you would like to use for overlay
> networking.
> -     The options are ``geneve`` or ``stt``. Your kernel must have support
> for
> -     your chosen ``$ENCAP_TYPE``. Both ``geneve`` and ``stt`` are part of
> the
> -     Open vSwitch kernel module that is compiled from this repo. If you
> use the
> -     Open vSwitch kernel module from upstream Linux, you will need a
> minimum
> -     kernel version of 3.18 for ``geneve``. There is no ``stt`` support in
> -     upstream Linux. You can verify whether you have the support in your
> kernel
> -     as follows::
> -
> -         $ lsmod | grep $ENCAP_TYPE
> -
> -   In addition, each Open vSwitch instance in an OVN deployment needs a
> unique,
> -   persistent identifier, called the ``system-id``.  If you install OVS
> from
> -   distribution packaging for Open vSwitch (e.g. .deb or .rpm packages),
> or if
> -   you use the ovs-ctl utility included with Open vSwitch, it
> automatically
> -   configures a system-id.  If you start Open vSwitch manually, you
> should set
> -   one up yourself. For example::
> -
> -       $ id_file=/etc/openvswitch/system-id.conf
> -       $ test -e $id_file || uuidgen > $id_file
> -       $ ovs-vsctl set Open_vSwitch . external_ids:system-id=$(cat
> $id_file)
> -
> -3. Start the ``ovn-controller``.
> -
> -   You need to run the below command on every boot::
> -
> -       $ /usr/share/openvswitch/scripts/ovn-ctl start_controller
> -
> -4. Start the Open vSwitch network driver.
> -
> -   By default Docker uses Linux bridge for networking. But it has support
> for
> -   external drivers. To use Open vSwitch instead of the Linux bridge, you
> will
> -   need to start the Open vSwitch driver.
> -
> -   The Open vSwitch driver uses the Python's flask module to listen to
> Docker's
> -   networking api calls. So, if your host does not have Python's flask
> module,
> -   install it::
> -
> -       $ sudo pip install Flask
> -
> -   Start the Open vSwitch driver on every host where you plan to create
> your
> -   containers. Refer to the note on ``$OVS_PYTHON_LIBS_PATH`` that is
> used below
> -   at the end of this document::
> -
> -       $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-overlay-driver
> --detach
> -
> -   .. note::
> -
> -     The ``$OVS_PYTHON_LIBS_PATH`` variable should point to the directory
> where
> -     Open vSwitch Python modules are installed. If you installed Open
> vSwitch
> -     Python modules via the Debian package of ``python-openvswitch`` or
> via pip
> -     by running ``pip install ovs``, you do not need to specify the PATH.
> If
> -     you installed it by following the instructions in
> -     :doc:`/intro/install/general`, then you should specify the PATH. In
> this
> -     case, the PATH depends on the options passed to ``./configure``. It
> is
> -     usually either ``/usr/share/openvswitch/python`` or
> -     ``/usr/local/share/openvswitch/python``
> -
> -Docker has inbuilt primitives that closely match OVN's logical switches
> and
> -logical port concepts. Consult Docker's documentation for all the possible
> -commands. Here are some examples.
> -
> -Create a logical switch
> -~~~~~~~~~~~~~~~~~~~~~~~
> -
> -To create a logical switch with name 'foo', on subnet '192.168.1.0/24',
> run::
> -
> -    $ NID=`docker network create -d openvswitch --subnet=192.168.1.0/24
> foo`
> -
> -List all logical switches
> -~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    $ docker network ls
> -
> -You can also look at this logical switch in OVN's northbound database by
> -running the following command::
> -
> -    $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 ls-list
> -
> -Delete a logical switch
> -~~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    $ docker network rm bar
> -
> -
> -Create a logical port
> -~~~~~~~~~~~~~~~~~~~~~
> -
> -Docker creates your logical port and attaches it to the logical network
> in a
> -single step. For example, to attach a logical port to network ``foo``
> inside
> -container busybox, run::
> -
> -    $ docker run -itd --net=foo --name=busybox busybox
> -
> -List all logical ports
> -~~~~~~~~~~~~~~~~~~~~~~
> -
> -Docker does not currently have a CLI command to list all logical ports
> but you
> -can look at them in the OVN database by running::
> -
> -    $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 lsp-list $NID
> -
> -Create and attach a logical port to a running container
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    $ docker network create -d openvswitch --subnet=192.168.2.0/24 bar
> -    $ docker network connect bar busybox
> -
> -Detach and delete a logical port from a running container
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -You can delete your logical port and detach it from a running container
> -by running:
> -
> -::
> -
> -    $ docker network disconnect bar busybox
> -
> -.. _docker-underlay:
> -
> -The "underlay" mode
> --------------------
> -
> -.. note::
> -
> -  This mode requires that you have a OpenStack setup pre-installed with
> -  OVN providing the underlay networking.
> -
> -1. One time setup
> -
> -   A OpenStack tenant creates a VM with a single network interface (or
> multiple)
> -   that belongs to management logical networks. The tenant needs to fetch
> the
> -   port-id associated with the interface via which he plans to send the
> container
> -   traffic inside the spawned VM. This can be obtained by running the
> below
> -   command to fetch the 'id' associated with the VM::
> -
> -       $ nova list
> -
> -   and then by running::
> -
> -       $ neutron port-list --device_id=$id
> -
> -   Inside the VM, download the OpenStack RC file that contains the tenant
> -   information (henceforth referred to as ``openrc.sh``). Edit the file
> and add the
> -   previously obtained port-id information to the file by appending the
> following
> -   line::
> -
> -       $ export OS_VIF_ID=$port_id
> -
> -   After this edit, the file will look something like::
> -
> -       #!/bin/bash
> -       export OS_AUTH_URL=http://10.33.75.122:5000/v2.0
> -       export OS_TENANT_ID=fab106b215d943c3bad519492278443d
> -       export OS_TENANT_NAME="demo"
> -       export OS_USERNAME="demo"
> -       export OS_VIF_ID=e798c371-85f4-4f2d-ad65-d09dd1d3c1c9
> -
> -2. Create the Open vSwitch bridge
> -
> -   If your VM has one ethernet interface (e.g.: 'eth0'), you will need to
> add
> -   that device as a port to an Open vSwitch bridge 'breth0' and move its
> IP
> -   address and route related information to that bridge. (If it has
> multiple
> -   network interfaces, you will need to create and attach an Open vSwitch
> -   bridge for the interface via which you plan to send your container
> -   traffic.)
> -
> -   If you use DHCP to obtain an IP address, then you should kill the DHCP
> -   client that was listening on the physical Ethernet interface (e.g.
> eth0) and
> -   start one listening on the Open vSwitch bridge (e.g. breth0).
> -
> -   Depending on your VM, you can make the above step persistent across
> reboots.
> -   For example, if your VM is Debian/Ubuntu-based, read
> -   `openvswitch-switch.README.Debian` found in `debian` folder. If your
> VM is
> -   RHEL-based, refer to :doc:`/intro/install/rhel`.
> -
> -3. Start the Open vSwitch network driver
> -
> -   The Open vSwitch driver uses the Python's flask module to listen to
> Docker's
> -   networking api calls. The driver also uses OpenStack's
> -   ``python-neutronclient`` libraries. If your host does not have Python's
> -   ``flask`` module or ``python-neutronclient`` you must install them. For
> -   example::
> -
> -       $ pip install python-neutronclient
> -       $ pip install Flask
> -
> -   Once installed, source the ``openrc`` file::
> -
> -       $ . ./openrc.sh
> -
> -   Start the network driver and provide your OpenStack tenant password
> when
> -   prompted::
> -
> -       $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-underlay-driver \
> -           --bridge breth0 --detach
> -
> -From here-on you can use the same Docker commands as described in
> -`docker-overlay`_.
> -
> -Refer to the ovs-architecture man pages (``man ovn-architecture``) to
> -understand OVN's architecture in detail.
> diff --git a/Documentation/howto/firewalld.rst
> b/Documentation/howto/firewalld.rst
> deleted file mode 100644
> index 0dc455ea8..000000000
> --- a/Documentation/howto/firewalld.rst
> +++ /dev/null
> @@ -1,107 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -===================================
> -Open Virtual Network With firewalld
> -===================================
> -
> -firewalld is a service that allows for easy administration of firewalls.
> OVN
> -ships with a set of service files that can be used with firewalld to allow
> -for remote connections to the northbound and southbound databases.
> -
> -This guide will describe how you can use these files with your existing
> -firewalld setup. Setup and administration of firewalld is outside the
> scope
> -of this document.
> -
> -Installation
> -------------
> -
> -If you have installed OVN from an RPM, then the service files for
> firewalld
> -will automatically be installed in ``/usr/lib/firewalld/services``.
> -Installation from RPM includes installation from the yum or dnf package
> -managers.
> -
> -If you have installed OVN from source, then from the top level source
> -directory, issue the following commands to copy the firewalld service
> files:
> -
> -::
> -
> -    $ cp rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml
> \
> -    /etc/firewalld/services/
> -    $ cp rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml \
> -    /etc/firewalld/services/
> -
> -
> -Activation
> -----------
> -
> -Assuming you are already running firewalld, you can issue the following
> -commands to enable the OVN services.
> -
> -On the central server (the one running ``ovn-northd``), issue the
> following::
> -
> -$ firewall-cmd --zone=public --add-service=ovn-central-firewall-service
> -
> -This will open TCP ports 6641 and 6642, allowing for remote connections
> to the
> -northbound and southbound databases.
> -
> -On the OVN hosts (the ones running ``ovn-controller``), issue the
> following::
> -
> -$ firewall-cmd --zone=public --add-service=ovn-host-firewall-service
> -
> -This will open UDP port 6081, allowing for geneve traffic to flow between
> the
> -controllers.
> -
> -Variations
> -----------
> -
> -When installing the XML service files, you have the choice of copying
> them to
> -``/etc/firewalld/services`` or ``/usr/lib/firewalld/services``. The
> former is
> -recommend since the latter can be overwritten if firewalld is upgraded.
> -
> -The above commands assumed your underlay network interfaces are in the
> -"public" firewalld zone. If your underlay network interfaces are in a
> separate
> -zone, then adjust the above commands accordingly.
> -
> -The ``--permanent`` option may be passed to the above firewall-cmd
> invocations
> -in order for the services to be permanently added to the firewalld
> -configuration. This way it is not necessary to re-issue the commands each
> -time the firewalld service restarts.
> -
> -The ovn-host-firewall-service only opens port 6081. This is because the
> -default protocol for OVN tunnels is geneve. If you are using a different
> -encapsulation protocol, you will need to modify the XML service file to
> open
> -the appropriate port(s). For VXLAN, open port 4789. For STT, open port
> 7471.
> -
> -Recommendations
> ----------------
> -
> -The firewalld service files included with the OVS repo are meant as a
> -convenience for firewalld users. All that the service files do is to open
> -the common ports used by OVN. No additional security is provided. To
> ensure a
> -more secure environment, it is a good idea to do the following
> -
> -* Use tools such as iptables or nftables to restrict access to known
> hosts.
> -* Use SSL for all remote connections to OVN databases.
> -* Use role-based access control for connections to the OVN southbound
> -  database.
> diff --git a/Documentation/howto/index.rst b/Documentation/howto/index.rst
> index 9a3487be3..60fb8a717 100644
> --- a/Documentation/howto/index.rst
> +++ b/Documentation/howto/index.rst
> @@ -50,12 +50,3 @@ OVS
>     sflow
>     dpdk
>
> -OVN
> ----
> -
> -.. toctree::
> -   :maxdepth: 1
> -
> -   docker
> -   openstack-containers
> -   firewalld
> diff --git a/Documentation/howto/openstack-containers.rst
> b/Documentation/howto/openstack-containers.rst
> deleted file mode 100644
> index 692fe25e5..000000000
> --- a/Documentation/howto/openstack-containers.rst
> +++ /dev/null
> @@ -1,135 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -================================================
> -Integration of Containers with OVN and OpenStack
> -================================================
> -
> -Isolation between containers is weaker than isolation between VMs, so some
> -environments deploy containers for different tenants in separate VMs as an
> -additional security measure.  This document describes creation of
> containers
> -inside VMs and how they can be made part of the logical networks
> securely.  The
> -created logical network can include VMs, containers and physical machines
> as
> -endpoints.  To better understand the proposed integration of containers
> with
> -OVN and OpenStack, this document describes the end to end workflow with an
> -example.
> -
> -* A OpenStack tenant creates a VM (say VM-A) with a single network
> interface
> -  that belongs to a management logical network.  The VM is meant to host
> -  containers.  OpenStack Nova chooses the hypervisor on which VM-A is
> created.
> -
> -* A Neutron port may have been created in advance and passed in to Nova
> with
> -  the request to create a new VM.  If not, Nova will issue a request to
> Neutron
> -  to create a new port.  The ID of the logical port from Neutron will
> also be
> -  used as the vif-id for the virtual network interface (VIF) of VM-A.
> -
> -* When VM-A is created on a hypervisor, its VIF gets added to the Open
> vSwitch
> -  integration bridge.  This creates a row in the Interface table of the
> -  ``Open_vSwitch`` database.  As explained in the :doc:`integration guide
> -  </topics/integration>`, the vif-id associated with the VM network
> interface
> -  gets added in the ``external_ids:iface-id`` column of the newly created
> row
> -  in the Interface table.
> -
> -* Since VM-A belongs to a logical network, it gets an IP address.  This IP
> -  address is used to spawn containers (either manually or through
> container
> -  orchestration systems) inside that VM and to monitor the health of the
> -  created containers.
> -
> -* The vif-id associated with the VM's network interface can be obtained by
> -  making a call to Neutron using tenant credentials.
> -
> -* This flow assumes a component called a "container network plugin".  If
> you
> -  take Docker as an example for containers, you could envision the plugin
> to be
> -  either a wrapper around Docker or a feature of Docker itself that
> understands
> -  how to perform part of this workflow to get a container connected to a
> -  logical network managed by Neutron.  The rest of the flow refers to this
> -  logical component that does not yet exist as the "container network
> plugin".
> -
> -* All the calls to Neutron will need tenant credentials.  These calls can
> -  either be made from inside the tenant VM as part of a container network
> -  plugin or from outside the tenant VM (if the tenant is not comfortable
> using
> -  temporary Keystone tokens from inside the tenant VMs).  For simplicity,
> this
> -  document explains the work flow using the former method.
> -
> -* The container hosting VM will need Open vSwitch installed in it.  The
> only
> -  work for Open vSwitch inside the VM is to tag network traffic coming
> from
> -  containers.
> -
> -* When a container needs to be created inside the VM with a container
> network
> -  interface that is expected to be attached to a particular logical
> switch, the
> -  network plugin in that VM chooses any unused VLAN (This VLAN tag only
> needs
> -  to be unique inside that VM.  This limits the number of container
> interfaces
> -  to 4096 inside a single VM).  This VLAN tag is stripped out in the
> hypervisor
> -  by OVN and is only useful as a context (or metadata) for OVN.
> -
> -* The container network plugin then makes a call to Neutron to create a
> logical
> -  port.  In addition to all the inputs that a call to create a port in
> Neutron
> -  that are currently needed, it sends the vif-id and the VLAN tag as
> inputs.
> -
> -* Neutron in turn will verify that the vif-id belongs to the tenant in
> question
> -  and then uses the OVN specific plugin to create a new row in the
> -  Logical_Switch_Port table of the OVN Northbound Database.  Neutron
> responds
> -  back with an IP address and MAC address for that network interface.  So
> -  Neutron becomes the IPAM system and provides unique IP and MAC addresses
> -  across VMs and containers in the same logical network.
> -
> -* The Neutron API call above to create a logical port for the container
> could
> -  add a relatively significant amount of time to container creation.
> However,
> -  an optimization is possible here.  Logical ports could be created in
> advance
> -  and reused by the container system doing container orchestration.
> Additional
> -  Neutron API calls would only be needed if the port needs to be attached
> to a
> -  different logical network.
> -
> -* When a container is eventually deleted, the network plugin in that VM
> may
> -  make a call to Neutron to delete that port.  Neutron in turn will
> delete the
> -  entry in the ``Logical_Switch_Port`` table of the OVN Northbound
> Database.
> -
> -As an example, consider Docker containers.  Since Docker currently does
> not
> -have a network plugin feature, this example uses a hypothetical wrapper
> around
> -Docker to make calls to Neutron.
> -
> -* Create a Logical switch::
> -
> -      $ ovn-docker --cred=cca86bd13a564ac2a63ddf14bf45d37f create network
> LS1
> -
> -  The above command will make a call to Neutron with the credentials to
> create
> -  a logical switch.  The above is optional if the logical switch has
> already
> -  been created from outside the VM.
> -
> -* List networks available to the tenant::
> -
> -      $ ovn-docker --cred=cca86bd13a564ac2a63ddf14bf45d37f list networks
> -
> -* Create a container and attach a interface to the previously created
> switch as
> -  a logical port::
> -
> -      $ ovn-docker --cred=cca86bd13a564ac2a63ddf14bf45d37f
> --vif-id=$VIF_ID \
> -          --network=LS1 run -d --net=none ubuntu:14.04 /bin/sh -c \
> -          "while true; do echo hello world; sleep 1; done"
> -
> -  The above command will make a call to Neutron with all the inputs it
> -  currently needs to create a logical port.  In addition, it passes the
> $VIF_ID
> -  and a unused VLAN.  Neutron will add that information in OVN and return
> back
> -  a MAC address and IP address for that interface.  ovn-docker will then
> create
> -  a veth pair, insert one end inside the container as 'eth0' and the
> other end
> -  as a port of a local OVS bridge as an access port of the chosen VLAN.
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index bace34dbf..f18f8df1c 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -33,22 +33,20 @@ How the Documentation is Organised
>  The Open vSwitch documentation is organised into multiple sections:
>
>  - :doc:`Installation guides </intro/install/index>` guide you through
> -  installing Open vSwitch (OVS) and Open Virtual Network (OVN) on a
> variety of
> -  different platforms
> +  installing Open vSwitch (OVS) on a variety of different platforms
>  - :doc:`Tutorials </tutorials/index>` take you through a series of steps
> to
> -  configure OVS and OVN in sandboxed environments
> -- :doc:`Topic guides </topics/index>` provide a high level overview of
> OVS and
> -  OVN internals and operation
> -- :doc:`How-to guides </howto/index>` are recipes or use-cases for OVS
> and OVN.
> +  configure OVS in sandboxed environments
> +- :doc:`Topic guides </topics/index>` provide a high level overview of OVS
> +  internals and operation
> +- :doc:`How-to guides </howto/index>` are recipes or use-cases for OVS.
>    They are more advanced than the tutorials.
>  - :doc:`Frequently Asked Questions </faq/index>` provide general insight
> into
> -  a variety of topics related to configuration and operation of OVS and
> OVN.
> +  a variety of topics related to configuration and operation of OVS.
>
>  First Steps
>  -----------
>
> -Getting started with Open vSwitch (OVS) or Open Virtual Network (OVN) for
> Open
> -vSwitch? Start here.
> +Getting started with Open vSwitch (OVS)? Start here.
>
>  - **Overview:** :doc:`intro/what-is-ovs` |
>    :doc:`intro/why-ovs`
> @@ -64,12 +62,8 @@ vSwitch? Start here.
>
>  - **Tutorials:** :doc:`tutorials/faucet` |
>    :doc:`tutorials/ovs-advanced` |
> -  :doc:`tutorials/ovn-sandbox` |
> -  :doc:`tutorials/ovn-openstack` |
>    :doc:`tutorials/ovs-conntrack` |
>    :doc:`tutorials/ipsec` |
> -  :doc:`tutorials/ovn-ipsec` |
> -  :doc:`tutorials/ovn-rbac`
>
>  Deeper Dive
>  -----------
> diff --git a/Documentation/intro/install/fedora.rst
> b/Documentation/intro/install/fedora.rst
> index 4e1a99766..f11d05a01 100644
> --- a/Documentation/intro/install/fedora.rst
> +++ b/Documentation/intro/install/fedora.rst
> @@ -119,16 +119,6 @@ tests.  This can take several minutes.
>
>      $ make rpm-fedora RPMBUILD_OPT="--with check"
>
> -To build OVN RPMs, execute the following from the directory in which
> -`./configure` was executed:
> -
> -::
> -
> -    $ make rpm-fedora-ovn
> -
> -This will create the RPMs `ovn`, `ovn-common`, `ovn-central`, `ovn-host`,
> -`ovn-docker` and `ovn-vtep`.
> -
>
>  Kernel OVS Tree Datapath RPM
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> @@ -165,8 +155,6 @@ In most cases only the `openvswitch` RPM will need to
> be installed. The
>  `openvswitch-debuginfo` RPMs are optional unless required for a specific
>  purpose.
>
> -The `ovn-*` packages are only needed when using OVN.
> -
>  Refer to the `RHEL README`__ for additional usage and configuration
>  information.
>
> diff --git a/Documentation/intro/install/index.rst
> b/Documentation/intro/install/index.rst
> index c27a9c9d1..586ced95f 100644
> --- a/Documentation/intro/install/index.rst
> +++ b/Documentation/intro/install/index.rst
> @@ -62,14 +62,6 @@ provided below.
>     fedora
>     rhel
>
> -Upgrades
> ---------
> -
> -.. toctree::
> -   :maxdepth: 2
> -
> -   ovn-upgrades
> -
>  Others
>  ------
>
> diff --git a/Documentation/intro/install/ovn-upgrades.rst
> b/Documentation/intro/install/ovn-upgrades.rst
> deleted file mode 100644
> index 3e6cd984e..000000000
> --- a/Documentation/intro/install/ovn-upgrades.rst
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -============
> -OVN Upgrades
> -============
> -
> -Since OVN is a distributed system, special consideration must be given to
> -the process used to upgrade OVN across a deployment.  This document
> discusses
> -the recommended upgrade process.
> -
> -Release Notes
> --------------
> -
> -You should always check the OVS and OVN release notes (NEWS file) for any
> -release specific notes on upgrades.
> -
> -OVS
> ----
> -
> -OVN depends on and is included with OVS.  It's expected that OVS and OVN
> are
> -upgraded together, partly for convenience.  OVN is included in OVS
> releases
> -so it's easiest to upgrade them together.  OVN may also make use of new
> -features of OVS only available in that release.
> -
> -Upgrade ovn-controller
> -----------------------
> -
> -You should start by upgrading ovn-controller on each host it's running on.
> -First, you upgrade the OVS and OVN packages.  Then, restart the
> -ovn-controller service.  You can restart with ovn-ctl::
> -
> -    $ sudo /usr/share/openvswitch/scripts/ovn-ctl restart_controller
> -
> -or with systemd::
> -
> -    $ sudo systemd restart ovn-controller
> -
> -Upgrade OVN Databases and ovn-northd
> -------------------------------------
> -
> -The OVN databases and ovn-northd should be upgraded next.  Since
> ovn-controller
> -has already been upgraded, it will be ready to operate on any new
> functionality
> -specified by the database or logical flows created by ovn-northd.
> -
> -Upgrading the OVN packages installs everything needed for an upgrade.
> The only
> -step required after upgrading the packages is to restart ovn-northd, which
> -automatically restarts the databases and upgrades the database schema, as
> well.
> -
> -You may perform this restart using the ovn-ctl script::
> -
> -    $ sudo /usr/share/openvswitch/scripts/ovn-ctl restart_northd
> -
> -or if you're using a Linux distribution with systemd::
> -
> -    $ sudo systemctl restart ovn-northd
> -
> -Schema Change
> -^^^^^^^^^^^^^
> -
> -During database upgrading, if there is schema change, the DB file will be
> -converted to the new schema automatically, if the schema change is
> backward
> -compatible.  OVN tries the best to keep the DB schemas backward
> compatible.
> -
> -However, there can be situations that an incompatible change is
> reasonble.  An
> -example of such case is to add constraints in the table to ensure
> correctness.
> -If there were already data that violates the new constraints got added
> somehow,
> -it will result in DB upgrade failures.  In this case, user should manually
> -correct data using ovn-nbctl (for north-bound DB) or ovn-sbctl (for south-
> -bound DB), and then upgrade again following previous steps.  Below is a
> list
> -of known impactible schema changes and how to fix when error encountered.
> -
> -#. Release 2.11: index [type, ip] added for Encap table of south-bound DB
> to
> -   prevent duplicated IPs being used for same tunnel type.  If there are
> -   duplicated data added already (e.g. due to improper chassis
> management),
> -   a convenient way to fix is to find the chassis that is using the IP
> -   with command::
> -
> -    $ ovn-sbctl show
> -
> -   Then delete the chassis with command::
> -
> -    $ ovn-sbctl chassis-del <chassis>
> -
> -
> -Upgrading OVN Integration
> --------------------------
> -
> -Lastly, you may also want to upgrade integration with OVN that you may be
> -using.  For example, this could be the OpenStack Neutron driver or
> -ovn-kubernetes.
> -
> -OVN's northbound database schema is a backwards compatible interface, so
> -you should be able to safely complete an OVN upgrade before upgrading
> -any integration in use.
> diff --git a/Documentation/ref/ovs-sim.1.rst
> b/Documentation/ref/ovs-sim.1.rst
> index 4382598e1..f59cd7af7 100644
> --- a/Documentation/ref/ovs-sim.1.rst
> +++ b/Documentation/ref/ovs-sim.1.rst
> @@ -142,103 +142,3 @@ with ``main`` directly.
>      must already have been created by a previous invocation of
>      ``net_add``. The default sandbox must not be ``main``.
>
> -OVN Commands
> -------------
> -
> -These commands interact with OVN, the Open Virtual Network.
> -
> -``ovn_start`` [*options*]
> -    Creates and initializes the central OVN databases (both
> -    ``ovn-sb(5)`` and ``ovn-nb(5)``) and starts an instance of
> -    ``ovsdb-server`` for each one.  Also starts an instance of
> -    ``ovn-northd``.
> -
> -    The following options are available:
> -
> -       ``--nbdb-model`` *model*
> -           Uses the given database model for the northbound database.
> -           The *model* may be ``standalone`` (the default), ``backup``,
> -           or ``clustered``.
> -
> -       ``--nbdb-servers`` *n*
> -           For a clustered northbound database, the number of servers in
> -           the cluster.  The default is 3.
> -
> -       ``--sbdb-model`` *model*
> -           Uses the given database model for the southbound database.
> -           The *model* may be ``standalone`` (the default), ``backup``,
> -           or ``clustered``.
> -
> -       ``--sbdb-servers`` *n*
> -           For a clustered southbound database, the number of servers in
> -           the cluster.  The default is 3.
> -
> -``ovn_attach`` *network* *bridge* *ip* [*masklen*]
> -    First, this command attaches bridge to interconnection network
> -    network, just like ``net_attach`` *network* *bridge*.  Second, it
> -    configures (simulated) IP address *ip* (with network mask length
> -    *masklen*, which defaults to 24) on *bridge*. Finally, it
> -    configures the Open vSwitch database to work with OVN and starts
> -    ``ovn-controller``.
> -
> -Examples
> -========
> -
> -The following creates a pair of Open vSwitch instances ``hv0`` and
> -``hv1``, adds a port named ``vif0`` or ``vif1``, respectively, to each
> -one, and then connects the two through an interconnection network
> -``n1``::
> -
> -    net_add n1
> -    for i in 0 1; do
> -        sim_add hv$i
> -        as hv$i ovs-vsctl add-br br0 -- add-port br0 vif$i
> -        as hv$i net_attach n1 br0
> -    done
> -
> -Here’s an extended version that also starts OVN::
> -
> -    ovn_start
> -    ovn-nbctl ls-add lsw0
> -    net_add n1
> -    for i in 0 1; do
> -        sim_add hv$i
> -        as hv$i
> -        ovs-vsctl add-br br-phys
> -        ovn_attach n1 br-phys 192.168.0.`expr $i + 1`
> -        ovs-vsctl add-port br-int vif$i -- set Interface vif$i
> external-ids:iface-id=lp$i
> -        ovn-nbctl lsp-add lsw0 lp$i
> -        ovn-nbctl lsp-set-addresses lp$i f0:00:00:00:00:0$i
> -    done
> -
> -Here’s a primitive OVN "scale test" (adjust the scale by changing
> -``n`` in the first line::
> -
> -    n=200; export n
> -    ovn_start --sbdb-model=clustered
> -    net_add n1
> -    ovn-nbctl ls-add br0
> -    for i in `seq $n`; do
> -        (sim_add hv$i
> -        as hv$i
> -        ovs-vsctl add-br br-phys
> -        y=$(expr $i / 256)
> -        x=$(expr $i % 256)
> -        ovn_attach n1 br-phys 192.168.$y.$x
> -        ovs-vsctl add-port br-int vif$i -- set Interface vif$i
> external-ids:iface-id=lp$i) &
> -        case $i in
> -            *50|*00) echo $i; wait ;;
> -        esac
> -    done
> -    wait
> -    for i in `seq $n`; do
> -        yy=$(printf %02x $(expr $i / 256))
> -        xx=$(printf $02x $(expr $i % 256))
> -        ovn-nbctl lsp-add br0 lp$i
> -        ovn-nbctl lsp-set-addresses lp$i f0:00:00:00:$yy:$xx
> -    done
> -
> -When the scale test has finished initializing, you can watch the
> -logical ports come up with a command like this::
> -
> -    watch 'for i in `seq $n`; do if test `ovn-nbctl lsp-get-up lp$i` !=
> up; then echo $i; fi; done'
> diff --git a/Documentation/topics/high-availability.rst
> b/Documentation/topics/high-availability.rst
> deleted file mode 100644
> index a5cb76383..000000000
> --- a/Documentation/topics/high-availability.rst
> +++ /dev/null
> @@ -1,440 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -==================================
> -OVN Gateway High Availability Plan
> -==================================
> -
> -::
> -
> -    OVN Gateway
> -
> -         +---------------------------+
> -         |                           |
> -         |     External Network      |
> -         |                           |
> -         +-------------^-------------+
> -                       |
> -                       |
> -                 +-----------+
> -                 |           |
> -                 |  Gateway  |
> -                 |           |
> -                 +-----------+
> -                       ^
> -                       |
> -                       |
> -         +-------------v-------------+
> -         |                           |
> -         |    OVN Virtual Network    |
> -         |                           |
> -         +---------------------------+
> -
> -The OVN gateway is responsible for shuffling traffic between the tunneled
> -overlay network (governed by ovn-northd), and the legacy physical
> network.  In
> -a naive implementation, the gateway is a single x86 server, or hardware
> VTEP.
> -For most deployments, a single system has enough forwarding capacity to
> service
> -the entire virtualized network, however, it introduces a single point of
> -failure.  If this system dies, the entire OVN deployment becomes
> unavailable.
> -To mitigate this risk, an HA solution is critical -- by spreading
> -responsibility across multiple systems, no single server failure can take
> down
> -the network.
> -
> -An HA solution is both critical to the manageability of the system, and
> -extremely difficult to get right.  The purpose of this document, is to
> propose
> -a plan for OVN Gateway High Availability which takes into account our past
> -experience building similar systems.  It should be considered a fluid
> changing
> -proposal, not a set-in-stone decree.
> -
> -.. note::
> -    This document describes a range of options OVN could take to provide
> -    high availability for gateways.  The current implementation provides
> L3
> -    gateway high availability by the "Router Specific Active/Backup"
> -    approach described in this document.
> -
> -Basic Architecture
> -------------------
> -
> -In an OVN deployment, the set of hypervisors and network elements
> operating
> -under the guidance of ovn-northd are in what's called "logical space".
> These
> -servers use VXLAN, STT, or Geneve to communicate, oblivious to the
> details of
> -the underlying physical network.  When these systems need to communicate
> with
> -legacy networks, traffic must be routed through a Gateway which
> translates from
> -OVN controlled tunnel traffic, to raw physical network traffic.
> -
> -Since the gateway is typically the only system with a connection to the
> -physical network all traffic between logical space and the WAN must travel
> -through it.  This makes it a critical single point of failure -- if the
> gateway
> -dies, communication with the WAN ceases for all systems in logical space.
> -
> -To mitigate this risk, multiple gateways should be run in a "High
> Availability
> -Cluster" or "HA Cluster".  The HA cluster will be responsible for
> performing
> -the duties of a gateways,  while being able to recover gracefully from
> -individual member failures.
> -
> -::
> -
> -    OVN Gateway HA Cluster
> -
> -             +---------------------------+
> -             |                           |
> -             |     External Network      |
> -             |                           |
> -             +-------------^-------------+
> -                           |
> -                           |
> -    +----------------------v----------------------+
> -    |                                             |
> -    |          High Availability Cluster          |
> -    |                                             |
> -    | +-----------+  +-----------+  +-----------+ |
> -    | |           |  |           |  |           | |
> -    | |  Gateway  |  |  Gateway  |  |  Gateway  | |
> -    | |           |  |           |  |           | |
> -    | +-----------+  +-----------+  +-----------+ |
> -    +----------------------^----------------------+
> -                           |
> -                           |
> -             +-------------v-------------+
> -             |                           |
> -             |    OVN Virtual Network    |
> -             |                           |
> -             +---------------------------+
> -
> -L2 vs L3 High Availability
> -~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -In order to achieve this goal, there are two broad approaches one can
> take.
> -The HA cluster can appear to the network like a giant Layer 2 Ethernet
> Switch,
> -or like a giant IP Router. These approaches are called L2HA, and L3HA
> -respectively.  L2HA allows ethernet broadcast domains to extend into
> logical
> -space, a significant advantage, but this comes at a cost.  The need to
> avoid
> -transient L2 loops during failover significantly complicates their
> design.  On
> -the other hand, L3HA works for most use cases, is simpler, and fails more
> -gracefully.  For these reasons, it is suggested that OVN supports an L3HA
> -model, leaving L2HA for future work (or third party VTEP providers).  Both
> -models are discussed further below.
> -
> -L3HA
> -----
> -
> -In this section, we'll work through a basic simple L3HA implementation,
> on top
> -of which we'll gradually build more sophisticated features explaining
> their
> -motivations and implementations as we go.
> -
> -Naive active-backup
> -~~~~~~~~~~~~~~~~~~~
> -
> -Let's assume that there are a collection of logical routers which a
> tenant has
> -asked for, our task is to schedule these logical routers on one of N
> gateways,
> -and gracefully redistribute the routers on gateways which have failed.
> The
> -absolute simplest way to achieve this is what we'll call
> "naive-active-backup".
> -
> -::
> -
> -    Naive Active Backup HA Implementation
> -
> -    +----------------+   +----------------+
> -    | Leader         |   | Backup         |
> -    |                |   |                |
> -    |      A B C     |   |                |
> -    |                |   |                |
> -    +----+-+-+-+----++   +-+--------------+
> -         ^ ^ ^ ^    |      |
> -         | | | |    |      |
> -         | | | |  +-+------+---+
> -         + + + +  | ovn-northd |
> -         Traffic  +------------+
> -
> -In a naive active-backup, one of the Gateways is chosen (arbitrarily) as a
> -leader.  All logical routers (A, B, C in the figure), are scheduled on
> this
> -leader gateway and all traffic flows through it.  ovn-northd monitors this
> -gateway via OpenFlow echo requests (or some equivalent), and if the
> gateway
> -dies, it recreates the routers on one of the backups.
> -
> -This approach basically works in most cases and should likely be the
> starting
> -point for OVN -- it's strictly better than no HA solution and is a good
> -foundation for more sophisticated solutions.  That said, it's not without
> it's
> -limitations. Specifically, this approach doesn't coordinate with the
> physical
> -network to minimize disruption during failures, and it tightly couples
> failover
> -to ovn-northd (we'll discuss why this is bad in a bit), and wastes
> resources by
> -leaving backup gateways completely unutilized.
> -
> -Router Failover
> -+++++++++++++++
> -
> -When ovn-northd notices the leader has died and decides to migrate
> routers to a
> -backup gateway, the physical network has to be notified to direct traffic
> to
> -the new gateway.  Otherwise, traffic could be blackholed for longer than
> -necessary making failovers worse than they need to be.
> -
> -For now, let's assume that OVN requires all gateways to be on the same IP
> -subnet on the physical network.  If this isn't the case, gateways would
> need to
> -participate in routing protocols to orchestrate failovers, something
> which is
> -difficult and out of scope of this document.
> -
> -Since all gateways are on the same IP subnet, we simply need to worry
> about
> -updating the MAC learning tables of the Ethernet switches on that subnet.
> -Presumably, they all have entries for each logical router pointing to the
> old
> -leader.  If these entries aren't updated, all traffic will be sent to the
> (now
> -defunct) old leader, instead of the new one.
> -
> -In order to mitigate this issue, it's recommended that the new gateway
> sends a
> -Reverse ARP (RARP) onto the physical network for each logical router it
> now
> -controls.  A Reverse ARP is a benign protocol used by many hypervisors
> when
> -virtual machines migrate to update L2 forwarding tables.  In this case,
> the
> -ethernet source address of the RARP is that of the logical router it
> -corresponds to, and its destination is the broadcast address.  This
> causes the
> -RARP to travel to every L2 switch in the broadcast domain, updating
> forwarding
> -tables accordingly.  This strategy is recommended in all failover
> mechanisms
> -discussed in this document -- when a router newly boots on a new leader,
> it
> -should RARP its MAC address.
> -
> -Controller Independent Active-backup
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    Controller Independent Active-Backup Implementation
> -
> -    +----------------+   +----------------+
> -    | Leader         |   | Backup         |
> -    |                |   |                |
> -    |      A B C     |   |                |
> -    |                |   |                |
> -    +----------------+   +----------------+
> -         ^ ^ ^ ^
> -         | | | |
> -         | | | |
> -         + + + +
> -         Traffic
> -
> -The fundamental problem with naive active-backup, is it tightly couples
> the
> -failover solution to ovn-northd.  This can significantly increase
> downtime in
> -the event of a failover as the (often already busy) ovn-northd controller
> has
> -to recompute state for the new leader. Worse, if ovn-northd goes down, we
> can't
> -perform gateway failover at all.  This violates the principle that control
> -plane outages should have no impact on dataplane functionality.
> -
> -In a controller independent active-backup configuration, ovn-northd is
> -responsible for initial configuration while the HA cluster is responsible
> for
> -monitoring the leader, and failing over to a backup if necessary.
> ovn-northd
> -sets HA policy, but doesn't actively participate when failovers occur.
> -
> -Of course, in this model, ovn-northd is not without some responsibility.
> Its
> -role is to pre-plan what should happen in the event of a failure, leaving
> it to
> -the individual switches to execute this plan.  It does this by assigning
> each
> -gateway a unique leadership priority.  Once assigned, it communicates this
> -priority to each node it controls.  Nodes use the leadership priority to
> -determine which gateway in the cluster is the active leader by using a
> simple
> -metric: the leader is the gateway that is healthy, with the highest
> priority.
> -If that gateway goes down, leadership falls to the next highest priority,
> and
> -conversely, if a new gateway comes up with a higher priority, it takes
> over
> -leadership.
> -
> -Thus, in this model, leadership of the HA cluster is determined simply by
> the
> -status of its members.  Therefore if we can communicate the status of each
> -gateway to each transport node, they can individually figure out which is
> the
> -leader, and direct traffic accordingly.
> -
> -Tunnel Monitoring
> -+++++++++++++++++
> -
> -Since in this model leadership is determined exclusively by the health
> status
> -of member gateways, a key problem is how do we communicate this
> information to
> -the relevant transport nodes.  Luckily, we can do this fairly cheaply
> using
> -tunnel monitoring protocols like BFD.
> -
> -The basic idea is pretty straightforward.  Each transport node maintains a
> -tunnel to every gateway in the HA cluster (not just the leader).  These
> tunnels
> -are monitored using the BFD protocol to see which are alive.  Given this
> -information, hypervisors can trivially compute the highest priority live
> -gateway, and thus the leader.
> -
> -In practice, this leadership computation can be performed trivially using
> the
> -bundle or group action.  Rather than using OpenFlow to simply output to
> the
> -leader, all gateways could be listed in an active-backup bundle action
> ordered
> -by their priority.  The bundle action will automatically take into
> account the
> -tunnel monitoring status to output the packet to the highest priority live
> -gateway.
> -
> -Inter-Gateway Monitoring
> -++++++++++++++++++++++++
> -
> -One somewhat subtle aspect of this model, is that failovers are not
> globally
> -atomic.  When a failover occurs, it will take some time for all
> hypervisors to
> -notice and adjust accordingly.  Similarly, if a new high priority Gateway
> comes
> -up, it may take some time for all hypervisors to switch over to the new
> leader.
> -In order to avoid confusing the physical network, under these
> circumstances
> -it's important for the backup gateways to drop traffic they've received
> -erroneously.  In order to do this, each Gateway must know whether or not
> it is,
> -in fact active.  This can be achieved by creating a mesh of tunnels
> between
> -gateways.  Each gateway monitors the other gateways its cluster to
> determine
> -which are alive, and therefore whether or not that gateway happens to be
> the
> -leader.  If leading, the gateway forwards traffic normally, otherwise it
> drops
> -all traffic.
> -
> -We should note that this method works well under the assumption that there
> -are no inter-gateway connectivity failures, in such case this method
> would fail
> -to elect a single master. The simplest example is two gateways which stop
> seeing
> -each other but can still reach the hypervisors. Protocols like VRRP or
> CARP
> -have the same issue. A mitigation for this type of failure mode could be
> -achieved by having all network elements (hypervisors and gateways)
> periodically
> -share their link status to other endpoints.
> -
> -Gateway Leadership Resignation
> -++++++++++++++++++++++++++++++
> -
> -Sometimes a gateway may be healthy, but still may not be suitable to lead
> the
> -HA cluster.  This could happen for several reasons including:
> -
> -* The physical network is unreachable
> -
> -* BFD (or ping) has detected the next hop router is unreachable
> -
> -* The Gateway recently booted and isn't fully configured
> -
> -In this case, the Gateway should resign leadership by holding its tunnels
> down
> -using the ``other_config:cpath_down`` flag.  This indicates to
> participating
> -hypervisors and Gateways that this gateway should be treated as if it's
> down,
> -even though its tunnels are still healthy.
> -
> -Router Specific Active-Backup
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    Router Specific Active-Backup
> -
> -    +----------------+ +----------------+
> -    |                | |                |
> -    |      A C       | |     B D E      |
> -    |                | |                |
> -    +----------------+ +----------------+
> -                  ^ ^   ^ ^
> -                  | |   | |
> -                  | |   | |
> -                  + +   + +
> -                   Traffic
> -
> -Controller independent active-backup is a great advance over naive
> -active-backup, but it still has one glaring problem -- it under-utilizes
> the
> -backup gateways.  In ideal scenario, all traffic would split evenly among
> the
> -live set of gateways.  Getting all the way there is somewhat tricky, but
> as a
> -step in the direction, one could use the "Router Specific Active-Backup"
> -algorithm.  This algorithm looks a lot like active-backup on a per logical
> -router basis, with one twist.  It chooses a different active Gateway for
> each
> -logical router.  Thus, in situations where there are several logical
> routers,
> -all with somewhat balanced load, this algorithm performs better.
> -
> -Implementation of this strategy is quite straightforward if built on top
> of
> -basic controller independent active-backup.  On a per logical router
> basis, the
> -algorithm is the same, leadership is determined by the liveness of the
> -gateways.  The key difference here is that the gateways must have a
> different
> -leadership priority for each logical router.  These leadership priorities
> can
> -be computed by ovn-northd just as they had been in the controller
> independent
> -active-backup model.
> -
> -Once we have these per logical router priorities, they simply need be
> -communicated to the members of the gateway cluster and the hypervisors.
> The
> -hypervisors in particular, need simply have an active-backup bundle
> action (or
> -group action) per logical router listing the gateways in priority order
> for
> -*that router*, rather than having a single bundle action shared for all
> the
> -routers.
> -
> -Additionally, the gateways need to be updated to take into account
> individual
> -router priorities.  Specifically, each gateway should drop traffic of
> backup
> -routers it's running, and forward traffic of active gateways, instead of
> simply
> -dropping or forwarding everything.  This should likely be done by having
> -ovn-controller recompute OpenFlow for the gateway, though other options
> exist.
> -
> -The final complication is that ovn-northd's logic must be updated to
> choose
> -these per logical router leadership priorities in a more sophisticated
> manner.
> -It doesn't matter much exactly what algorithm it chooses to do this,
> beyond
> -that it should provide good balancing in the common case.  I.E. each
> logical
> -routers priorities should be different enough that routers balance to
> different
> -gateways even when failures occur.
> -
> -Preemption
> -++++++++++
> -
> -In an active-backup setup, one issue that users will run into is that of
> -gateway leader preemption.  If a new Gateway is added to a cluster, or
> for some
> -reason an existing gateway is rebooted, we could end up in a situation
> where
> -the newly activated gateway has higher priority than any other in the HA
> -cluster.  In this case, as soon as that gateway appears, it will preempt
> -leadership from the currently active leader causing an unnecessary
> failover.
> -Since failover can be quite expensive, this preemption may be undesirable.
> -
> -The controller can optionally avoid preemption by cleverly tweaking the
> -leadership priorities.  For each router, new gateways should be assigned
> -priorities that put them second in line or later when they eventually
> come up.
> -Furthermore, if a gateway goes down for a significant period of time, its
> old
> -leadership priorities should be revoked and new ones should be assigned
> as if
> -it's a brand new gateway.  Note that this should only happen if a gateway
> has
> -been down for a while (several minutes), otherwise a flapping gateway
> could
> -have wide ranging, unpredictable, consequences.
> -
> -Note that preemption avoidance should be optional depending on the
> deployment.
> -One necessarily sacrifices optimal load balancing to satisfy these
> requirements
> -as new gateways will get no traffic on boot.  Thus, this feature
> represents a
> -trade-off which must be made on a per installation basis.
> -
> -Fully Active-Active HA
> -~~~~~~~~~~~~~~~~~~~~~~
> -
> -::
> -
> -    Fully Active-Active HA
> -
> -    +----------------+ +----------------+
> -    |                | |                |
> -    |   A B C D E    | |    A B C D E   |
> -    |                | |                |
> -    +----------------+ +----------------+
> -                  ^ ^   ^ ^
> -                  | |   | |
> -                  | |   | |
> -                  + +   + +
> -                   Traffic
> -
> -The final step in L3HA is to have true active-active HA.  In this
> scenario each
> -router has an instance on each Gateway, and a mechanism similar to ECMP
> is used
> -to distribute traffic evenly among all instances.  This mechanism would
> require
> -Gateways to participate in routing protocols with the physical network to
> -attract traffic and alert of failures.  It is out of scope of this
> document,
> -but may eventually be necessary.
> -
> -L2HA
> -----
> -
> -L2HA is very difficult to get right.  Unlike L3HA, where the consequences
> of
> -problems are minor, in L2HA if two gateways are both transiently active,
> an L2
> -loop triggers and a broadcast storm results.  In practice to get around
> this,
> -gateways end up implementing an overly conservative "when in doubt drop
> all
> -traffic" policy, or they implement something like MLAG.
> -
> -MLAG has multiple gateways work together to pretend to be a single L2
> switch
> -with a large LACP bond.  In principle, it's the right solution to the
> problem
> -as it solves the broadcast storm problem, and has been deployed
> successfully in
> -other contexts.  That said, it's difficult to get right and not
> recommended.
> diff --git a/Documentation/topics/index.rst
> b/Documentation/topics/index.rst
> index 057649dd7..fcb741637 100644
> --- a/Documentation/topics/index.rst
> +++ b/Documentation/topics/index.rst
> @@ -27,7 +27,7 @@
>  Deep Dive
>  =========
>
> -How Open vSwitch and OVN are implemented and, where necessary, why it was
> +How Open vSwitch is implemented and, where necessary, why it was
>  implemented that way.
>
>  OVS
> @@ -52,19 +52,3 @@ OVS
>     tracing
>     idl-compound-indexes
>
> -OVN
> ----
> -
> -.. toctree::
> -   :maxdepth: 2
> -
> -   high-availability
> -   role-based-access-control
> -   ovn-news-2.8
> -
> -.. list-table::
> -
> -   * - ovn-architecture(7)
> -     - `(pdf) <
> http://openvswitch.org/support/dist-docs/ovn-architecture.7.pdf>`__
> -     - `(html) <
> http://openvswitch.org/support/dist-docs/ovn-architecture.7.html>`__
> -     - `(plain text) <
> http://openvswitch.org/support/dist-docs/ovn-architecture.7.txt>`__
> diff --git a/Documentation/topics/ovn-news-2.8.rst
> b/Documentation/topics/ovn-news-2.8.rst
> deleted file mode 100644
> index fae0a4278..000000000
> --- a/Documentation/topics/ovn-news-2.8.rst
> +++ /dev/null
> @@ -1,278 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -===============================
> -What's New with OVS and OVN 2.8
> -===============================
> -
> -This document is about what was added in Open vSwitch 2.8, which was
> released
> -at the end of August 2017, concentrating on the new features in OVN.  It
> also
> -covers some of what is coming up in Open vSwitch and OVN 2.9, which is
> due to
> -be released in February 2018.  OVN has many features, and this document
> does
> -not cover every new or enhanced feature (but contributions are welcome).
> -
> -This document assumes a basic familiarity with Open vSwitch, OVN, and
> their
> -associated tools.  For more information, please refer to the Open vSwitch
> and
> -OVN documentation, such as the ``ovn-architecture``\(7) manpage.
> -
> -Debugging and Troubleshooting
> ------------------------------
> -
> -Before version 2.8, Open vSwitch command-line tools were far more painful
> to
> -use than they needed to be.  This section covers the improvements made to
> the
> -CLI in the 2.8 release.
> -
> -User-Hostile UUIDs
> -~~~~~~~~~~~~~~~~~~
> -
> -The OVN CLI, through ``ovn-nbctl``, ``ovn-nbctl``, and ``ovn-trace``, used
> -full-length UUIDs almost everywhere.  It didn't even provide any
> assistance
> -with completion, etc., which in practice meant always cutting and pasting
> UUIDs
> -from one command or window to another.  This problem wasn't limited to the
> -places where one would expect to have to see or use a UUID, either.  In
> many
> -places where one would expect to be able to use a network, router, or port
> -name, a UUID was required instead.  In many places where one would want
> to see
> -a name, the UUID was displayed instead.  More than anything else, these
> -shortcomings made the CLI user-hostile.
> -
> -There was an underlying problem that the southbound database didn't
> actually
> -contain all the information needed to provide a decent user interface.
> In some
> -cases, for example, the human-friendly names that one would want to use
> for
> -entities simply weren't part of the database.  These names weren't
> necessary
> -for correctness, only for usability.
> -
> -OVN 2.8 eased many of these problems.  Most parts of the CLI now allow
> the user
> -to abbreviate UUIDs, as long as the abbreviations are unique within the
> -database.  Some parts of the CLI where full-length UUIDs make output hard
> to
> -read now abbreviate them themselves.  Perhaps more importantly, in many
> places
> -the OVN CLI now displays and accepts human-friendly names for networks,
> -routers, ports, and other entities.  In the places where the names were
> not
> -previously available, OVN (through ``ovn-northd``) now copies the names
> into
> -the southbound database.
> -
> -The CLIs for layers below OVN, at the OpenFlow and datapath layers with
> -``ovs-ofctl`` and ``ovs-dpctl``, respectively, had some similar problems
> in
> -which numbers were used for entities that had human-friendly names.  Open
> -vSwitch 2.8 also solves some of those problems.  Other than that, the most
> -notable enhancement in this area was the ``--no-stats`` option to
> ``ovs-ofctl
> -dump-flows``, which made that command's output more readable for the cases
> -where per-flow statistics were not interesting to the reader.
> -
> -Connections Between Levels
> -~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -OVN and Open vSwitch work almost like a stack of compilers: the OVN
> Neutron
> -plugin translates Neutron configuration into OVN northbound configuration,
> -which ``ovn-northd`` translates into logical flows, which
> ``ovn-controller``
> -translates into OpenFlow flows, which ``ovs-vswitchd`` translates into
> datapath
> -flows.  For debugging and troubleshooting it is often necessary to
> understand
> -exactly how these translations work.  The relationship from a logical
> flow to
> -its OpenFlow flows, or in the other direction, from an OpenFlow flow back
> to
> -the logical flow that produced it, was often of particular interest, but
> OVN
> -didn't provide good tools for the job.
> -
> -OVN 2.8 added some new features that ease these jobs.  ``ovn-sbctl
> lflow-list``
> -has a new option ``--ovs`` that lists the OpenFlow flows on a particular
> -chassis that were generated from the logical flows that it lists.
> -``ovn-trace`` also added a similar ``--ovs`` option that applies to the
> logical
> -flows it traces.
> -
> -In the other direction, OVN 2.8 added a new utility ``ovn-detrace`` that,
> given
> -an Open vSwitch trace of OpenFlow flows, annotates it with the logical
> flows
> -that yielded those OpenFlow flows.
> -
> -Distributed Firewall
> -~~~~~~~~~~~~~~~~~~~~
> -
> -OVN supports a distributed firewall with stateful connection tracking to
> ensure
> -that only packets for established connections, or those that the
> configuration
> -explicitly allows, can ingress a given VM or container.  Neutron uses this
> -feature by default.  Most packets in an OpenStack environment pass
> through it
> -twice, once after egress from the packet's source VM and once before
> ingress
> -into its destination VM.  Before OVN 2.8, the ``ovn-trace`` program, which
> -shows the path of a packet through an OVN logical network, did not
> support the
> -logical firewall, which in practice made it almost useless for Neutron.
> -
> -In OVN 2.8, ``ovn-trace`` adds support for the logical firewall.  By
> default it
> -assumes that packets are part of an established connection, which is
> usually
> -what the user wants as part of the trace.  It also accepts command-line
> options
> -to override that assumption, which allows the user to discover the
> treatment of
> -packets that the firewall should drop.
> -
> -At the next level deeper, prior to Open vSwitch 2.8, the OpenFlow tracing
> -command ``ofproto/trace`` also supported neither the connection tracking
> -feature underlying the OVN distributed firewall nor the "recirculation"
> feature
> -that accompanied it.  This meant that, even if the user tried to look
> deeper
> -into the distributed firewall mechanism, he or she would encounter a
> further
> -roadblock.  Open vSwitch 2.8 added support for both of these features as
> well.
> -
> -Summary Display
> -~~~~~~~~~~~~~~~
> -
> -``ovn-nbctl show`` and ``ovn-sbctl show``, for showing an overview of the
> OVN
> -configuration, didn't show a lot of important information.  OVN adds some
> more
> -useful information here.
> -
> -DNS, and IPAM
> --------------
> -
> -OVN 2.8 adds a built-in DNS server designed for assigning names to VMs and
> -containers within an OVN logical network.  DNS names are assigned using
> records
> -in the OVN northbound database and, like other OVN features, translated
> into
> -logical flows at the OVN southbound layer.  DNS requests directed to the
> OVN
> -DNS server never leave the hypervisor from which the request is sent;
> instead,
> -OVN processes and replies to the request from its ``ovn-controller`` local
> -agent.  The OVN DNS server is not a general-purpose DNS server and cannot
> be
> -used for that purpose.
> -
> -OVN includes simple built-in support for IP address management (IPAM), in
> which
> -OVN assigns IP addresses to VMs or containers from a pool or pools of IP
> -addresses delegated to it by the administrator.  Before OVN 2.8, OVN IPAM
> only
> -supported IPv4 addresses; OVN 2.8 adds support for IPv6.  OVN 2.8 also
> enhances
> -the address pool support to allow specific addresses to be excluded.
> Neutron
> -assigns IP addresses itself and does not use OVN IPAM.
> -
> -High Availability
> ------------------
> -
> -As a distributed system, in OVN a lot can go wrong.  As OVN advances, it
> adds
> -redundancy in places where currently a single failure could disrupt the
> -functioning of the system as a whole.  OVN 2.8 adds two new kinds of high
> -availability.
> -
> -ovn-northd HA
> -~~~~~~~~~~~~~
> -
> -The ``ovn-northd`` program sits between the OVN northbound and southbound
> -databases and translates from a logical network configuration into logical
> -flows.  If ``ovn-northd`` itself or the host on which it runs fails, then
> -updates to the OVN northbound configuration will not propagate to the
> -hypervisors and the OVN configuration freezes in place until
> ``ovn-northd``
> -restarts.
> -
> -OVN 2.8 adds support for active-backup HA to ``ovn-northd``.  When more
> than
> -one ``ovn-northd`` instance runs, it uses an OVSDB locking feature to
> -automatically choose a single active instance.  When that instance dies or
> -becomes nonresponsive, the OVSDB server automatically choose one of the
> -remaining instance(s) to take over.
> -
> -L3 Gateway HA
> -~~~~~~~~~~~~~
> -
> -In OVN 2.8, multiple chassis may now be specified for L3 gateways.  When
> more
> -than one chassis is specified, OVN manages high availability for that
> gateway.
> -Each hypervisor uses the BFD protocol to keep track of the gateway nodes
> that
> -are currently up.  At any given time, a hypervisor uses the
> highest-priority
> -gateway node that is currently up.
> -
> -OVSDB
> ------
> -
> -The OVN architecture relies heavily on OVSDB, the Open vSwitch database,
> for
> -hosting the northbound and southbound databases.  OVSDB was originally
> selected
> -for this purpose because it was already used in Open vSwitch for
> configuring
> -OVS itself and, thus, it was well integrated with OVS and well supported
> in C
> -and Python, the two languages that are used in Open vSwitch.
> -
> -OVSDB was well designed for its original purpose of configuring Open
> vSwitch.
> -It supports ACID transactions, has a small, efficient server, a flexible
> schema
> -system, and good support for troubleshooting and debugging.  However, it
> lacked
> -several features that are important for OVN but not for Open vSwitch.  As
> OVN
> -advances, these missing features have become more and more of a problem.
> One
> -option would be to switch to a different database that already has many of
> -these features, but despite a careful search, no ideal existing database
> was
> -identified, so the project chose instead to improve OVSDB where necessary
> to
> -bring it up to speed.  The following sections talk more about recent and
> future
> -improvements.
> -
> -High Availability
> -~~~~~~~~~~~~~~~~~
> -
> -When ``ovsdb-server`` was only used for OVS configuration, high
> availability
> -was not important.  ``ovsdb-server`` was capable of restarting itself
> -automatically if it crashed, and if the whole system went down then Open
> -vSwitch itself was dead too, so the database server's failure was not
> -important.
> -
> -In contrast, the northbound and southbound databases are centralized
> components
> -of a distributed system, so it is important that they not be a single
> point of
> -failure for the system as a whole.  In released versions of OVN,
> -``ovsdb-server`` supports only "active-backup replication" across a pair
> of
> -servers.  This means that if one server goes down, the other can pick it
> back
> -up approximately where the other one left off.  The servers do not have
> -built-in support for deciding at any given time which is the active and
> which
> -the backup, so the administrator must configure an external agent to do
> this
> -management.
> -
> -Active-backup replication is not entirely satisfactory, for multiple
> reasons.
> -Replication is only approximate.  Configuring the external agent requires
> extra
> -work.  There is no benefit from the backup server except when the active
> server
> -fails.  At most two servers can be used.
> -
> -A new form of high availability for OVSDB is under development for the
> OVN 2.9
> -release, based on the Raft algorithm for distributed consensus.  Whereas
> -replication uses two servers, clustering using Raft requires three or more
> -(typically an odd number) and continues functioning as long as more than
> half
> -of the servers are up.  The clustering implementation is built into
> -``ovsdb-server`` and does not require an external agent.  Clustering
> preserves
> -the ACID properties of the database, so that a transaction that commits is
> -guaranteed to persist.  Finally, reads (which are the bulk of the OVN
> workload)
> -scale with the size of the cluster, so that adding more servers should
> improve
> -performance as the number of hypervisors in an OVN deployment increases.
> As of
> -this writing, OVSDB support for clustering is undergoing development and
> early
> -deployment testing.
> -
> -RBAC security
> -~~~~~~~~~~~~~
> -
> -Until Open vSwitch 2.8, ``ovsdb-server`` had little support for access
> control
> -within a database.  If an OVSDB client could modify the database at all,
> it
> -could make arbitrary changes.  This was sufficient for most uses case to
> that
> -point.
> -
> -Hypervisors in an OVN deployment need access to the OVN southbound
> database.
> -Most of their access is reads, to find out about the OVN configuration.
> -Hypervisors do need some write access to the southbound database,
> primarily to
> -let the other hypervisors know what VMs and containers they are running
> and how
> -to reach them.  Thus, OVN gives all of the hypervisors in the OVN
> deployment
> -write access to the OVN southbound database.  This is fine when all is
> well,
> -but if any of the hypervisors were compromised then they could disrupt the
> -entire OVN deployment by corrupting the database.
> -
> -The OVN developers considered a few ways to solve this problem.  One way
> would
> -be to introduce a new central service (perhaps in ``ovn-northd``) that
> provided
> -only the kinds of writes that the hypervisors legitimately need, and then
> grant
> -hypervisors direct access to the southbound database only for reads.  But
> -ultimately the developers decided to introduce a new form of more access
> -control for OVSDB, called the OVSDB RBAC (role-based access control)
> feature.
> -OVSDB RBAC allows for granular enough control over access that
> hypervisors can
> -be granted only the ability to add, modify, and delete the records that
> relate
> -to themselves, preventing them from corrupting the database as a whole.
> -
> -Further Directions
> -------------------
> -
> -For more information about new features in OVN and Open vSwitch, please
> refer
> -to the NEWS file distributed with the source tree.  If you have questions
> about
> -Open vSwitch or OVN features, please feel free to write to the Open
> vSwitch
> -discussion mailing list at ovs-discuss at openvswitch.org.
> diff --git a/Documentation/topics/role-based-access-control.rst
> b/Documentation/topics/role-based-access-control.rst
> deleted file mode 100644
> index 8f2a3a998..000000000
> --- a/Documentation/topics/role-based-access-control.rst
> +++ /dev/null
> @@ -1,101 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -=========================
> -Role Based Access Control
> -=========================
> -
> -Where SSL provides authentication when connecting to an OVS database, role
> -based access control (RBAC) provides authorization to operations performed
> -by clients connecting to an OVS database. RBAC allows for administrators
> to
> -restrict the database operations a client may perform and thus enhance the
> -security already provided by SSL.
> -
> -In theory, any OVS database could define RBAC roles and permissions, but
> at
> -present only the OVN southbound database has the appropriate tables
> defined to
> -facilitate RBAC.
> -
> -Mechanics
> ----------
> -RBAC is intended to supplement SSL. In order to enable RBAC, the
> connection to
> -the database must use SSL. Some permissions in RBAC are granted based on
> the
> -certificate common name (CN) of the connecting client.
> -
> -RBAC is controlled with two database tables, RBAC_Role and
> RBAC_Permission.
> -The RBAC_Permission table contains records that describe a set of
> permissions
> -for a given table in the database.
> -
> -The RBAC_Permission table contains the following columns:
> -
> -table
> -  The table in the database for which permissions are being described.
> -insert_delete
> -  Describes whether insertion and deletion of records is allowed.
> -update
> -  A list of columns that are allowed to be updated.
> -authorization
> -  A list of column names. One of the listed columns must match the SSL
> -  certificate CN in order for the attempted operation on the table to
> -  succeed. If a key-value pair is provided, then the key is the column
> name,
> -  and the value is the name of a key in that column. An empty string gives
> -  permission to all clients to perform operations.
> -
> -The RBAC_Role table contains the following columns:
> -
> -name
> -  The name of the role being defined
> -permissions
> -  A list of key-value pairs. The key is the name of a table in the
> database,
> -  and the value is a UUID of a record in the RBAC_Permission table that
> -  describes the permissions the role has for that table.
> -
> -.. note::
> -
> -   All tables not explicitly referenced in an RBAC_Role record are
> read-only
> -
> -In order to enable RBAC, specify the role name as an argument to the
> -set-connection command for the database. As an example, to enable the
> -"ovn-controller" role on the OVN southbound database, use the following
> -command:
> -
> -::
> -
> -   $ ovn-sbctl set-connection role=ovn-controller ssl:192.168.0.1:6642
> -
> -Pre-defined Roles
> ------------------
> -This section describes roles that have been defined internally by OVS/OVN.
> -
> -ovn-controller
> -~~~~~~~~~~~~~~
> -The ovn-controller role is specified in the OVN southbound database and is
> -intended for use by hypervisors running the ovn-controller daemon.
> -ovn-controller connects to the OVN southbound database mostly to read
> -information, but there are a few cases where ovn-controller also needs to
> -write. The ovn-controller role was designed to allow for ovn-controllers
> -to write to the southbound database only in places where it makes sense
> to do
> -so. This way, if an intruder were to take over a hypervisor running
> -ovn-controller, it is more difficult to compromise the entire overlay
> network.
> -
> -It is strongly recommended to set the ovn-controller role for the OVN
> -southbound database to enhance security.
> diff --git a/Documentation/tutorials/index.rst
> b/Documentation/tutorials/index.rst
> index 35340ee56..5ec62beab 100644
> --- a/Documentation/tutorials/index.rst
> +++ b/Documentation/tutorials/index.rst
> @@ -27,8 +27,7 @@
>  Tutorials
>  =========
>
> -Getting started with Open vSwitch (OVS) and Open Virtual Network (OVN)
> for Open
> -vSwitch.
> +Getting started with Open vSwitch (OVS).
>
>  .. TODO(stephenfin): We could really do with a few basic tutorials, along
> with
>     some more specialized ones (DPDK, XenServer, Windows). The latter could
> @@ -42,8 +41,4 @@ vSwitch.
>     faucet
>     ipsec
>     ovs-advanced
> -   ovn-sandbox
> -   ovn-openstack
> -   ovn-rbac
> -   ovn-ipsec
>     ovs-conntrack
> diff --git a/Documentation/tutorials/ovn-ipsec.rst
> b/Documentation/tutorials/ovn-ipsec.rst
> deleted file mode 100644
> index feb695ea3..000000000
> --- a/Documentation/tutorials/ovn-ipsec.rst
> +++ /dev/null
> @@ -1,146 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -==================
> -OVN IPsec Tutorial
> -==================
> -
> -This document provides a step-by-step guide for encrypting tunnel traffic
> with
> -IPsec in Open Virtual Network (OVN). OVN tunnel traffic is transported by
> -physical routers and switches. These physical devices could be untrusted
> -(devices in public network) or might be compromised.  Enabling IPsec
> encryption
> -for the tunnel traffic can prevent the traffic data from being monitored
> and
> -manipulated. More details about the OVN IPsec design can be found in
> -``ovn-architecture``\(7) manpage.
> -
> -This document assumes OVN is installed in your system and runs normally.
> Also,
> -you need to install OVS IPsec packages in each chassis (refer to
> -:ref:`install-ovs-ipsec`).
> -
> -Generating Certificates and Keys
> ---------------------------------
> -
> -OVN chassis uses CA-signed certificate to authenticate peer chassis for
> -building IPsec tunnel. If you have enabled Role-Based Access Control
> (RBAC) in
> -OVN, you can use the RBAC SSL certificates and keys to set up OVN IPsec.
> Or you
> -can generate separate certificates and keys with ``ovs-pki`` (refer to
> -:ref:`gen-certs-keys`).
> -
> -.. note::
> -
> -   OVN IPsec requires x.509 version 3 certificate with the subjectAltName
> DNS
> -   field setting the same string as the common name (CN) field. CN should
> be
> -   set as the chassis name.  ``ovs-pki`` in Open vSwitch 2.10.90 and later
> -   generates such certificates.  Please generate compatible certificates
> if you
> -   use another PKI tool, or an older version of ``ovs-pki``, to manage
> -   certificates.
> -
> -Configuring OVN IPsec
> ----------------------
> -
> -You need to install the CA certificate, chassis certificate and private
> key in
> -each chassis. Use the following command::
> -
> -    $ ovs-vsctl set Open_vSwitch . \
> -            other_config:certificate=/path/to/chassis-cert.pem \
> -            other_config:private_key=/path/to/chassis-privkey.pem \
> -            other_config:ca_cert=/path/to/cacert.pem
> -
> -Enabling OVN IPsec
> -------------------
> -
> -To enable OVN IPsec, set ``ipsec`` column in ``NB_Global`` table of the
> -northbound database to true::
> -
> -    $ ovn-nbctl set nb_global . ipsec=true
> -
> -With OVN IPsec enabled, all tunnel traffic in OVN will be encrypted with
> IPsec.
> -To disable it, set ``ipsec`` column in ``NB_Global`` table of the
> northbound
> -database to false::
> -
> -    $ ovn-nbctl set nb_global . ipsec=false
> -
> -Troubleshooting
> ----------------
> -
> -The ``ovs-monitor-ipsec`` daemon in each chassis manages and monitors the
> IPsec
> -tunnel state. Use the following ``ovs-appctl`` command to view
> -``ovs-monitor-ipsec`` internal representation of tunnel configuration::
> -
> -    $ ovs-appctl -t ovs-monitor-ipsec tunnels/show
> -
> -If there is a misconfiguration, then ``ovs-appctl`` should indicate why.
> -For example::
> -
> -   Interface name: ovn-host_2-0 v1 (CONFIGURED) <--- Should be set
> -                                             to CONFIGURED. Otherwise,
> -                                             error message will be
> -                                             provided
> -   Tunnel Type:    geneve
> -   Remote IP:      2.2.2.2
> -   SKB mark:       None
> -   Local cert:     /path/to/chassis-cert.pem
> -   Local name:     host_1
> -   Local key:      /path/to/chassis-privkey.pem
> -   Remote cert:    None
> -   Remote name:    host_2
> -   CA cert:        /path/to/cacert.pem
> -   PSK:            None
> -   Ofport:         2          <--- Whether ovs-vswitchd has assigned
> Ofport
> -                                   number to this Tunnel Port
> -   CFM state:      Disabled     <--- Whether CFM declared this tunnel
> healthy
> -   Kernel policies installed:
> -   ...                          <--- IPsec policies for this OVS tunnel in
> -                                     Linux Kernel installed by strongSwan
> -   Kernel security associations installed:
> -   ...                          <--- IPsec security associations for this
> OVS
> -                                     tunnel in Linux Kernel installed by
> -                                     strongswan
> -   IPsec connections that are active:
> -   ...                          <--- IPsec "connections" for this OVS
> -                                     tunnel
> -
> -If you don't see any active connections, try to run the following command
> to
> -refresh the ``ovs-monitor-ipsec`` daemon::
> -
> -    $ ovs-appctl -t ovs-monitor-ipsec refresh
> -
> -You can also check the logs of the ``ovs-monitor-ipsec`` daemon and the
> IKE
> -daemon to locate issues.  ``ovs-monitor-ipsec`` outputs log messages to
> -``/var/log/openvswitch/ovs-monitor-ipsec.log``.
> -
> -Bug Reporting
> --------------
> -
> -If you think you may have found a bug with security implications, like
> -
> -1. IPsec protected tunnel accepted packets that came unencrypted; OR
> -2. IPsec protected tunnel allowed packets to leave unencrypted;
> -
> -Then report such bugs according to :doc:`/internals/security`.
> -
> -If bug does not have security implications, then report it according to
> -instructions in :doc:`/internals/bugs`.
> -
> -If you have suggestions to improve this tutorial, please send a email to
> -ovs-discuss at openvswitch.org.
> diff --git a/Documentation/tutorials/ovn-openstack.rst
> b/Documentation/tutorials/ovn-openstack.rst
> deleted file mode 100644
> index c6dff5e55..000000000
> --- a/Documentation/tutorials/ovn-openstack.rst
> +++ /dev/null
> @@ -1,1922 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -======================
> -OVN OpenStack Tutorial
> -======================
> -
> -This tutorial demonstrates how OVN works in an OpenStack "DevStack"
> -environment.  It was tested with the "master" branches of DevStack and
> -Open vSwitch near the beginning of May 2017.  Anyone using an earlier
> -version is likely to encounter some differences.  In particular, we
> -noticed some shortcomings in OVN utilities while writing the tutorial
> -and pushed out some improvements, so it's best to use recent Open
> -vSwitch at least from that point of view.
> -
> -The goal of this tutorial is to demonstrate OVN in an end-to-end way,
> -that is, to show how it works from the cloud management system at the
> -top (in this case, OpenStack and specifically its Neutron networking
> -subsystem), through the OVN northbound and southbound databases, to
> -the bottom at the OVN local controller and Open vSwitch data plane.
> -We hope that this demonstration makes it easier for users and
> -potential users to understand how OVN works and how to debug and
> -troubleshoot it.
> -
> -In addition to new material, this tutorial incorporates content from
> -``testing.rst`` in OpenStack networking-ovn, by Russell Bryant and
> -others.  Without that example, this tutorial could not have been
> -written.
> -
> -We provide enough details in the tutorial that you should be able to
> -fully follow along, by creating a DevStack VM and cloning DevStack and
> -so on.  If you want to do this, start out from `Setting Up DevStack`_
> -below.
> -
> -Setting Up DevStack
> --------------------
> -
> -This section explains how to install DevStack, a kind of OpenStack
> -packaging for developers, in a way that allows you to follow along
> -with the tutorial in full.
> -
> -Unless you have a spare computer laying about, it's easiest to install
> -DevStacck in a virtual machine.  This tutorial was built using a VM
> -implemented by KVM and managed by virt-manager.  I recommend
> -configuring the VM configured for the x86-64 architecture, 4 GB RAM, 2
> -VCPUs, and a 20 GB virtual disk.
> -
> -.. note::
> -
> -   If you happen to run your Linux-based host with 32-bit userspace,
> -   then you will have some special issues, even if you use a 64-bit
> -   kernel:
> -
> -   * You may find that you can get 32-bit DevStack VMs to work to some
> -     extent, but I personally got tired of finding workarounds.  I
> -     recommend running your VMs in 64-bit mode.  To get this to work,
> -     I had to go to the CPUs tab for the VM configuration in
> -     virt-manager and change the CPU model from the one originally
> -     listed to "Hypervisor Default' (it is curious that this is not
> -     the default!).
> -
> -   * On a host with 32-bit userspace, KVM supports VMs with at most
> -     2047 MB RAM.  This is adequate, barely, to start DevStack, but it
> -     is not enough to run multiple (nested) VMs.  To prevent
> -     out-of-memory failures, set up extra swap space in the guest.
> -     For example, to add 2 GB swap::
> -
> -       $ sudo dd if=/dev/zero of=/swapfile bs=1M count=2048
> -       $ sudo mkswap /swapfile
> -       $ sudo swapon /swapfile
> -
> -     and then add a line like this to ``/etc/fstab`` to add the new
> -     swap automatically upon reboot::
> -
> -       /swapfile swap swap defaults 0 0
> -
> -Here are step-by-step instructions to get started:
> -
> -1. Install a VM.
> -
> -   I tested these instructions with Centos 7.3.  Download the "minimal
> -   install" ISO and booted it.  The install is straightforward.  Be
> -   sure to enable networking, and set a host name, such as
> -   "ovn-devstack-1".  Add a regular (non-root) user, and check the box
> -   "Make this user administrator".  Also, set your time zone.
> -
> -2. You can SSH into the DevStack VM, instead of running from a
> -   console.  I recommend it because it's easier to cut and paste
> -   commands into a terminal than a VM console.  You might also
> -   consider using a very wide terminal, perhaps 160 columns, to keep
> -   tables from wrapping.
> -
> -   To improve convenience further, you can make it easier to log in
> -   with the following steps, which are optional:
> -
> -   a. On your host, edit your ``~/.ssh/config``, adding lines like
> -      the following::
> -
> -        Host ovn-devstack-1
> -              Hostname VMIP
> -              User VMUSER
> -
> -      where VMIP is the VM's IP address and VMUSER is your username
> -      inside the VM.  (You can omit the ``User`` line if your
> -      username is the same in the host and the VM.)  After you do
> -      this, you can SSH to the VM by name, e.g. ``ssh
> -      ovn-devstack-1``, and if command-line completion is set up in
> -      your host shell, you can shorten that to something like ``ssh
> -      ovn`` followed by hitting the Tab key.
> -
> -   b. If you have SSH public key authentication set up, with an SSH
> -      agent, run on your host::
> -
> -        $ ssh-copy-id ovn-devstack-1
> -
> -      and type your password once.  Afterward, you can log in without
> -      typing your password again.
> -
> -      (If you don't already use SSH public key authentication and an
> -      agent, consider looking into it--it will save you time in the
> -      long run.)
> -
> -   c. Optionally, inside the VM, append the following to your
> -      ``~/.bash_profile``::
> -
> -        . $HOME/devstack/openrc admin
> -
> -      It will save you running it by hand each time you log in.  But
> -      it also prints garbage to the console, which can screw up
> -      services like ``ssh-copy-id``, so be careful.
> -
> -2. Boot into the installed system and log in as the regular user, then
> -   install Git::
> -
> -     $ sudo yum install git
> -
> -   .. note::
> -
> -      If you installed a 32-bit i386 guest (against the advice above),
> -      install a non-PAE kernel and reboot into it at this point::
> -
> -           $ sudo yum install kernel-core kernel-devel
> -           $ sudo reboot
> -
> -      Be sure to select the non-PAE kernel from the list at boot.
> -      Without this step, DevStack will fail to install properly later.
> -
> -3. Get copies of DevStack and OVN and set them up::
> -
> -     $ git clone http://git.openstack.org/openstack-dev/devstack.git
> -     $ git clone http://git.openstack.org/openstack/networking-ovn.git
> -     $ cd devstack
> -     $ cp ../networking-ovn/devstack/local.conf.sample local.conf
> -
> -   .. note::
> -
> -      If you installed a 32-bit i386 guest (against the advice above),
> -      at this point edit ``local.conf`` to add the following line::
> -
> -        CIRROS_ARCH=i386
> -
> -4. Initialize DevStack::
> -
> -     $ ./stack.sh
> -
> -   This will spew many screenfuls of text, and the first time you run
> -   it, it will download lots of software from the Internet.  The
> -   output should eventually end with something like this::
> -
> -     This is your host IP address: 172.16.189.6
> -     This is your host IPv6 address: ::1
> -     Horizon is now available at http://172.16.189.6/dashboard
> -     Keystone is serving at http://172.16.189.6/identity/
> -     The default users are: admin and demo
> -     The password: password
> -     2017-03-09 15:10:54.117 | stack.sh completed in 2110 seconds.
> -
> -   If there's some kind of failure, you can restart by running
> -   ``./stack.sh`` again.  It won't restart exactly where it left off,
> -   but steps up to the one where it failed will skip the download
> -   steps.  (Sometimes blindly restarting after a failure will allow it
> -   to succeed.)  If you reboot your VM, you need to rerun this
> -   command.  (If you run into trouble with ``stack.sh`` after
> -   rebooting your VM, try running ``./unstack.sh``.)
> -
> -   At this point you can navigate a web browser on your host to the
> -   Horizon dashboard URL.  Many OpenStack operations can be initiated
> -   from this UI.  Feel free to explore, but this tutorial focuses on
> -   the alternative command-line interfaces because they are easier to
> -   explain and to cut and paste.
> -
> -5. As of this writing, you need to run the following to fix a problem
> -   with using VM consoles from the OpenStack web instance::
> -
> -     $ (cd /opt/stack/noVNC && git checkout v0.6.0)
> -
> -   See
> -
> https://serenity-networks.com/how-to-fix-setkeycodes-00-and-unknown-key-pressed-console-errors-on-openstack/
> -   for more details.
> -
> -6. The firewall in the VM by default allows SSH access but not HTTP.
> -   You will probably want HTTP access to use the OpenStack web
> -   interface.  The following command enables that.  (It also enables
> -   every other kind of network access, so if you're concerned about
> -   security then you might want to find a more targeted approach.)
> -
> -   ::
> -
> -      $ sudo iptables -F
> -
> -   (You need to re-run this if you reboot the VM.)
> -
> -7. To use OpenStack command line utilities in the tutorial, run::
> -
> -     $ . ~/devstack/openrc admin
> -
> -   This needs to be re-run each time you log in (but see the following
> -   section).
> -
> -DevStack preliminaries
> -----------------------
> -
> -Before we really jump in, let's set up a couple of things in DevStack.
> -This is the first real test that DevStack is working, so if you get
> -errors from any of these commands, it's a sign that ``stack.sh``
> -didn't finish properly, or perhaps that you didn't run the ``openrc
> -admin`` command at the end of the previous instructions.
> -
> -If you stop and restart DevStack via ``unstack.sh`` followed by
> -``stack.sh``, you have to rerun these steps.
> -
> -1. For SSH access to the VMs we're going to create, we'll need a SSH
> -   keypair.  Later on, we'll get OpenStack to install this keypair
> -   into VMs.  Create one with::
> -
> -     $ openstack keypair create demo > ~/id_rsa_demo
> -     $ chmod 600 ~/id_rsa_demo
> -
> -2. By default, DevStack security groups drop incoming traffic, but to
> -   test networking in a reasonable way we need to enable it.  You only
> -   need to actually edit one particular security group, but DevStack
> -   creates multiple and it's somewhat difficult to figure out which
> -   one is important because all of them are named "default".  So, the
> -   following adds rules to allow SSH and ICMP traffic into **every**
> -   security group::
> -
> -     $ for group in $(openstack security group list -f value -c ID); do \
> -     openstack security group rule create --ingress --ethertype IPv4
> --dst-port 22 --protocol tcp $group; \
> -     openstack security group rule create --ingress --ethertype IPv4
> --protocol ICMP $group; \
> -     done
> -
> -3. Later on, we're going to create some VMs and we'll need an
> -   operating system image to install.  DevStack comes with a very
> -   simple image built-in, called "cirros", which works fine.  We need
> -   to get the UUID for this image.  Our later commands assume shell
> -   variable ``IMAGE_ID`` holds this UUID.  You can set this by hand,
> -   e.g.::
> -
> -     $ openstack image list
> -
>  +--------------------------------------+--------------------------+--------+
> -     | ID                                   | Name                     |
> Status |
> -
>  +--------------------------------------+--------------------------+--------+
> -     | 77f37d2c-3d6b-4e99-a01b-1fa5d78d1fa1 | cirros-0.3.5-x86_64-disk |
> active |
> -
>  +--------------------------------------+--------------------------+--------+
> -     $ IMAGE_ID=73ca34f3-63c4-4c10-a62f-4540afc24eaa
> -
> -   or by parsing CLI output::
> -
> -     $ IMAGE_ID=$(openstack image list -f value -c ID)
> -
> -   .. note::
> -
> -      Your image ID will differ from the one above, as will every UUID
> -      in this tutorial.  They will also change every time you run
> -      ``stack.sh``.  The UUIDs are generated randomly.
> -
> -Shortening UUIDs
> -----------------
> -
> -OpenStack, OVN, and Open vSwitch all really like UUIDs.  These are
> -great for uniqueness, but 36-character strings are terrible for
> -readability.  Statistically, just the first few characters are enough
> -for uniqueness in small environments, so let's define a helper to make
> -things more readable::
> -
> -  $ abbrev() { a='[0-9a-fA-F]' b=$a$a c=$b$b; sed
> "s/$b-$c-$c-$c-$c$c$c//g"; }
> -
> -You can use this as a filter to abbreviate UUIDs.  For example, use it
> -to abbreviate the above image list::
> -
> -  $ openstack image list -f yaml | abbrev
> -  - ID: 77f37d
> -    Name: cirros-0.3.5-x86_64-disk
> -    Status: active
> -
> -The command above also adds ``-f yaml`` to switch to YAML output
> -format, because abbreviating UUIDs screws up the default table-based
> -formatting and because YAML output doesn't produce wrap columns across
> -lines and therefore is easier to cut and paste.
> -
> -Overview
> ---------
> -
> -Now that DevStack is ready, with OVN set up as the networking
> -back-end, here's an overview of what we're going to do in the
> -remainder of the demo, all via OpenStack:
> -
> -1. Switching: Create an OpenStack network ``n1`` and VMs ``a`` and
> -   ``b`` attached to it.
> -
> -   An OpenStack network is a virtual switch; it corresponds to an OVN
> -   logical switch.
> -
> -2. Routing: Create a second OpenStack network ``n2`` and VM ``c``
> -   attached to it, then connect it to network ``n1`` by creating an
> -   OpenStack router and attaching ``n1`` and ``n2`` to it.
> -
> -3. Gateways: Make VMs ``a`` and ``b`` available via an external network.
> -
> -4. IPv6: Add IPv6 addresses to our VMs to demonstrate OVN support for
> -   IPv6 routing.
> -
> -5. ACLs: Add and modify OpenStack stateless and stateful rules in
> -   security groups.
> -
> -6. DHCP: How it works in OVN.
> -
> -7. Further directions: Adding more compute nodes.
> -
> -At each step, we will take a look at how the features in question work
> -from OpenStack's Neutron networking layer at the top to the data plane
> -layer at the bottom.  From the highest to lowest level, these layers
> -and the software components that connect them are:
> -
> -* OpenStack Neutron, which as the top level in the system is the
> -  authoritative source of the virtual network configuration.
> -
> -  We will use OpenStack's ``openstack`` utility to observe and modify
> -  Neutron and other OpenStack configuration.
> -
> -* networking-ovn, the Neutron driver that interfaces with OVN and
> -  translates the internal Neutron representation of the virtual
> -  network into OVN's representation and pushes that representation
> -  down the OVN northbound database.
> -
> -  In this tutorial it's rarely worth distinguishing Neutron from
> -  networking-ovn, so we usually don't break out this layer separately.
> -
> -* The OVN Northbound database, aka NB DB.  This is an instance of
> -  OVSDB, a simple general-purpose database that is used for multiple
> -  purposes in Open vSwitch and OVN.  The NB DB's schema is in terms of
> -  networking concepts such as switches and routers.  The NB DB serves
> -  the purpose that in other systems might be filled by some kind of
> -  API; for example, in place of calling an API to create or delete a
> -  logical switch, networking-ovn performs these operations by
> -  inserting or deleting a row in the NB DB's Logical_Switch table.
> -
> -  We will use OVN's ``ovn-nbctl`` utility to observe the NB DB.  (We
> -  won't directly modify data at this layer or below.  Because
> -  configuration trickles down from Neutron through the stack, the
> -  right way to make changes is to use the ``openstack`` utility or
> -  another OpenStack interface and then wait for them to percolate
> -  through to lower layers.)
> -
> -* The ovn-northd daemon, a program that runs centrally and translates
> -  the NB DB's network representation into the lower-level
> -  representation used by the OVN Southbound database in the next
> -  layer.  The details of this daemon are usually not of interest,
> -  although without it OVN will not work, so this tutorial does not
> -  often mention it.
> -
> -* The OVN Southbound database, aka SB DB, which is also an OVSDB
> -  database.  Its schema is very different from the NB DB.  Instead of
> -  familiar networking concepts, the SB DB defines the network in terms
> -  of collections of match-action rules called "logical flows", which
> -  while similar in concept to OpenFlow flows use logical concepts, such
> -  as virtual machine instances, in place of physical concepts like
> -  physical Ethernet ports.
> -
> -  We will use OVN's ``ovn-sbctl`` utility to observe the SB DB.
> -
> -* The ovn-controller daemon.  A copy of ovn-controller runs on each
> -  hypervisor.  It reads logical flows from the SB DB, translates them
> -  into OpenFlow flows, and sends them to Open vSwitch's ovs-vswitchd
> -  daemon.  Like ovn-northd, usually the details of what this daemon
> -  are not of interest, even though it's important to the operation of
> -  the system.
> -
> -* ovs-vswitchd.  This program runs on each hypervisor.  It is the core
> -  of Open vSwitch, which processes packets according to the OpenFlow
> -  flows set up by ovn-controller.
> -
> -* Open vSwitch datapath.  This is essentially a cache designed to
> -  accelerate packet processing.  Open vSwitch includes a few different
> -  datapaths but OVN installations typically use one based on the Open
> -  vSwitch Linux kernel module.
> -
> -Switching
> ----------
> -
> -Switching is the basis of networking in the real world and in virtual
> -networking as well.  OpenStack calls its concept of a virtual switch a
> -"network", and OVN calls its corresponding concept a "logical switch".
> -
> -In this step, we'll create an OpenStack network ``n1``, then create
> -VMs ``a`` and ``b`` and attach them to ``n1``.
> -
> -Creating network ``n1``
> -~~~~~~~~~~~~~~~~~~~~~~~
> -
> -Let's start by creating the network::
> -
> -  $ openstack network create --project admin --provider-network-type
> geneve n1
> -
> -OpenStack needs to know the subnets that a network serves.  We inform
> -it by creating subnet objects.  To keep it simple, let's give our
> -network a single subnet for the 10.1.1.0/24 network.  We have to give
> -it a name, in this case ``n1subnet``::
> -
> -  $ openstack subnet create --subnet-range 10.1.1.0/24 --network n1
> n1subnet
> -
> -If you ask Neutron to show us the available networks, we see ``n1`` as
> -well as the two networks that DevStack creates by default::
> -
> -  $ openstack network list -f yaml | abbrev
> -  - ID: 5b6baf
> -    Name: n1
> -    Subnets: 5e67e7
> -  - ID: c02c4d
> -    Name: private
> -    Subnets: d88a34, fd87f9
> -  - ID: d1ac28
> -    Name: public
> -    Subnets: 0b1e79, c87dc1
> -
> -Neutron pushes this network setup down to the OVN northbound
> -database.  We can use ``ovn-nbctl show`` to see an overview of what's
> -in the NB DB::
> -
> -  $ ovn-nbctl show | abbrev
> -  switch 5b3d5f (neutron-c02c4d) (aka private)
> -      port b256dd
> -          type: router
> -          router-port: lrp-b256dd
> -      port f264e7
> -          type: router
> -          router-port: lrp-f264e7
> -  switch 2579f4 (neutron-d1ac28) (aka public)
> -      port provnet-d1ac28
> -          type: localnet
> -          addresses: ["unknown"]
> -      port ae9b52
> -          type: router
> -          router-port: lrp-ae9b52
> -  switch 3eb263 (neutron-5b6baf) (aka n1)
> -  router c59ad2 (neutron-9b057f) (aka router1)
> -      port lrp-ae9b52
> -          mac: "fa:16:3e:b2:d2:67"
> -          networks: ["172.24.4.9/24", "2001:db8::b/64"]
> -      port lrp-b256dd
> -          mac: "fa:16:3e:35:33:db"
> -          networks: ["fdb0:5860:4ba8::1/64"]
> -      port lrp-f264e7
> -          mac: "fa:16:3e:fc:c8:da"
> -          networks: ["10.0.0.1/26"]
> -      nat 80914c
> -          external ip: "172.24.4.9"
> -          logical ip: "10.0.0.0/26"
> -          type: "snat"
> -
> -This output shows that OVN has three logical switches, each of which
> -corresponds to a Neutron network, and a logical router that
> -corresponds to the Neutron router that DevStack creates by default.
> -The logical switch that corresponds to our new network ``n1`` has no
> -ports yet, because we haven't added any.  The ``public`` and
> -``private`` networks that DevStack creates by default have router
> -ports that connect to the logical router.
> -
> -Using ovn-northd, OVN translates the NB DB's high-level switch and
> -router concepts into lower-level concepts of "logical datapaths" and
> -logical flows.  There's one logical datapath for each logical switch
> -or router::
> -
> -  $ ovn-sbctl list datapath_binding | abbrev
> -  _uuid               : 0ad69d
> -  external_ids        : {logical-switch="5b3d5f", name="neutron-c02c4d",
> "name2"=private}
> -  tunnel_key          : 1
> -
> -  _uuid               : a8a758
> -  external_ids        : {logical-switch="3eb263", name="neutron-5b6baf",
> "name2"="n1"}
> -  tunnel_key          : 4
> -
> -  _uuid               : 191256
> -  external_ids        : {logical-switch="2579f4", name="neutron-d1ac28",
> "name2"=public}
> -  tunnel_key          : 3
> -
> -  _uuid               : b87bec
> -  external_ids        : {logical-router="c59ad2", name="neutron-9b057f",
> "name2"="router1"}
> -  tunnel_key          : 2
> -
> -This output lists the NB DB UUIDs in external_ids:logical-switch and
> -Neutron UUIDs in externals_ids:uuid.  We can dive in deeper by viewing
> -the OVN logical flows that implement a logical switch.  Our new
> -logical switch is a simple and almost pathological example given that
> -it doesn't yet have any ports attached to it.  We'll look at the
> -details a bit later::
> -
> -  $ ovn-sbctl lflow-list n1 | abbrev
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: ingress
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(eth.src[40]),
> action=(drop;)
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(vlan.present),
> action=(drop;)
> -  ...
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: egress
> -    table=0 (ls_out_pre_lb      ), priority=0    , match=(1),
> action=(next;)
> -    table=1 (ls_out_pre_acl     ), priority=0    , match=(1),
> action=(next;)
> -  ...
> -
> -We have one hypervisor (aka "compute node", in OpenStack parlance),
> -which is the one where we're running all these commands.  On this
> -hypervisor, ovn-controller is translating OVN logical flows into
> -OpenFlow flows ("physical flows").  It makes sense to go deeper, to
> -see the OpenFlow flows that get generated from this datapath.  By
> -adding ``--ovs`` to the ``ovn-sbctl`` command, we can see OpenFlow
> -flows listed just below their logical flows.  We also need to use
> -``sudo`` because connecting to Open vSwitch is privileged.  Go ahead
> -and try it::
> -
> -  $ sudo ovn-sbctl --ovs lflow-list n1 | abbrev
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: ingress
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(eth.src[40]),
> action=(drop;)
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(vlan.present),
> action=(drop;)
> -  ...
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: egress
> -    table=0 (ls_out_pre_lb      ), priority=0    , match=(1),
> action=(next;)
> -    table=1 (ls_out_pre_acl     ), priority=0    , match=(1),
> action=(next;)
> -  ...
> -
> -You were probably disappointed: the output didn't change, and no
> -OpenFlow flows were printed.  That's because no OpenFlow flows are
> -installed for this logical datapath, which in turn is because there
> -are no VIFs for this logical datapath on the local hypervisor.  For a
> -better example, you can try ``ovn-sbctl --ovs`` on one of the other
> -logical datapaths.
> -
> -Attaching VMs
> -~~~~~~~~~~~~~
> -
> -A switch without any ports is not very interesting.  Let's create a
> -couple of VMs and attach them to the switch.  Run the following
> -commands, which create VMs named ``a`` and ``b`` and attaches them to
> -our network ``n1`` with IP addresses 10.1.1.5 and 10.1.1.6,
> -respectively.  It is not actually necessary to manually assign IP
> -address assignments, since OpenStack is perfectly happy to assign them
> -itself from the subnet's IP address range, but predictable addresses
> -are useful for our discussion::
> -
> -  $ openstack server create --nic net-id=n1,v4-fixed-ip=10.1.1.5 --flavor
> m1.nano --image $IMAGE_ID --key-name demo a
> -  $ openstack server create --nic net-id=n1,v4-fixed-ip=10.1.1.6 --flavor
> m1.nano --image $IMAGE_ID --key-name demo b
> -
> -These commands return before the VMs are really finished being built.
> -You can run ``openstack server list`` a few times until each of them
> -is shown in the state ACTIVE, which means that they're not just built
> -but already running on the local hypervisor.
> -
> -These operations had the side effect of creating separate "port"
> -objects, but without giving those ports any easy-to-read names.  It'll
> -be easier to deal with them later if we can refer to them by name, so
> -let's name ``a``'s port ``ap`` and ``b``'s port ``bp``::
> -
> -  $ openstack port set --name ap $(openstack port list --server a -f
> value -c ID)
> -  $ openstack port set --name bp $(openstack port list --server b -f
> value -c ID)
> -
> -We'll need to refer to these ports' MAC addresses a few times, so
> -let's put them in variables::
> -
> -  $ AP_MAC=$(openstack port show -f value -c mac_address ap)
> -  $ BP_MAC=$(openstack port show -f value -c mac_address bp)
> -
> -At this point you can log into the consoles of the VMs if you like.
> -You can do that from the OpenStack web interface or get a direct URL
> -to paste into a web browser using a command like::
> -
> -  $ openstack console url show -f yaml a
> -
> -(The option ``-f yaml`` keeps the URL in the output from being broken
> -into noncontiguous pieces on a 80-column console.)
> -
> -The VMs don't have many tools in them but ``ping`` and ``ssh`` from
> -one to the other should work fine.  The VMs do not have any external
> -network access or DNS configuration.
> -
> -Let's chase down what's changed in OVN.  Start with the NB DB at the
> -top of the system.  It's clear that our logical switch now has the two
> -logical ports attached to it::
> -
> -  $ ovn-nbctl show | abbrev
> -  ...
> -  switch 3eb263 (neutron-5b6baf) (aka n1)
> -      port c29d41 (aka bp)
> -          addresses: ["fa:16:3e:99:7a:17 10.1.1.6"]
> -      port 820c08 (aka ap)
> -          addresses: ["fa:16:3e:a9:4c:c7 10.1.1.5"]
> -  ...
> -
> -We can get some more details on each of these by looking at their NB
> -DB records in the Logical_Switch_Port table.  Each port has addressing
> -information, port security enabled, and a pointer to DHCP
> -configuration (which we'll look at much later in `DHCP`_)::
> -
> -  $ ovn-nbctl list logical_switch_port ap bp | abbrev
> -  _uuid               : ef17e5
> -  addresses           : ["fa:16:3e:a9:4c:c7 10.1.1.5"]
> -  dhcpv4_options      : 165974
> -  dhcpv6_options      : []
> -  dynamic_addresses   : []
> -  enabled             : true
> -  external_ids        : {"neutron:port_name"=ap}
> -  name                : "820c08"
> -  options             : {}
> -  parent_name         : []
> -  port_security       : ["fa:16:3e:a9:4c:c7 10.1.1.5"]
> -  tag                 : []
> -  tag_request         : []
> -  type                : ""
> -  up                  : true
> -
> -  _uuid               : e8af12
> -  addresses           : ["fa:16:3e:99:7a:17 10.1.1.6"]
> -  dhcpv4_options      : 165974
> -  dhcpv6_options      : []
> -  dynamic_addresses   : []
> -  enabled             : true
> -  external_ids        : {"neutron:port_name"=bp}
> -  name                : "c29d41"
> -  options             : {}
> -  parent_name         : []
> -  port_security       : ["fa:16:3e:99:7a:17 10.1.1.6"]
> -  tag                 : []
> -  tag_request         : []
> -  type                : ""
> -  up                  : true
> -
> -Now that the logical switch is less pathological, it's worth taking
> -another look at the SB DB logical flow table.  Try a command like
> -this::
> -
> -  $ ovn-sbctl lflow-list n1 | abbrev | less -S
> -
> -and then glance through the flows.  Packets that egress a VM into the
> -logical switch travel through the flow table's ingress pipeline
> -starting from table 0.  At each table, the switch finds the
> -highest-priority logical flow that matches and executes its actions,
> -or if there's no matching flow then the packet is dropped.  The
> -``ovn-sb``\(5) manpage gives all the details, but with a little
> -thought it's possible to guess a lot without reading the manpage.  For
> -example, consider the flows in ingress pipeline table 0, which are the
> -first flows encountered by a packet traversing the switch::
> -
> -  table=0 (ls_in_port_sec_l2  ), priority=100  , match=(eth.src[40]),
> action=(drop;)
> -  table=0 (ls_in_port_sec_l2  ), priority=100  , match=(vlan.present),
> action=(drop;)
> -  table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport ==
> "820c08" && eth.src == {fa:16:3e:a9:4c:c7}), action=(next;)
> -  table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport ==
> "c29d41" && eth.src == {fa:16:3e:99:7a:17}), action=(next;)
> -
> -The first two flows, with priority 100, immediately drop two kinds of
> -invalid packets: those with a multicast or broadcast Ethernet source
> -address (since multicast is only for packet destinations) and those
> -with a VLAN tag (because OVN doesn't yet support VLAN tags inside
> -logical networks).  The next two flows implement L2 port security:
> -they advance to the next table for packets with the correct Ethernet
> -source addresses for their ingress ports.  A packet that does not
> -match any flow is implicitly dropped, so there's no need for flows to
> -deal with mismatches.
> -
> -The logical flow table includes many other flows, some of which we
> -will look at later.  For now, it's most worth looking at ingress table
> -13::
> -
> -  table=13(ls_in_l2_lkup      ), priority=100  , match=(eth.mcast),
> action=(outport = "_MC_flood"; output;)
> -  table=13(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> fa:16:3e:99:7a:17), action=(outport = "c29d41"; output;)
> -  table=13(ls_in_l2_lkup      ), priority=50   , match=(eth.dst ==
> fa:16:3e:a9:4c:c7), action=(outport = "820c08"; output;)
> -
> -The first flow in table 13 checks whether the packet is an Ethernet
> -multicast or broadcast and, if so, outputs it to a special port that
> -egresses to every logical port (other than the ingress port).
> -Otherwise the packet is output to the port corresponding to its
> -Ethernet destination address.  Packets addressed to any other Ethernet
> -destination are implicitly dropped.
> -
> -(It's common for an OVN logical switch to know all the MAC addresses
> -supported by its logical ports, like this one.  That's why there's no
> -logic here for MAC learning or flooding packets to unknown MAC
> -addresses.  OVN does support unknown MAC handling but that's not in
> -play in our example.)
> -
> -.. note::
> -
> -  If you're interested in the details for the multicast group, you can
> -  run a command like the following and then look at the row for the
> -  correct datapath::
> -
> -    $ ovn-sbctl find multicast_group name=_MC_flood | abbrev
> -
> -Now if you want to look at the OpenFlow flows, you can actually see
> -them.  For example, here's the beginning of the output that lists the
> -first four logical flows, which we already looked at above, and their
> -corresponding OpenFlow flows.  If you want to know more about the
> -syntax, the ``ovs-fields``\(7) manpage explains OpenFlow matches and
> -``ovs-ofctl``\(8) explains OpenFlow actions::
> -
> -  $ sudo ovn-sbctl --ovs lflow-list n1 | abbrev
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: ingress
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(eth.src[40]),
> action=(drop;)
> -      table=8 metadata=0x4,dl_src=01:00:00:00:00:00/01:00:00:00:00:00
> actions=drop
> -    table=0 (ls_in_port_sec_l2  ), priority=100  , match=(vlan.present),
> action=(drop;)
> -      table=8 metadata=0x4,vlan_tci=0x1000/0x1000 actions=drop
> -    table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport ==
> "820c08" && eth.src == {fa:16:3e:a9:4c:c7}), action=(next;)
> -      table=8 reg14=0x1,metadata=0x4,dl_src=fa:16:3e:a9:4c:c7
> actions=resubmit(,9)
> -    table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport ==
> "c29d41" && eth.src == {fa:16:3e:99:7a:17}), action=(next;)
> -      table=8 reg14=0x2,metadata=0x4,dl_src=fa:16:3e:99:7a:17
> actions=resubmit(,9)
> -  ...
> -
> -Logical Tracing
> -+++++++++++++++
> -
> -Let's go a level deeper.  So far, everything we've done has been
> -fairly general.  We can also look at something more specific: the path
> -that a particular packet would take through OVN, logically, and Open
> -vSwitch, physically.
> -
> -Let's use OVN's ovn-trace utility to see what happens to packets from
> -a logical point of view.  The ``ovn-trace``\(8) manpage has a lot of
> -detail on how to do that, but let's just start by building up from a
> -simple example.  You can start with a command that just specifies the
> -logical datapath, an input port, and nothing else; unspecified fields
> -default to all-zeros.  This doesn't do much::
> -
> -  $ ovn-trace n1 'inport == "ap"'
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2: no match (implicit drop)
> -
> -We see that the packet was dropped in logical table 0,
> -"ls_in_port_sec_l2", the L2 port security stage (as we discussed
> -earlier).  That's because we didn't use the right Ethernet source
> -address for ``a``.  Let's see what happens if we do::
> -
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "ap" && eth.src ==
> {fa:16:3e:a9:4c:c7}, priority 50, uuid 6dcc418a
> -      next;
> -  13. ls_in_l2_lkup: no match (implicit drop)
> -
> -Now the packet passes through L2 port security and skips through
> -several other tables until it gets dropped in the L2 lookup stage
> -(because the destination is unknown).  Let's add the Ethernet
> -destination for ``b``::
> -
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC' && eth.dst ==
> '$BP_MAC
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "ap" && eth.src ==
> {fa:16:3e:a9:4c:c7}, priority 50, uuid 6dcc418a
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:99:7a:17,
> priority 50, uuid 57a4c46f
> -      outport = "bp";
> -      output;
> -
> -  egress(dp="n1", inport="ap", outport="bp")
> -  ------------------------------------------
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "bp" && eth.dst
> == {fa:16:3e:99:7a:17}, priority 50, uuid 8aa6426d
> -      output;
> -      /* output to "bp", type "" */
> -
> -You can see that in this case the packet gets properly switched from
> -``a`` to ``b``.
> -
> -Physical Tracing for Hypothetical Packets
> -+++++++++++++++++++++++++++++++++++++++++
> -
> -ovn-trace showed us how a hypothetical packet would travel through the
> -system in a logical fashion, that is, without regard to how VMs are
> -distributed across the physical network.  This is a convenient
> -representation for understanding how OVN is **supposed** to work
> -abstractly, but sometimes we might want to know more about how it
> -actually works in the real systems where it is running.  For this, we
> -can use the tracing tool that Open vSwitch provides, which traces
> -a hypothetical packet through the OpenFlow tables.
> -
> -We can actually get two levels of detail.  Let's start with the
> -version that's easier to interpret, by physically tracing a packet
> -that looks like the one we logically traced before.  One obstacle is
> -that we need to know the OpenFlow port number of the input port.  One
> -way to do that is to look for a port whose "attached-mac" is the one
> -we expect and print its ofport number::
> -
> -  $ AP_PORT=$(ovs-vsctl --bare --columns=ofport find  interface
> external-ids:attached-mac=\"$AP_MAC\")
> -  $ echo $AP_PORT
> -  3
> -
> -(You could also just do a plain ``ovs-vsctl list interface`` and then
> -look through for the right row and pick its ``ofport`` value.)
> -
> -Now we can feed this input port number into ``ovs-appctl
> -ofproto/trace`` along with the correct Ethernet source and
> -destination addresses and get a physical trace::
> -
> -  $ sudo ovs-appctl ofproto/trace br-int
> in_port=$AP_PORT,dl_src=$AP_MAC,dl_dst=$BP_MAC
> -  Flow:
> in_port=3,vlan_tci=0x0000,dl_src=fa:16:3e:a9:4c:c7,dl_dst=fa:16:3e:99:7a:17,dl_type=0x0000
> -
> -  bridge("br-int")
> -  ----------------
> -   0. in_port=3, priority 100
> -      set_field:0x8->reg13
> -      set_field:0x9->reg11
> -      set_field:0xa->reg12
> -      set_field:0x4->metadata
> -      set_field:0x1->reg14
> -      resubmit(,8)
> -   8. reg14=0x1,metadata=0x4,dl_src=fa:16:3e:a9:4c:c7, priority 50,
> cookie 0x6dcc418a
> -      resubmit(,9)
> -   9. metadata=0x4, priority 0, cookie 0x8fe8689e
> -      resubmit(,10)
> -  10. metadata=0x4, priority 0, cookie 0x719549d1
> -      resubmit(,11)
> -  11. metadata=0x4, priority 0, cookie 0x39c99e6f
> -      resubmit(,12)
> -  12. metadata=0x4, priority 0, cookie 0x838152a3
> -      resubmit(,13)
> -  13. metadata=0x4, priority 0, cookie 0x918259e3
> -      resubmit(,14)
> -  14. metadata=0x4, priority 0, cookie 0xcad14db2
> -      resubmit(,15)
> -  15. metadata=0x4, priority 0, cookie 0x7834d912
> -      resubmit(,16)
> -  16. metadata=0x4, priority 0, cookie 0x87745210
> -      resubmit(,17)
> -  17. metadata=0x4, priority 0, cookie 0x34951929
> -      resubmit(,18)
> -  18. metadata=0x4, priority 0, cookie 0xd7a8c9fb
> -      resubmit(,19)
> -  19. metadata=0x4, priority 0, cookie 0xd02e9578
> -      resubmit(,20)
> -  20. metadata=0x4, priority 0, cookie 0x42d35507
> -      resubmit(,21)
> -  21. metadata=0x4,dl_dst=fa:16:3e:99:7a:17, priority 50, cookie
> 0x57a4c46f
> -      set_field:0x2->reg15
> -      resubmit(,32)
> -  32. priority 0
> -      resubmit(,33)
> -  33. reg15=0x2,metadata=0x4, priority 100
> -      set_field:0xb->reg13
> -      set_field:0x9->reg11
> -      set_field:0xa->reg12
> -      resubmit(,34)
> -  34. priority 0
> -      set_field:0->reg0
> -      set_field:0->reg1
> -      set_field:0->reg2
> -      set_field:0->reg3
> -      set_field:0->reg4
> -      set_field:0->reg5
> -      set_field:0->reg6
> -      set_field:0->reg7
> -      set_field:0->reg8
> -      set_field:0->reg9
> -      resubmit(,40)
> -  40. metadata=0x4, priority 0, cookie 0xde9f3899
> -      resubmit(,41)
> -  41. metadata=0x4, priority 0, cookie 0x74074eff
> -      resubmit(,42)
> -  42. metadata=0x4, priority 0, cookie 0x7789c8b1
> -      resubmit(,43)
> -  43. metadata=0x4, priority 0, cookie 0xa6b002c0
> -      resubmit(,44)
> -  44. metadata=0x4, priority 0, cookie 0xaeab2b45
> -      resubmit(,45)
> -  45. metadata=0x4, priority 0, cookie 0x290cc4d4
> -      resubmit(,46)
> -  46. metadata=0x4, priority 0, cookie 0xa3223b88
> -      resubmit(,47)
> -  47. metadata=0x4, priority 0, cookie 0x7ac2132e
> -      resubmit(,48)
> -  48. reg15=0x2,metadata=0x4,dl_dst=fa:16:3e:99:7a:17, priority 50,
> cookie 0x8aa6426d
> -      resubmit(,64)
> -  64. priority 0
> -      resubmit(,65)
> -  65. reg15=0x2,metadata=0x4, priority 100
> -      output:4
> -
> -  Final flow:
> reg11=0x9,reg12=0xa,reg13=0xb,reg14=0x1,reg15=0x2,metadata=0x4,in_port=3,vlan_tci=0x0000,dl_src=fa:16:3e:a9:4c:c7,dl_dst=fa:16:3e:99:7a:17,dl_type=0x0000
> -  Megaflow:
> recirc_id=0,ct_state=-new-est-rel-rpl-inv-trk,ct_label=0/0x1,in_port=3,vlan_tci=0x0000/0x1000,dl_src=fa:16:3e:a9:4c:c7,dl_dst=fa:16:3e:99:7a:17,dl_type=0x0000
> -  Datapath actions: 4
> -
> -There's a lot there, which you can read through if you like, but the
> -important part is::
> -
> -  65. reg15=0x2,metadata=0x4, priority 100
> -      output:4
> -
> -which means that the packet is ultimately being output to OpenFlow
> -port 4.  That's port ``b``, which you can confirm with::
> -
> -  $ sudo ovs-vsctl find interface ofport=4
> -  _uuid               : 840a5aca-ea8d-4c16-a11b-a94e0f408091
> -  admin_state         : up
> -  bfd                 : {}
> -  bfd_status          : {}
> -  cfm_fault           : []
> -  cfm_fault_status    : []
> -  cfm_flap_count      : []
> -  cfm_health          : []
> -  cfm_mpid            : []
> -  cfm_remote_mpids    : []
> -  cfm_remote_opstate  : []
> -  duplex              : full
> -  error               : []
> -  external_ids        : {attached-mac="fa:16:3e:99:7a:17",
> iface-id="c29d4120-20a4-4c44-bd83-8d91f5f447fd", iface-status=active,
> vm-id="2db969ca-ca2a-4d9a-b49e-f287d39c5645"}
> -  ifindex             : 9
> -  ingress_policing_burst: 0
> -  ingress_policing_rate: 0
> -  lacp_current        : []
> -  link_resets         : 1
> -  link_speed          : 10000000
> -  link_state          : up
> -  lldp                : {}
> -  mac                 : []
> -  mac_in_use          : "fe:16:3e:99:7a:17"
> -  mtu                 : 1500
> -  mtu_request         : []
> -  name                : "tapc29d4120-20"
> -  ofport              : 4
> -  ofport_request      : []
> -  options             : {}
> -  other_config        : {}
> -  statistics          : {collisions=0, rx_bytes=4254, rx_crc_err=0,
> rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=39,
> tx_bytes=4188, tx_dropped=0, tx_errors=0, tx_packets=39}
> -  status              : {driver_name=tun, driver_version="1.6",
> firmware_version=""}
> -  type                : ""
> -
> -or::
> -
> -  $ BP_PORT=$(ovs-vsctl --bare --columns=ofport find  interface
> external-ids:attached-mac=\"$BP_MAC\")
> -  $ echo $BP_PORT
> -  4
> -
> -Physical Tracing for Real Packets
> -+++++++++++++++++++++++++++++++++
> -
> -In the previous sections we traced a hypothetical L2 packet, one
> -that's honestly not very realistic: we didn't even supply an Ethernet
> -type, so it defaulted to zero, which isn't anything one would see on a
> -real network.  We could refine our packet so that it becomes a more
> -realistic TCP or UDP or ICMP, etc. packet, but let's try a different
> -approach: working from a real packet.
> -
> -Pull up a console for VM ``a`` and start ``ping 10.1.1.6``, then leave
> -it running for the rest of our experiment.
> -
> -Now go back to your DevStack session and run::
> -
> -  $ sudo watch ovs-dpctl dump-flows
> -
> -We're working with a new program.  ovn-dpctl is an interface to Open
> -vSwitch datapaths, in this case to the Linux kernel datapath.  Its
> -``dump-flows`` command displays the contents of the in-kernel flow
> -cache, and by running it under the ``watch`` program we see a new
> -snapshot of the flow table every 2 seconds.
> -
> -Look through the output for a flow that begins with ``recirc_id(0)``
> -and matches the Ethernet source address for ``a``.  There is one flow
> -per line, but the lines are very long, so it's easier to read if you
> -make the window very wide.  This flow's packet counter should be
> -increasing at a rate of 1 packet per second.  It looks something like
> -this::
> -
> -
> recirc_id(0),in_port(3),eth(src=fa:16:3e:f5:2a:90),eth_type(0x0800),ipv4(src=10.1.1.5,frag=no),
> packets:388, bytes:38024, used:0.977s, actions:ct(zone=8),recirc(0x18)
> -
> -We can hand the first part of this (everything up to the first space)
> -to ``ofproto/trace``, and it will tell us what happens::
> -
> -  $ sudo ovs-appctl ofproto/trace
> 'recirc_id(0),in_port(3),eth(src=fa:16:3e:a9:4c:c7),eth_type(0x0800),ipv4(src=10.1.1.5,dst=
> 10.1.0.0/255.255.0.0,frag=no)'
> -  Flow:
> ip,in_port=3,vlan_tci=0x0000,dl_src=fa:16:3e:a9:4c:c7,dl_dst=00:00:00:00:00:00,nw_src=10.1.1.5,nw_dst=10.1.0.0,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
> -
> -  bridge("br-int")
> -  ----------------
> -   0. in_port=3, priority 100
> -      set_field:0x8->reg13
> -      set_field:0x9->reg11
> -      set_field:0xa->reg12
> -      set_field:0x4->metadata
> -      set_field:0x1->reg14
> -      resubmit(,8)
> -   8. reg14=0x1,metadata=0x4,dl_src=fa:16:3e:a9:4c:c7, priority 50,
> cookie 0x6dcc418a
> -      resubmit(,9)
> -   9. ip,reg14=0x1,metadata=0x4,dl_src=fa:16:3e:a9:4c:c7,nw_src=10.1.1.5,
> priority 90, cookie 0x343af48c
> -      resubmit(,10)
> -  10. metadata=0x4, priority 0, cookie 0x719549d1
> -      resubmit(,11)
> -  11. ip,metadata=0x4, priority 100, cookie 0x46c089e6
> -      load:0x1->NXM_NX_XXREG0[96]
> -      resubmit(,12)
> -  12. metadata=0x4, priority 0, cookie 0x838152a3
> -      resubmit(,13)
> -  13. ip,reg0=0x1/0x1,metadata=0x4, priority 100, cookie 0xd1941634
> -      ct(table=22,zone=NXM_NX_REG13[0..15])
> -      drop
> -
> -  Final flow:
> ip,reg0=0x1,reg11=0x9,reg12=0xa,reg13=0x8,reg14=0x1,metadata=0x4,in_port=3,vlan_tci=0x0000,dl_src=fa:16:3e:a9:4c:c7,dl_dst=00:00:00:00:00:00,nw_src=10.1.1.5,nw_dst=10.1.0.0,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
> -  Megaflow:
> recirc_id=0,ip,in_port=3,vlan_tci=0x0000/0x1000,dl_src=fa:16:3e:a9:4c:c7,nw_src=10.1.1.5,nw_dst=
> 10.1.0.0/16,nw_frag=no
> -  Datapath actions: ct(zone=8),recirc(0xb)
> -
> -.. note::
> -   Be careful cutting and pasting ``ovs-dpctl dump-flows`` output into
> -   ``ofproto/trace`` because the latter has terrible error reporting.
> -   If you add an extra line break, etc., it will likely give you a
> -   useless error message.
> -
> -There's no ``output`` action in the output, but there are ``ct`` and
> -``recirc`` actions (which you can see in the ``Datapath actions`` at
> -the end).  The ``ct`` action tells the kernel to pass the packet
> -through the kernel connection tracking for firewalling purposes and
> -the ``recirc`` says to go back to the flow cache for another pass
> -based on the firewall results.  The ``0xb`` value inside the
> -``recirc`` gives us a hint to look at the kernel flows for a cached
> -flow with ``recirc_id(0xb)``.  Indeed, there is one::
> -
> -
> recirc_id(0xb),in_port(3),ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:a9:4c:c7,dst=fa:16:3e:99:7a:17),eth_type(0x0800),ipv4(dst=
> 10.1.1.4/255.255.255.252,frag=no), packets:171, bytes:16758, used:0.271s,
> actions:ct(zone=11),recirc(0xc)
> -
> -We can then repeat our command with the match part of this kernel
> -flow::
> -
> -  $ sudo ovs-appctl ofproto/trace
> 'recirc_id(0xb),in_port(3),ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:a9:4c:c7,dst=fa:16:3e:99:7a:17),eth_type(0x0800),ipv4(dst=
> 10.1.1.4/255.255.255.252,frag=no)'
> -  ...
> -  Datapath actions: ct(zone=11),recirc(0xc)
> -
> -In other words, the flow passes through the connection tracker a
> -second time.  The first time was for ``a``'s outgoing firewall; this
> -second time is for ``b``'s incoming firewall.  Again, we continue
> -tracing with ``recirc_id(0xc)``::
> -
> -  $ sudo ovs-appctl ofproto/trace
> 'recirc_id(0xc),in_port(3),ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:a9:4c:c7,dst=fa:16:3e:99:7a:17),eth_type(0x0800),ipv4(dst=10.1.1.6,proto=1,frag=no)'
> -  ...
> -  Datapath actions: 4
> -
> -It was took multiple hops, but we finally came to the end of the line
> -where the packet was output to ``b`` after passing through both
> -firewalls.  The port number here is a datapath port number, which is
> -usually different from an OpenFlow port number.  To check that it is
> -``b``'s port, we first list the datapath ports to get the name
> -corresponding to the port number::
> -
> -  $ sudo ovs-dpctl show
> -  system at ovs-system:
> -          lookups: hit:1994 missed:56 lost:0
> -          flows: 6
> -          masks: hit:2340 total:4 hit/pkt:1.14
> -          port 0: ovs-system (internal)
> -          port 1: br-int (internal)
> -          port 2: br-ex (internal)
> -          port 3: tap820c0888-13
> -          port 4: tapc29d4120-20
> -
> -and then confirm that this is the port we think it is with a command
> -like this::
> -
> -  $ ovs-vsctl --columns=external-ids list interface tapc29d4120-20
> -  external_ids        : {attached-mac="fa:16:3e:99:7a:17",
> iface-id="c29d4120-20a4-4c44-bd83-8d91f5f447fd", iface-status=active,
> vm-id="2db969ca-ca2a-4d9a-b49e-f287d39c5645"}
> -
> -Finally, we can relate the OpenFlow flows from our traces back to OVN
> -logical flows.  For individual flows, cut and paste a "cookie" value
> -from ``ofproto/trace`` output into ``ovn-sbctl lflow-list``, e.g.::
> -
> -  $ ovn-sbctl lflow-list 0x6dcc418a|abbrev
> -  Datapath: "neutron-5b6baf" aka "n1" (a8a758)  Pipeline: ingress
> -    table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport ==
> "820c08" && eth.src == {fa:16:3e:a9:4c:c7}), action=(next;)
> -
> -Or, you can pipe ``ofproto/trace`` output through ``ovn-detrace`` to
> -annotate every flow::
> -
> -  $ sudo ovs-appctl ofproto/trace
> 'recirc_id(0xc),in_port(3),ct_state(-new+est-rel-rpl-inv+trk),ct_label(0/0x1),eth(src=fa:16:3e:a9:4c:c7,dst=fa:16:3e:99:7a:17),eth_type(0x0800),ipv4(dst=10.1.1.6,proto=1,frag=no)'
> | ovn-detrace
> -  ...
> -
> -Routing
> --------
> -
> -Previously we set up a pair of VMs ``a`` and ``b`` on a network ``n1``
> -and demonstrated how packets make their way between them.  In this
> -step, we'll set up a second network ``n2`` with a new VM ``c``,
> -connect a router ``r`` to both networks, and demonstrate how routing
> -works in OVN.
> -
> -There's nothing really new for the network and the VM so let's just go
> -ahead and create them::
> -
> -  $ openstack network create --project admin --provider-network-type
> geneve n2
> -  $ openstack subnet create --subnet-range 10.1.2.0/24 --network n2
> n2subnet
> -  $ openstack server create --nic net-id=n2,v4-fixed-ip=10.1.2.7 --flavor
> m1.nano --image $IMAGE_ID --key-name demo c
> -  $ openstack port set --name cp $(openstack port list --server c -f
> value -c ID)
> -  $ CP_MAC=$(openstack port show -f value -c mac_address cp)
> -
> -The new network ``n2`` is not yet connected to ``n1`` in any way.  You
> -can try tracing a broadcast packet from ``a`` to see, for example,
> -that it doesn't make it to ``c``::
> -
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC' && eth.dst ==
> '$CP_MAC
> -  ...
> -
> -Now create an OpenStack router and connect it to ``n1`` and ``n2``::
> -
> -  $ openstack router create r
> -  $ openstack router add subnet r n1subnet
> -  $ openstack router add subnet r n2subnet
> -
> -Now ``a``, ``b``, and ``c`` should all be able to reach other.  You
> -can get some verification that routing is taking place by running you
> -``ping`` between ``c`` and one of the other VMs: the reported TTL
> -should be one less than between ``a`` and ``b`` (63 instead of 64).
> -
> -Observe via ``ovn-nbctl`` the new OVN logical switch and router and
> -then ports that connect them together::
> -
> -  $ ovn-nbctl show|abbrev
> -  ...
> -  switch f51234 (neutron-332346) (aka n2)
> -      port 82b983
> -          type: router
> -          router-port: lrp-82b983
> -      port 2e585f (aka cp)
> -          addresses: ["fa:16:3e:89:f2:36 10.1.2.7"]
> -  switch 3eb263 (neutron-5b6baf) (aka n1)
> -      port c29d41 (aka bp)
> -          addresses: ["fa:16:3e:99:7a:17 10.1.1.6"]
> -      port 820c08 (aka ap)
> -          addresses: ["fa:16:3e:a9:4c:c7 10.1.1.5"]
> -      port 17d870
> -          type: router
> -          router-port: lrp-17d870
> -  ...
> -  router dde06c (neutron-f88ebc) (aka r)
> -      port lrp-82b983
> -          mac: "fa:16:3e:19:9f:46"
> -          networks: ["10.1.2.1/24"]
> -      port lrp-17d870
> -          mac: "fa:16:3e:f6:e2:8f"
> -          networks: ["10.1.1.1/24"]
> -
> -We have not yet looked at the logical flows for an OVN logical router.
> -You might find it of interest to look at them on your own::
> -
> -  $ ovn-sbctl lflow-list r | abbrev | less -S
> -  ...
> -
> -Let's grab the ``n1subnet`` router porter MAC address to simplify
> -later commands::
> -
> -  $ N1SUBNET_MAC=$(ovn-nbctl --bare --columns=mac find
> logical_router_port networks=10.1.1.1/24)
> -
> -Let's see what happens at the logical flow level for an ICMP packet
> -from ``a`` to ``c``.  This generates a long trace but an interesting
> -one, so we'll look at it bit by bit.  The first three stanzas in the
> -output show the packet's ingress into ``n1`` and processing through
> -the firewall on that side (via the "ct_next" connection-tracking
> -action), and then the selection of the port that leads to router ``r``
> -as the output port::
> -
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC' && eth.dst ==
> '$N1SUBNET_MAC' && ip4.src == 10.1.1.5 && ip4.dst == 10.1.2.7 && ip.ttl ==
> 64 && icmp4.type == 8'
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "ap" && eth.src ==
> {fa:16:3e:a9:4c:c7}, priority 50, uuid 6dcc418a
> -      next;
> -   1. ls_in_port_sec_ip (ovn-northd.c:2364): inport == "ap" && eth.src ==
> fa:16:3e:a9:4c:c7 && ip4.src == {10.1.1.5}, priority 90, uuid 343af48c
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2646): ip, priority 100, uuid 46c089e6
> -      reg0[0] = 1;
> -      next;
> -   5. ls_in_pre_stateful (ovn-northd.c:2764): reg0[0] == 1, priority 100,
> uuid d1941634
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   6. ls_in_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (inport == "ap" && ip4), priority 2002, uuid
> a12b39f0
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:f6:e2:8f,
> priority 50, uuid c43ead31
> -      outport = "17d870";
> -      output;
> -
> -  egress(dp="n1", inport="ap", outport="17d870")
> -  ----------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2626): ip && outport == "17d870",
> priority 110, uuid 60395450
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "17d870",
> priority 50, uuid 91b5cab0
> -      output;
> -      /* output to "17d870", type "patch" */
> -
> -The next two stanzas represent processing through logical router
> -``r``.  The processing in table 5 is the core of the routing
> -implementation: it recognizes that the packet is destined for an
> -attached subnet, decrements the TTL and updates the Ethernet source
> -address.  Table 6 then selects the Ethernet destination address based
> -on the IP destination.  The packet then passes to switch ``n2`` via an
> -OVN "logical patch port"::
> -
> -  ingress(dp="r", inport="lrp-17d870")
> -  ------------------------------------
> -   0. lr_in_admission (ovn-northd.c:4071): eth.dst == fa:16:3e:f6:e2:8f
> && inport == "lrp-17d870", priority 50, uuid fa5270b0
> -      next;
> -   5. lr_in_ip_routing (ovn-northd.c:3782): ip4.dst == 10.1.2.0/24,
> priority 49, uuid 5f9d469f
> -      ip.ttl--;
> -      reg0 = ip4.dst;
> -      reg1 = 10.1.2.1;
> -      eth.src = fa:16:3e:19:9f:46;
> -      outport = "lrp-82b983";
> -      flags.loopback = 1;
> -      next;
> -   6. lr_in_arp_resolve (ovn-northd.c:5088): outport == "lrp-82b983" &&
> reg0 == 10.1.2.7, priority 100, uuid 03d506d3
> -      eth.dst = fa:16:3e:89:f2:36;
> -      next;
> -   8. lr_in_arp_request (ovn-northd.c:5260): 1, priority 0, uuid 6dacdd82
> -      output;
> -
> -  egress(dp="r", inport="lrp-17d870", outport="lrp-82b983")
> -  ---------------------------------------------------------
> -   3. lr_out_delivery (ovn-northd.c:5288): outport == "lrp-82b983",
> priority 100, uuid 00bea4f2
> -      output;
> -      /* output to "lrp-82b983", type "patch" */
> -
> -Finally the logical switch for ``n2`` runs through the same logic as
> -``n1`` and the packet is delivered to VM ``c``::
> -
> -  ingress(dp="n2", inport="82b983")
> -  ---------------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "82b983", priority
> 50, uuid 9a789e06
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2624): ip && inport == "82b983",
> priority 110, uuid ab52f21a
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:89:f2:36,
> priority 50, uuid dcafb3e9
> -      outport = "cp";
> -      output;
> -
> -  egress(dp="n2", inport="82b983", outport="cp")
> -  ----------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2648): ip, priority 100, uuid cd9cfa74
> -      reg0[0] = 1;
> -      next;
> -   2. ls_out_pre_stateful (ovn-northd.c:2766): reg0[0] == 1, priority
> 100, uuid 9e8e22c5
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   4. ls_out_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (outport == "cp" && ip4 && ip4.src ==
> $as_ip4_0fc1b6cf_f925_49e6_8f00_6dd13beca9dc), priority 2002, uuid a746fa0d
> -      next;
> -   7. ls_out_port_sec_ip (ovn-northd.c:2364): outport == "cp" && eth.dst
> == fa:16:3e:89:f2:36 && ip4.dst == {255.255.255.255, 224.0.0.0/4,
> 10.1.2.7}, priority 90, uuid 4d9862b5
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "cp" && eth.dst
> == {fa:16:3e:89:f2:36}, priority 50, uuid 0242cdc3
> -      output;
> -      /* output to "cp", type "" */
> -
> -Physical Tracing
> -~~~~~~~~~~~~~~~~
> -
> -It's possible to use ``ofproto/trace``, just as before, to trace a
> -packet through OpenFlow tables, either for a hypothetical packet or
> -one that you get from a real test case using ``ovs-dpctl``.  The
> -process is just the same as before and the output is almost the same,
> -too.  Using a router doesn't actually introduce any interesting new
> -wrinkles, so we'll skip over this for this case and for the remainder
> -of the tutorial, but you can follow the steps on your own if you like.
> -
> -Adding a Gateway
> -----------------
> -
> -The VMs that we've created can access each other but they are isolated
> -from the physical world.  In OpenStack, the dominant way to connect a
> -VM to external networks is by creating what is called a "floating IP
> -address", which uses network address translation to connect an
> -external address to an internal one.
> -
> -DevStack created a pair of networks named "private" and "public".  To
> -use a floating IP address from a VM, we first add a port to the VM
> -with an IP address from the "private" network, then we create a
> -floating IP address on the "public" network, then we associate the
> -port with the floating IP address.
> -
> -Let's add a new VM ``d`` with a floating IP::
> -
> -  $ openstack server create --nic net-id=private --flavor m1.nano --image
> $IMAGE_ID --key-name demo d
> -  $ openstack port set --name dp $(openstack port list --server d -f
> value -c ID)
> -  $ DP_MAC=$(openstack port show -f value -c mac_address dp)
> -  $ openstack floating ip create --floating-ip-address 172.24.4.8 public
> -  $ openstack server add floating ip d 172.24.4.8
> -
> -(We specified a particular floating IP address to make the examples
> -easier to follow, but without that OpenStack will automatically
> -allocate one.)
> -
> -It's also necessary to configure the "public" network because DevStack
> -does not do it automatically::
> -
> -  $ sudo ip link set br-ex up
> -  $ sudo ip route add 172.24.4.0/24 dev br-ex
> -  $ sudo ip addr add 172.24.4.1/24 dev br-ex
> -
> -Now you should be able to "ping" VM ``d`` from the OpenStack host::
> -
> -  $ ping 172.24.4.8
> -  PING 172.24.4.8 (172.24.4.8) 56(84) bytes of data.
> -  64 bytes from 172.24.4.8: icmp_seq=1 ttl=63 time=56.0 ms
> -  64 bytes from 172.24.4.8: icmp_seq=2 ttl=63 time=1.44 ms
> -  64 bytes from 172.24.4.8: icmp_seq=3 ttl=63 time=1.04 ms
> -  64 bytes from 172.24.4.8: icmp_seq=4 ttl=63 time=0.403 ms
> -  ^C
> -  --- 172.24.4.8 ping statistics ---
> -  4 packets transmitted, 4 received, 0% packet loss, time 3003ms
> -  rtt min/avg/max/mdev = 0.403/14.731/56.028/23.845 ms
> -
> -You can also SSH in with the key that we created during setup::
> -
> -  $ ssh -i ~/id_rsa_demo cirros at 172.24.4.8
> -
> -Let's dive in and see how this gets implemented in OVN.  First, the
> -relevant parts of the NB DB for the "public" and "private" networks
> -and the router between them::
> -
> -  $ ovn-nbctl show | abbrev
> -  switch 2579f4 (neutron-d1ac28) (aka public)
> -      port provnet-d1ac28
> -          type: localnet
> -          addresses: ["unknown"]
> -      port ae9b52
> -          type: router
> -          router-port: lrp-ae9b52
> -  switch 5b3d5f (neutron-c02c4d) (aka private)
> -      port b256dd
> -          type: router
> -          router-port: lrp-b256dd
> -      port f264e7
> -          type: router
> -          router-port: lrp-f264e7
> -      port cae25b (aka dp)
> -          addresses: ["fa:16:3e:c1:f5:a2 10.0.0.6
> fdb0:5860:4ba8:0:f816:3eff:fec1:f5a2"]
> -  ...
> -  router c59ad2 (neutron-9b057f) (aka router1)
> -      port lrp-ae9b52
> -          mac: "fa:16:3e:b2:d2:67"
> -          networks: ["172.24.4.9/24", "2001:db8::b/64"]
> -      port lrp-b256dd
> -          mac: "fa:16:3e:35:33:db"
> -          networks: ["fdb0:5860:4ba8::1/64"]
> -      port lrp-f264e7
> -          mac: "fa:16:3e:fc:c8:da"
> -          networks: ["10.0.0.1/26"]
> -      nat 788c6d
> -          external ip: "172.24.4.8"
> -          logical ip: "10.0.0.6"
> -          type: "dnat_and_snat"
> -      nat 80914c
> -          external ip: "172.24.4.9"
> -          logical ip: "10.0.0.0/26"
> -          type: "snat"
> -  ...
> -
> -What we see is:
> -
> -* VM ``d`` is on the "private" switch under its private IP address
> -  10.0.0.8.  The "private" switch is connected to "router1" via two
> -  router ports (one for IPv4, one for IPv6).
> -
> -* The "public" switch is connected to "router1" and to the physical
> -  network via a "localnet" port.
> -
> -* "router1" is in the middle between "private" and "public".  In
> -  addition to the router ports that connect to these switches, it has
> -  "nat" entries that direct network address translation.  The
> -  translation between floating IP address 172.24.4.8 and private
> -  address 10.0.0.8 makes perfect sense.
> -
> -When the NB DB gets translated into logical flows at the southbound
> -layer, the "nat" entries get translated into IP matches that then
> -invoke "ct_snat" and "ct_dnat" actions.  The details are intricate,
> -but you can get some of the idea by just looking for relevant flows::
> -
> -  $ ovn-sbctl lflow-list router1 | abbrev | grep nat | grep -E
> '172.24.4.8|10.0.0.8'
> -    table=3 (lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst
> == 172.24.4.8 && inport == "lrp-ae9b52" &&
> is_chassis_resident("cr-lrp-ae9b52")), action=(ct_snat;)
> -    table=3 (lr_in_unsnat       ), priority=50   , match=(ip && ip4.dst
> == 172.24.4.8), action=(reg9[0] = 1; next;)
> -    table=4 (lr_in_dnat         ), priority=100  , match=(ip && ip4.dst
> == 172.24.4.8 && inport == "lrp-ae9b52" &&
> is_chassis_resident("cr-lrp-ae9b52")), action=(ct_dnat(10.0.0.6);)
> -    table=4 (lr_in_dnat         ), priority=50   , match=(ip && ip4.dst
> == 172.24.4.8), action=(reg9[0] = 1; next;)
> -    table=1 (lr_out_snat        ), priority=33   , match=(ip && ip4.src
> == 10.0.0.6 && outport == "lrp-ae9b52" &&
> is_chassis_resident("cr-lrp-ae9b52")), action=(ct_snat(172.24.4.8);)
> -
> -Let's take a look at how a packet passes through this whole gauntlet.
> -The first two stanzas just show the packet traveling through the
> -"public" network and being forwarded to the "router1" network::
> -
> -  $ ovn-trace public 'inport ==
> "provnet-d1ac2896-18a7-4bca-8f46-b21e2370e5b1" && eth.src ==
> 00:01:02:03:04:05 && eth.dst == fa:16:3e:b2:d2:67 && ip4.src == 172.24.4.1
> && ip4.dst == 172.24.4.8 && ip.ttl == 64 && icmp4.type==8'
> -  ...
> -  ingress(dp="public", inport="provnet-d1ac28")
> -  ---------------------------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "provnet-d1ac28",
> priority 50, uuid 8d86fb06
> -      next;
> -  10. ls_in_arp_rsp (ovn-northd.c:3266): inport == "provnet-d1ac28",
> priority 100, uuid 21313eff
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3571): eth.dst == fa:16:3e:b2:d2:67 &&
> is_chassis_resident("cr-lrp-ae9b52"), priority 50, uuid 7f28f51f
> -      outport = "ae9b52";
> -      output;
> -
> -  egress(dp="public", inport="provnet-d1ac28", outport="ae9b52")
> -  --------------------------------------------------------------
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "ae9b52",
> priority 50, uuid 72fea396
> -      output;
> -      /* output to "ae9b52", type "patch" */
> -
> -In "router1", first the ``ct_snat`` action without an argument
> -attempts to "un-SNAT" the packet.  ovn-trace treats this as a no-op,
> -because it doesn't have any state for tracking connections.  As an
> -alternative, it invokes ``ct_dnat(10.0.0.8)`` to NAT the destination
> -IP::
> -
> -  ingress(dp="router1", inport="lrp-ae9b52")
> -  ------------------------------------------
> -   0. lr_in_admission (ovn-northd.c:4071): eth.dst == fa:16:3e:b2:d2:67
> && inport == "lrp-ae9b52" && is_chassis_resident("cr-lrp-ae9b52"), priority
> 50, uuid 8c6945c2
> -      next;
> -   3. lr_in_unsnat (ovn-northd.c:4591): ip && ip4.dst == 172.24.4.8 &&
> inport == "lrp-ae9b52" && is_chassis_resident("cr-lrp-ae9b52"), priority
> 100, uuid e922f541
> -      ct_snat;
> -
> -  ct_snat /* assuming no un-snat entry, so no change */
> -  -----------------------------------------------------
> -   4. lr_in_dnat (ovn-northd.c:4649): ip && ip4.dst == 172.24.4.8 &&
> inport == "lrp-ae9b52" && is_chassis_resident("cr-lrp-ae9b52"), priority
> 100, uuid 02f41b79
> -      ct_dnat(10.0.0.6);
> -
> -Still in "router1", the routing and output steps transmit the packet
> -to the "private" network::
> -
> -  ct_dnat(ip4.dst=10.0.0.6)
> -  -------------------------
> -   5. lr_in_ip_routing (ovn-northd.c:3782): ip4.dst == 10.0.0.0/26,
> priority 53, uuid 86e005b0
> -      ip.ttl--;
> -      reg0 = ip4.dst;
> -      reg1 = 10.0.0.1;
> -      eth.src = fa:16:3e:fc:c8:da;
> -      outport = "lrp-f264e7";
> -      flags.loopback = 1;
> -      next;
> -   6. lr_in_arp_resolve (ovn-northd.c:5088): outport == "lrp-f264e7" &&
> reg0 == 10.0.0.6, priority 100, uuid 2963d67c
> -      eth.dst = fa:16:3e:c1:f5:a2;
> -      next;
> -   8. lr_in_arp_request (ovn-northd.c:5260): 1, priority 0, uuid eea419b7
> -      output;
> -
> -  egress(dp="router1", inport="lrp-ae9b52", outport="lrp-f264e7")
> -  ---------------------------------------------------------------
> -   3. lr_out_delivery (ovn-northd.c:5288): outport == "lrp-f264e7",
> priority 100, uuid 42dadc23
> -      output;
> -      /* output to "lrp-f264e7", type "patch" */
> -
> -In the "private" network, the packet passes through VM ``d``'s
> -firewall and is output to ``d``::
> -
> -  ingress(dp="private", inport="f264e7")
> -  --------------------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "f264e7", priority
> 50, uuid 5b721214
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2624): ip && inport == "f264e7",
> priority 110, uuid 5bdc3209
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:c1:f5:a2,
> priority 50, uuid 7957f80f
> -      outport = "dp";
> -      output;
> -
> -  egress(dp="private", inport="f264e7", outport="dp")
> -  ---------------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2648): ip, priority 100, uuid 4981c79d
> -      reg0[0] = 1;
> -      next;
> -   2. ls_out_pre_stateful (ovn-northd.c:2766): reg0[0] == 1, priority
> 100, uuid 247e02eb
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   4. ls_out_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (outport == "dp" && ip4 && ip4.src == 0.0.0.0/0
> && icmp4), priority 2002, uuid b860fc9f
> -      next;
> -   7. ls_out_port_sec_ip (ovn-northd.c:2364): outport == "dp" && eth.dst
> == fa:16:3e:c1:f5:a2 && ip4.dst == {255.255.255.255, 224.0.0.0/4,
> 10.0.0.6}, priority 90, uuid 15655a98
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "dp" && eth.dst
> == {fa:16:3e:c1:f5:a2}, priority 50, uuid 5916f94b
> -      output;
> -      /* output to "dp", type "" */
> -
> -IPv6
> -----
> -
> -OVN supports IPv6 logical routing.  Let's try it out.
> -
> -The first step is to add an IPv6 subnet to networks ``n1`` and ``n2``,
> -then attach those subnets to our router ``r``.  As usual, though
> -OpenStack can assign addresses itself, we use fixed ones to make the
> -discussion easier::
> -
> -  $ openstack subnet create --ip-version 6 --subnet-range fc11::/64
> --network n1 n1subnet6
> -  $ openstack subnet create --ip-version 6 --subnet-range fc22::/64
> --network n2 n2subnet6
> -  $ openstack router add subnet r n1subnet6
> -  $ openstack router add subnet r n2subnet6
> -
> -Then we add an IPv6 address to each of our VMs::
> -
> -  $ A_PORT_ID=$(openstack port list --server a -f value -c ID)
> -  $ openstack port set --fixed-ip subnet=n1subnet6,ip-address=fc11::5
> $A_PORT_ID
> -  $ B_PORT_ID=$(openstack port list --server b -f value -c ID)
> -  $ openstack port set --fixed-ip subnet=n1subnet6,ip-address=fc11::6
> $B_PORT_ID
> -  $ C_PORT_ID=$(openstack port list --server c -f value -c ID)
> -  $ openstack port set --fixed-ip subnet=n2subnet6,ip-address=fc22::7
> $C_PORT_ID
> -
> -At least for me, the new IPv6 addresses didn't automatically get
> -propagated into the VMs.  To do it by hand, pull up the console for
> -``a`` and run::
> -
> -  $ sudo ip addr add fc11::5/64 dev eth0
> -  $ sudo ip route add via fc11::1
> -
> -Then in ``b``::
> -
> -  $ sudo ip addr add fc11::6/64 dev eth0
> -  $ sudo ip route add via fc11::1
> -
> -Finally in ``c``::
> -
> -  $ sudo ip addr add fc22::7/64 dev eth0
> -  $ sudo ip route add via fc22::1
> -
> -Now you should have working IPv6 routing through router ``r``.  The
> -relevant parts of the NB DB look like the following.  The interesting
> -parts are the new ``fc11::`` and ``fc22::`` addresses on the ports in
> -``n1`` and ``n2`` and the new IPv6 router ports in ``r``::
> -
> -  $ ovn-nbctl show | abbrev
> -  ...
> -  switch f51234 (neutron-332346) (aka n2)
> -      port 1a8162
> -          type: router
> -          router-port: lrp-1a8162
> -      port 82b983
> -          type: router
> -          router-port: lrp-82b983
> -      port 2e585f (aka cp)
> -          addresses: ["fa:16:3e:89:f2:36 10.1.2.7 fc22::7"]
> -  switch 3eb263 (neutron-5b6baf) (aka n1)
> -      port ad952e
> -          type: router
> -          router-port: lrp-ad952e
> -      port c29d41 (aka bp)
> -          addresses: ["fa:16:3e:99:7a:17 10.1.1.6 fc11::6"]
> -      port 820c08 (aka ap)
> -          addresses: ["fa:16:3e:a9:4c:c7 10.1.1.5 fc11::5"]
> -      port 17d870
> -          type: router
> -          router-port: lrp-17d870
> -  ...
> -  router dde06c (neutron-f88ebc) (aka r)
> -      port lrp-1a8162
> -          mac: "fa:16:3e:06:de:ad"
> -          networks: ["fc22::1/64"]
> -      port lrp-82b983
> -          mac: "fa:16:3e:19:9f:46"
> -          networks: ["10.1.2.1/24"]
> -      port lrp-ad952e
> -          mac: "fa:16:3e:ef:2f:8b"
> -          networks: ["fc11::1/64"]
> -      port lrp-17d870
> -          mac: "fa:16:3e:f6:e2:8f"
> -          networks: ["10.1.1.1/24"]
> -
> -Try tracing a packet from ``a`` to ``c``.  The results correspond
> -closely to those for IPv4 which we already discussed back under
> -`Routing`_::
> -
> -  $ N1SUBNET6_MAC=$(ovn-nbctl --bare --columns=mac find
> logical_router_port networks=\"fc11::1/64\")
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC' && eth.dst ==
> '$N1SUBNET6_MAC' && ip6.src == fc11::5 && ip6.dst == fc22::7 && ip.ttl ==
> 64 && icmp6.type == 8'
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "ap" && eth.src ==
> {fa:16:3e:a9:4c:c7}, priority 50, uuid 6dcc418a
> -      next;
> -   1. ls_in_port_sec_ip (ovn-northd.c:2390): inport == "ap" && eth.src ==
> fa:16:3e:a9:4c:c7 && ip6.src == {fe80::f816:3eff:fea9:4cc7, fc11::5},
> priority 90, uuid 604810ea
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2646): ip, priority 100, uuid 46c089e6
> -      reg0[0] = 1;
> -      next;
> -   5. ls_in_pre_stateful (ovn-northd.c:2764): reg0[0] == 1, priority 100,
> uuid d1941634
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   6. ls_in_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (inport == "ap" && ip6), priority 2002, uuid
> 7fdd607e
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:ef:2f:8b,
> priority 50, uuid e1d87fc5
> -      outport = "ad952e";
> -      output;
> -
> -  egress(dp="n1", inport="ap", outport="ad952e")
> -  ----------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2626): ip && outport == "ad952e",
> priority 110, uuid 88f68988
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "ad952e",
> priority 50, uuid 5935755e
> -      output;
> -      /* output to "ad952e", type "patch" */
> -
> -  ingress(dp="r", inport="lrp-ad952e")
> -  ------------------------------------
> -   0. lr_in_admission (ovn-northd.c:4071): eth.dst == fa:16:3e:ef:2f:8b
> && inport == "lrp-ad952e", priority 50, uuid ddfeb712
> -      next;
> -   5. lr_in_ip_routing (ovn-northd.c:3782): ip6.dst == fc22::/64,
> priority 129, uuid cc2130ec
> -      ip.ttl--;
> -      xxreg0 = ip6.dst;
> -      xxreg1 = fc22::1;
> -      eth.src = fa:16:3e:06:de:ad;
> -      outport = "lrp-1a8162";
> -      flags.loopback = 1;
> -      next;
> -   6. lr_in_arp_resolve (ovn-northd.c:5122): outport == "lrp-1a8162" &&
> xxreg0 == fc22::7, priority 100, uuid bcf75288
> -      eth.dst = fa:16:3e:89:f2:36;
> -      next;
> -   8. lr_in_arp_request (ovn-northd.c:5260): 1, priority 0, uuid 6dacdd82
> -      output;
> -
> -  egress(dp="r", inport="lrp-ad952e", outport="lrp-1a8162")
> -  ---------------------------------------------------------
> -   3. lr_out_delivery (ovn-northd.c:5288): outport == "lrp-1a8162",
> priority 100, uuid 5260dfc5
> -      output;
> -      /* output to "lrp-1a8162", type "patch" */
> -
> -  ingress(dp="n2", inport="1a8162")
> -  ---------------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "1a8162", priority
> 50, uuid 10957d1b
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2624): ip && inport == "1a8162",
> priority 110, uuid a27ebd00
> -      next;
> -  13. ls_in_l2_lkup (ovn-northd.c:3529): eth.dst == fa:16:3e:89:f2:36,
> priority 50, uuid dcafb3e9
> -      outport = "cp";
> -      output;
> -
> -  egress(dp="n2", inport="1a8162", outport="cp")
> -  ----------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2648): ip, priority 100, uuid cd9cfa74
> -      reg0[0] = 1;
> -      next;
> -   2. ls_out_pre_stateful (ovn-northd.c:2766): reg0[0] == 1, priority
> 100, uuid 9e8e22c5
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   4. ls_out_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (outport == "cp" && ip6 && ip6.src ==
> $as_ip6_0fc1b6cf_f925_49e6_8f00_6dd13beca9dc), priority 2002, uuid 12fc96f9
> -      next;
> -   7. ls_out_port_sec_ip (ovn-northd.c:2390): outport == "cp" && eth.dst
> == fa:16:3e:89:f2:36 && ip6.dst == {fe80::f816:3eff:fe89:f236, ff00::/8,
> fc22::7}, priority 90, uuid c622596a
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "cp" && eth.dst
> == {fa:16:3e:89:f2:36}, priority 50, uuid 0242cdc3
> -      output;
> -      /* output to "cp", type "" */
> -
> -ACLs
> -----
> -
> -Let's explore how ACLs work in OpenStack and OVN.  In OpenStack, ACL
> -rules are part of "security groups", which are "default deny", that
> -is, packets are not allowed by default and the rules added to security
> -groups serve to allow different classes of packets.  The default group
> -(named "default") that is assigned to each of our VMs so far allows
> -all traffic from our other VMs, which isn't very interesting for
> -testing.  So, let's create a new security group, which we'll name
> -"custom", add rules to it that allow incoming SSH and ICMP traffic,
> -and apply this security group to VM ``c``::
> -
> -  $ openstack security group create custom
> -  $ openstack security group rule create --dst-port 22 custom
> -  $ openstack security group rule create --protocol icmp custom
> -  $ openstack server remove security group c default
> -  $ openstack server add security group c custom
> -
> -Now we can do some experiments to test security groups.  From the
> -console on ``a`` or ``b``, it should now be possible to "ping" ``c``
> -or to SSH to it, but attempts to initiate connections on other ports
> -should be blocked.  (You can try to connect on another port with
> -``ssh -p PORT IP`` or ``nc PORT IP``.)  Connection attempts should
> -time out rather than receive the "connection refused" or "connection
> -reset" error that you would see between ``a`` and ``b``.
> -
> -It's also possible to test ACLs via ovn-trace, with one new wrinkle.
> -ovn-trace can't simulate connection tracking state in the network, so
> -by default it assumes that every packet represents an established
> -connection.  That's good enough for what we've been doing so far, but
> -for checking properties of security groups we want to look at more
> -detail.
> -
> -If you look back at the VM-to-VM traces we've done until now, you can
> -see that they execute two ``ct_next`` actions:
> -
> -* The first of these is for the packet passing outward through the
> -  source VM's firewall.  We can tell ovn-trace to treat the packet as
> -  starting a new connection or adding to an established connection by
> -  adding a ``--ct`` option: ``--ct new`` or ``--ct est``,
> -  respectively.  The latter is the default and therefore what we've
> -  been using so far.  We can also use ``--ct est,rpl``, which in
> -  addition to ``--ct est`` means that the connection was initiated by
> -  the destination VM rather than by the VM sending this packet.
> -
> -* The second is for the packet passing inward through the destination
> -  VM's firewall.  For this one, it makes sense to tell ovn-trace that
> -  the packet is starting a new connection, with ``--ct new``, or that
> -  it is a packet sent in reply to a connection established by the
> -  destination VM, with ``--ct est,rpl``.
> -
> -ovn-trace uses the ``--ct`` options in order, so if we want to
> -override the second ``ct_next`` behavior we have to specify two
> -options.
> -
> -Another useful ovn-trace option for this testing is ``--minimal``,
> -which reduces the amount of output.  In this case we're really just
> -interested in finding out whether the packet reaches the destination
> -VM, that is, whether there's an eventual ``output`` action to ``c``,
> -so ``--minimal`` works fine and the output is easier to read.
> -
> -Try a few traces.  For example:
> -
> -* VM ``a`` initiates a new SSH connection to ``c``::
> -
> -    $ ovn-trace --ct new --ct new --minimal n1 'inport == "ap" && eth.src
> == '$AP_MAC' && eth.dst == '$N1SUBNET6_MAC' && ip4.src == 10.1.1.5 &&
> ip4.dst == 10.1.2.7 && ip.ttl == 64 && tcp.dst == 22'
> -    ...
> -    ct_next(ct_state=new|trk) {
> -        ip.ttl--;
> -        eth.src = fa:16:3e:19:9f:46;
> -        eth.dst = fa:16:3e:89:f2:36;
> -        ct_next(ct_state=new|trk) {
> -            output("cp");
> -        };
> -    };
> -
> -  This succeeds, as you can see since there is an ``output`` action.
> -
> -* VM ``a`` initiates a new Telnet connection to ``c``::
> -
> -    $ ovn-trace --ct new --ct new --minimal n1 'inport == "ap" && eth.src
> == '$AP_MAC' && eth.dst == '$N1SUBNET6_MAC' && ip4.src == 10.1.1.5 &&
> ip4.dst == 10.1.2.7 && ip.ttl == 64 && tcp.dst == 23'
> -    ct_next(ct_state=new|trk) {
> -        ip.ttl--;
> -        eth.src = fa:16:3e:19:9f:46;
> -        eth.dst = fa:16:3e:89:f2:36;
> -        ct_next(ct_state=new|trk);
> -    };
> -
> -  This fails, as you can see from the lack of an ``output`` action.
> -
> -* VM ``a`` replies to a packet that is part of a Telnet connection
> -  originally initiated by ``c``::
> -
> -    $ ovn-trace --ct est,rpl --ct est,rpl --minimal n1 'inport == "ap" &&
> eth.src == '$AP_MAC' && eth.dst == '$N1SUBNET6_MAC' && ip4.src == 10.1.1.5
> && ip4.dst == 10.1.2.7 && ip.ttl == 64 && tcp.dst == 23'
> -    ...
> -    ct_next(ct_state=est|rpl|trk) {
> -        ip.ttl--;
> -        eth.src = fa:16:3e:19:9f:46;
> -        eth.dst = fa:16:3e:89:f2:36;
> -        ct_next(ct_state=est|rpl|trk) {
> -            output("cp");
> -        };
> -    };
> -
> -  This succeeds, as you can see from the ``output`` action, since
> -  traffic received in reply to an outgoing connection is always
> -  allowed.
> -
> -DHCP
> -----
> -
> -As a final demonstration of the OVN architecture, let's examine the
> -DHCP implementation.  Like switching, routing, and NAT, the OVN
> -implementation of DHCP involves configuration in the NB DB and logical
> -flows in the SB DB.
> -
> -Let's look at the DHCP support for ``a``'s port ``ap``.  The port's
> -Logical_Switch_Port record shows that ``ap`` has DHCPv4 options::
> -
> -  $ ovn-nbctl list logical_switch_port ap | abbrev
> -  _uuid               : ef17e5
> -  addresses           : ["fa:16:3e:a9:4c:c7 10.1.1.5 fc11::5"]
> -  dhcpv4_options      : 165974
> -  dhcpv6_options      : 26f7cd
> -  dynamic_addresses   : []
> -  enabled             : true
> -  external_ids        : {"neutron:port_name"=ap}
> -  name                : "820c08"
> -  options             : {}
> -  parent_name         : []
> -  port_security       : ["fa:16:3e:a9:4c:c7 10.1.1.5 fc11::5"]
> -  tag                 : []
> -  tag_request         : []
> -  type                : ""
> -  up                  : true
> -
> -We can then list them either by UUID or, more easily, by port name::
> -
> -  $ ovn-nbctl list dhcp_options ap | abbrev
> -  _uuid               : 165974
> -  cidr                : "10.1.1.0/24"
> -  external_ids        : {subnet_id="5e67e7"}
> -  options             : {lease_time="43200", mtu="1442",
> router="10.1.1.1", server_id="10.1.1.1", server_mac="fa:16:3e:bb:94:72"}
> -
> -These options show the basic DHCP configuration for the subnet.  They
> -do not include the IP address itself, which comes from the
> -Logical_Switch_Port record.  This allows a whole Neutron subnet to
> -share a single DHCP_Options record.  You can see this sharing in
> -action, if you like, by listing the record for port ``bp``, which is
> -on the same subnet as ``ap``, and see that it is the same record as
> before::
> -
> -  $ ovn-nbctl list dhcp_options bp | abbrev
> -  _uuid               : 165974
> -  cidr                : "10.1.1.0/24"
> -  external_ids        : {subnet_id="5e67e7"}
> -  options             : {lease_time="43200", mtu="1442",
> router="10.1.1.1", server_id="10.1.1.1", server_mac="fa:16:3e:bb:94:72"}
> -
> -You can take another look at the southbound flow table if you like,
> -but the best demonstration is to trace a DHCP packet.  The following
> -is a trace of a DHCP request inbound from ``ap``.  The first part is
> -just the usual travel through the firewall::
> -
> -  $ ovn-trace n1 'inport == "ap" && eth.src == '$AP_MAC' && eth.dst ==
> ff:ff:ff:ff:ff:ff && ip4.dst == 255.255.255.255 && udp.src == 68 && udp.dst
> == 67 && ip.ttl == 1'
> -  ...
> -  ingress(dp="n1", inport="ap")
> -  -----------------------------
> -   0. ls_in_port_sec_l2 (ovn-northd.c:3234): inport == "ap" && eth.src ==
> {fa:16:3e:a9:4c:c7}, priority 50, uuid 6dcc418a
> -      next;
> -   1. ls_in_port_sec_ip (ovn-northd.c:2325): inport == "ap" && eth.src ==
> fa:16:3e:a9:4c:c7 && ip4.src == 0.0.0.0 && ip4.dst == 255.255.255.255 &&
> udp.src == 68 && udp.dst == 67, priority 90, uuid e46bed6f
> -      next;
> -   3. ls_in_pre_acl (ovn-northd.c:2646): ip, priority 100, uuid 46c089e6
> -      reg0[0] = 1;
> -      next;
> -   5. ls_in_pre_stateful (ovn-northd.c:2764): reg0[0] == 1, priority 100,
> uuid d1941634
> -      ct_next;
> -
> -The next part is the new part.  First, an ACL in table 6 allows a DHCP
> -request to pass through.  In table 11, the special ``put_dhcp_opts``
> -action replaces a DHCPDISCOVER or DHCPREQUEST packet by a
> -reply.  Table 12 flips the packet's source and destination and sends
> -it back the way it came in::
> -
> -   6. ls_in_acl (ovn-northd.c:2925): !ct.new && ct.est && !ct.rpl &&
> ct_label.blocked == 0 && (inport == "ap" && ip4 && ip4.dst ==
> {255.255.255.255, 10.1.1.0/24} && udp && udp.src == 68 && udp.dst == 67),
> priority 2002, uuid 9c90245d
> -      next;
> -  11. ls_in_dhcp_options (ovn-northd.c:3409): inport == "ap" && eth.src
> == fa:16:3e:a9:4c:c7 && ip4.src == 0.0.0.0 && ip4.dst == 255.255.255.255 &&
> udp.src == 68 && udp.dst == 67, priority 100, uuid 8d63f29c
> -      reg0[3] = put_dhcp_opts(offerip = 10.1.1.5, lease_time = 43200, mtu
> = 1442, netmask = 255.255.255.0, router = 10.1.1.1, server_id = 10.1.1.1);
> -      /* We assume that this packet is DHCPDISCOVER or DHCPREQUEST. */
> -      next;
> -  12. ls_in_dhcp_response (ovn-northd.c:3438): inport == "ap" && eth.src
> == fa:16:3e:a9:4c:c7 && ip4 && udp.src == 68 && udp.dst == 67 && reg0[3],
> priority 100, uuid 995eeaa9
> -      eth.dst = eth.src;
> -      eth.src = fa:16:3e:bb:94:72;
> -      ip4.dst = 10.1.1.5;
> -      ip4.src = 10.1.1.1;
> -      udp.src = 67;
> -      udp.dst = 68;
> -      outport = inport;
> -      flags.loopback = 1;
> -      output;
> -
> -Then the last part is just traveling back through the firewall to VM
> -``a``::
> -
> -  egress(dp="n1", inport="ap", outport="ap")
> -  ------------------------------------------
> -   1. ls_out_pre_acl (ovn-northd.c:2648): ip, priority 100, uuid 3752b746
> -      reg0[0] = 1;
> -      next;
> -   2. ls_out_pre_stateful (ovn-northd.c:2766): reg0[0] == 1, priority
> 100, uuid 0c066ea1
> -      ct_next;
> -
> -  ct_next(ct_state=est|trk /* default (use --ct to customize) */)
> -  ---------------------------------------------------------------
> -   4. ls_out_acl (ovn-northd.c:3008): outport == "ap" && eth.src ==
> fa:16:3e:bb:94:72 && ip4.src == 10.1.1.1 && udp && udp.src == 67 && udp.dst
> == 68, priority 34000, uuid 0b383e77
> -      ct_commit;
> -      next;
> -   7. ls_out_port_sec_ip (ovn-northd.c:2364): outport == "ap" && eth.dst
> == fa:16:3e:a9:4c:c7 && ip4.dst == {255.255.255.255, 224.0.0.0/4,
> 10.1.1.5}, priority 90, uuid 7b8cbcd5
> -      next;
> -   8. ls_out_port_sec_l2 (ovn-northd.c:3654): outport == "ap" && eth.dst
> == {fa:16:3e:a9:4c:c7}, priority 50, uuid b874ece8
> -      output;
> -      /* output to "ap", type "" */
> -
> -Further Directions
> -------------------
> -
> -We've looked at a fair bit of how OVN works and how it interacts with
> -OpenStack.  If you still have some interest, then you might want to
> -explore some of these directions:
> -
> -* Adding more than one hypervisor ("compute node", in OpenStack
> -  parlance).  OVN connects compute nodes by tunneling packets with the
> -  STT or Geneve protocols.  OVN scales to 1000 compute nodes or more,
> -  but two compute nodes demonstrate the principle.  All of the tools
> -  and techniques we demonstrated also work with multiple compute
> -  nodes.
> -
> -* Container support.  OVN supports seamlessly connecting VMs to
> -  containers, whether the containers are hosted on "bare metal" or
> -  nested inside VMs.  OpenStack support for containers, however, is
> -  still evolving, and too difficult to incorporate into the tutorial
> -  at this point.
> -
> -* Other kinds of gateways.  In addition to floating IPs with NAT, OVN
> -  supports directly attaching VMs to a physical network and connecting
> -  logical switches to VTEP hardware.
> diff --git a/Documentation/tutorials/ovn-rbac.rst
> b/Documentation/tutorials/ovn-rbac.rst
> deleted file mode 100644
> index ec163e2df..000000000
> --- a/Documentation/tutorials/ovn-rbac.rst
> +++ /dev/null
> @@ -1,134 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -=============================================
> -OVN Role-Based Access Control (RBAC) Tutorial
> -=============================================
> -
> -This document provides a step-by-step guide for setting up role-based
> access
> -control (RBAC) in OVN. In OVN, hypervisors (chassis) read and write the
> -southbound database to do configuration. Without restricting hypervisor's
> -access to the southbound database, a compromised hypervisor might disrupt
> the
> -entire OVN deployment by corrupting the database. RBAC ensures that each
> -hypervisor can only modify its own data and thus improves the security of
> OVN.
> -More details about the RBAC design can be found in
> ``ovn-architecture``\(7)
> -manpage.
> -
> -This document assumes OVN is installed in your system and runs normally.
> -
> -.. _gen-certs-keys:
> -
> -Generating Certificates and Keys
> ---------------------------------
> -
> -In the OVN RBAC deployment, ovn-controller connects to the southbound
> database
> -via SSL connection. The southbound database uses CA-signed certificate to
> -authenticate ovn-controller.
> -
> -Suppose there are three machines in your deployment. `machine_1` runs
> -`chassis_1` and has IP address `machine_1-ip`. `machine_2` runs
> `chassis_2` and
> -has IP address `machine_2-ip`. `machine_3` hosts southbound database and
> has IP
> -address `machine_3-ip`. `machine_3` also hosts public key infrastructure
> (PKI).
> -
> -1. Initiate PKI.
> -
> -   In `machine_3`::
> -
> -      $ ovs-pki init
> -
> -2. Generate southbound database's certificate request. Sign the
> certificate
> -   request with the CA key.
> -
> -   In `machine_3`::
> -
> -      $ ovs-pki req -u sbdb
> -      $ ovs-pki sign sbdb switch
> -
> -3. Generate chassis certificate requests. Copy the certificate requests to
> -   `machine_3`.
> -
> -   In `machine_1`::
> -
> -      $ ovs-pki req -u chassis_1
> -      $ scp chassis_1-req.pem \
> -                    machine_3 at machine_3-ip:/path/to/chassis_1-req.pem
> -
> -   In `machine_2`::
> -
> -      $ ovs-pki req -u chassis_2
> -      $ scp chassis_2-req.pem \
> -                    machine_3 at machine_3-ip:/path/to/chassis_2-req.pem
> -
> -   .. note::
> -
> -     chassis_1 must be the same string as ``external_ids:system-id`` in
> the
> -     Open_vSwitch table (the chassis name) of machine_1. Same applies for
> -     chassis_2.
> -
> -4. Sign the chassis certificate requests with the CA key. Copy
> `chassis_1`'s
> -   signed certificate and the CA certificate to `machine_1`. Copy
> `chassis_2`'s
> -   signed certificate and the CA certificate to `machine_2`.
> -
> -   In `machine_3`::
> -
> -      $ ovs-pki sign chassis_1 switch
> -      $ ovs-pki sign chassis_2 switch
> -      $ scp chassis_1-cert.pem \
> -                    machine_1 at machine_1-ip:/path/to/chassis_1-cert.pem
> -      $ scp /var/lib/openvswitch/pki/switchca/cacert.pem \
> -                    machine_1 at machine_1-ip:/path/to/cacert.pem
> -      $ scp chassis_2-cert.pem \
> -                    machine_2 at machine_2-ip:/path/to/chassis_2-cert.pem
> -      $ scp /var/lib/openvswitch/pki/switchca/cacert.pem \
> -                    machine_2 at machine_2-ip:/path/to/cacert.pem
> -
> -Configuring RBAC
> -----------------
> -
> -1. Set certificate, private key, and CA certificate for the southbound
> -   database. Configure the southbound database to listen on SSL
> connection and
> -   enforce role-based access control.
> -
> -   In `machine_3`::
> -
> -      $ ovn-sbctl set-ssl /path/to/sbdb-privkey.pem \
> -                          /path/to/sbdb-cert.pem /path/to/cacert.pem
> -      $ ovn-sbctl set-connection role=ovn-controller pssl:6642
> -
> -2. Set certificate, private key, and CA certificate for `chassis_1` and
> -   `chassis_2`. Configure `chassis_1` and `chassis_2` to connect
> southbound
> -   database via SSL.
> -
> -   In `machine_1`::
> -
> -      $ ovs-vsctl set-ssl /path/to/chassis_1-privkey.pem \
> -                    /path/to/chassis_1-cert.pem /path/to/cacert.pem
> -      $ ovs-vsctl set open_vswitch . \
> -                    external_ids:ovn-remote=ssl:machine_3-ip:6642
> -
> -   In `machine_2`::
> -
> -      $ ovs-vsctl set-ssl /path/to/chassis_2-privkey.pem \
> -                    /path/to/chassis_2-cert.pem /path/to/cacert.pem
> -      $ ovs-vsctl set open_vswitch . \
> -                    external_ids:ovn-remote=ssl:machine_3-ip:6642
> diff --git a/Documentation/tutorials/ovn-sandbox.rst
> b/Documentation/tutorials/ovn-sandbox.rst
> deleted file mode 100644
> index b906b799d..000000000
> --- a/Documentation/tutorials/ovn-sandbox.rst
> +++ /dev/null
> @@ -1,177 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -===========
> -OVN Sandbox
> -===========
> -
> -This tutorial shows you how to explore features using ``ovs-sandbox`` as a
> -simulated test environment.  It's assumed that you have an understanding
> of OVS
> -before going through this tutorial. Detail about OVN is covered in
> -ovn-architecture_, but this tutorial lets you quickly see it in action.
> -
> -Getting Started
> ----------------
> -
> -For some general information about ``ovs-sandbox``, see the "Getting
> Started"
> -section of :doc:`ovs-advanced`.
> -
> -``ovs-sandbox`` does not include OVN support by default.  To enable OVN,
> you
> -must pass the ``--ovn`` flag.  For example, if running it straight from
> the OVS
> -git tree you would run::
> -
> -    $ make sandbox SANDBOXFLAGS="--ovn"
> -
> -Running the sandbox with OVN enabled does the following additional steps
> to the
> -environment:
> -
> -1. Creates the ``OVN_Northbound`` and ``OVN_Southbound`` databases as
> described in
> -   `ovn-nb(5)`_ and `ovn-sb(5)`_.
> -
> -2. Creates a backup server for ``OVN_Southbond`` database. Sandbox launch
> -   screen provides the instructions on accessing the backup database.
> However
> -   access to the backup server is not required to go through the tutorial.
> -
> -3. Creates the ``hardware_vtep`` database as described in `vtep(5)`_.
> -
> -4. Runs the `ovn-northd(8)`_, `ovn-controller(8)`_, and
> -   `ovn-controller-vtep(8)`_ daemons.
> -
> -5. Makes OVN and VTEP utilities available for use in the environment,
> including
> -   `vtep-ctl(8)`_, `ovn-nbctl(8)`_, and `ovn-sbctl(8)`_.
> -
> -Using GDB
> ----------
> -
> -GDB support is not required to go through the tutorial. See the "Using
> GDB"
> -section of :doc:`ovs-advanced` for more info. Additional flags exist for
> -launching the debugger for the OVN programs::
> -
> -    --gdb-ovn-northd
> -    --gdb-ovn-controller
> -    --gdb-ovn-controller-vtep
> -
> -Creating OVN Resources
> -----------------------
> -
> -Once you have ``ovs-sandbox`` running with OVN enabled, you can start
> using OVN
> -utilities to create resources in OVN.  As an example, we will create an
> -environment that has two logical switches connected by a logical router.
> -
> -Create the first logical switch with one port::
> -
> -    $ ovn-nbctl ls-add sw0
> -    $ ovn-nbctl lsp-add sw0 sw0-port1
> -    $ ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01
> 192.168.0.2"
> -
> -Create the second logical switch with one port::
> -
> -    $ ovn-nbctl ls-add sw1
> -    $ ovn-nbctl lsp-add sw1 sw1-port1
> -    $ ovn-nbctl lsp-set-addresses sw1-port1 "50:54:00:00:00:03 11.0.0.2"
> -
> -Create the logical router and attach both logical switches::
> -
> -    $ ovn-nbctl lr-add lr0
> -    $ ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:ff:01 192.168.0.1/24
> -    $ ovn-nbctl lsp-add sw0 lrp0-attachment
> -    $ ovn-nbctl lsp-set-type lrp0-attachment router
> -    $ ovn-nbctl lsp-set-addresses lrp0-attachment 00:00:00:00:ff:01
> -    $ ovn-nbctl lsp-set-options lrp0-attachment router-port=lrp0
> -    $ ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:ff:02 11.0.0.1/24
> -    $ ovn-nbctl lsp-add sw1 lrp1-attachment
> -    $ ovn-nbctl lsp-set-type lrp1-attachment router
> -    $ ovn-nbctl lsp-set-addresses lrp1-attachment 00:00:00:00:ff:02
> -    $ ovn-nbctl lsp-set-options lrp1-attachment router-port=lrp1
> -
> -View a summary of OVN's current logical configuration::
> -
> -    $ ovn-nbctl show
> -        switch 1396cf55-d176-4082-9a55-1c06cef626e4 (sw1)
> -            port lrp1-attachment
> -                addresses: ["00:00:00:00:ff:02"]
> -            port sw1-port1
> -                addresses: ["50:54:00:00:00:03 11.0.0.2"]
> -        switch 2c9d6d03-09fc-4e32-8da6-305f129b0d53 (sw0)
> -            port lrp0-attachment
> -                addresses: ["00:00:00:00:ff:01"]
> -            port sw0-port1
> -                addresses: ["50:54:00:00:00:01 192.168.0.2"]
> -        router f8377e8c-f75e-4fc8-8751-f3ea03c6dd98 (lr0)
> -            port lrp0
> -                mac: "00:00:00:00:ff:01"
> -                networks: ["192.168.0.1/24"]
> -            port lrp1
> -                mac: "00:00:00:00:ff:02"
> -                networks: ["11.0.0.1/24"]
> -
> -The ``tutorial`` directory of the OVS source tree includes a script
> -that runs all of the commands for you::
> -
> -    $ ./ovn-setup.sh
> -
> -Using ovn-trace
> ----------------
> -
> -Once you have configured resources in OVN, try using ``ovn-trace`` to see
> -how OVN would process a sample packet through its logical pipeline.
> -
> -For example, we can trace an IP packet from ``sw0-port1`` to
> ``sw1-port1``.
> -The ``--minimal`` output shows each visible action performed on the
> packet,
> -which includes:
> -
> -#. The logical router will decrement the IP TTL field.
> -#. The logical router will change the source and destination
> -   MAC addresses to reflect the next hop.
> -#. The packet will be output to ``sw1-port1``.
> -
> -::
> -
> -    $ ovn-trace --minimal sw0 'inport == "sw0-port1" \
> -    > && eth.src == 50:54:00:00:00:01 && ip4.src == 192.168.0.2 \
> -    > && eth.dst == 00:00:00:00:ff:01 && ip4.dst == 11.0.0.2 \
> -    > && ip.ttl == 64'
> -
> -    #
> ip,reg14=0x1,vlan_tci=0x0000,dl_src=50:54:00:00:00:01,dl_dst=00:00:00:00:ff:01,nw_src=192.168.0.2,nw_dst=11.0.0.2,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=64
> -    ip.ttl--;
> -    eth.src = 00:00:00:00:ff:02;
> -    eth.dst = 50:54:00:00:00:03;
> -    output("sw1-port1");
> -
> -The ``ovn-trace`` utility can also provide much more detail on how the
> packet
> -would be processed through OVN's logical pipeline, as well as correlate
> that
> -to OpenFlow flows programmed by ``ovn-controller``.  See the
> `ovn-trace(8)`_
> -man page for more detail.
> -
> -
> -.. _ovn-architecture:
> http://openvswitch.org/support/dist-docs/ovn-architecture.7.html
> -.. _ovn-nb(5): http://openvswitch.org/support/dist-docs/ovn-nb.5.html
> -.. _ovn-sb(5): http://openvswitch.org/support/dist-docs/ovn-sb.5.html
> -.. _vtep(5): http://openvswitch.org/support/dist-docs/vtep.5.html
> -.. _ovn-northd(8):
> http://openvswitch.org/support/dist-docs/ovn-northd.8.html
> -.. _ovn-controller(8):
> http://openvswitch.org/support/dist-docs/ovn-controller.8.html
> -.. _ovn-controller-vtep(8):
> http://openvswitch.org/support/dist-docs/ovn-controller-vtep.8.html
> -.. _vtep-ctl(8): http://openvswitch.org/support/dist-docs/vtep-ctl.8.html
> -.. _ovn-nbctl(8):
> http://openvswitch.org/support/dist-docs/ovn-nbctl.8.html
> -.. _ovn-sbctl(8):
> http://openvswitch.org/support/dist-docs/ovn-sbctl.8.html
> -.. _ovn-trace(8):
> http://openvswitch.org/support/dist-docs/ovn-trace.8.html
> diff --git a/Makefile.am b/Makefile.am
> index ff1f94b48..a03a0351a 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -495,6 +495,5 @@ include vtep/automake.mk
>  include datapath-windows/automake.mk
>  include datapath-windows/include/automake.mk
>  include windows/automake.mk
> -include ovn/automake.mk
>  include selinux/automake.mk
>  include build-aux/automake.mk
> diff --git a/NEWS b/NEWS
> index c5caa13d6..289d1e4f9 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,6 +1,9 @@
>  Post-v2.12.0
>  ---------------------
> -
> +   - OVN:
> +     * OVN has been removed from this repository. It now exists as a
> +       separate project. You can find it at
> +       https://github.com/ovn-org/ovn.git
>
>  v2.12.0 - xx xxx xxxx
>  ---------------------
> diff --git a/configure.ac b/configure.ac
> index e3603926b..64ce4cef0 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -148,7 +148,6 @@ AC_CONFIG_FILES([
>      ofproto/libofproto.sym
>      lib/libsflow.sym
>      lib/libopenvswitch.sym
> -    ovn/lib/libovn.sym
>      vtep/libvtep.sym])
>
>  OVS_ENABLE_OPTION([-Wall])
> @@ -210,8 +209,6 @@ dnl This makes sure that include/openflow gets created
> in the build directory.
>  AC_CONFIG_COMMANDS([include/openflow/openflow.h.stamp])
>
>  AC_CONFIG_COMMANDS([utilities/bugtool/dummy], [:])
> -AC_CONFIG_COMMANDS([ovn/dummy], [:])
> -AC_CONFIG_COMMANDS([ovn/utilities/dummy], [:])
>  AC_CONFIG_COMMANDS([ipsec/dummy], [:])
>
>  m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES])
> diff --git a/debian/.gitignore b/debian/.gitignore
> index 9ec70eb9c..441a8ffb7 100644
> --- a/debian/.gitignore
> +++ b/debian/.gitignore
> @@ -22,10 +22,5 @@
>  /openvswitch-test
>  /openvswitch-testcontroller
>  /openvswitch-vtep
> -/ovn-common
> -/ovn-controller-vtep
> -/ovn-host
> -/ovn-central
> -/ovn-docker
>  /python-openvswitch
>  /tmp
> diff --git a/debian/automake.mk b/debian/automake.mk
> index 8a8d43c9f..03a1d68c2 100644
> --- a/debian/automake.mk
> +++ b/debian/automake.mk
> @@ -52,28 +52,6 @@ EXTRA_DIST += \
>         debian/openvswitch-vtep.init \
>         debian/openvswitch-vtep.install \
>         debian/openvswitch-vtep.manpages \
> -       debian/ovn-central.dirs \
> -       debian/ovn-central.init \
> -       debian/ovn-central.install \
> -       debian/ovn-central.manpages \
> -       debian/ovn-central.postinst \
> -       debian/ovn-central.postrm \
> -       debian/ovn-central.template \
> -       debian/ovn-controller-vtep.init \
> -       debian/ovn-controller-vtep.install \
> -       debian/ovn-controller-vtep.manpages \
> -       debian/ovn-common.install \
> -       debian/ovn-common.manpages \
> -       debian/ovn-common.postinst \
> -       debian/ovn-common.postrm \
> -       debian/ovn-docker.install \
> -       debian/ovn-host.dirs \
> -       debian/ovn-host.init \
> -       debian/ovn-host.install \
> -       debian/ovn-host.manpages \
> -       debian/ovn-host.postinst \
> -       debian/ovn-host.postrm \
> -       debian/ovn-host.template \
>         debian/python-openvswitch.dirs \
>         debian/python-openvswitch.install \
>         debian/rules \
> diff --git a/debian/control b/debian/control
> index b97e99b92..2ad35f2ce 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -119,84 +119,6 @@ Description: Open vSwitch switch implementations
>   openvswitch-switch provides the userspace components and utilities for
>   the Open vSwitch kernel-based switch.
>
> -Package: ovn-common
> -Architecture: linux-any
> -Depends: openvswitch-common (= ${binary:Version}),
> -         ${misc:Depends},
> -         ${shlibs:Depends}
> -Description: OVN common components
> - OVN, the Open Virtual Network, is a system to support virtual network
> - abstraction.  OVN complements the existing capabilities of OVS to add
> - native support for virtual network abstractions, such as virtual L2 and
> L3
> - overlays and security groups.
> - .
> - ovn-common provides components required by other OVN packages.
> -
> -Package: ovn-controller-vtep
> -Architecture: linux-any
> -Depends: ovn-common (= ${binary:Version}),
> -         ${misc:Depends},
> -         ${shlibs:Depends}
> -Description: OVN vtep controller
> - ovn-controller-vtep is the local controller daemon in
> - OVN, the Open Virtual Network, for VTEP enabled physical switches.
> - It connects up to the OVN Southbound database over the OVSDB protocol,
> - and down to the VTEP database over the OVSDB protocol.
> - .
> - ovn-controller-vtep provides the ovn-controller-vtep binary for
> controlling
> - vtep gateways.
> -
> -
> -Package: ovn-host
> -Architecture: linux-any
> -Depends: openvswitch-switch (= ${binary:Version}),
> -         openvswitch-common (= ${binary:Version}),
> -         ovn-common (= ${binary:Version}),
> -         ${misc:Depends},
> -         ${shlibs:Depends}
> -Description: OVN host components
> - OVN, the Open Virtual Network, is a system to support virtual network
> - abstraction.  OVN complements the existing capabilities of OVS to add
> - native support for virtual network abstractions, such as virtual L2 and
> L3
> - overlays and security groups.
> - .
> - ovn-host provides the userspace components and utilities for
> - OVN that can be run on every host/hypervisor.
> -
> -Package: ovn-central
> -Architecture: linux-any
> -Depends: openvswitch-switch (= ${binary:Version}),
> -         openvswitch-common (= ${binary:Version}),
> -         ovn-common (= ${binary:Version}),
> -         ${misc:Depends},
> -         ${shlibs:Depends}
> -Description: OVN central components
> - OVN, the Open Virtual Network, is a system to support virtual network
> - abstraction.  OVN complements the existing capabilities of OVS to add
> - native support for virtual network abstractions, such as virtual L2 and
> L3
> - overlays and security groups.
> - .
> - ovn-central provides the userspace daemons, utilities and
> - databases for OVN that is run at a central location.
> -
> -Package: ovn-docker
> -Architecture: linux-any
> -Depends: openvswitch-switch (= ${binary:Version}),
> -         openvswitch-common (= ${binary:Version}),
> -         python (>= 2.7),
> -         python-openvswitch (= ${source:Version}),
> -         ovn-common (= ${binary:Version}),
> -         ${misc:Depends},
> -         ${python:Depends},
> -         ${shlibs:Depends}
> -Description: OVN Docker drivers
> - OVN, the Open Virtual Network, is a system to support virtual network
> - abstraction.  OVN complements the existing capabilities of OVS to add
> - native support for virtual network abstractions, such as virtual L2 and
> L3
> - overlays and security groups.
> - .
> - ovn-docker provides the docker drivers for OVN.
> -
>  Package: openvswitch-pki
>  Architecture: all
>  Depends: openvswitch-common (<< ${source:Version}.1~),
> diff --git a/debian/ovn-central.dirs b/debian/ovn-central.dirs
> deleted file mode 100644
> index 6394883ce..000000000
> --- a/debian/ovn-central.dirs
> +++ /dev/null
> @@ -1 +0,0 @@
> -/usr/share/ovn/central
> diff --git a/debian/ovn-central.init b/debian/ovn-central.init
> deleted file mode 100755
> index 60cee95a3..000000000
> --- a/debian/ovn-central.init
> +++ /dev/null
> @@ -1,60 +0,0 @@
> -#! /bin/sh
> -#
> -### BEGIN INIT INFO
> -# Provides:          ovn-central
> -# Required-Start:    openvswitch-switch $remote_fs $syslog
> -# Required-Stop:     $remote_fs
> -# Default-Start:     2 3 4 5
> -# Default-Stop:      0 1 6
> -# Short-Description: OVN central components
> -# Description:       ovn-central provides the userspace daemons,
> -#                    utilities and databases for OVN that is run at a
> central
> -#                    location.
> -### END INIT INFO
> -
> -test -x /usr/bin/ovn-northd  || exit 0
> -test -x /usr/share/openvswitch/scripts/ovn-ctl || exit 0
> -
> -_SYSTEMCTL_SKIP_REDIRECT=yes
> -SYSTEMCTL_SKIP_REDIRECT=yes
> -
> -. /usr/share/openvswitch/scripts/ovs-lib
> -if [ -e /etc/default/ovn-central ]; then
> -    . /etc/default/ovn-central
> -fi
> -
> -start () {
> -    set /usr/share/openvswitch/scripts/ovn-ctl ${1-start_northd}
> -    set "$@" $OVN_CTL_OPTS
> -    "$@" || exit $?
> -}
> -
> -stop_northd () {
> -    set /usr/share/openvswitch/scripts/ovn-ctl ${1-stop_northd}
> -    set "$@" $OVN_CTL_OPTS
> -    "$@" || exit $?
> -}
> -
> -case $1 in
> -    start)
> -        start
> -        ;;
> -    stop)
> -        stop_northd
> -        ;;
> -    restart)
> -        start restart_northd
> -        ;;
> -    reload | force-reload)
> -        ;;
> -    status)
> -        /usr/share/openvswitch/scripts/ovn-ctl status_northd
> -        exit $?
> -        ;;
> -    *)
> -        echo "Usage: $0 {start|stop|reload|force-reload|restart|status}"
> >&2
> -        exit 1
> -        ;;
> -esac
> -
> -exit 0
> diff --git a/debian/ovn-central.install b/debian/ovn-central.install
> deleted file mode 100644
> index 733d3fc5e..000000000
> --- a/debian/ovn-central.install
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -usr/bin/ovn-northd
> -usr/share/openvswitch/ovn-nb.ovsschema
> -usr/share/openvswitch/ovn-sb.ovsschema
> diff --git a/debian/ovn-central.manpages b/debian/ovn-central.manpages
> deleted file mode 100644
> index 2ddb43784..000000000
> --- a/debian/ovn-central.manpages
> +++ /dev/null
> @@ -1 +0,0 @@
> -ovn/northd/ovn-northd.8
> diff --git a/debian/ovn-central.postinst b/debian/ovn-central.postinst
> deleted file mode 100755
> index 10e07ece4..000000000
> --- a/debian/ovn-central.postinst
> +++ /dev/null
> @@ -1,49 +0,0 @@
> -#!/bin/sh
> -# postinst script for ovn-central
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -# summary of how this script can be called:
> -#        * <postinst> `configure' <most-recently-configured-version>
> -#        * <old-postinst> `abort-upgrade' <new version>
> -#        * <conflictor's-postinst> `abort-remove' `in-favour' <package>
> -#          <new-version>
> -#        * <postinst> `abort-remove'
> -#        * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
> -#          <failed-install-package> <version> `removing'
> -#          <conflicting-package> <version>
> -# for details, see http://www.debian.org/doc/debian-policy/ or
> -# the debian-policy package
> -
> -
> -case "$1" in
> -    configure)
> -        DEFAULT=/etc/default/ovn-central
> -        TEMPLATE=/usr/share/ovn/central/default.template
> -        if ! test -e $DEFAULT; then
> -            cp $TEMPLATE $DEFAULT
> -        else
> -            for var in $(awk -F'[ :]' '/^# [_A-Z0-9]+:/{print $2}'
> $TEMPLATE)
> -            do
> -                if ! grep $var $DEFAULT >/dev/null 2>&1; then
> -                    echo >> $DEFAULT
> -                    sed -n "/$var:/,/$var=/p" $TEMPLATE >> $DEFAULT
> -                fi
> -            done
> -        fi
> -        ;;
> -
> -    abort-upgrade|abort-remove|abort-deconfigure)
> -        ;;
> -
> -    *)
> -        echo "postinst called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -#DEBHELPER#
> -
> -exit 0
> diff --git a/debian/ovn-central.postrm b/debian/ovn-central.postrm
> deleted file mode 100755
> index 0e654a37a..000000000
> --- a/debian/ovn-central.postrm
> +++ /dev/null
> @@ -1,48 +0,0 @@
> -#!/bin/sh
> -# postrm script for ovn-central
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -# summary of how this script can be called:
> -#        * <postrm> `remove'
> -#        * <postrm> `purge'
> -#        * <old-postrm> `upgrade' <new-version>
> -#        * <new-postrm> `failed-upgrade' <old-version>
> -#        * <new-postrm> `abort-install'
> -#        * <new-postrm> `abort-install' <old-version>
> -#        * <new-postrm> `abort-upgrade' <old-version>
> -#        * <disappearer's-postrm> `disappear' <overwriter>
> -#          <overwriter-version>
> -# for details, see http://www.debian.org/doc/debian-policy/ or
> -# the debian-policy package
> -
> -
> -case "$1" in
> -    purge)
> -        rm -f /etc/default/ovn-central
> -        rm -f /etc/openvswitch/ovnnb.db*
> -        rm -f /etc/openvswitch/.ovnnb.db.~lock~
> -        rm -f /etc/openvswitch/ovnsb.db*
> -        rm -f /etc/openvswitch/.ovnsb.db.~lock~
> -        rm -f /var/log/openvswitch/ovn-northd.log* || true
> -        ;;
> -
> -    remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
> -        ;;
> -
> -    *)
> -        echo "postrm called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -# dh_installdeb will replace this with shell code automatically
> -# generated by other debhelper scripts.
> -
> -#DEBHELPER#
> -
> -exit 0
> -
> -
> diff --git a/debian/ovn-central.template b/debian/ovn-central.template
> deleted file mode 100644
> index 7cea13e50..000000000
> --- a/debian/ovn-central.template
> +++ /dev/null
> @@ -1,5 +0,0 @@
> -# This is a POSIX shell fragment                -*- sh -*-
> -
> -# OVN_CTL_OPTS: Extra options to pass to ovs-ctl.  This is, for example,
> -# a suitable place to specify --ovn-northd-wrapper=valgrind.
> -# OVN_CTL_OPTS=
> diff --git a/debian/ovn-common.install b/debian/ovn-common.install
> deleted file mode 100644
> index 90484d2fe..000000000
> --- a/debian/ovn-common.install
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -usr/bin/ovn-nbctl
> -usr/bin/ovn-sbctl
> -usr/bin/ovn-trace
> -usr/bin/ovn-detrace
> -usr/share/openvswitch/scripts/ovn-ctl
> -usr/share/openvswitch/scripts/ovndb-servers.ocf
> -usr/lib/*/libovn*.so.*
> diff --git a/debian/ovn-common.manpages b/debian/ovn-common.manpages
> deleted file mode 100644
> index 249349ed8..000000000
> --- a/debian/ovn-common.manpages
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -ovn/ovn-architecture.7
> -ovn/ovn-nb.5
> -ovn/ovn-sb.5
> -ovn/utilities/ovn-ctl.8
> -ovn/utilities/ovn-nbctl.8
> -ovn/utilities/ovn-sbctl.8
> -ovn/utilities/ovn-trace.8
> -ovn/utilities/ovn-detrace.1
> diff --git a/debian/ovn-common.postinst b/debian/ovn-common.postinst
> deleted file mode 100644
> index 588044fbc..000000000
> --- a/debian/ovn-common.postinst
> +++ /dev/null
> @@ -1,24 +0,0 @@
> -#!/bin/sh
> -# postinst script for ovn-common
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -case "$1" in
> -    configure)
> -        mkdir -p /usr/lib/ocf/resource.d/ovn
> -        ln -sf /usr/share/openvswitch/scripts/ovndb-servers.ocf
> /usr/lib/ocf/resource.d/ovn/ovndb-servers
> -        ;;
> -    abort-upgrade|abort-remove|abort-deconfigure)
> -        ;;
> -
> -    *)
> -        echo "postinst called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -#DEBHELPER#
> -
> -exit 0
> diff --git a/debian/ovn-common.postrm b/debian/ovn-common.postrm
> deleted file mode 100644
> index 9face726b..000000000
> --- a/debian/ovn-common.postrm
> +++ /dev/null
> @@ -1,23 +0,0 @@
> -#!/bin/sh
> -# postrm script for openvswitch-testcontroller
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -case "$1" in
> -    purge|remove)
> -        rm -rf /usr/lib/ocf/resource.d/ovn
> -        ;;
> -    upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
> -        ;;
> -
> -    *)
> -        echo "postrm called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -#DEBHELPER#
> -
> -exit 0
> diff --git a/debian/ovn-controller-vtep.init
> b/debian/ovn-controller-vtep.init
> deleted file mode 100755
> index be0a24358..000000000
> --- a/debian/ovn-controller-vtep.init
> +++ /dev/null
> @@ -1,54 +0,0 @@
> -#! /bin/sh
> -#
> -### BEGIN INIT INFO
> -# Provides:          ovn-controller-vtep
> -# Required-Start:    openvswitch-switch $remote_fs $syslog
> -# Required-Stop:     $remote_fs
> -# Default-Start:     2 3 4 5
> -# Default-Stop:      0 1 6
> -# Short-Description: OVN Controller for VTEP enabled devices
> -# Description:       ovn-controller-vtep provides the userspace
> -#                    components and utilities for OVN that can be run on
> -#                    hosts taht connect to VTEP enabled devices.
> -### END INIT INFO
> -
> -test -x /usr/bin/ovn-controller-vtep  || exit 0
> -test -x /usr/share/openvswitch/scripts/ovn-ctl || exit 0
> -
> -_SYSTEMCTL_SKIP_REDIRECT=yes
> -SYSTEMCTL_SKIP_REDIRECT=yes
> -
> -. /usr/share/openvswitch/scripts/ovs-lib
> -if [ -e /etc/default/ovn-controller-vtep ]; then
> -    . /etc/default/ovn-controller-vtep
> -fi
> -
> -start () {
> -    set /usr/share/openvswitch/scripts/ovn-ctl ${1-start_controller_vtep}
> -    set "$@" $OVN_CTL_OPTS
> -    "$@" || exit $?
> -}
> -
> -case $1 in
> -    start)
> -        start
> -        ;;
> -    stop | force-stop)
> -        /usr/share/openvswitch/scripts/ovn-ctl stop_controller_vtep
> -        ;;
> -    restart)
> -        start restart_controller_vtep
> -        ;;
> -    status)
> -        /usr/share/openvswitch/scripts/ovn-ctl status_controller_vtep
> -        exit $?
> -        ;;
> -    reload | force-reload)
> -        ;;
> -    *)
> -        echo "Usage: $0 {start|stop|reload|force-reload|restart|status}"
> >&2
> -        exit 1
> -        ;;
> -esac
> -
> -exit 0
> diff --git a/debian/ovn-controller-vtep.install
> b/debian/ovn-controller-vtep.install
> deleted file mode 100644
> index 1d208f37e..000000000
> --- a/debian/ovn-controller-vtep.install
> +++ /dev/null
> @@ -1 +0,0 @@
> -usr/bin/ovn-controller-vtep
> diff --git a/debian/ovn-controller-vtep.manpages
> b/debian/ovn-controller-vtep.manpages
> deleted file mode 100644
> index 787301704..000000000
> --- a/debian/ovn-controller-vtep.manpages
> +++ /dev/null
> @@ -1 +0,0 @@
> -ovn/controller-vtep/ovn-controller-vtep.8
> diff --git a/debian/ovn-docker.install b/debian/ovn-docker.install
> deleted file mode 100644
> index 583306732..000000000
> --- a/debian/ovn-docker.install
> +++ /dev/null
> @@ -1,2 +0,0 @@
> -usr/bin/ovn-docker-overlay-driver
> -usr/bin/ovn-docker-underlay-driver
> diff --git a/debian/ovn-host.dirs b/debian/ovn-host.dirs
> deleted file mode 100644
> index 7d3c761e1..000000000
> --- a/debian/ovn-host.dirs
> +++ /dev/null
> @@ -1 +0,0 @@
> -/usr/share/ovn/host
> diff --git a/debian/ovn-host.init b/debian/ovn-host.init
> deleted file mode 100755
> index 39c3bcf16..000000000
> --- a/debian/ovn-host.init
> +++ /dev/null
> @@ -1,54 +0,0 @@
> -#! /bin/sh
> -#
> -### BEGIN INIT INFO
> -# Provides:          ovn-host
> -# Required-Start:    openvswitch-switch $remote_fs $syslog
> -# Required-Stop:     $remote_fs
> -# Default-Start:     2 3 4 5
> -# Default-Stop:      0 1 6
> -# Short-Description: OVN host components
> -# Description:       ovn-host provides the userspace
> -#                    components and utilities for OVN that can be run on
> -#                    every host/hypervisor.
> -### END INIT INFO
> -
> -test -x /usr/bin/ovn-controller  || exit 0
> -test -x /usr/share/openvswitch/scripts/ovn-ctl || exit 0
> -
> -_SYSTEMCTL_SKIP_REDIRECT=yes
> -SYSTEMCTL_SKIP_REDIRECT=yes
> -
> -. /usr/share/openvswitch/scripts/ovs-lib
> -if [ -e /etc/default/ovn-host ]; then
> -    . /etc/default/ovn-host
> -fi
> -
> -start () {
> -    set /usr/share/openvswitch/scripts/ovn-ctl ${1-start_controller}
> -    set "$@" $OVN_CTL_OPTS
> -    "$@" || exit $?
> -}
> -
> -case $1 in
> -    start)
> -        start
> -        ;;
> -    stop | force-stop)
> -        /usr/share/openvswitch/scripts/ovn-ctl stop_controller
> -        ;;
> -    restart)
> -        start restart_controller
> -        ;;
> -    status)
> -        /usr/share/openvswitch/scripts/ovn-ctl status_controller
> -        exit $?
> -        ;;
> -    reload | force-reload)
> -        ;;
> -    *)
> -        echo "Usage: $0 {start|stop|reload|force-reload|restart|status}"
> >&2
> -        exit 1
> -        ;;
> -esac
> -
> -exit 0
> diff --git a/debian/ovn-host.install b/debian/ovn-host.install
> deleted file mode 100644
> index d2de82fd9..000000000
> --- a/debian/ovn-host.install
> +++ /dev/null
> @@ -1 +0,0 @@
> -usr/bin/ovn-controller
> diff --git a/debian/ovn-host.manpages b/debian/ovn-host.manpages
> deleted file mode 100644
> index 4f9e7bc90..000000000
> --- a/debian/ovn-host.manpages
> +++ /dev/null
> @@ -1 +0,0 @@
> -ovn/controller/ovn-controller.8
> diff --git a/debian/ovn-host.postinst b/debian/ovn-host.postinst
> deleted file mode 100755
> index 4b3edeb75..000000000
> --- a/debian/ovn-host.postinst
> +++ /dev/null
> @@ -1,49 +0,0 @@
> -#!/bin/sh
> -# postinst script for ovn-host
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -# summary of how this script can be called:
> -#        * <postinst> `configure' <most-recently-configured-version>
> -#        * <old-postinst> `abort-upgrade' <new version>
> -#        * <conflictor's-postinst> `abort-remove' `in-favour' <package>
> -#          <new-version>
> -#        * <postinst> `abort-remove'
> -#        * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
> -#          <failed-install-package> <version> `removing'
> -#          <conflicting-package> <version>
> -# for details, see http://www.debian.org/doc/debian-policy/ or
> -# the debian-policy package
> -
> -
> -case "$1" in
> -    configure)
> -        DEFAULT=/etc/default/ovn-host
> -        TEMPLATE=/usr/share/ovn/host/default.template
> -        if ! test -e $DEFAULT; then
> -            cp $TEMPLATE $DEFAULT
> -        else
> -            for var in $(awk -F'[ :]' '/^# [_A-Z0-9]+:/{print $2}'
> $TEMPLATE)
> -            do
> -                if ! grep $var $DEFAULT >/dev/null 2>&1; then
> -                    echo >> $DEFAULT
> -                    sed -n "/$var:/,/$var=/p" $TEMPLATE >> $DEFAULT
> -                fi
> -            done
> -        fi
> -        ;;
> -
> -    abort-upgrade|abort-remove|abort-deconfigure)
> -        ;;
> -
> -    *)
> -        echo "postinst called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -#DEBHELPER#
> -
> -exit 0
> diff --git a/debian/ovn-host.postrm b/debian/ovn-host.postrm
> deleted file mode 100755
> index 4cceb9087..000000000
> --- a/debian/ovn-host.postrm
> +++ /dev/null
> @@ -1,44 +0,0 @@
> -#!/bin/sh
> -# postrm script for ovn-host
> -#
> -# see: dh_installdeb(1)
> -
> -set -e
> -
> -# summary of how this script can be called:
> -#        * <postrm> `remove'
> -#        * <postrm> `purge'
> -#        * <old-postrm> `upgrade' <new-version>
> -#        * <new-postrm> `failed-upgrade' <old-version>
> -#        * <new-postrm> `abort-install'
> -#        * <new-postrm> `abort-install' <old-version>
> -#        * <new-postrm> `abort-upgrade' <old-version>
> -#        * <disappearer's-postrm> `disappear' <overwriter>
> -#          <overwriter-version>
> -# for details, see http://www.debian.org/doc/debian-policy/ or
> -# the debian-policy package
> -
> -
> -case "$1" in
> -    purge)
> -        rm -f /etc/default/ovn-host
> -        rm -f /var/log/openvswitch/ovn-controller.log* || true
> -        ;;
> -
> -    remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
> -        ;;
> -
> -    *)
> -        echo "postrm called with unknown argument \`$1'" >&2
> -        exit 1
> -        ;;
> -esac
> -
> -# dh_installdeb will replace this with shell code automatically
> -# generated by other debhelper scripts.
> -
> -#DEBHELPER#
> -
> -exit 0
> -
> -
> diff --git a/debian/ovn-host.template b/debian/ovn-host.template
> deleted file mode 100644
> index 7fd54efda..000000000
> --- a/debian/ovn-host.template
> +++ /dev/null
> @@ -1,5 +0,0 @@
> -# This is a POSIX shell fragment                -*- sh -*-
> -
> -# OVN_CTL_OPTS: Extra options to pass to ovs-ctl.  This is, for example,
> -# a suitable place to specify --ovn-controller-wrapper=valgrind.
> -# OVN_CTL_OPTS=
> diff --git a/debian/rules b/debian/rules
> index 9d0a81f1a..77f3a1984 100755
> --- a/debian/rules
> +++ b/debian/rules
> @@ -53,12 +53,6 @@ override_dh_install-arch:
>         # openvswitch-switch
>         cp debian/openvswitch-switch.template
> debian/openvswitch-switch/usr/share/openvswitch/switch/default.template
>
> -       # ovn-host
> -       cp debian/ovn-host.template
> debian/ovn-host/usr/share/ovn/host/default.template
> -
> -       # ovn-central
> -       cp debian/ovn-central.template
> debian/ovn-central/usr/share/ovn/central/default.template
> -
>  override_dh_install-indep:
>         dh_install
>
> diff --git a/include/automake.mk b/include/automake.mk
> index 01031e88d..e982da87d 100644
> --- a/include/automake.mk
> +++ b/include/automake.mk
> @@ -11,7 +11,6 @@ include/odp-netlink-macros.h: include/odp-netlink.h \
>  EXTRA_DIST += build-aux/extract-odp-netlink-h
> build-aux/extract-odp-netlink-macros-h
>  CLEANFILES += include/odp-netlink.h include/odp-netlink-macros.h
>
> -include include/ovn/automake.mk
>  include include/openflow/automake.mk
>  include include/openvswitch/automake.mk
>  include include/sparse/automake.mk
> diff --git a/include/ovn/actions.h b/include/ovn/actions.h
> deleted file mode 100644
> index 63d3907d8..000000000
> --- a/include/ovn/actions.h
> +++ /dev/null
> @@ -1,622 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_ACTIONS_H
> -#define OVN_ACTIONS_H 1
> -
> -#include <stdbool.h>
> -#include <stdint.h>
> -#include "compiler.h"
> -#include "expr.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/uuid.h"
> -#include "util.h"
> -
> -struct expr;
> -struct lexer;
> -struct ofpbuf;
> -struct shash;
> -struct simap;
> -struct ovn_extend_table;
> -
> -/* List of OVN logical actions.
> - *
> - * This macro is used directly only internally by this header and its
> - * corresponding .c file, but the list is still of interest to developers.
> - *
> - * Each OVNACT invocation has the following parameters:
> - *
> - * 1. <ENUM>, used below in the enum definition of OVNACT_<ENUM>, and
> - *    elsewhere.
> - *
> - * 2. <STRUCT> corresponding to a structure "struct <STRUCT>", that must
> be a
> - *    defined below.  This structure must be an abstract definition of the
> - *    action.  Its first member must have type "struct ovnact" and name
> - *    "ovnact".  The structure must have a fixed length, that is, it may
> not
> - *    end with a flexible array member.
> - */
> -#define OVNACTS                                       \
> -    OVNACT(OUTPUT,            ovnact_null)            \
> -    OVNACT(NEXT,              ovnact_next)            \
> -    OVNACT(LOAD,              ovnact_load)            \
> -    OVNACT(MOVE,              ovnact_move)            \
> -    OVNACT(EXCHANGE,          ovnact_move)            \
> -    OVNACT(DEC_TTL,           ovnact_null)            \
> -    OVNACT(CT_NEXT,           ovnact_ct_next)         \
> -    OVNACT(CT_COMMIT,         ovnact_ct_commit)       \
> -    OVNACT(CT_DNAT,           ovnact_ct_nat)          \
> -    OVNACT(CT_SNAT,           ovnact_ct_nat)          \
> -    OVNACT(CT_LB,             ovnact_ct_lb)           \
> -    OVNACT(CT_CLEAR,          ovnact_null)            \
> -    OVNACT(CLONE,             ovnact_nest)            \
> -    OVNACT(ARP,               ovnact_nest)            \
> -    OVNACT(ICMP4,             ovnact_nest)            \
> -    OVNACT(ICMP4_ERROR,       ovnact_nest)            \
> -    OVNACT(ICMP6,             ovnact_nest)            \
> -    OVNACT(IGMP,              ovnact_null)            \
> -    OVNACT(TCP_RESET,         ovnact_nest)            \
> -    OVNACT(ND_NA,             ovnact_nest)            \
> -    OVNACT(ND_NA_ROUTER,      ovnact_nest)            \
> -    OVNACT(GET_ARP,           ovnact_get_mac_bind)    \
> -    OVNACT(PUT_ARP,           ovnact_put_mac_bind)    \
> -    OVNACT(GET_ND,            ovnact_get_mac_bind)    \
> -    OVNACT(PUT_ND,            ovnact_put_mac_bind)    \
> -    OVNACT(PUT_DHCPV4_OPTS,   ovnact_put_opts)        \
> -    OVNACT(PUT_DHCPV6_OPTS,   ovnact_put_opts)        \
> -    OVNACT(SET_QUEUE,         ovnact_set_queue)       \
> -    OVNACT(DNS_LOOKUP,        ovnact_dns_lookup)      \
> -    OVNACT(LOG,               ovnact_log)             \
> -    OVNACT(PUT_ND_RA_OPTS,    ovnact_put_opts)        \
> -    OVNACT(ND_NS,             ovnact_nest)            \
> -    OVNACT(SET_METER,         ovnact_set_meter)       \
> -    OVNACT(OVNFIELD_LOAD,     ovnact_load)            \
> -    OVNACT(CHECK_PKT_LARGER,  ovnact_check_pkt_larger) \
> -    OVNACT(TRIGGER_EVENT,     ovnact_controller_event)
> -
> -/* enum ovnact_type, with a member OVNACT_<ENUM> for each action. */
> -enum OVS_PACKED_ENUM ovnact_type {
> -#define OVNACT(ENUM, STRUCT) OVNACT_##ENUM,
> -    OVNACTS
> -#undef OVNACT
> -};
> -
> -/* Define N_OVNACTS to the number of types of ovnacts. */
> -enum {
> -#define OVNACT(ENUM, STRUCT) + 1
> -    N_OVNACTS = OVNACTS
> -#undef OVNACT
> -};
> -
> -/* Header for an action.
> - *
> - * Each action is a structure "struct ovnact_*" that begins with "struct
> - * ovnact", usually followed by other data that describes the action.
> Actions
> - * are padded out to a multiple of OVNACT_ALIGNTO bytes in length.
> - */
> -struct ovnact {
> -    /* We want the space advantage of an 8-bit type here on every
> -     * implementation, without giving up the advantage of having a useful
> type
> -     * on implementations that support packed enums. */
> -#ifdef HAVE_PACKED_ENUM
> -    enum ovnact_type type;      /* OVNACT_*. */
> -#else
> -    uint8_t type;               /* OVNACT_* */
> -#endif
> -    uint8_t pad;                /* Pad to multiple of 16 bits. */
> -
> -    uint16_t len;               /* Length of the action, in bytes,
> including
> -                                 * struct ovnact, excluding padding. */
> -};
> -BUILD_ASSERT_DECL(sizeof(struct ovnact) == 4);
> -
> -/* Alignment. */
> -#define OVNACT_ALIGNTO 8
> -#define OVNACT_ALIGN(SIZE) ROUND_UP(SIZE, OVNACT_ALIGNTO)
> -
> -/* Returns the ovnact following 'ovnact'. */
> -static inline struct ovnact *
> -ovnact_next(const struct ovnact *ovnact)
> -{
> -    return (void *) ((uint8_t *) ovnact + OVNACT_ALIGN(ovnact->len));
> -}
> -
> -struct ovnact *ovnact_next_flattened(const struct ovnact *);
> -
> -static inline struct ovnact *
> -ovnact_end(const struct ovnact *ovnacts, size_t ovnacts_len)
> -{
> -    return (void *) ((uint8_t *) ovnacts + ovnacts_len);
> -}
> -
> -/* Assigns POS to each ovnact, in turn, in the OVNACTS_LEN bytes of
> ovnacts
> - * starting at OVNACTS. */
> -#define OVNACT_FOR_EACH(POS, OVNACTS, OVNACTS_LEN)                      \
> -    for ((POS) = (OVNACTS); (POS) < ovnact_end(OVNACTS, OVNACTS_LEN);  \
> -         (POS) = ovnact_next(POS))
> -
> -static inline int
> -ovnacts_count(const struct ovnact *ovnacts, size_t ovnacts_len)
> -{
> -    uint8_t n_ovnacts = 0;
> -    if (ovnacts) {
> -        const struct ovnact *a;
> -
> -        OVNACT_FOR_EACH (a, ovnacts, ovnacts_len) {
> -            n_ovnacts++;
> -        }
> -    }
> -    return n_ovnacts;
> -}
> -
> -/* Action structure for each OVNACT_*. */
> -
> -/* Action structure for actions that do not have any extra data beyond the
> - * action type. */
> -struct ovnact_null {
> -    struct ovnact ovnact;
> -};
> -
> -/* Logical pipeline in which a set of actions is executed. */
> -enum ovnact_pipeline {
> -    OVNACT_P_INGRESS,
> -    OVNACT_P_EGRESS,
> -};
> -
> -/* OVNACT_NEXT. */
> -struct ovnact_next {
> -    struct ovnact ovnact;
> -
> -    /* Arguments. */
> -    uint8_t ltable;                /* Logical table ID of next table. */
> -    enum ovnact_pipeline pipeline; /* Pipeline of next table. */
> -
> -    /* Information about the flow that the action is in.  This does not
> affect
> -     * behavior, since the implementation of "next" doesn't depend on the
> -     * source table or pipeline.  It does affect how ovnacts_format()
> prints
> -     * the action. */
> -    uint8_t src_ltable;                /* Logical table ID of source
> table. */
> -    enum ovnact_pipeline src_pipeline; /* Pipeline of source table. */
> -};
> -
> -/* OVNACT_LOAD. */
> -struct ovnact_load {
> -    struct ovnact ovnact;
> -    struct expr_field dst;
> -    union expr_constant imm;
> -};
> -
> -/* OVNACT_MOVE, OVNACT_EXCHANGE. */
> -struct ovnact_move {
> -    struct ovnact ovnact;
> -    struct expr_field lhs;
> -    struct expr_field rhs;
> -};
> -
> -/* OVNACT_CT_NEXT. */
> -struct ovnact_ct_next {
> -    struct ovnact ovnact;
> -    uint8_t ltable;                /* Logical table ID of next table. */
> -};
> -
> -/* OVNACT_CT_COMMIT. */
> -struct ovnact_ct_commit {
> -    struct ovnact ovnact;
> -    uint32_t ct_mark, ct_mark_mask;
> -    ovs_be128 ct_label, ct_label_mask;
> -};
> -
> -/* OVNACT_CT_DNAT, OVNACT_CT_SNAT. */
> -struct ovnact_ct_nat {
> -    struct ovnact ovnact;
> -    ovs_be32 ip;
> -    uint8_t ltable;             /* Logical table ID of next table. */
> -};
> -
> -struct ovnact_ct_lb_dst {
> -    int family;
> -    union {
> -        struct in6_addr ipv6;
> -        ovs_be32 ipv4;
> -    };
> -    uint16_t port;
> -};
> -
> -/* OVNACT_CT_LB. */
> -struct ovnact_ct_lb {
> -    struct ovnact ovnact;
> -    struct ovnact_ct_lb_dst *dsts;
> -    size_t n_dsts;
> -    uint8_t ltable;             /* Logical table ID of next table. */
> -};
> -
> -/* OVNACT_ARP, OVNACT_ND_NA, OVNACT_CLONE. */
> -struct ovnact_nest {
> -    struct ovnact ovnact;
> -    struct ovnact *nested;
> -    size_t nested_len;
> -};
> -
> -/* OVNACT_GET_ARP, OVNACT_GET_ND. */
> -struct ovnact_get_mac_bind {
> -    struct ovnact ovnact;
> -    struct expr_field port;     /* Logical port name. */
> -    struct expr_field ip;       /* 32-bit or 128-bit IP address. */
> -};
> -
> -/* OVNACT_PUT_ARP, ONVACT_PUT_ND. */
> -struct ovnact_put_mac_bind {
> -    struct ovnact ovnact;
> -    struct expr_field port;     /* Logical port name. */
> -    struct expr_field ip;       /* 32-bit or 128-bit IP address. */
> -    struct expr_field mac;      /* 48-bit Ethernet address. */
> -};
> -
> -struct ovnact_gen_option {
> -    const struct gen_opts_map *option;
> -    struct expr_constant_set value;
> -};
> -
> -/* OVNACT_PUT_DHCPV4_OPTS, OVNACT_PUT_DHCPV6_OPTS. */
> -struct ovnact_put_opts {
> -    struct ovnact ovnact;
> -    struct expr_field dst;      /* 1-bit destination field. */
> -    struct ovnact_gen_option *options;
> -    size_t n_options;
> -};
> -
> -/* Valid arguments to SET_QUEUE action.
> - *
> - * QDISC_MIN_QUEUE_ID is the default queue, so user-defined queues should
> - * start at QDISC_MIN_QUEUE_ID+1. */
> -#define QDISC_MIN_QUEUE_ID  0
> -#define QDISC_MAX_QUEUE_ID  0xf000
> -
> -/* OVNACT_SET_QUEUE. */
> -struct ovnact_set_queue {
> -    struct ovnact ovnact;
> -    uint16_t queue_id;
> -};
> -
> -/* OVNACT_DNS_LOOKUP. */
> -struct ovnact_dns_lookup {
> -    struct ovnact ovnact;
> -    struct expr_field dst;      /* 1-bit destination field. */
> -};
> -
> -/* OVNACT_LOG. */
> -struct ovnact_log {
> -    struct ovnact ovnact;
> -    uint8_t verdict;            /* One of LOG_VERDICT_*. */
> -    uint8_t severity;           /* One of LOG_SEVERITY_*. */
> -    char *name;
> -    char *meter;
> -};
> -
> -/* OVNACT_SET_METER. */
> -struct ovnact_set_meter {
> -    struct ovnact ovnact;
> -    uint64_t rate;                   /* rate field, in kbps. */
> -    uint64_t burst;                  /* burst rate field, in kbps. */
> -};
> -
> -/* OVNACT_CHECK_IP_PKT_LARGER. */
> -struct ovnact_check_pkt_larger {
> -    struct ovnact ovnact;
> -    uint16_t pkt_len;
> -    struct expr_field dst;      /* 1-bit destination field. */
> -};
> -
> -/* OVNACT_EVENT. */
> -struct ovnact_controller_event {
> -    struct ovnact ovnact;
> -    int event_type;   /* controller event type */
> -    struct ovnact_gen_option *options;
> -    size_t n_options;
> -};
> -
> -/* Internal use by the helpers below. */
> -void ovnact_init(struct ovnact *, enum ovnact_type, size_t len);
> -void *ovnact_put(struct ofpbuf *, enum ovnact_type, size_t len);
> -
> -/* For each OVNACT_<ENUM> with a corresponding struct <STRUCT>, this
> defines
> - * the following commonly useful functions:
> - *
> - *   struct <STRUCT> *ovnact_put_<ENUM>(struct ofpbuf *ovnacts);
> - *
> - *     Appends a new 'ovnact', of length OVNACT_<ENUM>_SIZE, to 'ovnacts',
> - *     initializes it with ovnact_init_<ENUM>(), and returns it.  Also
> sets
> - *     'ovnacts->header' to the returned action.
> - *
> - *   struct <STRUCT> *ovnact_get_<ENUM>(const struct ovnact *ovnact);
> - *
> - *     Returns 'ovnact' cast to "struct <STRUCT> *".  'ovnact->type' must
> be
> - *     OVNACT_<ENUM>.
> - *
> - * as well as the following more rarely useful definitions:
> - *
> - *   void ovnact_init_<ENUM>(struct <STRUCT> *ovnact);
> - *
> - *     Initializes the parts of 'ovnact' that identify it as having type
> - *     OVNACT_<ENUM> and length OVNACT_<ENUM>_SIZE and zeros the rest.
> - *
> - *   <ENUM>_SIZE
> - *
> - *     The size of the action structure, that is, sizeof(struct <STRUCT>)
> - *     rounded up to a multiple of OVNACT_ALIGNTO.
> - */
> -#define OVNACT(ENUM, STRUCT)                                            \
> -    BUILD_ASSERT_DECL(offsetof(struct STRUCT, ovnact) == 0);            \
> -                                                                        \
> -    enum { OVNACT_##ENUM##_SIZE = OVNACT_ALIGN(sizeof(struct STRUCT)) }; \
> -                                                                        \
> -    static inline struct STRUCT *                                       \
> -    ovnact_get_##ENUM(const struct ovnact *ovnact)                      \
> -    {                                                                   \
> -        ovs_assert(ovnact->type == OVNACT_##ENUM);                      \
> -        return ALIGNED_CAST(struct STRUCT *, ovnact);                   \
> -    }                                                                   \
> -                                                                        \
> -    static inline struct STRUCT *                                       \
> -    ovnact_put_##ENUM(struct ofpbuf *ovnacts)                           \
> -    {                                                                   \
> -        return ovnact_put(ovnacts, OVNACT_##ENUM,                       \
> -                          OVNACT_##ENUM##_SIZE);                        \
> -    }                                                                   \
> -                                                                        \
> -    static inline void                                                  \
> -    ovnact_init_##ENUM(struct STRUCT *ovnact)                           \
> -    {                                                                   \
> -        ovnact_init(&ovnact->ovnact, OVNACT_##ENUM,                     \
> -                    OVNACT_##ENUM##_SIZE);                              \
> -    }
> -OVNACTS
> -#undef OVNACT
> -
> -enum action_opcode {
> -    /* "arp { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_ARP,
> -
> -    /* "put_arp(port, ip, mac)"
> -     *
> -     * Arguments are passed through the packet metadata and data, as
> follows:
> -     *
> -     *     MFF_REG0 = ip
> -     *     MFF_LOG_INPORT = port
> -     *     MFF_ETH_SRC = mac
> -     */
> -    ACTION_OPCODE_PUT_ARP,
> -
> -    /* "result = put_dhcp_opts(offer_ip, option, ...)".
> -     *
> -     * Arguments follow the action_header, in this format:
> -     *   - A 32-bit or 64-bit OXM header designating the result field.
> -     *   - A 32-bit integer specifying a bit offset within the result
> field.
> -     *   - The 32-bit DHCP offer IP.
> -     *   - Any number of DHCP options.
> -     */
> -    ACTION_OPCODE_PUT_DHCP_OPTS,
> -
> -    /* "nd_na { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_ND_NA,
> -
> -    /* "put_nd(port, ip6, mac)"
> -     *
> -     * Arguments are passed through the packet metadata and data, as
> follows:
> -     *
> -     *     MFF_XXREG0 = ip6
> -     *     MFF_LOG_INPORT = port
> -     *     MFF_ETH_SRC = mac
> -     */
> -    ACTION_OPCODE_PUT_ND,
> -
> -    /* "result = put_dhcpv6_opts(option, ...)".
> -     *
> -     * Arguments follow the action_header, in this format:
> -     *   - A 32-bit or 64-bit OXM header designating the result field.
> -     *   - A 32-bit integer specifying a bit offset within the result
> field.
> -     *   - Any number of DHCPv6 options.
> -     */
> -    ACTION_OPCODE_PUT_DHCPV6_OPTS,
> -
> -    /* "result = dns_lookup()".
> -     * Arguments follow the action_header, in this format:
> -     *   - A 32-bit or 64-bit OXM header designating the result field.
> -     *   - A 32-bit integer specifying a bit offset within the result
> field.
> -     *
> -     */
> -    ACTION_OPCODE_DNS_LOOKUP,
> -
> -    /* "log(arguments)".
> -     *
> -     * Arguments are as follows:
> -     *   - An 8-bit verdict.
> -     *   - An 8-bit severity.
> -     *   - A variable length string containing the name.
> -     */
> -    ACTION_OPCODE_LOG,
> -
> -    /* "result = put_nd_ra_opts(option, ...)".
> -     * Arguments follow the action_header, in this format:
> -     *   - A 32-bit or 64-bit OXM header designating the result field.
> -     *   - A 32-bit integer specifying a bit offset within the result
> field.
> -     *   - Any number of ICMPv6 options.
> -     */
> -    ACTION_OPCODE_PUT_ND_RA_OPTS,
> -
> -    /* "nd_ns { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_ND_NS,
> -
> -    /* "icmp4 { ...actions... } and icmp6 { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_ICMP,
> -
> -    /* "tcp_reset { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_TCP_RESET,
> -
> -    /* "nd_na_router { ...actions... }" with rso flag 'ND_RSO_ROUTER' set.
> -        *
> -        * The actions, in OpenFlow 1.3 format, follow the action_header.
> -        */
> -    ACTION_OPCODE_ND_NA_ROUTER,
> -
> -     /* MTU value (to put in the icmp4 header field - frag_mtu) follow the
> -     * action header. */
> -    ACTION_OPCODE_PUT_ICMP4_FRAG_MTU,
> -
> -    /* "icmp4_error { ...actions... }".
> -     *
> -     * The actions, in OpenFlow 1.3 format, follow the action_header.
> -     */
> -    ACTION_OPCODE_ICMP4_ERROR,
> -
> -    /* "trigger_event (event_type)" */
> -    ACTION_OPCODE_EVENT,
> -
> -    /* "igmp".
> -     *
> -     * Snoop IGMP, learn the multicast participants
> -     */
> -    ACTION_OPCODE_IGMP,
> -};
> -
> -/* Header. */
> -struct action_header {
> -    ovs_be32 opcode;            /* One of ACTION_OPCODE_* */
> -    uint8_t pad[4];
> -};
> -BUILD_ASSERT_DECL(sizeof(struct action_header) == 8);
> -
> -OVS_PACKED(
> -struct ovnfield_act_header {
> -    ovs_be16 id; /* one of enum ovnfield_id. */
> -    ovs_be16 len; /* Length of the ovnfield data. */
> -});
> -
> -struct ovnact_parse_params {
> -    /* A table of "struct expr_symbol"s to support (as one would provide
> to
> -     * expr_parse()). */
> -    const struct shash *symtab;
> -
> -    /* hmap of 'struct gen_opts_map' to support 'put_dhcp_opts' action */
> -    const struct hmap *dhcp_opts;
> -
> -    /* hmap of 'struct gen_opts_map'  to support 'put_dhcpv6_opts' action
> */
> -    const struct hmap *dhcpv6_opts;
> -
> -    /* hmap of 'struct gen_opts_map' to support 'put_nd_ra_opts' action */
> -    const struct hmap *nd_ra_opts;
> -
> -    /* Array of hmap of 'struct gen_opts_map' to support 'trigger_event'
> -     * action */
> -    const struct controller_event_options *controller_event_opts;
> -
> -    /* Each OVN flow exists in a logical table within a logical pipeline.
> -     * These parameters express this context for a set of OVN actions
> being
> -     * parsed:
> -     *
> -     *     - 'n_tables' is the number of tables in the logical ingress and
> -     *        egress pipelines, that is, "next" may specify a table less
> than
> -     *        or equal to 'n_tables'.  If 'n_tables' is 0 then "next" is
> -     *        disallowed entirely.
> -     *
> -     *     - 'cur_ltable' is the logical table of the current flow, within
> -     *       'pipeline'.  If cur_ltable + 1 < n_tables, then this defines
> the
> -     *       default table that "next" will jump to.
> -     *
> -     *     - 'pipeline' is the logical pipeline.  It is the default
> pipeline to
> -     *       which 'next' will jump.  If 'pipeline' is OVNACT_P_EGRESS,
> then
> -     *       'next' will also be able to jump into the ingress pipeline,
> but
> -     *       the reverse is not true. */
> -    enum ovnact_pipeline pipeline; /* Logical pipeline. */
> -    uint8_t n_tables;              /* Number of logical flow tables. */
> -    uint8_t cur_ltable;            /* 0 <= cur_ltable < n_tables. */
> -};
> -
> -bool ovnacts_parse(struct lexer *, const struct ovnact_parse_params *,
> -                    struct ofpbuf *ovnacts, struct expr **prereqsp);
> -char *ovnacts_parse_string(const char *s, const struct
> ovnact_parse_params *,
> -                           struct ofpbuf *ovnacts, struct expr **prereqsp)
> -    OVS_WARN_UNUSED_RESULT;
> -
> -void ovnacts_format(const struct ovnact[], size_t ovnacts_len, struct ds
> *);
> -
> -struct ovnact_encode_params {
> -    /* Looks up logical port 'port_name'.  If found, stores its port
> number in
> -     * '*portp' and returns true; otherwise, returns false. */
> -    bool (*lookup_port)(const void *aux, const char *port_name,
> -                        unsigned int *portp);
> -    const void *aux;
> -
> -    /* 'true' if the flow is for a switch. */
> -    bool is_switch;
> -
> -    /* A struct to figure out the group_id for group actions. */
> -    struct ovn_extend_table *group_table;
> -
> -    /* A struct to figure out the meter_id for meter actions. */
> -    struct ovn_extend_table *meter_table;
> -
> -    /* The logical flow uuid that drove this action. */
> -    struct uuid lflow_uuid;
> -
> -    /* OVN maps each logical flow table (ltable), one-to-one, onto a
> physical
> -     * OpenFlow flow table (ptable).  A number of parameters describe this
> -     * mapping and data related to flow tables:
> -     *
> -     *     - 'pipeline' is the logical pipeline in which the actions are
> -     *       executing.
> -     *
> -     *     - 'ingress_ptable' is the OpenFlow table that corresponds to
> OVN
> -     *       ingress table 0.
> -     *
> -     *     - 'egress_ptable' is the OpenFlow table that corresponds to OVN
> -     *       egress table 0.
> -     *
> -     *     - 'output_ptable' should be the OpenFlow table to which the
> logical
> -     *       "output" action will resubmit.
> -     *
> -     *     - 'mac_bind_ptable' should be the OpenFlow table used to track
> MAC
> -     *       bindings. */
> -    enum ovnact_pipeline pipeline; /* Logical pipeline. */
> -    uint8_t ingress_ptable;     /* First OpenFlow ingress table. */
> -    uint8_t egress_ptable;      /* First OpenFlow egress table. */
> -    uint8_t output_ptable;      /* OpenFlow table for 'output' to
> resubmit. */
> -    uint8_t mac_bind_ptable;    /* OpenFlow table for 'get_arp'/'get_nd'
> to
> -                                   resubmit. */
> -};
> -
> -void ovnacts_encode(const struct ovnact[], size_t ovnacts_len,
> -                    const struct ovnact_encode_params *,
> -                    struct ofpbuf *ofpacts);
> -
> -void ovnacts_free(struct ovnact[], size_t ovnacts_len);
> -
> -#endif /* ovn/actions.h */
> diff --git a/include/ovn/automake.mk b/include/ovn/automake.mk
> deleted file mode 100644
> index 54b0e2c0e..000000000
> --- a/include/ovn/automake.mk
> +++ /dev/null
> @@ -1,6 +0,0 @@
> -ovnincludedir = $(includedir)/ovn
> -ovninclude_HEADERS = \
> -       include/ovn/actions.h \
> -       include/ovn/expr.h \
> -       include/ovn/lex.h  \
> -       include/ovn/logical-fields.h
> diff --git a/include/ovn/expr.h b/include/ovn/expr.h
> deleted file mode 100644
> index 22f633e57..000000000
> --- a/include/ovn/expr.h
> +++ /dev/null
> @@ -1,518 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_EXPR_H
> -#define OVN_EXPR_H 1
> -
> -/* OVN matching expression tree
> - * ============================
> - *
> - * The data structures here form an abstract expression tree for matching
> - * expressions in OVN.
> - *
> - * The abstract syntax tree representation of a matching expression is
> one of:
> - *
> - *    - A Boolean literal ("true" or "false").
> - *
> - *    - A comparison of a field (or part of a field) against a constant
> - *      with one of the operators == != < <= > >=.
> - *
> - *    - The logical AND or OR of two or more matching expressions.
> - *
> - * Literals and comparisons are called "terminal" nodes, logical AND and
> OR
> - * nodes are "nonterminal" nodes.
> - *
> - * The syntax for expressions includes a few other concepts that are not
> part
> - * of the abstract syntax tree.  In these examples, x is a field, a, b,
> and c
> - * are constants, and e1 and e2 are arbitrary expressions:
> - *
> - *    - Logical NOT.  The parser implements NOT by inverting the sense of
> the
> - *      operand: !(x == a) becomes x != a, !(e1 && e2) becomes !e1 ||
> !e2, and
> - *      so on.
> - *
> - *    - Set membership.  The parser translates x == {a, b, c} into
> - *      x == a || x == b || x == c.
> - *
> - *    - Reversed comparisons.  The parser translates a < x into x > a.
> - *
> - *    - Range expressions.  The parser translates a < x < b into
> - *      x > a && x < b.
> - */
> -
> -#include "classifier.h"
> -#include "lex.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/list.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/meta-flow.h"
> -#include "logical-fields.h"
> -
> -struct ds;
> -struct expr;
> -struct flow;
> -struct ofpbuf;
> -struct shash;
> -struct simap;
> -struct sset;
> -
> -/* "Measurement level" of a field.  See "Level of Measurement" in the
> large
> - * comment on struct expr_symbol below for more information. */
> -enum expr_level {
> -    EXPR_L_NOMINAL,
> -
> -    /* Boolean values are nominal, however because of their simple nature
> OVN
> -     * can allow both equality and inequality tests on them. */
> -    EXPR_L_BOOLEAN,
> -
> -    /* Ordinal values can at least be ordered on a scale.  OVN allows
> equality
> -     * and inequality and relational tests on ordinal values.  These are
> the
> -     * fields on which OVS allows bitwise matching. */
> -    EXPR_L_ORDINAL
> -};
> -
> -const char *expr_level_to_string(enum expr_level);
> -
> -/* A symbol.
> - *
> - *
> - * Name
> - * ====
> - *
> - * Every symbol must have a name.  To be useful, the name must satisfy the
> - * lexer's syntax for an identifier.
> - *
> - *
> - * Width
> - * =====
> - *
> - * Every symbol has a width.  For integer symbols, this is the number of
> bits
> - * in the value; for string symbols, this is 0.
> - *
> - *
> - * Types
> - * =====
> - *
> - * There are three kinds of symbols:
> - *
> - *   Fields:
> - *
> - *     One might, for example, define a field named "vlan.tci" to refer to
> - *     MFF_VLAN_TCI.  'field' specifies the field.
> - *
> - *     'parent' and 'predicate' are NULL, and 'parent_ofs' is 0.
> - *
> - *     Integer fields can be nominal or ordinal (see below).  String
> fields are
> - *     always nominal.
> - *
> - *   Subfields:
> - *
> - *     'parent' specifies the field (which may itself be a subfield,
> - *     recursively) in which the subfield is embedded, and 'parent_ofs' a
> - *     bitwise offset from the least-significant bit of the parent.  The
> - *     subfield can contain a subset of the bits of the parent or all of
> them
> - *     (in the latter case the subfield is really just a synonym for the
> - *     parent).
> - *
> - *     'field' and 'predicate' are NULL.
> - *
> - *     Only ordinal fields (see below) may have subfields, and subfields
> are
> - *     always ordinal.
> - *
> - *   Predicates:
> - *
> - *     A predicate is an arbitrary Boolean expression that can be used in
> an
> - *     expression much like a 1-bit field.  'predicate' specifies the
> Boolean
> - *     expression, e.g. "ip4" might expand to "eth.type == 0x800".  The
> - *     epxression might refer to other predicates, e.g. "icmp4" might
> expand to
> - *     "ip4 && ip4.proto == 1".
> - *
> - *     'field' and 'parent' are NULL, and 'parent_ofs' is 0.
> - *
> - *     A predicate that refers to any nominal field or predicate (see
> below) is
> - *     nominal; other predicates have Boolean level of measurement.
> - *
> - *
> - * Level of Measurement
> - * ====================
> - *
> - * See http://en.wikipedia.org/wiki/Level_of_measurement for the
> statistical
> - * concept on which this classification is based.  There are three levels:
> - *
> - *   Ordinal:
> - *
> - *     In statistics, ordinal values can be ordered on a scale.  Here, we
> - *     consider a field (or subfield) to be ordinal if its bits can be
> examined
> - *     individually.  This is true for the OpenFlow fields that OpenFlow
> or
> - *     Open vSwitch makes "maskable".
> - *
> - *     OVN supports all the usual arithmetic relations (== != < <= > >=)
> on
> - *     ordinal fields and their subfields, because all of these can be
> - *     implemented as collections of bitwise tests.
> - *
> - *   Nominal:
> - *
> - *     In statistics, nominal values cannot be usefully compared except
> for
> - *     equality.  This is true of OpenFlow port numbers, Ethernet types,
> and IP
> - *     protocols are examples: all of these are just identifiers assigned
> - *     arbitrarily with no deeper meaning.  In OpenFlow and Open vSwitch,
> bits
> - *     in these fields generally aren't individually addressable.
> - *
> - *     OVN only supports arithmetic tests for equality on nominal fields,
> - *     because OpenFlow and Open vSwitch provide no way for a flow to
> - *     efficiently implement other comparisons on them.  (A test for
> inequality
> - *     can be sort of built out of two flows with different priorities,
> but OVN
> - *     matching expressions always generate flows with a single priority.)
> - *
> - *     String fields are always nominal.
> - *
> - *   Boolean:
> - *
> - *     A nominal field that has only two values, 0 and 1, is somewhat
> - *     exceptional, since it is easy to support both equality and
> inequality
> - *     tests on such a field: either one can be implemented as a test for
> 0 or
> - *     1.
> - *
> - *     Only predicates (see above) have a Boolean level of measurement.
> - *
> - *     This isn't a standard level of measurement.
> - *
> - *
> - * Prerequisites
> - * =============
> - *
> - * Any symbol can have prerequisites, which are specified as a string
> giving an
> - * additional expression that must be true whenever the symbol is
> referenced.
> - * For example, the "icmp4.type" symbol might have prerequisite "icmp4",
> which
> - * would cause an expression "icmp4.type == 0" to be interpreted as
> "icmp4.type
> - * == 0 && icmp4", which would in turn expand to "icmp4.type == 0 &&
> eth.type
> - * == 0x800 && ip4.proto == 1" (assuming "icmp4" is a predicate defined as
> - * suggested under "Types" above).
> - *
> - *
> - * Crossproducting
> - * ===============
> - *
> - * Ordinarily OVN is willing to consider using any field as a dimension
> in the
> - * Open vSwitch "conjunctive match" extension (see ovs-ofctl(8)).
> However,
> - * some fields can't actually be used that way because they are necessary
> as
> - * prerequisites.  For example, from an expression like "tcp.src ==
> {1,2,3}
> - * && tcp.dst == {4,5,6}", OVN might naturally generate flows like this:
> - *
> - *     conj_id=1,actions=...
> - *     ip,actions=conjunction(1,1/3)
> - *     ip6,actions=conjunction(1,1/3)
> - *     tp_src=1,actions=conjunction(1,2/3)
> - *     tp_src=2,actions=conjunction(1,2/3)
> - *     tp_src=3,actions=conjunction(1,2/3)
> - *     tp_dst=4,actions=conjunction(1,3/3)
> - *     tp_dst=5,actions=conjunction(1,3/3)
> - *     tp_dst=6,actions=conjunction(1,3/3)
> - *
> - * but that's not valid because any flow that matches on tp_src or tp_dst
> must
> - * also match on either ip or ip6.  Thus, one would mark eth.type as "must
> - * crossproduct", to force generating flows like this:
> - *
> - *     conj_id=1,actions=...
> - *     ip,tp_src=1,actions=conjunction(1,1/2)
> - *     ip,tp_src=2,actions=conjunction(1,1/2)
> - *     ip,tp_src=3,actions=conjunction(1,1/2)
> - *     ip6,tp_src=1,actions=conjunction(1,1/2)
> - *     ip6,tp_src=2,actions=conjunction(1,1/2)
> - *     ip6,tp_src=3,actions=conjunction(1,1/2)
> - *     ip,tp_dst=4,actions=conjunction(1,2/2)
> - *     ip,tp_dst=5,actions=conjunction(1,2/2)
> - *     ip,tp_dst=6,actions=conjunction(1,2/2)
> - *     ip6,tp_dst=4,actions=conjunction(1,2/2)
> - *     ip6,tp_dst=5,actions=conjunction(1,2/2)
> - *     ip6,tp_dst=6,actions=conjunction(1,2/2)
> - *
> - * which are acceptable.
> - */
> -struct expr_symbol {
> -    char *name;
> -    int width;
> -
> -    const struct mf_field *field;     /* Fields only, otherwise NULL. */
> -    const struct ovn_field *ovn_field;  /* OVN Fields only, otherwise
> NULL. */
> -    const struct expr_symbol *parent; /* Subfields only, otherwise NULL.
> */
> -    int parent_ofs;                   /* Subfields only, otherwise 0. */
> -    char *predicate;                  /* Predicates only, otherwise NULL.
> */
> -
> -    enum expr_level level;
> -
> -    char *prereqs;
> -    bool must_crossproduct;
> -    bool rw;
> -};
> -
> -void expr_symbol_format(const struct expr_symbol *, struct ds *);
> -
> -/* A reference to a symbol or a subfield of a symbol.
> - *
> - * For string fields, ofs and n_bits are 0. */
> -struct expr_field {
> -    const struct expr_symbol *symbol; /* The symbol. */
> -    int ofs;                          /* Starting bit offset. */
> -    int n_bits;                       /* Number of bits. */
> -};
> -
> -bool expr_field_parse(struct lexer *, const struct shash *symtab,
> -                      struct expr_field *, struct expr **prereqsp);
> -void expr_field_format(const struct expr_field *, struct ds *);
> -
> -struct expr_symbol *expr_symtab_add_field(struct shash *symtab,
> -                                          const char *name, enum
> mf_field_id,
> -                                          const char *prereqs,
> -                                          bool must_crossproduct);
> -struct expr_symbol *expr_symtab_add_subfield(struct shash *symtab,
> -                                             const char *name,
> -                                             const char *prereqs,
> -                                             const char *subfield);
> -struct expr_symbol *expr_symtab_add_string(struct shash *symtab,
> -                                           const char *name, enum
> mf_field_id,
> -                                           const char *prereqs);
> -struct expr_symbol *expr_symtab_add_predicate(struct shash *symtab,
> -                                              const char *name,
> -                                              const char *expansion);
> -struct expr_symbol *expr_symtab_add_ovn_field(struct shash *symtab,
> -                                              const char *name,
> -                                              enum ovn_field_id id);
> -void expr_symtab_destroy(struct shash *symtab);
> -
> -/* Expression type. */
> -enum expr_type {
> -    EXPR_T_CMP,                 /* Compare symbol with constant. */
> -    EXPR_T_AND,                 /* Logical AND of 2 or more
> subexpressions. */
> -    EXPR_T_OR,                  /* Logical OR of 2 or more
> subexpressions. */
> -    EXPR_T_BOOLEAN,             /* True or false constant. */
> -    EXPR_T_CONDITION,           /* Conditional to be evaluated in the
> -                                 * controller during expr_simplify(),
> -                                 * prior to constructing OpenFlow
> matches. */
> -};
> -
> -/* Expression condition type. */
> -enum expr_cond_type {
> -    EXPR_COND_CHASSIS_RESIDENT, /* Check if specified logical port name is
> -                                 * resident on the controller chassis. */
> -};
> -
> -/* Relational operator. */
> -enum expr_relop {
> -    EXPR_R_EQ,                  /* == */
> -    EXPR_R_NE,                  /* != */
> -    EXPR_R_LT,                  /* < */
> -    EXPR_R_LE,                  /* <= */
> -    EXPR_R_GT,                  /* > */
> -    EXPR_R_GE,                  /* >= */
> -};
> -const char *expr_relop_to_string(enum expr_relop);
> -bool expr_relop_from_token(enum lex_type type, enum expr_relop *relop);
> -
> -/* An abstract syntax tree for a matching expression.
> - *
> - * The expression code maintains and relies on a few important invariants:
> - *
> - *     - An EXPR_T_AND or EXPR_T_OR node never has a child of the same
> type.
> - *       (Any such children could be merged into their parent.)  A node
> may
> - *       have grandchildren of its own type.
> - *
> - *       As a consequence, every nonterminal node at the same distance
> from the
> - *       root has the same type.
> - *
> - *     - EXPR_T_AND and EXPR_T_OR nodes must have at least two children.
> - *
> - *     - An EXPR_T_CMP node always has a nonzero mask, and never has a
> 1-bit
> - *       in its value in a position where the mask is a 0-bit.
> - *
> - * The expr_honors_invariants() function can check invariants. */
> -struct expr {
> -    struct ovs_list node;       /* In parent EXPR_T_AND or EXPR_T_OR if
> any. */
> -    enum expr_type type;        /* Expression type. */
> -
> -    union {
> -        /* EXPR_T_CMP.
> -         *
> -         * The symbol is on the left, e.g. "field < constant". */
> -        struct {
> -            const struct expr_symbol *symbol;
> -            enum expr_relop relop;
> -
> -            union {
> -                char *string;
> -                struct {
> -                    union mf_subvalue value;
> -                    union mf_subvalue mask;
> -                };
> -            };
> -        } cmp;
> -
> -        /* EXPR_T_AND, EXPR_T_OR. */
> -        struct ovs_list andor;
> -
> -        /* EXPR_T_BOOLEAN. */
> -        bool boolean;
> -
> -        /* EXPR_T_CONDITION. */
> -        struct {
> -            enum expr_cond_type type;
> -            bool not;
> -            /* XXX Should arguments for conditions be generic? */
> -            char *string;
> -        } cond;
> -    };
> -};
> -
> -struct expr *expr_create_boolean(bool b);
> -struct expr *expr_create_andor(enum expr_type);
> -struct expr *expr_combine(enum expr_type, struct expr *a, struct expr *b);
> -
> -static inline struct expr *
> -expr_from_node(const struct ovs_list *node)
> -{
> -    return CONTAINER_OF(node, struct expr, node);
> -}
> -
> -void expr_format(const struct expr *, struct ds *);
> -void expr_print(const struct expr *);
> -struct expr *expr_parse(struct lexer *, const struct shash *symtab,
> -                        const struct shash *addr_sets,
> -                        const struct shash *port_groups,
> -                        struct sset *addr_sets_ref);
> -struct expr *expr_parse_string(const char *, const struct shash *symtab,
> -                               const struct shash *addr_sets,
> -                               const struct shash *port_groups,
> -                               struct sset *addr_sets_ref,
> -                               char **errorp);
> -
> -struct expr *expr_clone(struct expr *);
> -void expr_destroy(struct expr *);
> -
> -struct expr *expr_annotate(struct expr *, const struct shash *symtab,
> -                           char **errorp);
> -struct expr *expr_simplify(struct expr *,
> -                           bool (*is_chassis_resident)(const void *c_aux,
> -                                                       const char
> *port_name),
> -                           const void *c_aux);
> -struct expr *expr_normalize(struct expr *);
> -
> -bool expr_honors_invariants(const struct expr *);
> -bool expr_is_simplified(const struct expr *);
> -bool expr_is_normalized(const struct expr *);
> -
> -char *expr_parse_microflow(const char *, const struct shash *symtab,
> -                           const struct shash *addr_sets,
> -                           const struct shash *port_groups,
> -                           bool (*lookup_port)(const void *aux,
> -                                               const char *port_name,
> -                                               unsigned int *portp),
> -                           const void *aux, struct flow *uflow)
> -    OVS_WARN_UNUSED_RESULT;
> -
> -bool expr_evaluate(const struct expr *, const struct flow *uflow,
> -                   bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                       unsigned int *portp),
> -                   const void *aux);
> -
> -/* Converting expressions to OpenFlow flows. */
> -
> -/* An OpenFlow match generated from a Boolean expression.  See
> - * expr_to_matches() for more information. */
> -struct expr_match {
> -    struct hmap_node hmap_node;
> -    struct match match;
> -    struct cls_conjunction *conjunctions;
> -    size_t n, allocated;
> -};
> -
> -uint32_t expr_to_matches(const struct expr *,
> -                         bool (*lookup_port)(const void *aux,
> -                                             const char *port_name,
> -                                             unsigned int *portp),
> -                         const void *aux,
> -                         struct hmap *matches);
> -void expr_matches_destroy(struct hmap *matches);
> -void expr_matches_print(const struct hmap *matches, FILE *);
> -
> -/* Action parsing helper. */
> -
> -char *expr_type_check(const struct expr_field *, int n_bits, bool rw)
> -    OVS_WARN_UNUSED_RESULT;
> -struct mf_subfield expr_resolve_field(const struct expr_field *);
> -
> -/* Type of a "union expr_constant" or "struct expr_constant_set". */
> -enum expr_constant_type {
> -    EXPR_C_INTEGER,
> -    EXPR_C_STRING
> -};
> -
> -/* A string or integer constant (one must know which from context). */
> -union expr_constant {
> -    /* Integer constant.
> -     *
> -     * The width of a constant isn't always clear, e.g. if you write "1",
> -     * there's no way to tell whether you mean for that to be a 1-bit
> constant
> -     * or a 128-bit constant or somewhere in between. */
> -    struct {
> -        union mf_subvalue value;
> -        union mf_subvalue mask; /* Only initialized if 'masked'. */
> -        bool masked;
> -
> -        enum lex_format format; /* From the constant's lex_token. */
> -    };
> -
> -    /* Null-terminated string constant. */
> -    char *string;
> -};
> -
> -bool expr_constant_parse(struct lexer *, const struct expr_field *,
> -                         union expr_constant *);
> -void expr_constant_format(const union expr_constant *,
> -                          enum expr_constant_type, struct ds *);
> -void expr_constant_destroy(const union expr_constant *,
> -                           enum expr_constant_type);
> -
> -/* A collection of "union expr_constant"s of the same type. */
> -struct expr_constant_set {
> -    union expr_constant *values;  /* Constants. */
> -    size_t n_values;              /* Number of constants. */
> -    enum expr_constant_type type; /* Type of the constants. */
> -    bool in_curlies;              /* Whether the constants were in {}. */
> -};
> -
> -bool expr_constant_set_parse(struct lexer *, struct expr_constant_set *);
> -void expr_constant_set_format(const struct expr_constant_set *, struct ds
> *);
> -void expr_constant_set_destroy(struct expr_constant_set *cs);
> -
> -
> -/* Constant sets.
> - *
> - * For example, instead of referring to a set of IP addresses as:
> - *    {addr1, addr2, ..., addrN}
> - * You can register a set of values and refer to them as:
> - *    $name
> - *
> - * If convert_to_integer is true, the set must contain
> - * integer/masked-integer values. The values that don't qualify
> - * are ignored.
> - */
> -
> -void expr_const_sets_add(struct shash *const_sets, const char *name,
> -                         const char * const *values, size_t n_values,
> -                         bool convert_to_integer);
> -void expr_const_sets_remove(struct shash *const_sets, const char *name);
> -void expr_const_sets_destroy(struct shash *const_sets);
> -
> -#endif /* ovn/expr.h */
> diff --git a/include/ovn/lex.h b/include/ovn/lex.h
> deleted file mode 100644
> index 8d5585766..000000000
> --- a/include/ovn/lex.h
> +++ /dev/null
> @@ -1,152 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_LEX_H
> -#define OVN_LEX_H 1
> -
> -/* OVN lexical analyzer
> - * ====================
> - *
> - * This is a simple lexical analyzer (or tokenizer) for OVN match
> expressions
> - * and ACLs. */
> -
> -#include "openvswitch/meta-flow.h"
> -
> -struct ds;
> -
> -/* Token type. */
> -enum lex_type {
> -    LEX_T_END,                  /* end of input */
> -
> -    /* Tokens with auxiliary data. */
> -    LEX_T_ID,                   /* foo */
> -    LEX_T_STRING,               /* "foo" */
> -    LEX_T_INTEGER,              /* 12345 or 1.2.3.4 or ::1 or
> 01:02:03:04:05 */
> -    LEX_T_MASKED_INTEGER,       /* 12345/10 or 1.2.0.0/16 or ::2/127
> or... */
> -    LEX_T_MACRO,                /* $NAME */
> -    LEX_T_PORT_GROUP,            /* @NAME */
> -    LEX_T_ERROR,                /* invalid input */
> -
> -    /* Bare tokens. */
> -    LEX_T_LPAREN,               /* ( */
> -    LEX_T_RPAREN,               /* ) */
> -    LEX_T_LCURLY,               /* { */
> -    LEX_T_RCURLY,               /* } */
> -    LEX_T_LSQUARE,              /* [ */
> -    LEX_T_RSQUARE,              /* ] */
> -    LEX_T_EQ,                   /* == */
> -    LEX_T_NE,                   /* != */
> -    LEX_T_LT,                   /* < */
> -    LEX_T_LE,                   /* <= */
> -    LEX_T_GT,                   /* > */
> -    LEX_T_GE,                   /* >= */
> -    LEX_T_LOG_NOT,              /* ! */
> -    LEX_T_LOG_AND,              /* && */
> -    LEX_T_LOG_OR,               /* || */
> -    LEX_T_ELLIPSIS,             /* .. */
> -    LEX_T_COMMA,                /* , */
> -    LEX_T_SEMICOLON,            /* ; */
> -    LEX_T_EQUALS,               /* = */
> -    LEX_T_EXCHANGE,             /* <-> */
> -    LEX_T_DECREMENT,            /* -- */
> -    LEX_T_COLON,                /* : */
> -};
> -
> -/* Subtype for LEX_T_INTEGER and LEX_T_MASKED_INTEGER tokens.
> - *
> - * These do not change the semantics of a token; instead, they determine
> the
> - * format used when a token is serialized back to a text form.  That's
> - * important because 3232268289 is meaningless to a human whereas
> 192.168.128.1
> - * has some actual significance. */
> -enum lex_format {
> -    LEX_F_DECIMAL,
> -    LEX_F_HEXADECIMAL,
> -    LEX_F_IPV4,
> -    LEX_F_IPV6,
> -    LEX_F_ETHERNET,
> -};
> -const char *lex_format_to_string(enum lex_format);
> -
> -/* A token. */
> -struct lex_token {
> -    /* One of LEX_*. */
> -    enum lex_type type;
> -
> -    /* Meaningful for LEX_T_ID, LEX_T_STRING, LEX_T_ERROR, LEX_T_MACRO
> only.
> -     * For these token types, 's' may point to 'buffer'; otherwise, it
> points
> -     * to malloc()ed memory owned by the token.
> -     *
> -     * Must be NULL for other token types.
> -     *
> -     * For LEX_T_MACRO, 's' does not include the leading $. */
> -    char *s;
> -
> -    /* LEX_T_INTEGER, LEX_T_MASKED_INTEGER only. */
> -    enum lex_format format;
> -
> -    union {
> -        /* LEX_T_INTEGER, LEX_T_MASKED_INTEGER only. */
> -        struct {
> -            union mf_subvalue value; /* LEX_T_INTEGER,
> LEX_T_MASKED_INTEGER. */
> -            union mf_subvalue mask;  /* LEX_T_MASKED_INTEGER only. */
> -        };
> -
> -        /* LEX_T_ID, LEX_T_STRING, LEX_T_ERROR, LEX_T_MACRO only. */
> -        char buffer[256];
> -    };
> -};
> -
> -void lex_token_init(struct lex_token *);
> -void lex_token_destroy(struct lex_token *);
> -void lex_token_swap(struct lex_token *, struct lex_token *);
> -void lex_token_strcpy(struct lex_token *, const char *s, size_t length);
> -void lex_token_strset(struct lex_token *, char *s);
> -void lex_token_vsprintf(struct lex_token *, const char *format, va_list
> args);
> -
> -void lex_token_format(const struct lex_token *, struct ds *);
> -const char *lex_token_parse(struct lex_token *, const char *input,
> -                            const char **startp);
> -
> -/* A lexical analyzer. */
> -struct lexer {
> -    const char *input;          /* Remaining input (not owned by lexer).
> */
> -    const char *start;          /* Start of current token in 'input'. */
> -    struct lex_token token;     /* Current token (owned by lexer). */
> -    char *error;                /* Error message, if any (owned by
> lexer). */
> -};
> -
> -void lexer_init(struct lexer *, const char *input);
> -void lexer_destroy(struct lexer *);
> -
> -enum lex_type lexer_get(struct lexer *);
> -enum lex_type lexer_lookahead(const struct lexer *);
> -bool lexer_match(struct lexer *, enum lex_type);
> -bool lexer_force_match(struct lexer *, enum lex_type);
> -bool lexer_match_id(struct lexer *, const char *id);
> -bool lexer_is_int(const struct lexer *);
> -bool lexer_get_int(struct lexer *, int *value);
> -bool lexer_force_int(struct lexer *, int *value);
> -
> -bool lexer_force_end(struct lexer *);
> -
> -void lexer_error(struct lexer *, const char *message, ...)
> -    OVS_PRINTF_FORMAT(2, 3);
> -void lexer_syntax_error(struct lexer *, const char *message, ...)
> -    OVS_PRINTF_FORMAT(2, 3);
> -
> -char *lexer_steal_error(struct lexer *);
> -
> -#endif /* ovn/lex.h */
> diff --git a/include/ovn/logical-fields.h b/include/ovn/logical-fields.h
> deleted file mode 100644
> index 9bac8e027..000000000
> --- a/include/ovn/logical-fields.h
> +++ /dev/null
> @@ -1,130 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_LOGICAL_FIELDS_H
> -#define OVN_LOGICAL_FIELDS_H 1
> -
> -#include "openvswitch/meta-flow.h"
> -
> -struct shash;
> -
> -enum ovn_controller_event {
> -    OVN_EVENT_EMPTY_LB_BACKENDS = 0,
> -    OVN_EVENT_MAX,
> -};
> -
> -/* Logical fields.
> - *
> - * These values are documented in ovn-architecture(7), please update the
> - * documentation if you change any of them. */
> -#define MFF_LOG_DATAPATH MFF_METADATA /* Logical datapath (64 bits). */
> -#define MFF_LOG_FLAGS      MFF_REG10  /* One of MLF_* (32 bits). */
> -#define MFF_LOG_DNAT_ZONE  MFF_REG11  /* conntrack dnat zone for gateway
> router
> -                                       * (32 bits). */
> -#define MFF_LOG_SNAT_ZONE  MFF_REG12  /* conntrack snat zone for gateway
> router
> -                                       * (32 bits). */
> -#define MFF_LOG_CT_ZONE    MFF_REG13  /* Logical conntrack zone for lports
> -                                       * (32 bits). */
> -#define MFF_LOG_INPORT     MFF_REG14  /* Logical input port (32 bits). */
> -#define MFF_LOG_OUTPORT    MFF_REG15  /* Logical output port (32 bits). */
> -
> -/* Logical registers.
> - *
> - * Make sure these don't overlap with the logical fields! */
> -#define MFF_LOG_REG0 MFF_REG0
> -#define MFF_N_LOG_REGS 10
> -
> -void ovn_init_symtab(struct shash *symtab);
> -
> -/* MFF_LOG_FLAGS_REG bit assignments */
> -enum mff_log_flags_bits {
> -    MLF_ALLOW_LOOPBACK_BIT = 0,
> -    MLF_RCV_FROM_VXLAN_BIT = 1,
> -    MLF_FORCE_SNAT_FOR_DNAT_BIT = 2,
> -    MLF_FORCE_SNAT_FOR_LB_BIT = 3,
> -    MLF_LOCAL_ONLY_BIT = 4,
> -    MLF_NESTED_CONTAINER_BIT = 5,
> -};
> -
> -/* MFF_LOG_FLAGS_REG flag assignments */
> -enum mff_log_flags {
> -    /* Allow outputting back to inport. */
> -    MLF_ALLOW_LOOPBACK = (1 << MLF_ALLOW_LOOPBACK_BIT),
> -
> -    /* Indicate that a packet was received from a VXLAN tunnel to
> -     * compensate for the lack of egress port information available in
> -     * VXLAN encapsulation.  Egress port information is available for
> -     * Geneve and STT tunnel types. */
> -    MLF_RCV_FROM_VXLAN = (1 << MLF_RCV_FROM_VXLAN_BIT),
> -
> -    /* Indicate that a packet needs a force SNAT in the gateway router
> when
> -     * DNAT has taken place. */
> -    MLF_FORCE_SNAT_FOR_DNAT = (1 << MLF_FORCE_SNAT_FOR_DNAT_BIT),
> -
> -    /* Indicate that a packet needs a force SNAT in the gateway router
> when
> -     * load-balancing has taken place. */
> -    MLF_FORCE_SNAT_FOR_LB = (1 << MLF_FORCE_SNAT_FOR_LB_BIT),
> -
> -    /* Indicate that a packet that should be distributed across multiple
> -     * hypervisors should instead only be output to local targets
> -     */
> -    MLF_LOCAL_ONLY = (1 << MLF_LOCAL_ONLY_BIT),
> -
> -    /* Indicate that a packet was received from a nested container. */
> -    MLF_NESTED_CONTAINER = (1 << MLF_NESTED_CONTAINER_BIT),
> -};
> -
> -/* OVN logical fields
> - * ===================
> - * These are the fields which OVN supports modifying which gets translated
> - * to OFFlow controller action.
> - *
> - * OpenvSwitch doesn't support modifying these fields yet. If a field is
> - * supported later by OpenvSwitch, it can be deleted from here.
> - */
> -
> -enum ovn_field_id {
> -    /*
> -     * Name: "icmp4.frag_mtu" -
> -     * Type: be16
> -     * Description: Sets the low-order 16 bits of the ICMP4 header field
> -     * (that is labelled "unused" in the ICMP specification) of the ICMP4
> -     * packet as per the RFC 1191.
> -     */
> -    OVN_ICMP4_FRAG_MTU,
> -
> -    OVN_FIELD_N_IDS
> -};
> -
> -struct ovn_field {
> -    enum ovn_field_id id;
> -    const char *name;
> -    unsigned int n_bytes;       /* Width of the field in bytes. */
> -    unsigned int n_bits;        /* Number of significant bits in field. */
> -};
> -
> -static inline const struct ovn_field *
> -ovn_field_from_id(enum ovn_field_id id)
> -{
> -    extern const struct ovn_field ovn_fields[OVN_FIELD_N_IDS];
> -    ovs_assert((unsigned int) id < OVN_FIELD_N_IDS);
> -    return &ovn_fields[id];
> -}
> -
> -const char *event_to_string(enum ovn_controller_event event);
> -int string_to_event(const char *s);
> -const struct ovn_field *ovn_field_from_name(const char *name);
> -void ovn_destroy_ovnfields(void);
> -#endif /* ovn/lib/logical-fields.h */
> diff --git a/lib/db-ctl-base.xml b/lib/db-ctl-base.xml
> index a5fcc901c..10124c3ad 100644
> --- a/lib/db-ctl-base.xml
> +++ b/lib/db-ctl-base.xml
> @@ -40,8 +40,8 @@
>      <dd>
>        Either a universally unique identifier in the style of RFC 4122,
>        e.g. <code>f81d4fae-7dec-11d0-a765-00a0c91e6bf6</code>, or an
> <code>@</code><var>name</var>
> -      defined by a <code>get</code> or <code>create</code> command within
> the same <code>ovn-nbctl</code>
> -      invocation.
> +      defined by a <code>get</code> or <code>create</code> command within
> the
> +      same <code>ovs-vsctl</code> invocation.
>      </dd>
>
>    </dl>
> @@ -177,7 +177,7 @@
>        </p>
>
>        <p>
> -        The UUIDs shown for rows created in the same
> <code>ovn-nbctl</code>
> +        The UUIDs shown for rows created in the same
> <code>ovs-vsctl</code>
>          invocation will be wrong.
>        </p>
>
> @@ -199,7 +199,7 @@
>        </p>
>        <p>
>          If <code>@</code><var>name</var> is specified, then the UUID for
> <var>record</var> may be
> -        referred to by that name later in the same <code>ovn-nbctl</code>
> +        referred to by that name later in the same <code>ovs-vsctl</code>
>          invocation in contexts where a UUID is expected.
>        </p>
>        <p>
> @@ -379,8 +379,8 @@
>        </dl>
>        <p>
>          Consider specifying <code>--timeout=0</code> along with
> -        <code>--wait-until</code>, to prevent <code>ovn-nbctl</code> from
> terminating
> -        after waiting only at most 5 seconds.
> +        <code>--wait-until</code>, to prevent <code>ovs-vsctl</code> from
> +        terminating after waiting only at most 5 seconds.
>        </p>
>      </dd>
>
> diff --git a/manpages.mk b/manpages.mk
> index 5f43aa387..5012977aa 100644
> --- a/manpages.mk
> +++ b/manpages.mk
> @@ -1,33 +1,5 @@
>  # Generated automatically -- do not modify!    -*- buffer-read-only: t -*-
>
> -ovn/utilities/ovn-detrace.1: \
> -       ovn/utilities/ovn-detrace.1.in \
> -       lib/common-syn.man \
> -       lib/common.man \
> -       lib/ovs.tmac
> -ovn/utilities/ovn-detrace.1.in:
> -lib/common-syn.man:
> -lib/common.man:
> -lib/ovs.tmac:
> -
> -ovn/utilities/ovn-sbctl.8: \
> -       ovn/utilities/ovn-sbctl.8.in \
> -       lib/common.man \
> -       lib/db-ctl-base.man \
> -       lib/ovs.tmac \
> -       lib/ssl-bootstrap.man \
> -       lib/ssl.man \
> -       lib/table.man \
> -       lib/vlog.man
> -ovn/utilities/ovn-sbctl.8.in:
> -lib/common.man:
> -lib/db-ctl-base.man:
> -lib/ovs.tmac:
> -lib/ssl-bootstrap.man:
> -lib/ssl.man:
> -lib/table.man:
> -lib/vlog.man:
> -
>  ovsdb/ovsdb-client.1: \
>         ovsdb/ovsdb-client.1.in \
>         lib/common-syn.man \
> diff --git a/ovn/.gitignore b/ovn/.gitignore
> deleted file mode 100644
> index d971938aa..000000000
> --- a/ovn/.gitignore
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -/ovn-architecture.7
> -/ovn-nb.5
> -/ovn-nb.gv
> -/ovn-nb.pic
> -/ovn-sb.5
> -/ovn-sb.gv
> -/ovn-sb.pic
> -/*.ovsschema.stamp
> diff --git a/ovn/TODO.rst b/ovn/TODO.rst
> deleted file mode 100644
> index 33489174f..000000000
> --- a/ovn/TODO.rst
> +++ /dev/null
> @@ -1,147 +0,0 @@
> -..
> -      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> -      not use this file except in compliance with the License. You may
> obtain
> -      a copy of the License at
> -
> -          http://www.apache.org/licenses/LICENSE-2.0
> -
> -      Unless required by applicable law or agreed to in writing, software
> -      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> -      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> -      License for the specific language governing permissions and
> limitations
> -      under the License.
> -
> -      Convention for heading levels in Open vSwitch documentation:
> -
> -      =======  Heading 0 (reserved for the title in a document)
> -      -------  Heading 1
> -      ~~~~~~~  Heading 2
> -      +++++++  Heading 3
> -      '''''''  Heading 4
> -
> -      Avoid deeper levels because they do not render well.
> -
> -==============
> -OVN To-do List
> -==============
> -
> -* Get incremental updates in ovn-controller and ovn-northd in some
> -  sensible way.
> -
> -* Live migration.
> -
> -  Russell Bryant: "When you're ready to have the destination take over,
> you
> -  have to remove the iface-id from the source and add it at the
> destination and
> -  I think it'd typically be configured on both ends, since it's a clone
> of the
> -  source VM (and it's config)."
> -
> -* VLAN trunk ports.
> -
> -  Russell Bryant: "Today that would require creating 4096 ports for the
> VM and
> -  attach to 4096 OVN networks, so doable, but not quite ideal."
> -
> -* Service function chaining.
> -
> -* MAC learning.
> -
> -  Han Zhou: "To support VMs that hosts workloads with their own macs, e.g.
> -  containers, if not using OVN native container support."
> -
> -* Finish up ARP/ND support: re-checking bindings, expiring bindings.
> -
> -* Hitless upgrade, especially for data plane.
> -
> -* Use OpenFlow "bundles" for transactional data plane updates.
> -
> -* Dynamic IP to MAC binding enhancements.
> -
> -  OVN has basic support for establishing IP to MAC bindings dynamically,
> using
> -  ARP.
> -
> -  * Ratelimiting.
> -
> -    From casual observation, Linux appears to generate at most one ARP per
> -    second per destination.
> -
> -    This might be supported by adding a new OVN logical action for
> -    rate-limiting.
> -
> -  * Tracking queries
> -
> -     It's probably best to only record in the database responses to
> queries
> -     actually issued by an L3 logical router, so somehow they have to be
> -     tracked, probably by putting a tentative binding without a MAC
> address
> -     into the database.
> -
> -  * Renewal and expiration.
> -
> -    Something needs to make sure that bindings remain valid and expire
> those
> -    that become stale.
> -
> -    One way to do this might be to add some support for time to the
> database
> -    server itself.
> -
> -  * Table size limiting.
> -
> -    The table of MAC bindings must not be allowed to grow unreasonably
> large.
> -
> -  * MTU handling (fragmentation on output)
> -
> -* ovsdb-server
> -
> -  ovsdb-server should have adequate features for OVN but it probably
> needs work
> -  for scale and possibly for availability as deployments grow.  Here are
> some
> -  thoughts.
> -
> -  * Multithreading.
> -
> -    If it turns out that other changes don't let ovsdb-server scale
> -    adequately, we can multithread ovsdb-server.  Initially one might
> -    only break protocol handling into separate threads, leaving the
> -    actual database work serialized through a lock.
> -
> -  * Reducing startup time.
> -
> -    As-is, if ovsdb-server restarts, every client will fetch a fresh copy
> of
> -    the part of the database that it cares about.  With hundreds of
> clients,
> -    this could cause heavy CPU load on ovsdb-server and use excessive
> network
> -    bandwidth.  It would be better to allow incremental updates even
> across
> -    connection loss.  One way might be to use "Difference Digests" as
> described
> -    in Epstein et al., "What's the Difference? Efficient Set
> Reconciliation
> -    Without Prior Context".  (I'm not yet aware of previous non-academic
> use of
> -    this technique.)
> -
> -* Support multiple tunnel encapsulations in Chassis.
> -
> -  So far, both ovn-controller and ovn-controller-vtep only allow chassis
> to
> -  have one tunnel encapsulation entry.  We should extend the
> implementation
> -  to support multiple tunnel encapsulations.
> -
> -* Update learned MAC addresses from VTEP to OVN
> -
> -  The VTEP gateway stores all MAC addresses learned from its physical
> -  interfaces in the 'Ucast_Macs_Local' and the 'Mcast_Macs_Local' tables.
> -  ovn-controller-vtep should be able to update that information back to
> -  ovn-sb database, so that other chassis know where to send packets
> destined
> -  to the extended external network instead of broadcasting.
> -
> -* Translate ovn-sb Multicast_Group table into VTEP config
> -
> -  The ovn-controller-vtep daemon should be able to translate the
> -  Multicast_Group table entry in ovn-sb database into Mcast_Macs_Remote
> table
> -  configuration in VTEP database.
> -
> -* OVN OCF pacemaker script to support Active / Passive HA for OVN dbs
> provides
> -  the option to configure the inactivity_probe value. The default 5
> seconds
> -  inactivity_probe value is not sufficient and ovsdb-server drops the
> client
> -  IDL connections for openstack deployments when the neutron server is
> heavily
> -  loaded.
> -
> -  We need to find a proper solution to solve this issue instead of
> increasing
> -  the inactivity_probe value.
> -
> -* ACL
> -
> -  * Support FTP ALGs.
> -
> -  * Support reject action.
> diff --git a/ovn/automake.mk b/ovn/automake.mk
> deleted file mode 100644
> index b33112ef1..000000000
> --- a/ovn/automake.mk
> +++ /dev/null
> @@ -1,92 +0,0 @@
> -# OVN southbound schema and IDL
> -EXTRA_DIST += ovn/ovn-sb.ovsschema
> -pkgdata_DATA += ovn/ovn-sb.ovsschema
> -
> -# OVN southbound E-R diagram
> -#
> -# If "python" or "dot" is not available, then we do not add graphical
> diagram
> -# to the documentation.
> -if HAVE_PYTHON
> -if HAVE_DOT
> -ovn/ovn-sb.gv: ovsdb/ovsdb-dot.in ovn/ovn-sb.ovsschema
> -       $(AM_V_GEN)$(OVSDB_DOT) --no-arrows $(srcdir)/ovn/ovn-sb.ovsschema
> > $@
> -ovn/ovn-sb.pic: ovn/ovn-sb.gv ovsdb/dot2pic
> -       $(AM_V_GEN)(dot -T plain < ovn/ovn-sb.gv | $(PYTHON)
> $(srcdir)/ovsdb/dot2pic -f 3) > $@.tmp && \
> -       mv $@.tmp $@
> -OVN_SB_PIC = ovn/ovn-sb.pic
> -OVN_SB_DOT_DIAGRAM_ARG = --er-diagram=$(OVN_SB_PIC)
> -CLEANFILES += ovn/ovn-sb.gv ovn/ovn-sb.pic
> -endif
> -endif
> -
> -# OVN southbound schema documentation
> -EXTRA_DIST += ovn/ovn-sb.xml
> -CLEANFILES += ovn/ovn-sb.5
> -man_MANS += ovn/ovn-sb.5
> -ovn/ovn-sb.5: \
> -       ovsdb/ovsdb-doc ovn/ovn-sb.xml ovn/ovn-sb.ovsschema $(OVN_SB_PIC)
> -       $(AM_V_GEN)$(OVSDB_DOC) \
> -               $(OVN_SB_DOT_DIAGRAM_ARG) \
> -               --version=$(VERSION) \
> -               $(srcdir)/ovn/ovn-sb.ovsschema \
> -               $(srcdir)/ovn/ovn-sb.xml > $@.tmp && \
> -       mv $@.tmp $@
> -
> -# OVN northbound schema and IDL
> -EXTRA_DIST += ovn/ovn-nb.ovsschema
> -pkgdata_DATA += ovn/ovn-nb.ovsschema
> -
> -# OVN northbound E-R diagram
> -#
> -# If "python" or "dot" is not available, then we do not add graphical
> diagram
> -# to the documentation.
> -if HAVE_PYTHON
> -if HAVE_DOT
> -ovn/ovn-nb.gv: ovsdb/ovsdb-dot.in ovn/ovn-nb.ovsschema
> -       $(AM_V_GEN)$(OVSDB_DOT) --no-arrows $(srcdir)/ovn/ovn-nb.ovsschema
> > $@
> -ovn/ovn-nb.pic: ovn/ovn-nb.gv ovsdb/dot2pic
> -       $(AM_V_GEN)(dot -T plain < ovn/ovn-nb.gv | $(PYTHON)
> $(srcdir)/ovsdb/dot2pic -f 3) > $@.tmp && \
> -       mv $@.tmp $@
> -OVN_NB_PIC = ovn/ovn-nb.pic
> -OVN_NB_DOT_DIAGRAM_ARG = --er-diagram=$(OVN_NB_PIC)
> -CLEANFILES += ovn/ovn-nb.gv ovn/ovn-nb.pic
> -endif
> -endif
> -
> -# OVN northbound schema documentation
> -EXTRA_DIST += ovn/ovn-nb.xml
> -CLEANFILES += ovn/ovn-nb.5
> -man_MANS += ovn/ovn-nb.5
> -ovn/ovn-nb.5: \
> -       ovsdb/ovsdb-doc ovn/ovn-nb.xml ovn/ovn-nb.ovsschema $(OVN_NB_PIC)
> -       $(AM_V_GEN)$(OVSDB_DOC) \
> -               $(OVN_NB_DOT_DIAGRAM_ARG) \
> -               --version=$(VERSION) \
> -               $(srcdir)/ovn/ovn-nb.ovsschema \
> -               $(srcdir)/ovn/ovn-nb.xml > $@.tmp && \
> -       mv $@.tmp $@
> -
> -man_MANS += ovn/ovn-architecture.7
> -EXTRA_DIST += ovn/ovn-architecture.7.xml
> -CLEANFILES += ovn/ovn-architecture.7
> -
> -EXTRA_DIST += \
> -       ovn/TODO.rst
> -
> -# Version checking for ovn-nb.ovsschema.
> -ALL_LOCAL += ovn/ovn-nb.ovsschema.stamp
> -ovn/ovn-nb.ovsschema.stamp: ovn/ovn-nb.ovsschema
> -       $(srcdir)/build-aux/cksum-schema-check $? $@
> -CLEANFILES += ovn/ovn-nb.ovsschema.stamp
> -
> -# Version checking for ovn-sb.ovsschema.
> -ALL_LOCAL += ovn/ovn-sb.ovsschema.stamp
> -ovn/ovn-sb.ovsschema.stamp: ovn/ovn-sb.ovsschema
> -       $(srcdir)/build-aux/cksum-schema-check $? $@
> -CLEANFILES += ovn/ovn-sb.ovsschema.stamp
> -
> -include ovn/controller/automake.mk
> -include ovn/controller-vtep/automake.mk
> -include ovn/lib/automake.mk
> -include ovn/northd/automake.mk
> -include ovn/utilities/automake.mk
> diff --git a/ovn/controller-vtep/.gitignore
> b/ovn/controller-vtep/.gitignore
> deleted file mode 100644
> index 3ec8072c7..000000000
> --- a/ovn/controller-vtep/.gitignore
> +++ /dev/null
> @@ -1,2 +0,0 @@
> -/ovn-controller-vtep
> -/ovn-controller-vtep.8
> diff --git a/ovn/controller-vtep/automake.mk b/ovn/controller-vtep/
> automake.mk
> deleted file mode 100644
> index 0c83dc70a..000000000
> --- a/ovn/controller-vtep/automake.mk
> +++ /dev/null
> @@ -1,14 +0,0 @@
> -bin_PROGRAMS += ovn/controller-vtep/ovn-controller-vtep
> -ovn_controller_vtep_ovn_controller_vtep_SOURCES = \
> -       ovn/controller-vtep/binding.c \
> -       ovn/controller-vtep/binding.h \
> -       ovn/controller-vtep/gateway.c \
> -       ovn/controller-vtep/gateway.h \
> -       ovn/controller-vtep/ovn-controller-vtep.c \
> -       ovn/controller-vtep/ovn-controller-vtep.h \
> -       ovn/controller-vtep/vtep.c \
> -       ovn/controller-vtep/vtep.h
> -ovn_controller_vtep_ovn_controller_vtep_LDADD = ovn/lib/libovn.la lib/
> libopenvswitch.la vtep/libvtep.la
> -man_MANS += ovn/controller-vtep/ovn-controller-vtep.8
> -EXTRA_DIST += ovn/controller-vtep/ovn-controller-vtep.8.xml
> -CLEANFILES += ovn/controller-vtep/ovn-controller-vtep.8
> diff --git a/ovn/controller-vtep/binding.c b/ovn/controller-vtep/binding.c
> deleted file mode 100644
> index 9cbfadc71..000000000
> --- a/ovn/controller-vtep/binding.c
> +++ /dev/null
> @@ -1,274 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "binding.h"
> -
> -#include "openvswitch/shash.h"
> -#include "lib/smap.h"
> -#include "lib/util.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn-controller-vtep.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "vtep/vtep-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(binding);
> -
> -/*
> - * This module scans through the Port_Binding table in ovnsb.  If there
> is a
> - * logical port binding entry for logical switch in vtep gateway chassis's
> - * 'vtep_logical_switches' column, sets the binding's chassis column to
> the
> - * corresponding vtep gateway chassis.
> - *
> - */
> -
> -
> -/* Returns true if the 'vtep_lswitch' specified in 'port_binding_rec'
> - * has already been bound to another port binding entry, and resets
> - * 'port_binding_rec''s chassis column.  Otherwise, updates 'ls_to_pb'
> - * and returns false. */
> -static bool
> -check_pb_conflict(struct shash *ls_to_pb,
> -                  const struct sbrec_port_binding *port_binding_rec,
> -                  const char *chassis_name,
> -                  const char *vtep_lswitch)
> -{
> -    const struct sbrec_port_binding *pb_conflict =
> -        shash_find_data(ls_to_pb, vtep_lswitch);
> -
> -    if (pb_conflict) {
> -        VLOG_WARN("logical switch (%s), on vtep gateway chassis "
> -                  "(%s) has already been associated with logical "
> -                  "port (%s), ignore logical port (%s)",
> -                  vtep_lswitch, chassis_name,
> -                  pb_conflict->logical_port,
> -                  port_binding_rec->logical_port);
> -        sbrec_port_binding_set_chassis(port_binding_rec, NULL);
> -
> -        return true;
> -    }
> -
> -    shash_add(ls_to_pb, vtep_lswitch, port_binding_rec);
> -    return false;
> -}
> -
> -/* Returns true if the 'vtep_lswitch' specified in 'port_binding_rec'
> - * has already been bound to a different datapath, and resets
> - * 'port_binding_rec''s chassis column.  Otherwise, updates 'ls_to_db' and
> - * returns false. */
> -static bool
> -check_db_conflict(struct shash *ls_to_db,
> -                  const struct sbrec_port_binding *port_binding_rec,
> -                  const char *chassis_name,
> -                  const char *vtep_lswitch)
> -{
> -    const struct sbrec_datapath_binding *db_conflict =
> -        shash_find_data(ls_to_db, vtep_lswitch);
> -
> -    if (db_conflict && db_conflict != port_binding_rec->datapath) {
> -        VLOG_WARN("logical switch (%s), on vtep gateway chassis "
> -                  "(%s) has already been associated with logical "
> -                  "datapath (with tunnel key %"PRId64"), ignore "
> -                  "logical port (%s) which belongs to logical "
> -                  "datapath (with tunnel key %"PRId64")",
> -                  vtep_lswitch, chassis_name,
> -                  db_conflict->tunnel_key,
> -                  port_binding_rec->logical_port,
> -                  port_binding_rec->datapath->tunnel_key);
> -        sbrec_port_binding_set_chassis(port_binding_rec, NULL);
> -
> -        return true;
> -    }
> -
> -    shash_replace(ls_to_db, vtep_lswitch, port_binding_rec->datapath);
> -    return false;
> -}
> -
> -/* Updates the 'port_binding_rec''s chassis column to 'chassis_rec'. */
> -static void
> -update_pb_chassis(const struct sbrec_port_binding *port_binding_rec,
> -                  const struct sbrec_chassis *chassis_rec)
> -{
> -    if (port_binding_rec->chassis != chassis_rec) {
> -        if (chassis_rec && port_binding_rec->chassis) {
> -            VLOG_DBG("Changing chassis association of logical "
> -                     "port (%s) from (%s) to (%s)",
> -                     port_binding_rec->logical_port,
> -                     port_binding_rec->chassis->name,
> -                     chassis_rec->name);
> -        }
> -        sbrec_port_binding_set_chassis(port_binding_rec, chassis_rec);
> -    }
> -}
> -
> -
> -/* Checks and updates logical port to vtep logical switch bindings for
> each
> - * physical switch in VTEP. */
> -void
> -binding_run(struct controller_vtep_ctx *ctx)
> -{
> -    if (!ctx->ovnsb_idl_txn) {
> -        return;
> -    }
> -
> -    /* 'ls_to_db'
> -     *
> -     * Maps vtep logical switch name to the datapath binding entry.  This
> is
> -     * used to guarantee that each vtep logical switch is only included
> -     * in only one ovn datapath (ovn logical switch).  See
> check_db_conflict()
> -     * for details.
> -     *
> -     * 'ls_to_pb'
> -     *
> -     * Maps vtep logical switch name to the port binding entry.  This is
> used
> -     * to guarantee that each vtep logical switch on a vtep physical
> switch
> -     * is only bound to one logical port.  See check_pb_conflict() for
> -     * details.
> -     *
> -     */
> -    struct shash ls_to_db = SHASH_INITIALIZER(&ls_to_db);
> -
> -    /* Stores the 'chassis' and the 'ls_to_pb' map related to
> -     * a vtep physcial switch. */
> -    struct ps {
> -        const struct sbrec_chassis *chassis_rec;
> -        struct shash ls_to_pb;
> -    };
> -    struct shash ps_map = SHASH_INITIALIZER(&ps_map);
> -    const struct vteprec_physical_switch *pswitch;
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (pswitch, ctx->vtep_idl) {
> -        const struct sbrec_chassis *chassis_rec
> -            = get_chassis_by_name(ctx->ovnsb_idl, pswitch->name);
> -        struct ps *ps = xmalloc(sizeof *ps);
> -        size_t i;
> -
> -        /* 'chassis_rec' must exist. */
> -        ovs_assert(chassis_rec);
> -        ps->chassis_rec = chassis_rec;
> -        shash_init(&ps->ls_to_pb);
> -        for (i = 0; i < chassis_rec->n_vtep_logical_switches; i++) {
> -            shash_add(&ps->ls_to_pb,
> chassis_rec->vtep_logical_switches[i],
> -                      NULL);
> -        }
> -        shash_add(&ps_map, chassis_rec->name, ps);
> -    }
> -
> -    ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn,
> -                              "ovn-controller-vtep: updating bindings");
> -
> -    const struct sbrec_port_binding *port_binding_rec;
> -    /* Port binding for vtep gateway chassis must have type "vtep",
> -     * and matched physical switch name and logical switch name. */
> -    SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->ovnsb_idl) {
> -        const char *type = port_binding_rec->type;
> -        const char *vtep_pswitch = smap_get(&port_binding_rec->options,
> -                                            "vtep-physical-switch");
> -        const char *vtep_lswitch = smap_get(&port_binding_rec->options,
> -                                            "vtep-logical-switch");
> -        struct ps *ps
> -            = vtep_pswitch ? shash_find_data(&ps_map, vtep_pswitch) :
> NULL;
> -        bool found_ls
> -            = ps && vtep_lswitch && shash_find(&ps->ls_to_pb,
> vtep_lswitch);
> -
> -        if (!strcmp(type, "vtep") && found_ls) {
> -            bool pb_conflict, db_conflict;
> -
> -            pb_conflict = check_pb_conflict(&ps->ls_to_pb,
> port_binding_rec,
> -                                            ps->chassis_rec->name,
> -                                            vtep_lswitch);
> -            db_conflict = check_db_conflict(&ls_to_db, port_binding_rec,
> -                                            ps->chassis_rec->name,
> -                                            vtep_lswitch);
> -            /* Updates port binding's chassis column when there
> -             * is no conflict. */
> -            if (!pb_conflict && !db_conflict) {
> -                update_pb_chassis(port_binding_rec, ps->chassis_rec);
> -            }
> -        } else if (port_binding_rec->chassis
> -                   && shash_find(&ps_map,
> port_binding_rec->chassis->name)) {
> -            /* Resets 'port_binding_rec' since it is no longer bound to
> -             * any vtep logical switch. */
> -            update_pb_chassis(port_binding_rec, NULL);
> -        }
> -    }
> -
> -    struct shash_node *iter, *next;
> -    SHASH_FOR_EACH_SAFE (iter, next, &ps_map) {
> -        struct ps *ps = iter->data;
> -        struct shash_node *node;
> -
> -        SHASH_FOR_EACH (node, &ps->ls_to_pb) {
> -            if (!node->data) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 5);
> -                VLOG_DBG_RL(&rl, "No port binding entry for logical
> switch (%s)"
> -                            "on vtep gateway chassis (%s)", node->name,
> -                            ps->chassis_rec->name);
> -            }
> -        }
> -        shash_delete(&ps_map, iter);
> -        shash_destroy(&ps->ls_to_pb);
> -        free(ps);
> -    }
> -    shash_destroy(&ls_to_db);
> -    shash_destroy(&ps_map);
> -}
> -
> -/* Removes all port binding association with vtep gateway chassis.
> - * Returns true when done (i.e. there is no change made to
> 'ctx->ovnsb_idl'),
> - * otherwise returns false. */
> -bool
> -binding_cleanup(struct controller_vtep_ctx *ctx)
> -{
> -    if (!ctx->ovnsb_idl_txn) {
> -        return false;
> -    }
> -
> -    struct shash ch_to_pb = SHASH_INITIALIZER(&ch_to_pb);
> -    const struct sbrec_port_binding *port_binding_rec;
> -    bool all_done = true;
> -    /* Hashs all port binding entries using the associated chassis name.
> */
> -    SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->ovnsb_idl) {
> -        if (port_binding_rec->chassis) {
> -            shash_add(&ch_to_pb, port_binding_rec->chassis->name,
> -                      port_binding_rec);
> -        }
> -    }
> -
> -    ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn,
> -                              "ovn-controller-vtep: removing bindings");
> -
> -    const struct vteprec_physical_switch *pswitch;
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (pswitch, ctx->vtep_idl) {
> -        const struct sbrec_chassis *chassis_rec
> -            = get_chassis_by_name(ctx->ovnsb_idl, pswitch->name);
> -
> -        if (!chassis_rec) {
> -            continue;
> -        }
> -
> -        for (;;) {
> -            port_binding_rec = shash_find_and_delete(&ch_to_pb,
> -                                                     chassis_rec->name);
> -            if (!port_binding_rec) {
> -                break;
> -            }
> -            all_done = false;
> -            update_pb_chassis(port_binding_rec, NULL);
> -        }
> -    }
> -    shash_destroy(&ch_to_pb);
> -
> -    return all_done;
> -}
> diff --git a/ovn/controller-vtep/binding.h b/ovn/controller-vtep/binding.h
> deleted file mode 100644
> index 374c1ccf8..000000000
> --- a/ovn/controller-vtep/binding.h
> +++ /dev/null
> @@ -1,27 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_BINDING_H
> -#define OVN_BINDING_H 1
> -
> -#include <stdbool.h>
> -
> -struct controller_vtep_ctx;
> -
> -void binding_run(struct controller_vtep_ctx *);
> -bool binding_cleanup(struct controller_vtep_ctx *);
> -
> -#endif /* ovn/controller-gw/binding.h */
> diff --git a/ovn/controller-vtep/gateway.c b/ovn/controller-vtep/gateway.c
> deleted file mode 100644
> index 619c3c49a..000000000
> --- a/ovn/controller-vtep/gateway.c
> +++ /dev/null
> @@ -1,230 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "gateway.h"
> -
> -#include "openvswitch/poll-loop.h"
> -#include "lib/simap.h"
> -#include "lib/sset.h"
> -#include "lib/util.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "vtep/vtep-idl.h"
> -#include "ovn-controller-vtep.h"
> -
> -VLOG_DEFINE_THIS_MODULE(gateway);
> -
> -/*
> - * Registers the physical switches in vtep to ovnsb as chassis.  For each
> - * physical switch in the vtep database, finds all vtep logical switches
> that
> - * are associated with the physical switch, and updates the corresponding
> - * chassis's 'vtep_logical_switches' column.
> - *
> - */
> -
> -/* Global revalidation sequence number, incremented at each call to
> - * 'revalidate_gateway()'. */
> -static unsigned int gw_reval_seq;
> -
> -/* Maps all chassis created by the gateway module to their own reval_seq.
> */
> -static struct simap gw_chassis_map = SIMAP_INITIALIZER(&gw_chassis_map);
> -
> -/* Creates and returns a new instance of 'struct sbrec_chassis'. */
> -static const struct sbrec_chassis *
> -create_chassis_rec(struct ovsdb_idl_txn *txn, const char *name,
> -                   const char *encap_ip)
> -{
> -    const struct sbrec_chassis *chassis_rec;
> -    struct sbrec_encap *encap_rec;
> -
> -    VLOG_INFO("add Chassis row for VTEP physical switch (%s)", name);
> -
> -    chassis_rec = sbrec_chassis_insert(txn);
> -    sbrec_chassis_set_name(chassis_rec, name);
> -    encap_rec = sbrec_encap_insert(txn);
> -    sbrec_encap_set_type(encap_rec, OVN_SB_ENCAP_TYPE);
> -    sbrec_encap_set_ip(encap_rec, encap_ip);
> -    const struct smap options = SMAP_CONST1(&options, "csum", "false");
> -    sbrec_encap_set_options(encap_rec, &options);
> -    sbrec_chassis_set_encaps(chassis_rec, &encap_rec, 1);
> -
> -    return chassis_rec;
> -}
> -
> -/* Revalidates chassis in ovnsb against vtep database.  Creates chassis
> for
> - * new vtep physical switch.  And removes chassis which no longer have
> - * physical switch in vtep.
> - *
> - * xxx: Support multiple tunnel encaps.
> - *
> - * */
> -static void
> -revalidate_gateway(struct controller_vtep_ctx *ctx)
> -{
> -    const struct vteprec_physical_switch *pswitch;
> -
> -    /* Increments the global revalidation sequence number. */
> -    gw_reval_seq++;
> -
> -    ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn,
> -                              "ovn-controller-vtep: updating vtep
> chassis");
> -
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (pswitch, ctx->vtep_idl) {
> -        const struct sbrec_chassis *chassis_rec;
> -        struct simap_node *gw_node;
> -        const char *encap_ip;
> -
> -        encap_ip = pswitch->n_tunnel_ips ? pswitch->tunnel_ips[0] : "";
> -        gw_node = simap_find(&gw_chassis_map, pswitch->name);
> -        chassis_rec = get_chassis_by_name(ctx->ovnsb_idl, pswitch->name);
> -        if (chassis_rec) {
> -            if (!gw_node &&
> -                (strcmp(chassis_rec->encaps[0]->type, OVN_SB_ENCAP_TYPE)
> -                 || strcmp(chassis_rec->encaps[0]->ip, encap_ip))) {
> -                VLOG_WARN("Chassis config changing on startup, make sure "
> -                          "multiple chassis are not configured :
> %s/%s->%s/%s",
> -                          chassis_rec->encaps[0]->type,
> -                          chassis_rec->encaps[0]->ip,
> -                          OVN_SB_ENCAP_TYPE, encap_ip);
> -            }
> -            /* Updates chassis's encap if anything changed. */
> -            if (strcmp(chassis_rec->encaps[0]->type, OVN_SB_ENCAP_TYPE)) {
> -                VLOG_WARN("Chassis for VTEP physical switch (%s) can only
> have "
> -                          "encap type \"%s\"", pswitch->name,
> OVN_SB_ENCAP_TYPE);
> -                sbrec_encap_set_type(chassis_rec->encaps[0],
> OVN_SB_ENCAP_TYPE);
> -            }
> -            if (strcmp(chassis_rec->encaps[0]->ip, encap_ip)) {
> -                sbrec_encap_set_ip(chassis_rec->encaps[0], encap_ip);
> -            }
> -            if (smap_get_bool(&chassis_rec->encaps[0]->options, "csum",
> true)) {
> -                const struct smap options = SMAP_CONST1(&options, "csum",
> -
> "false");
> -                sbrec_encap_set_options(chassis_rec->encaps[0], &options);
> -            }
> -        } else {
> -            if (gw_node) {
> -                VLOG_WARN("Chassis for VTEP physical switch (%s)
> disappears, "
> -                          "maybe deleted by ovn-sbctl, adding it back",
> -                          pswitch->name);
> -            }
> -            /* Creates a new chassis for the VTEP physical switch. */
> -            create_chassis_rec(ctx->ovnsb_idl_txn, pswitch->name,
> encap_ip);
> -        }
> -        /* Updates or creates the simap node for 'pswitch->name'. */
> -        simap_put(&gw_chassis_map, pswitch->name, gw_reval_seq);
> -    }
> -
> -    struct simap_node *iter, *next;
> -    /* For 'gw_node' in 'gw_chassis_map' whose data is not
> -     * 'gw_reval_seq', it means the corresponding physical switch no
> -     * longer exist.  So, garbage collects them. */
> -    SIMAP_FOR_EACH_SAFE (iter, next, &gw_chassis_map) {
> -        if (iter->data != gw_reval_seq) {
> -            const struct sbrec_chassis *chassis_rec;
> -
> -            chassis_rec = get_chassis_by_name(ctx->ovnsb_idl, iter->name);
> -            if (chassis_rec) {
> -                sbrec_chassis_delete(chassis_rec);
> -            }
> -            simap_delete(&gw_chassis_map, iter);
> -        }
> -    }
> -}
> -
> -/* Updates the 'vtep_logical_switches' column in the Chassis table based
> - * on vtep database configuration. */
> -static void
> -update_vtep_logical_switches(struct controller_vtep_ctx *ctx)
> -{
> -    const struct vteprec_physical_switch *pswitch;
> -
> -    ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn, "ovn-controller-vtep: "
> -                              "updating chassis's vtep_logical_switches");
> -
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (pswitch, ctx->vtep_idl) {
> -        const struct sbrec_chassis *chassis_rec =
> -            get_chassis_by_name(ctx->ovnsb_idl, pswitch->name);
> -        struct sset lswitches = SSET_INITIALIZER(&lswitches);
> -        size_t i;
> -
> -        for (i = 0; i < pswitch->n_ports; i++) {
> -            const struct vteprec_physical_port *port = pswitch->ports[i];
> -            size_t j;
> -
> -            for (j = 0; j < port->n_vlan_bindings; j++) {
> -                const struct vteprec_logical_switch *vtep_lswitch;
> -
> -                vtep_lswitch = port->value_vlan_bindings[j];
> -                /* If not already in 'lswitches', records it. */
> -                if (!sset_find(&lswitches, vtep_lswitch->name)) {
> -                    sset_add(&lswitches, vtep_lswitch->name);
> -                }
> -            }
> -        }
> -
> -        const char **ls_arr = sset_array(&lswitches);
> -        sbrec_chassis_set_vtep_logical_switches(chassis_rec, ls_arr,
> -                                                sset_count(&lswitches));
> -        free(ls_arr);
> -        sset_destroy(&lswitches);
> -    }
> -}
> -
> -
> -void
> -gateway_run(struct controller_vtep_ctx *ctx)
> -{
> -    if (!ctx->ovnsb_idl_txn) {
> -        return;
> -    }
> -
> -    revalidate_gateway(ctx);
> -    update_vtep_logical_switches(ctx);
> -}
> -
> -/* Destroys the chassis table entries for vtep physical switches.
> - * Returns true when done (i.e. there is no change made to
> 'ctx->ovnsb_idl'),
> - * otherwise returns false. */
> -bool
> -gateway_cleanup(struct controller_vtep_ctx *ctx)
> -{
> -    static bool simap_destroyed = false;
> -    const struct vteprec_physical_switch *pswitch;
> -
> -    if (!ctx->ovnsb_idl_txn) {
> -        return false;
> -    }
> -
> -    bool all_done = true;
> -    ovsdb_idl_txn_add_comment(ctx->ovnsb_idl_txn, "ovn-controller-vtep: "
> -                              "unregistering vtep chassis");
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (pswitch, ctx->vtep_idl) {
> -        const struct sbrec_chassis *chassis_rec;
> -
> -        chassis_rec = get_chassis_by_name(ctx->ovnsb_idl, pswitch->name);
> -        if (!chassis_rec) {
> -            continue;
> -        }
> -        all_done = false;
> -        sbrec_chassis_delete(chassis_rec);
> -    }
> -    if (!simap_destroyed) {
> -        simap_destroy(&gw_chassis_map);
> -        simap_destroyed = true;
> -    }
> -
> -    return all_done;
> -}
> diff --git a/ovn/controller-vtep/gateway.h b/ovn/controller-vtep/gateway.h
> deleted file mode 100644
> index 0086191d9..000000000
> --- a/ovn/controller-vtep/gateway.h
> +++ /dev/null
> @@ -1,26 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_GATEWAY_H
> -#define OVN_GATEWAY_H 1
> -
> -#include <stdbool.h>
> -
> -struct controller_vtep_ctx;
> -
> -void gateway_run(struct controller_vtep_ctx *);
> -bool gateway_cleanup(struct controller_vtep_ctx *);
> -
> -#endif /* ovn/controller-gw/gateway.h */
> diff --git a/ovn/controller-vtep/ovn-controller-vtep.8.xml
> b/ovn/controller-vtep/ovn-controller-vtep.8.xml
> deleted file mode 100644
> index 2c706e46e..000000000
> --- a/ovn/controller-vtep/ovn-controller-vtep.8.xml
> +++ /dev/null
> @@ -1,80 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-controller-vtep" section="8"
> title="ovn-controller-vtep">
> -    <h1>Name</h1>
> -    <p>ovn-controller-vtep -- Open Virtual Network local controller for
> -       vtep enabled physical switches.
> -    </p>
> -
> -    <h1>Synopsis</h1>
> -    <p><code>ovn-controller-vtep</code> [<var>options</var>]
> -    [<var>--vtep-db=vtep-database</var>]
> [<var>--ovnsb-db=ovnsb-database</var>]
> -    </p>
> -
> -    <h1>Description</h1>
> -    <p>
> -      <code>ovn-controller-vtep</code> is the local controller daemon in
> -      OVN, the Open Virtual Network, for VTEP enabled physical switches.
> -      It connects up to the OVN Southbound database (see
> -      <code>ovn-sb</code>(5)) over the OVSDB protocol, and down to the
> VTEP
> -      database (see <code>vtep</code>(5)) over the OVSDB protocol.
> -    </p>
> -
> -    <h2>PKI Options</h2>
> -    <p>
> -      PKI configuration is required in order to use SSL for the
> connections to
> -      the VTEP and Southbound databases.
> -    </p>
> -    <xi:include href="lib/ssl.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -    <xi:include href="lib/ssl-bootstrap.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -    <xi:include href="lib/ssl-peer-ca-cert.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h1>Configuration</h1>
> -    <p>
> -      <code>ovn-controller-vtep</code> retrieves its configuration
> -      information from both the ovnsb and the vtep database.  If the
> -      database locations are not given from command line, the default
> -      is the <code>db.sock</code> in local OVSDB's 'run' directory.
> -      The datapath location must take one of the following forms:
> -    </p>
> -    <ul>
> -      <li>
> -        <p>
> -          <code>ssl:<var>host</var>:<var>port</var></code>
> -        </p>
> -        <p>
> -          The specified SSL <var>port</var> on the give <var>host</var>,
> which
> -          can either be a DNS name (if built with unbound library) or an
> IP
> -          address (IPv4 or IPv6).  If <var>host</var> is an IPv6 address,
> then
> -          wrap <var>host</var> with square brackets, e.g.:
> <code>ssl:[::1]:6640</code>.
> -          The <code>--private-key</code>, <code>--certificate</code> and
> either
> -          of <code>--ca-cert</code> or <code>--bootstrap-ca-cert</code>
> options
> -          are mandatory when this form is used.
> -        </p>
> -      </li>
> -      <li>
> -        <p>
> -          <code>tcp:<var>host</var>:<var>port</var></code>
> -        </p>
> -        <p>
> -          Connect to the given TCP <var>port</var> on <var>host</var>,
> where
> -          <var>host</var> can be a DNS name (if built with unbound
> library) or
> -          IP address (IPv4 or IPv6). If <var>host</var> is an IPv6
> address,
> -          then wrap <var>host</var> with square brackets,
> -          e.g.: <code>tcp:[::1]:6640</code>.
> -        </p>
> -      </li>
> -      <li>
> -        <p>
> -          <code>unix:<var>file</var></code>
> -        </p>
> -        <p>
> -          On POSIX, connect to the Unix domain server socket named
> -          <var>file</var>.
> -        </p>
> -        <p>
> -          On Windows, connect to a localhost TCP port whose value is
> written
> -          in <var>file</var>.
> -        </p>
> -      </li>
> -    </ul>
> -</manpage>
> diff --git a/ovn/controller-vtep/ovn-controller-vtep.c
> b/ovn/controller-vtep/ovn-controller-vtep.c
> deleted file mode 100644
> index 292a3f464..000000000
> --- a/ovn/controller-vtep/ovn-controller-vtep.c
> +++ /dev/null
> @@ -1,272 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <errno.h>
> -#include <getopt.h>
> -#include <signal.h>
> -#include <stdlib.h>
> -#include <string.h>
> -
> -#include "command-line.h"
> -#include "compiler.h"
> -#include "daemon.h"
> -#include "dirs.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "fatal-signal.h"
> -#include "openvswitch/poll-loop.h"
> -#include "stream.h"
> -#include "stream-ssl.h"
> -#include "unixctl.h"
> -#include "util.h"
> -#include "openvswitch/vconn.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "vtep/vtep-idl.h"
> -
> -#include "binding.h"
> -#include "gateway.h"
> -#include "vtep.h"
> -#include "ovn-controller-vtep.h"
> -
> -static unixctl_cb_func ovn_controller_vtep_exit;
> -
> -static void parse_options(int argc, char *argv[]);
> -OVS_NO_RETURN static void usage(void);
> -
> -static char *vtep_remote;
> -static char *ovnsb_remote;
> -static char *default_db_;
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    struct unixctl_server *unixctl;
> -    bool exiting;
> -    int retval;
> -
> -    ovs_cmdl_proctitle_init(argc, argv);
> -    set_program_name(argv[0]);
> -    service_start(&argc, &argv);
> -    parse_options(argc, argv);
> -    fatal_ignore_sigpipe();
> -
> -    daemonize_start(false);
> -
> -    retval = unixctl_server_create(NULL, &unixctl);
> -    if (retval) {
> -        exit(EXIT_FAILURE);
> -    }
> -    unixctl_command_register("exit", "", 0, 0, ovn_controller_vtep_exit,
> -                             &exiting);
> -
> -    daemonize_complete();
> -
> -    /* Connect to VTEP database. */
> -    struct ovsdb_idl_loop vtep_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create(vtep_remote, &vteprec_idl_class, true, true));
> -    ovsdb_idl_get_initial_snapshot(vtep_idl_loop.idl);
> -
> -    /* Connect to OVN SB database. */
> -    struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create(ovnsb_remote, &sbrec_idl_class, true, true));
> -    ovsdb_idl_get_initial_snapshot(ovnsb_idl_loop.idl);
> -
> -    /* Main loop. */
> -    exiting = false;
> -    while (!exiting) {
> -        struct controller_vtep_ctx ctx = {
> -            .vtep_idl = vtep_idl_loop.idl,
> -            .vtep_idl_txn = ovsdb_idl_loop_run(&vtep_idl_loop),
> -            .ovnsb_idl = ovnsb_idl_loop.idl,
> -            .ovnsb_idl_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop),
> -        };
> -
> -        gateway_run(&ctx);
> -        binding_run(&ctx);
> -        vtep_run(&ctx);
> -        unixctl_server_run(unixctl);
> -
> -        unixctl_server_wait(unixctl);
> -        if (exiting) {
> -            poll_immediate_wake();
> -        }
> -        ovsdb_idl_loop_commit_and_wait(&vtep_idl_loop);
> -        ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop);
> -        poll_block();
> -        if (should_service_stop()) {
> -            exiting = true;
> -        }
> -    }
> -
> -    /* It's time to exit.  Clean up the databases. */
> -    bool done = false;
> -    while (!done) {
> -        struct controller_vtep_ctx ctx = {
> -            .vtep_idl = vtep_idl_loop.idl,
> -            .vtep_idl_txn = ovsdb_idl_loop_run(&vtep_idl_loop),
> -            .ovnsb_idl = ovnsb_idl_loop.idl,
> -            .ovnsb_idl_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop),
> -        };
> -
> -        /* Run all of the cleanup functions, even if one of them returns
> false.
> -         * We're done if all of them return true. */
> -        done = binding_cleanup(&ctx);
> -        done = gateway_cleanup(&ctx) && done;
> -        done = vtep_cleanup(&ctx) && done;
> -        if (done) {
> -            poll_immediate_wake();
> -        }
> -
> -        ovsdb_idl_loop_commit_and_wait(&vtep_idl_loop);
> -        ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop);
> -        poll_block();
> -    }
> -
> -    unixctl_server_destroy(unixctl);
> -
> -    ovsdb_idl_loop_destroy(&vtep_idl_loop);
> -    ovsdb_idl_loop_destroy(&ovnsb_idl_loop);
> -
> -    free(ovnsb_remote);
> -    free(vtep_remote);
> -    free(default_db_);
> -    service_stop();
> -
> -    exit(retval);
> -}
> -
> -static const char *
> -default_db(void)
> -{
> -    if (!default_db_) {
> -        default_db_ = xasprintf("unix:%s/db.sock", ovs_rundir());
> -    }
> -    return default_db_;
> -}
> -
> -static void
> -parse_options(int argc, char *argv[])
> -{
> -    enum {
> -        OPT_PEER_CA_CERT = UCHAR_MAX + 1,
> -        OPT_BOOTSTRAP_CA_CERT,
> -        VLOG_OPTION_ENUMS,
> -        DAEMON_OPTION_ENUMS,
> -        SSL_OPTION_ENUMS,
> -    };
> -
> -    static struct option long_options[] = {
> -        {"ovnsb-db", required_argument, NULL, 'd'},
> -        {"vtep-db", required_argument, NULL, 'D'},
> -        {"help", no_argument, NULL, 'h'},
> -        {"version", no_argument, NULL, 'V'},
> -        VLOG_LONG_OPTIONS,
> -        DAEMON_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {"peer-ca-cert", required_argument, NULL, OPT_PEER_CA_CERT},
> -        {"bootstrap-ca-cert", required_argument, NULL,
> OPT_BOOTSTRAP_CA_CERT},
> -        {NULL, 0, NULL, 0}
> -    };
> -    char *short_options =
> ovs_cmdl_long_options_to_short_options(long_options);
> -
> -    for (;;) {
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, long_options, NULL);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        switch (c) {
> -        case 'd':
> -            ovnsb_remote = xstrdup(optarg);
> -            break;
> -
> -        case 'D':
> -            vtep_remote = xstrdup(optarg);
> -            break;
> -
> -        case 'h':
> -            usage();
> -
> -        case 'V':
> -            ovs_print_version(OFP13_VERSION, OFP13_VERSION);
> -            exit(EXIT_SUCCESS);
> -
> -        VLOG_OPTION_HANDLERS
> -        DAEMON_OPTION_HANDLERS
> -        STREAM_SSL_OPTION_HANDLERS
> -
> -        case OPT_PEER_CA_CERT:
> -            stream_ssl_set_peer_ca_cert_file(optarg);
> -            break;
> -
> -        case OPT_BOOTSTRAP_CA_CERT:
> -            stream_ssl_set_ca_cert_file(optarg, true);
> -            break;
> -
> -        case '?':
> -            exit(EXIT_FAILURE);
> -
> -        default:
> -            abort();
> -        }
> -    }
> -    free(short_options);
> -
> -    if (!ovnsb_remote) {
> -        ovnsb_remote = xstrdup(default_sb_db());
> -    }
> -
> -    if (!vtep_remote) {
> -        vtep_remote = xstrdup(default_db());
> -    }
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN controller VTEP\n\
> -usage %s [OPTIONS]\n\
> -\n\
> -Options:\n\
> -  --vtep-db=DATABASE        connect to vtep database at DATABASE\n\
> -                            (default: %s)\n\
> -  --ovnsb-db=DATABASE       connect to ovn-sb database at DATABASE\n\
> -                            (default: %s)\n\
> -  -h, --help                display this help message\n\
> -  -o, --options             list available options\n\
> -  -V, --version             display version information\n\
> -", program_name, program_name, default_db(), default_sb_db());
> -    stream_usage("database", true, false, true);
> -    daemon_usage();
> -    vlog_usage();
> -    exit(EXIT_SUCCESS);
> -}
> -
> -
> -static void
> -ovn_controller_vtep_exit(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                       const char *argv[] OVS_UNUSED, void *exiting_)
> -{
> -    bool *exiting = exiting_;
> -    *exiting = true;
> -
> -    unixctl_command_reply(conn, NULL);
> -}
> diff --git a/ovn/controller-vtep/ovn-controller-vtep.h
> b/ovn/controller-vtep/ovn-controller-vtep.h
> deleted file mode 100644
> index 435a730d9..000000000
> --- a/ovn/controller-vtep/ovn-controller-vtep.h
> +++ /dev/null
> @@ -1,51 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_CONTROLLER_VTEP_H
> -#define OVN_CONTROLLER_VTEP_H 1
> -
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -struct ovsdb_idl;
> -struct ovsdb_idl_txn;
> -
> -struct controller_vtep_ctx {
> -    struct ovsdb_idl *ovnsb_idl;
> -    struct ovsdb_idl_txn *ovnsb_idl_txn;
> -
> -    struct ovsdb_idl *vtep_idl;
> -    struct ovsdb_idl_txn *vtep_idl_txn;
> -};
> -
> -/* VTEP needs what VTEP needs. */
> -#define OVN_SB_ENCAP_TYPE "vxlan"
> -#define VTEP_ENCAP_TYPE "vxlan_over_ipv4"
> -
> -static inline const struct sbrec_chassis *
> -get_chassis_by_name(struct ovsdb_idl *ovnsb_idl, const char *chassis_id)
> -{
> -    const struct sbrec_chassis *chassis_rec;
> -
> -    SBREC_CHASSIS_FOR_EACH(chassis_rec, ovnsb_idl) {
> -        if (!strcmp(chassis_rec->name, chassis_id)) {
> -            break;
> -        }
> -    }
> -
> -    return chassis_rec;
> -}
> -
> -#endif /* ovn/ovn-controller-vtep.h */
> diff --git a/ovn/controller-vtep/vtep.c b/ovn/controller-vtep/vtep.c
> deleted file mode 100644
> index a72b149eb..000000000
> --- a/ovn/controller-vtep/vtep.c
> +++ /dev/null
> @@ -1,600 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "vtep.h"
> -
> -#include "lib/hash.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/shash.h"
> -#include "lib/smap.h"
> -#include "lib/sset.h"
> -#include "lib/util.h"
> -#include "ovn-controller-vtep.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "vtep/vtep-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(vtep);
> -
> -struct vtep_rec_physical_locator_list_entry {
> -    struct ovs_list locators_node;
> -    const struct vteprec_physical_locator *vteprec_ploc;
> -};
> -
> -struct mmr_hash_node_data {
> -    const struct vteprec_mcast_macs_remote *mmr;
> -    struct shash physical_locators;
> -};
> -
> -/*
> - * Scans through the Binding table in ovnsb, and updates the vtep logical
> - * switch tunnel keys and the 'Ucast_Macs_Remote' table in the VTEP
> - * database.
> - *
> - */
> -
> -/* Searches the 'chassis_rec->encaps' for the first vtep tunnel
> - * configuration, returns the 'ip'.  Unless duplicated, the returned
> - * pointer cannot live past current vtep_run() execution. */
> -static const char *
> -get_chassis_vtep_ip(const struct sbrec_chassis *chassis_rec)
> -{
> -    if (chassis_rec) {
> -        size_t i;
> -
> -        for (i = 0; i < chassis_rec->n_encaps; i++) {
> -            if (!strcmp(chassis_rec->encaps[i]->type, "vxlan")) {
> -                return chassis_rec->encaps[i]->ip;
> -            }
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -/* Creates a new 'Ucast_Macs_Remote'. */
> -static struct vteprec_ucast_macs_remote *
> -create_umr(struct ovsdb_idl_txn *vtep_idl_txn, const char *mac,
> -           const struct vteprec_logical_switch *vtep_ls)
> -{
> -    struct vteprec_ucast_macs_remote *new_umr =
> -        vteprec_ucast_macs_remote_insert(vtep_idl_txn);
> -
> -    vteprec_ucast_macs_remote_set_MAC(new_umr, mac);
> -    vteprec_ucast_macs_remote_set_logical_switch(new_umr, vtep_ls);
> -
> -    return new_umr;
> -}
> -
> -/* Creates a new 'Physical_Locator'. */
> -static struct vteprec_physical_locator *
> -create_pl(struct ovsdb_idl_txn *vtep_idl_txn, const char *chassis_ip)
> -{
> -    struct vteprec_physical_locator *new_pl =
> -        vteprec_physical_locator_insert(vtep_idl_txn);
> -
> -    vteprec_physical_locator_set_dst_ip(new_pl, chassis_ip);
> -    vteprec_physical_locator_set_encapsulation_type(new_pl,
> VTEP_ENCAP_TYPE);
> -
> -    return new_pl;
> -}
> -
> -/* Creates a new 'Mcast_Macs_Remote'. */
> -static void
> -vtep_create_mmr(struct ovsdb_idl_txn *vtep_idl_txn, const char *mac,
> -                const struct vteprec_logical_switch *vtep_ls,
> -                const struct vteprec_physical_locator_set *ploc_set)
> -{
> -    struct vteprec_mcast_macs_remote *new_mmr =
> -       vteprec_mcast_macs_remote_insert(vtep_idl_txn);
> -
> -    vteprec_mcast_macs_remote_set_MAC(new_mmr, mac);
> -    vteprec_mcast_macs_remote_set_logical_switch(new_mmr, vtep_ls);
> -    vteprec_mcast_macs_remote_set_locator_set(new_mmr, ploc_set);
> -}
> -
> -/* Compares previous and new mmr locator sets and returns true if they
> - * differ and false otherwise. This function also preps a new locator
> - * set for database write.
> - *
> - * 'locators_list' is the new set of locators for the associated
> - * 'Mcast_Macs_Remote' entry passed in and is queried to generate the
> - * new set of locators in vtep database format. */
> -static bool
> -vtep_process_pls(const struct ovs_list *locators_list,
> -                 const struct mmr_hash_node_data *mmr_ext,
> -                 struct vteprec_physical_locator **locators)
> -{
> -    size_t n_locators_prev = 0;
> -    size_t n_locators_new = ovs_list_size(locators_list);
> -    bool locator_lists_differ = false;
> -
> -    if (mmr_ext) {
> -        n_locators_prev = mmr_ext->mmr->locator_set->n_locators;
> -    }
> -    if (n_locators_prev != n_locators_new) {
> -        locator_lists_differ = true;
> -    }
> -
> -    if (n_locators_new) {
> -        int i = 0;
> -        struct vtep_rec_physical_locator_list_entry *ploc_entry;
> -        LIST_FOR_EACH (ploc_entry, locators_node, locators_list) {
> -            locators[i] = (struct vteprec_physical_locator *)
> -                           ploc_entry->vteprec_ploc;
> -            if (mmr_ext && !shash_find_data(&mmr_ext->physical_locators,
> -                                            locators[i]->dst_ip)) {
> -                    locator_lists_differ = true;
> -            }
> -            i++;
> -        }
> -    }
> -
> -    return locator_lists_differ;
> -}
> -
> -/* Creates a new 'Mcast_Macs_Remote' entry if needed and also cleans up
> - * out-dated remote mcast mac entries as needed. */
> -static void
> -vtep_update_mmr(struct ovsdb_idl_txn *vtep_idl_txn,
> -                struct ovs_list *locators_list,
> -                const struct vteprec_logical_switch *vtep_ls,
> -                const struct mmr_hash_node_data *mmr_ext)
> -{
> -    struct vteprec_physical_locator **locators = NULL;
> -    size_t n_locators_new = ovs_list_size(locators_list);
> -    bool mmr_changed;
> -
> -    locators = xmalloc(n_locators_new * sizeof *locators);
> -
> -    mmr_changed = vtep_process_pls(locators_list, mmr_ext, locators);
> -
> -    if (mmr_ext && !n_locators_new) {
> -        vteprec_mcast_macs_remote_delete(mmr_ext->mmr);
> -    } else if ((mmr_ext && mmr_changed) ||
> -               (!mmr_ext && n_locators_new)) {
> -
> -        const struct vteprec_physical_locator_set *ploc_set =
> -            vteprec_physical_locator_set_insert(vtep_idl_txn);
> -
> -        vtep_create_mmr(vtep_idl_txn, "unknown-dst", vtep_ls, ploc_set);
> -
> -        vteprec_physical_locator_set_set_locators(ploc_set, locators,
> -                                                  n_locators_new);
> -    }
> -    free(locators);
> -}
> -
> -/* Updates the vtep Logical_Switch table entries' tunnel keys based
> - * on the port bindings. */
> -static void
> -vtep_lswitch_run(struct shash *vtep_pbs, struct sset *vtep_pswitches,
> -                 struct shash *vtep_lswitches)
> -{
> -    struct sset used_ls = SSET_INITIALIZER(&used_ls);
> -    struct shash_node *node;
> -
> -    /* Collects the logical switch bindings from port binding entries.
> -     * Since the binding module has already guaranteed that each vtep
> -     * logical switch is bound only to one ovn-sb logical datapath,
> -     * we can just iterate and assign tunnel key to vtep logical switch.
> */
> -    SHASH_FOR_EACH (node, vtep_pbs) {
> -        const struct sbrec_port_binding *port_binding_rec = node->data;
> -        const char *pswitch_name = smap_get(&port_binding_rec->options,
> -                                            "vtep-physical-switch");
> -        const char *lswitch_name = smap_get(&port_binding_rec->options,
> -                                            "vtep-logical-switch");
> -        const struct vteprec_logical_switch *vtep_ls;
> -
> -        /* If 'port_binding_rec->chassis' exists then 'pswitch_name'
> -         * and 'lswitch_name' must also exist. */
> -        if (!pswitch_name || !lswitch_name) {
> -            /* This could only happen when someone directly modifies the
> -             * database,  (e.g. using ovn-sbctl). */
> -            VLOG_ERR("logical port (%s) with no 'options:vtep-physical-"
> -                     "switch' or 'options:vtep-logical-switch' specified "
> -                     "is bound to chassis (%s).",
> -                     port_binding_rec->logical_port,
> -                     port_binding_rec->chassis->name);
> -            continue;
> -        }
> -        vtep_ls = shash_find_data(vtep_lswitches, lswitch_name);
> -        /* Also checks 'pswitch_name' since the same 'lswitch_name' could
> -         * exist in multiple vtep database instances and be bound to
> different
> -         * ovn logical networks. */
> -        if (vtep_ls && sset_find(vtep_pswitches, pswitch_name)) {
> -            int64_t tnl_key;
> -
> -            if (sset_find(&used_ls, lswitch_name)) {
> -                continue;
> -            }
> -
> -            tnl_key = port_binding_rec->datapath->tunnel_key;
> -            if (vtep_ls->n_tunnel_key
> -                && vtep_ls->tunnel_key[0] != tnl_key) {
> -                VLOG_DBG("set vtep logical switch (%s) tunnel key from "
> -                         "(%"PRId64") to (%"PRId64")", vtep_ls->name,
> -                         vtep_ls->tunnel_key[0], tnl_key);
> -            }
> -            vteprec_logical_switch_set_tunnel_key(vtep_ls, &tnl_key, 1);
> -
> -            /* OVN is expected to always use source node replication mode,
> -             * hence the replication mode is hard-coded for each logical
> -             * switch in the context of ovn-controller-vtep. */
> -            vteprec_logical_switch_set_replication_mode(vtep_ls,
> "source_node");
> -            sset_add(&used_ls, lswitch_name);
> -        }
> -    }
> -    /* Resets the tunnel keys for unused vtep logical switches. */
> -    SHASH_FOR_EACH (node, vtep_lswitches) {
> -        if (!sset_find(&used_ls, node->name)) {
> -            int64_t tnl_key = 0;
> -            vteprec_logical_switch_set_tunnel_key(node->data, &tnl_key,
> 1);
> -        }
> -    }
> -    sset_destroy(&used_ls);
> -}
> -
> -/* Updates the vtep 'Ucast_Macs_Remote' and 'Mcast_Macs_Remote' tables
> based
> - * on non-vtep port bindings. */
> -static void
> -vtep_macs_run(struct ovsdb_idl_txn *vtep_idl_txn, struct shash
> *ucast_macs_rmts,
> -              struct shash *mcast_macs_rmts, struct shash
> *physical_locators,
> -              struct shash *vtep_lswitches, struct shash *non_vtep_pbs)
> -{
> -    struct shash_node *node;
> -    struct hmap ls_map;
> -
> -    /* Maps from ovn logical datapath tunnel key (which is also the vtep
> -     * logical switch tunnel key) to the corresponding vtep logical switch
> -     * instance.  Also, the shash map 'added_macs' is used for checking
> -     * duplicated MAC addresses in the same ovn logical datapath.
> 'mmr_ext'
> -     * is used to track mmr info per LS that needs creation/update and
> -     * 'locators_list' collects the new physical locators to be bound for
> -     * an mmr_ext; 'physical_locators' is used to track existing locators
> and
> -     * filter duplicates per logical switch. */
> -    struct ls_hash_node {
> -        struct hmap_node hmap_node;
> -
> -        const struct vteprec_logical_switch *vtep_ls;
> -        struct shash added_macs;
> -
> -        struct ovs_list locators_list;
> -        struct shash physical_locators;
> -        struct mmr_hash_node_data *mmr_ext;
> -    };
> -
> -    hmap_init(&ls_map);
> -    SHASH_FOR_EACH (node, vtep_lswitches) {
> -        const struct vteprec_logical_switch *vtep_ls = node->data;
> -        struct ls_hash_node *ls_node;
> -
> -        if (!vtep_ls->n_tunnel_key) {
> -            continue;
> -        }
> -        ls_node = xmalloc(sizeof *ls_node);
> -        ls_node->vtep_ls = vtep_ls;
> -        shash_init(&ls_node->added_macs);
> -        shash_init(&ls_node->physical_locators);
> -        ovs_list_init(&ls_node->locators_list);
> -        ls_node->mmr_ext = NULL;
> -        hmap_insert(&ls_map, &ls_node->hmap_node,
> -                    hash_uint64((uint64_t) vtep_ls->tunnel_key[0]));
> -    }
> -
> -    SHASH_FOR_EACH (node, non_vtep_pbs) {
> -        const struct sbrec_port_binding *port_binding_rec = node->data;
> -        const struct sbrec_chassis *chassis_rec;
> -        struct ls_hash_node *ls_node;
> -        const char *chassis_ip;
> -        int64_t tnl_key;
> -        size_t i;
> -
> -        chassis_rec = port_binding_rec->chassis;
> -        if (!chassis_rec) {
> -            continue;
> -        }
> -
> -        tnl_key = port_binding_rec->datapath->tunnel_key;
> -        HMAP_FOR_EACH_WITH_HASH (ls_node, hmap_node,
> -                                 hash_uint64((uint64_t) tnl_key),
> -                                 &ls_map) {
> -            if (ls_node->vtep_ls->tunnel_key[0] == tnl_key) {
> -                break;
> -            }
> -        }
> -        /* If 'ls_node' is NULL, that means no vtep logical switch is
> -         * attached to the corresponding ovn logical datapath, so pass.
> -         */
> -        if (!ls_node) {
> -            continue;
> -        }
> -
> -        chassis_ip = get_chassis_vtep_ip(chassis_rec);
> -        /* Unreachable chassis, continue. */
> -        if (!chassis_ip) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -            VLOG_INFO_RL(&rl, "VTEP tunnel encap on chassis (%s) not
> found",
> -                         chassis_rec->name);
> -            continue;
> -        }
> -
> -        const struct vteprec_physical_locator *pl =
> -            shash_find_data(physical_locators, chassis_ip);
> -        if (!pl) {
> -            pl = create_pl(vtep_idl_txn, chassis_ip);
> -            shash_add(physical_locators, chassis_ip, pl);
> -        }
> -
> -        const struct vteprec_physical_locator *ls_pl =
> -            shash_find_data(&ls_node->physical_locators, chassis_ip);
> -        if (!ls_pl) {
> -            struct vtep_rec_physical_locator_list_entry *ploc_entry =
> -                xmalloc(sizeof *ploc_entry);
> -            ploc_entry->vteprec_ploc = pl;
> -            ovs_list_push_back(&ls_node->locators_list,
> -                               &ploc_entry->locators_node);
> -            shash_add(&ls_node->physical_locators, chassis_ip, pl);
> -        }
> -
> -        char *mac_tnlkey = xasprintf("%s_%"PRId64, "unknown-dst",
> tnl_key);
> -        ls_node->mmr_ext = shash_find_data(mcast_macs_rmts, mac_tnlkey);
> -
> -        if (ls_node->mmr_ext &&
> -            ls_node->mmr_ext->mmr->logical_switch == ls_node->vtep_ls) {
> -
> -            /* Delete the entry from the hash table so the mmr does not
> get
> -             * removed from the DB later on during stale checking. */
> -            shash_find_and_delete(mcast_macs_rmts, mac_tnlkey);
> -        }
> -        free(mac_tnlkey);
> -
> -        for (i = 0; i < port_binding_rec->n_mac; i++) {
> -            const struct vteprec_ucast_macs_remote *umr;
> -            const struct sbrec_port_binding *conflict;
> -            char *mac = port_binding_rec->mac[i];
> -
> -            /* Checks for duplicate MAC in the same vtep logical switch.
> */
> -            conflict = shash_find_data(&ls_node->added_macs, mac);
> -            if (conflict) {
> -                VLOG_WARN("MAC address (%s) has already been known to be "
> -                          "on logical port (%s) in the same logical "
> -                          "datapath, so just ignore this logical port
> (%s)",
> -                          mac, conflict->logical_port,
> -                          port_binding_rec->logical_port);
> -                continue;
> -            }
> -            shash_add(&ls_node->added_macs, mac, port_binding_rec);
> -
> -            char *mac_ip_tnlkey = xasprintf("%s_%s_%"PRId64, mac,
> chassis_ip,
> -                                            tnl_key);
> -            umr = shash_find_data(ucast_macs_rmts, mac_ip_tnlkey);
> -            /* If finds the 'umr' entry for the mac, ip, and tnl_key,
> deletes
> -             * the entry from shash so that it is not gargage collected.
> -             *
> -             * If not found, creates a new 'umr' entry. */
> -            if (umr && umr->logical_switch == ls_node->vtep_ls) {
> -                shash_find_and_delete(ucast_macs_rmts, mac_ip_tnlkey);
> -            } else {
> -                const struct vteprec_ucast_macs_remote *new_umr;
> -                new_umr = create_umr(vtep_idl_txn, mac, ls_node->vtep_ls);
> -                vteprec_ucast_macs_remote_set_locator(new_umr, pl);
> -            }
> -            free(mac_ip_tnlkey);
> -        }
> -    }
> -
> -    /* Removes all remaining 'umr's, since they do not exist anymore. */
> -    SHASH_FOR_EACH (node, ucast_macs_rmts) {
> -        vteprec_ucast_macs_remote_delete(node->data);
> -    }
> -    struct ls_hash_node *iter, *next;
> -    HMAP_FOR_EACH_SAFE (iter, next, hmap_node, &ls_map) {
> -        struct vtep_rec_physical_locator_list_entry *ploc_entry;
> -        vtep_update_mmr(vtep_idl_txn, &iter->locators_list,
> -                        iter->vtep_ls, iter->mmr_ext);
> -        LIST_FOR_EACH_POP(ploc_entry, locators_node,
> -                          &iter->locators_list) {
> -            free(ploc_entry);
> -        }
> -        hmap_remove(&ls_map, &iter->hmap_node);
> -        shash_destroy(&iter->added_macs);
> -        shash_destroy(&iter->physical_locators);
> -        free(iter);
> -    }
> -    hmap_destroy(&ls_map);
> -
> -    /* Clean stale 'Mcast_Macs_Remote' */
> -    struct mmr_hash_node_data *mmr_ext;
> -    SHASH_FOR_EACH (node, mcast_macs_rmts) {
> -        mmr_ext = node->data;
> -        vteprec_mcast_macs_remote_delete(mmr_ext->mmr);
> -    }
> -}
> -
> -/* Resets all logical switches' 'tunnel_key' to NULL */
> -static bool
> -vtep_lswitch_cleanup(struct ovsdb_idl *vtep_idl)
> -{
> -   const struct vteprec_logical_switch *vtep_ls;
> -    bool done = true;
> -
> -    VTEPREC_LOGICAL_SWITCH_FOR_EACH (vtep_ls, vtep_idl) {
> -        if (vtep_ls->n_tunnel_key) {
> -            vteprec_logical_switch_set_tunnel_key(vtep_ls, NULL, 0);
> -            done = false;
> -        }
> -    }
> -
> -    return done;
> -}
> -
> -/* Removes all entries in the 'Ucast_Macs_Remote' table in the vtep
> database.
> - * Returns true when all done (i.e. no entry to remove). */
> -static bool
> -vtep_ucast_macs_cleanup(struct ovsdb_idl *vtep_idl)
> -{
> -    const struct vteprec_ucast_macs_remote *umr;
> -
> -    VTEPREC_UCAST_MACS_REMOTE_FOR_EACH (umr, vtep_idl) {
> -        vteprec_ucast_macs_remote_delete(umr);
> -        return false;
> -    }
> -
> -    return true;
> -}
> -
> -/* Removes all entries in the 'Mcast_Macs_Remote' table in vtep database.
> - * Returns true when all done (i.e. no entry to remove). */
> -static bool
> -vtep_mcast_macs_cleanup(struct ovsdb_idl *vtep_idl)
> -{
> -    const struct vteprec_mcast_macs_remote *mmr;
> -
> -    VTEPREC_MCAST_MACS_REMOTE_FOR_EACH (mmr, vtep_idl) {
> -        vteprec_mcast_macs_remote_delete(mmr);
> -        return false;
> -    }
> -
> -    return true;
> -}
> -
> -/* Updates vtep logical switch tunnel keys. */
> -void
> -vtep_run(struct controller_vtep_ctx *ctx)
> -{
> -    if (!ctx->vtep_idl_txn) {
> -        return;
> -    }
> -
> -    struct sset vtep_pswitches = SSET_INITIALIZER(&vtep_pswitches);
> -    struct shash vtep_lswitches = SHASH_INITIALIZER(&vtep_lswitches);
> -    struct shash ucast_macs_rmts = SHASH_INITIALIZER(&ucast_macs_rmts);
> -    struct shash mcast_macs_rmts = SHASH_INITIALIZER(&mcast_macs_rmts);
> -    struct shash physical_locators =
> SHASH_INITIALIZER(&physical_locators);
> -    struct shash vtep_pbs = SHASH_INITIALIZER(&vtep_pbs);
> -    struct shash non_vtep_pbs = SHASH_INITIALIZER(&non_vtep_pbs);
> -    const struct vteprec_physical_switch *vtep_ps;
> -    const struct vteprec_logical_switch *vtep_ls;
> -    const struct vteprec_ucast_macs_remote *umr;
> -    const struct sbrec_port_binding *port_binding_rec;
> -    const struct vteprec_mcast_macs_remote *mmr;
> -    struct shash_node *node;
> -
> -    /* Collects 'Physical_Switch's. */
> -    VTEPREC_PHYSICAL_SWITCH_FOR_EACH (vtep_ps, ctx->vtep_idl) {
> -        sset_add(&vtep_pswitches, vtep_ps->name);
> -    }
> -
> -    /* Collects 'Logical_Switch's. */
> -    VTEPREC_LOGICAL_SWITCH_FOR_EACH (vtep_ls, ctx->vtep_idl) {
> -        shash_add(&vtep_lswitches, vtep_ls->name, vtep_ls);
> -    }
> -
> -    /* Collects 'Ucast_Macs_Remote's. */
> -    VTEPREC_UCAST_MACS_REMOTE_FOR_EACH (umr, ctx->vtep_idl) {
> -        char *mac_ip_tnlkey =
> -            xasprintf("%s_%s_%"PRId64, umr->MAC,
> -                      umr->locator ? umr->locator->dst_ip : "",
> -                      umr->logical_switch &&
> umr->logical_switch->n_tunnel_key
> -                          ? umr->logical_switch->tunnel_key[0] :
> INT64_MAX);
> -
> -        shash_add(&ucast_macs_rmts, mac_ip_tnlkey, umr);
> -        free(mac_ip_tnlkey);
> -    }
> -
> -    /* Collects 'Mcast_Macs_Remote's. */
> -    VTEPREC_MCAST_MACS_REMOTE_FOR_EACH (mmr, ctx->vtep_idl) {
> -        struct mmr_hash_node_data *mmr_ext = xmalloc(sizeof *mmr_ext);;
> -        char *mac_tnlkey =
> -            xasprintf("%s_%"PRId64, mmr->MAC,
> -                      mmr->logical_switch &&
> mmr->logical_switch->n_tunnel_key
> -                          ? mmr->logical_switch->tunnel_key[0] :
> INT64_MAX);
> -
> -        shash_add(&mcast_macs_rmts, mac_tnlkey, mmr_ext);
> -        mmr_ext->mmr = mmr;
> -
> -        shash_init(&mmr_ext->physical_locators);
> -        for (size_t i = 0; i < mmr->locator_set->n_locators; i++) {
> -            shash_add(&mmr_ext->physical_locators,
> -                      mmr->locator_set->locators[i]->dst_ip,
> -                      mmr->locator_set->locators[i]);
> -        }
> -
> -        free(mac_tnlkey);
> -    }
> -
> -    /* Collects 'Physical_Locator's. */
> -    const struct vteprec_physical_locator *pl;
> -    VTEPREC_PHYSICAL_LOCATOR_FOR_EACH (pl, ctx->vtep_idl) {
> -        shash_add(&physical_locators, pl->dst_ip, pl);
> -    }
> -
> -    /* Collects and classifies 'Port_Binding's. */
> -    SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->ovnsb_idl) {
> -        struct shash *target =
> -            !strcmp(port_binding_rec->type, "vtep") ? &vtep_pbs :
> &non_vtep_pbs;
> -
> -        if (!port_binding_rec->chassis) {
> -            continue;
> -        }
> -        shash_add(target, port_binding_rec->logical_port,
> port_binding_rec);
> -    }
> -
> -    ovsdb_idl_txn_add_comment(ctx->vtep_idl_txn,
> -                              "ovn-controller-vtep: update logical switch
> "
> -                              "tunnel keys and 'ucast_macs_remote's");
> -
> -    vtep_lswitch_run(&vtep_pbs, &vtep_pswitches, &vtep_lswitches);
> -    vtep_macs_run(ctx->vtep_idl_txn, &ucast_macs_rmts,
> -                  &mcast_macs_rmts, &physical_locators,
> -                  &vtep_lswitches, &non_vtep_pbs);
> -
> -    sset_destroy(&vtep_pswitches);
> -    shash_destroy(&vtep_lswitches);
> -    shash_destroy(&ucast_macs_rmts);
> -    SHASH_FOR_EACH (node, &mcast_macs_rmts) {
> -        struct mmr_hash_node_data *mmr_ext = node->data;
> -        shash_destroy(&mmr_ext->physical_locators);
> -        free(mmr_ext);
> -    }
> -    shash_destroy(&mcast_macs_rmts);
> -    shash_destroy(&physical_locators);
> -    shash_destroy(&vtep_pbs);
> -    shash_destroy(&non_vtep_pbs);
> -}
> -
> -/* Cleans up all related entries in vtep.  Returns true when done (i.e.
> there
> - * is no change made to 'ctx->vtep_idl'), otherwise returns false. */
> -bool
> -vtep_cleanup(struct controller_vtep_ctx *ctx)
> -{
> -    if (!ctx->vtep_idl_txn) {
> -        return false;
> -    }
> -
> -    bool all_done;
> -
> -    ovsdb_idl_txn_add_comment(ctx->vtep_idl_txn,
> -                              "ovn-controller-vtep: cleaning up vtep "
> -                              "configuration");
> -    all_done = vtep_lswitch_cleanup(ctx->vtep_idl);
> -    all_done = vtep_ucast_macs_cleanup(ctx->vtep_idl) && all_done;
> -    all_done = vtep_mcast_macs_cleanup(ctx->vtep_idl) && all_done;
> -
> -    return all_done;
> -}
> diff --git a/ovn/controller-vtep/vtep.h b/ovn/controller-vtep/vtep.h
> deleted file mode 100644
> index 97c87b7a7..000000000
> --- a/ovn/controller-vtep/vtep.h
> +++ /dev/null
> @@ -1,27 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_VTEP_H
> -#define OVN_VTEP_H 1
> -
> -#include <stdbool.h>
> -
> -struct controller_vtep_ctx;
> -
> -void vtep_run(struct controller_vtep_ctx *);
> -bool vtep_cleanup(struct controller_vtep_ctx *);
> -
> -#endif /* ovn/controller-vtep/vtep.h */
> diff --git a/ovn/controller/.gitignore b/ovn/controller/.gitignore
> deleted file mode 100644
> index 4199a3741..000000000
> --- a/ovn/controller/.gitignore
> +++ /dev/null
> @@ -1,2 +0,0 @@
> -/ovn-controller
> -/ovn-controller.8
> diff --git a/ovn/controller/automake.mk b/ovn/controller/automake.mk
> deleted file mode 100644
> index 193ea690b..000000000
> --- a/ovn/controller/automake.mk
> +++ /dev/null
> @@ -1,32 +0,0 @@
> -bin_PROGRAMS += ovn/controller/ovn-controller
> -ovn_controller_ovn_controller_SOURCES = \
> -       ovn/controller/bfd.c \
> -       ovn/controller/bfd.h \
> -       ovn/controller/binding.c \
> -       ovn/controller/binding.h \
> -       ovn/controller/chassis.c \
> -       ovn/controller/chassis.h \
> -       ovn/controller/encaps.c \
> -       ovn/controller/encaps.h \
> -       ovn/controller/ha-chassis.c \
> -       ovn/controller/ha-chassis.h \
> -       ovn/controller/ip-mcast.c \
> -       ovn/controller/ip-mcast.h \
> -       ovn/controller/lflow.c \
> -       ovn/controller/lflow.h \
> -       ovn/controller/lport.c \
> -       ovn/controller/lport.h \
> -       ovn/controller/ofctrl.c \
> -       ovn/controller/ofctrl.h \
> -       ovn/controller/pinctrl.c \
> -       ovn/controller/pinctrl.h \
> -       ovn/controller/patch.c \
> -       ovn/controller/patch.h \
> -       ovn/controller/ovn-controller.c \
> -       ovn/controller/ovn-controller.h \
> -       ovn/controller/physical.c \
> -       ovn/controller/physical.h
> -ovn_controller_ovn_controller_LDADD = ovn/lib/libovn.la lib/
> libopenvswitch.la
> -man_MANS += ovn/controller/ovn-controller.8
> -EXTRA_DIST += ovn/controller/ovn-controller.8.xml
> -CLEANFILES += ovn/controller/ovn-controller.8
> diff --git a/ovn/controller/bfd.c b/ovn/controller/bfd.c
> deleted file mode 100644
> index 22db00af7..000000000
> --- a/ovn/controller/bfd.c
> +++ /dev/null
> @@ -1,268 +0,0 @@
> -/* Copyright (c) 2017 Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "bfd.h"
> -#include "encaps.h"
> -#include "lport.h"
> -#include "ovn-controller.h"
> -
> -#include "lib/hash.h"
> -#include "lib/sset.h"
> -#include "lib/util.h"
> -#include "lib/vswitch-idl.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn-controller.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ovn_bfd);
> -
> -void
> -bfd_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    /* NOTE: this assumes that binding.c has added the
> -     * ovsrec_interface table */
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd_status);
> -}
> -
> -void
> -bfd_calculate_active_tunnels(const struct ovsrec_bridge *br_int,
> -                             struct sset *active_tunnels)
> -{
> -    int i;
> -
> -    if (!br_int) {
> -        /* Nothing to do if integration bridge doesn't exist. */
> -        return;
> -    }
> -
> -    for (i = 0; i < br_int->n_ports; i++) {
> -        const struct ovsrec_port *port_rec = br_int->ports[i];
> -
> -        if (!strcmp(port_rec->name, br_int->name)) {
> -            continue;
> -        }
> -
> -        int j;
> -        for (j = 0; j < port_rec->n_interfaces; j++) {
> -            const struct ovsrec_interface *iface_rec;
> -            iface_rec = port_rec->interfaces[j];
> -
> -            /* Check if this is a tunnel interface. */
> -            if (smap_get(&iface_rec->options, "remote_ip")) {
> -                /* Add ovn-chassis-id if the bfd_status of the tunnel
> -                 * is active */
> -                const char *bfd = smap_get(&iface_rec->bfd, "enable");
> -                if (bfd && !strcmp(bfd, "true")) {
> -                    const char *status = smap_get(&iface_rec->bfd_status,
> -                                                  "state");
> -                    if (status && !strcmp(status, "up")) {
> -                        const char *id = smap_get(&port_rec->external_ids,
> -                                                  "ovn-chassis-id");
> -                        if (id) {
> -                            char *chassis_name = NULL;
> -
> -                            if (encaps_tunnel_id_parse(id, &chassis_name,
> -                                                       NULL)) {
> -                                if (!sset_contains(active_tunnels,
> -                                                   chassis_name)) {
> -                                    sset_add(active_tunnels,
> chassis_name);
> -                                }
> -                                free(chassis_name);
> -                            }
> -                        }
> -                    }
> -                }
> -            }
> -        }
> -    }
> -}
> -
> -/* Loops through the HA chassis groups in the SB DB and returns
> - * the set of chassis which the call can establish the BFD sessions
> - * with.
> - * Eg.
> - * If there are 2 HA chassis groups.
> - * Group name - hapgrp1
> - *   - HA chassis - (HA1, HA2, HA3)
> - *   - ref chassis - (C1, C2)
> - *
> - * Group name - hapgrp2
> - *   - HA chassis - (HA1, HA4, HA5)
> - *   - ref chassis - (C1, C3, C4)
> - *
> - * If 'our_chassis' is HA1 then this function returns
> - *  bfd chassis set - (HA2, HA3, HA4 HA5, C1, C2, C3, C4)
> - *
> - * If 'our_chassis' is C1 then this function returns
> - *  bfd chassis set - (HA1, HA2, HA3, HA4, HA5)
> - *
> - * If 'our_chassis' is HA5 then this function returns
> - *  bfd chassis set - (HA1, HA4, C1, C3, C4)
> - *
> - * If 'our_chassis' is C2 then this function returns
> - *  bfd chassis set - (HA1, HA2, HA3)
> - *
> - * If 'our_chassis' is C5 then this function returns empty bfd set.
> - */
> -static void
> -bfd_calculate_chassis(
> -    const struct sbrec_chassis *our_chassis,
> -    const struct sbrec_ha_chassis_group_table *ha_chassis_grp_table,
> -    struct sset *bfd_chassis)
> -{
> -    const struct sbrec_ha_chassis_group *ha_chassis_grp;
> -    SBREC_HA_CHASSIS_GROUP_TABLE_FOR_EACH (ha_chassis_grp,
> -                                           ha_chassis_grp_table) {
> -        bool is_ha_chassis = false;
> -        struct sset grp_chassis = SSET_INITIALIZER(&grp_chassis);
> -        const struct sbrec_ha_chassis *ha_ch;
> -        bool bfd_setup_required = false;
> -        if (ha_chassis_grp->n_ha_chassis < 2) {
> -            /* No need to consider the chassis group for BFD if
> -             * there is  1 or no chassis in it. */
> -            continue;
> -        }
> -        for (size_t i = 0; i < ha_chassis_grp->n_ha_chassis; i++) {
> -            ha_ch = ha_chassis_grp->ha_chassis[i];
> -            if (!ha_ch->chassis) {
> -                continue;
> -            }
> -            sset_add(&grp_chassis, ha_ch->chassis->name);
> -            if (our_chassis == ha_ch->chassis) {
> -                is_ha_chassis = true;
> -                bfd_setup_required = true;
> -            }
> -        }
> -
> -        if (is_ha_chassis) {
> -            /* It's an HA chassis. So add the ref_chassis to the bfd set.
> */
> -            for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) {
> -                sset_add(&grp_chassis,
> ha_chassis_grp->ref_chassis[i]->name);
> -            }
> -        } else {
> -            /* This is not an HA chassis. Check if this chassis is present
> -             * in the ref_chassis list. If so add the ha_chassis to the
> -             * sset .*/
> -            for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) {
> -                if (our_chassis == ha_chassis_grp->ref_chassis[i]) {
> -                    bfd_setup_required = true;
> -                    break;
> -                }
> -            }
> -        }
> -
> -        if (bfd_setup_required) {
> -            const char *name;
> -            SSET_FOR_EACH (name, &grp_chassis) {
> -                sset_add(bfd_chassis, name);
> -            }
> -        }
> -        sset_destroy(&grp_chassis);
> -    }
> -}
> -
> -void
> -bfd_run(const struct ovsrec_interface_table *interface_table,
> -        const struct ovsrec_bridge *br_int,
> -        const struct sbrec_chassis *chassis_rec,
> -        const struct sbrec_ha_chassis_group_table *ha_chassis_grp_table,
> -        const struct sbrec_sb_global_table *sb_global_table)
> -{
> -    if (!chassis_rec) {
> -        return;
> -    }
> -    struct sset bfd_chassis = SSET_INITIALIZER(&bfd_chassis);
> -    bfd_calculate_chassis(chassis_rec, ha_chassis_grp_table,
> -                          &bfd_chassis);
> -
> -    /* Identify tunnels ports(connected to remote chassis id) to enable
> bfd */
> -    struct sset tunnels = SSET_INITIALIZER(&tunnels);
> -    struct sset bfd_ifaces = SSET_INITIALIZER(&bfd_ifaces);
> -    for (size_t k = 0; k < br_int->n_ports; k++) {
> -        const char *tunnel_id = smap_get(&br_int->ports[k]->external_ids,
> -                                          "ovn-chassis-id");
> -        if (tunnel_id) {
> -            char *chassis_name = NULL;
> -            char *port_name = br_int->ports[k]->name;
> -
> -            sset_add(&tunnels, port_name);
> -
> -            if (encaps_tunnel_id_parse(tunnel_id, &chassis_name, NULL)) {
> -                if (sset_contains(&bfd_chassis, chassis_name)) {
> -                    sset_add(&bfd_ifaces, port_name);
> -                }
> -                free(chassis_name);
> -            }
> -        }
> -    }
> -
> -    const struct sbrec_sb_global *sb
> -        = sbrec_sb_global_table_first(sb_global_table);
> -    struct smap bfd = SMAP_INITIALIZER(&bfd);
> -    smap_add(&bfd, "enable", "true");
> -
> -    if (sb) {
> -        const char *min_rx = smap_get(&sb->options, "bfd-min-rx");
> -        const char *decay_min_rx = smap_get(&sb->options,
> "bfd-decay-min-rx");
> -        const char *min_tx = smap_get(&sb->options, "bfd-min-tx");
> -        const char *mult = smap_get(&sb->options, "bfd-mult");
> -        if (min_rx) {
> -            smap_add(&bfd, "min_rx", min_rx);
> -        }
> -        if (decay_min_rx) {
> -            smap_add(&bfd, "decay_min_rx", decay_min_rx);
> -        }
> -        if (min_tx) {
> -            smap_add(&bfd, "min_tx", min_tx);
> -        }
> -        if (mult) {
> -            smap_add(&bfd, "mult", mult);
> -        }
> -    }
> -
> -    /* Enable or disable bfd */
> -    const struct ovsrec_interface *iface;
> -    OVSREC_INTERFACE_TABLE_FOR_EACH (iface, interface_table) {
> -        if (sset_contains(&tunnels, iface->name)) {
> -            if (sset_contains(&bfd_ifaces, iface->name)) {
> -                /* We need to enable BFD for this interface. Configure the
> -                 * BFD params if
> -                 *  - If BFD was disabled earlier
> -                 *  - Or if CMS has updated BFD config options.
> -                 */
> -                if (!smap_equal(&iface->bfd, &bfd)) {
> -                    ovsrec_interface_verify_bfd(iface);
> -                    ovsrec_interface_set_bfd(iface, &bfd);
> -                    VLOG_INFO("Enabled BFD on interface %s", iface->name);
> -                }
> -            } else {
> -                /* We need to disable BFD for this interface if it was
> enabled
> -                 * earlier. */
> -                if (smap_count(&iface->bfd)) {
> -                    ovsrec_interface_verify_bfd(iface);
> -                    ovsrec_interface_set_bfd(iface, NULL);
> -                    VLOG_INFO("Disabled BFD on interface %s",
> iface->name);
> -                }
> -            }
> -        }
> -    }
> -
> -    smap_destroy(&bfd);
> -    sset_destroy(&tunnels);
> -    sset_destroy(&bfd_ifaces);
> -    sset_destroy(&bfd_chassis);
> -}
> diff --git a/ovn/controller/bfd.h b/ovn/controller/bfd.h
> deleted file mode 100644
> index 17fab5323..000000000
> --- a/ovn/controller/bfd.h
> +++ /dev/null
> @@ -1,41 +0,0 @@
> -/* Copyright (c) 2017 Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_BFD_H
> -#define OVN_BFD_H 1
> -
> -struct hmap;
> -struct ovsdb_idl;
> -struct ovsdb_idl_index;
> -struct ovsrec_bridge;
> -struct ovsrec_interface_table;
> -struct ovsrec_open_vswitch_table;
> -struct sbrec_chassis;
> -struct sbrec_sb_global_table;
> -struct sbrec_ha_chassis_group_table;
> -struct sset;
> -
> -void bfd_register_ovs_idl(struct ovsdb_idl *);
> -
> -void bfd_run(const struct ovsrec_interface_table *,
> -             const struct ovsrec_bridge *,
> -             const struct sbrec_chassis *,
> -             const struct sbrec_ha_chassis_group_table *,
> -             const struct sbrec_sb_global_table *);
> -
> -void  bfd_calculate_active_tunnels(const struct ovsrec_bridge *br_int,
> -                                   struct sset *active_tunnels);
> -
> -#endif
> diff --git a/ovn/controller/binding.c b/ovn/controller/binding.c
> deleted file mode 100644
> index ace0f811b..000000000
> --- a/ovn/controller/binding.c
> +++ /dev/null
> @@ -1,764 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "binding.h"
> -#include "ha-chassis.h"
> -#include "lflow.h"
> -#include "lport.h"
> -
> -#include "lib/bitmap.h"
> -#include "openvswitch/poll-loop.h"
> -#include "lib/sset.h"
> -#include "lib/util.h"
> -#include "lib/netdev.h"
> -#include "lib/vswitch-idl.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn-controller.h"
> -
> -VLOG_DEFINE_THIS_MODULE(binding);
> -
> -#define OVN_QOS_TYPE "linux-htb"
> -
> -struct qos_queue {
> -    struct hmap_node node;
> -    uint32_t queue_id;
> -    uint32_t max_rate;
> -    uint32_t burst;
> -};
> -
> -void
> -binding_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_open_vswitch);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_open_vswitch_col_bridges);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_bridge);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_name);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_ports);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_port);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_interfaces);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_qos);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl,
> &ovsrec_interface_col_external_ids);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd_status);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_status);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_qos);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_qos_col_type);
> -}
> -
> -static void
> -get_local_iface_ids(const struct ovsrec_bridge *br_int,
> -                    struct shash *lport_to_iface,
> -                    struct sset *local_lports,
> -                    struct sset *egress_ifaces)
> -{
> -    int i;
> -
> -    for (i = 0; i < br_int->n_ports; i++) {
> -        const struct ovsrec_port *port_rec = br_int->ports[i];
> -        const char *iface_id;
> -        int j;
> -
> -        if (!strcmp(port_rec->name, br_int->name)) {
> -            continue;
> -        }
> -
> -        for (j = 0; j < port_rec->n_interfaces; j++) {
> -            const struct ovsrec_interface *iface_rec;
> -
> -            iface_rec = port_rec->interfaces[j];
> -            iface_id = smap_get(&iface_rec->external_ids, "iface-id");
> -            int64_t ofport = iface_rec->n_ofport ? *iface_rec->ofport : 0;
> -
> -            if (iface_id && ofport > 0) {
> -                shash_add(lport_to_iface, iface_id, iface_rec);
> -                sset_add(local_lports, iface_id);
> -            }
> -
> -            /* Check if this is a tunnel interface. */
> -            if (smap_get(&iface_rec->options, "remote_ip")) {
> -                const char *tunnel_iface
> -                    = smap_get(&iface_rec->status, "tunnel_egress_iface");
> -                if (tunnel_iface) {
> -                    sset_add(egress_ifaces, tunnel_iface);
> -                }
> -            }
> -        }
> -    }
> -}
> -
> -static void
> -add_local_datapath__(struct ovsdb_idl_index
> *sbrec_datapath_binding_by_key,
> -                     struct ovsdb_idl_index
> *sbrec_port_binding_by_datapath,
> -                     struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                     const struct sbrec_datapath_binding *datapath,
> -                     bool has_local_l3gateway, int depth,
> -                     struct hmap *local_datapaths)
> -{
> -    uint32_t dp_key = datapath->tunnel_key;
> -    struct local_datapath *ld = get_local_datapath(local_datapaths,
> dp_key);
> -    if (ld) {
> -        if (has_local_l3gateway) {
> -            ld->has_local_l3gateway = true;
> -        }
> -        return;
> -    }
> -
> -    ld = xzalloc(sizeof *ld);
> -    hmap_insert(local_datapaths, &ld->hmap_node, dp_key);
> -    ld->datapath = datapath;
> -    ld->localnet_port = NULL;
> -    ld->has_local_l3gateway = has_local_l3gateway;
> -
> -    if (depth >= 100) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "datapaths nested too deep");
> -        return;
> -    }
> -
> -    struct sbrec_port_binding *target =
> -        sbrec_port_binding_index_init_row(sbrec_port_binding_by_datapath);
> -    sbrec_port_binding_index_set_datapath(target, datapath);
> -
> -    const struct sbrec_port_binding *pb;
> -    SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target,
> -                                       sbrec_port_binding_by_datapath) {
> -        if (!strcmp(pb->type, "patch")) {
> -            const char *peer_name = smap_get(&pb->options, "peer");
> -            if (peer_name) {
> -                const struct sbrec_port_binding *peer;
> -
> -                peer = lport_lookup_by_name(sbrec_port_binding_by_name,
> -                                            peer_name);
> -
> -                if (peer && peer->datapath) {
> -                    add_local_datapath__(sbrec_datapath_binding_by_key,
> -                                         sbrec_port_binding_by_datapath,
> -                                         sbrec_port_binding_by_name,
> -                                         peer->datapath, false,
> -                                         depth + 1, local_datapaths);
> -                    ld->n_peer_ports++;
> -                    ld->peer_ports = xrealloc(ld->peer_ports,
> -                                              ld->n_peer_ports *
> -                                              sizeof *ld->peer_ports);
> -                    ld->peer_ports[ld->n_peer_ports - 1] = peer;
> -                }
> -            }
> -        }
> -    }
> -    sbrec_port_binding_index_destroy_row(target);
> -}
> -
> -static void
> -add_local_datapath(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -                   struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -                   struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                   const struct sbrec_datapath_binding *datapath,
> -                   bool has_local_l3gateway, struct hmap *local_datapaths)
> -{
> -    add_local_datapath__(sbrec_datapath_binding_by_key,
> -                         sbrec_port_binding_by_datapath,
> -                         sbrec_port_binding_by_name,
> -                         datapath, has_local_l3gateway, 0,
> local_datapaths);
> -}
> -
> -static void
> -get_qos_params(const struct sbrec_port_binding *pb, struct hmap
> *queue_map)
> -{
> -    uint32_t max_rate = smap_get_int(&pb->options, "qos_max_rate", 0);
> -    uint32_t burst = smap_get_int(&pb->options, "qos_burst", 0);
> -    uint32_t queue_id = smap_get_int(&pb->options, "qdisc_queue_id", 0);
> -
> -    if ((!max_rate && !burst) || !queue_id) {
> -        /* Qos is not configured for this port. */
> -        return;
> -    }
> -
> -    struct qos_queue *node = xzalloc(sizeof *node);
> -    hmap_insert(queue_map, &node->node, hash_int(queue_id, 0));
> -    node->max_rate = max_rate;
> -    node->burst = burst;
> -    node->queue_id = queue_id;
> -}
> -
> -static const struct ovsrec_qos *
> -get_noop_qos(struct ovsdb_idl_txn *ovs_idl_txn,
> -             const struct ovsrec_qos_table *qos_table)
> -{
> -    const struct ovsrec_qos *qos;
> -    OVSREC_QOS_TABLE_FOR_EACH (qos, qos_table) {
> -        if (!strcmp(qos->type, "linux-noop")) {
> -            return qos;
> -        }
> -    }
> -
> -    if (!ovs_idl_txn) {
> -        return NULL;
> -    }
> -    qos = ovsrec_qos_insert(ovs_idl_txn);
> -    ovsrec_qos_set_type(qos, "linux-noop");
> -    return qos;
> -}
> -
> -static bool
> -set_noop_qos(struct ovsdb_idl_txn *ovs_idl_txn,
> -             const struct ovsrec_port_table *port_table,
> -             const struct ovsrec_qos_table *qos_table,
> -             struct sset *egress_ifaces)
> -{
> -    if (!ovs_idl_txn) {
> -        return false;
> -    }
> -
> -    const struct ovsrec_qos *noop_qos = get_noop_qos(ovs_idl_txn,
> qos_table);
> -    if (!noop_qos) {
> -        return false;
> -    }
> -
> -    const struct ovsrec_port *port;
> -    size_t count = 0;
> -
> -    OVSREC_PORT_TABLE_FOR_EACH (port, port_table) {
> -        if (sset_contains(egress_ifaces, port->name)) {
> -            ovsrec_port_set_qos(port, noop_qos);
> -            count++;
> -        }
> -        if (sset_count(egress_ifaces) == count) {
> -            break;
> -        }
> -    }
> -    return true;
> -}
> -
> -static void
> -set_qos_type(struct netdev *netdev, const char *type)
> -{
> -    int error = netdev_set_qos(netdev, type, NULL);
> -    if (error) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "%s: could not set qdisc type \"%s\" (%s)",
> -                     netdev_get_name(netdev), type, ovs_strerror(error));
> -    }
> -}
> -
> -static void
> -setup_qos(const char *egress_iface, struct hmap *queue_map)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5);
> -    struct netdev *netdev_phy;
> -
> -    if (!egress_iface) {
> -        /* Queues cannot be configured. */
> -        return;
> -    }
> -
> -    int error = netdev_open(egress_iface, NULL, &netdev_phy);
> -    if (error) {
> -        VLOG_WARN_RL(&rl, "%s: could not open netdev (%s)",
> -                     egress_iface, ovs_strerror(error));
> -        return;
> -    }
> -
> -    /* Check current qdisc. */
> -    const char *qdisc_type;
> -    struct smap qdisc_details;
> -
> -    smap_init(&qdisc_details);
> -    if (netdev_get_qos(netdev_phy, &qdisc_type, &qdisc_details) != 0 ||
> -        qdisc_type[0] == '\0') {
> -        smap_destroy(&qdisc_details);
> -        netdev_close(netdev_phy);
> -        /* Qos is not supported. */
> -        return;
> -    }
> -    smap_destroy(&qdisc_details);
> -
> -    /* If we're not actually being requested to do any QoS:
> -     *
> -     *     - If the current qdisc type is OVN_QOS_TYPE, then we clear the
> qdisc
> -     *       type to "".  Otherwise, it's possible that our own leftover
> qdisc
> -     *       settings could cause strange behavior on egress.  Also, QoS
> is
> -     *       expensive and may waste CPU time even if it's not really in
> use.
> -     *
> -     *       OVN isn't the only software that can configure qdiscs, and
> -     *       physical interfaces are shared resources, so there is some
> risk in
> -     *       this strategy: we could disrupt some other program's QoS.
> -     *       Probably, to entirely avoid this possibility we would need
> to add
> -     *       a configuration setting.
> -     *
> -     *     - Otherwise leave the qdisc alone. */
> -    if (hmap_is_empty(queue_map)) {
> -        if (!strcmp(qdisc_type, OVN_QOS_TYPE)) {
> -            set_qos_type(netdev_phy, "");
> -        }
> -        netdev_close(netdev_phy);
> -        return;
> -    }
> -
> -    /* Configure qdisc. */
> -    if (strcmp(qdisc_type, OVN_QOS_TYPE)) {
> -        set_qos_type(netdev_phy, OVN_QOS_TYPE);
> -    }
> -
> -    /* Check and delete if needed. */
> -    struct netdev_queue_dump dump;
> -    unsigned int queue_id;
> -    struct smap queue_details;
> -    struct qos_queue *sb_info;
> -    struct hmap consistent_queues;
> -
> -    smap_init(&queue_details);
> -    hmap_init(&consistent_queues);
> -    NETDEV_QUEUE_FOR_EACH (&queue_id, &queue_details, &dump, netdev_phy) {
> -        bool is_queue_needed = false;
> -
> -        HMAP_FOR_EACH_WITH_HASH (sb_info, node, hash_int(queue_id, 0),
> -                                 queue_map) {
> -            is_queue_needed = true;
> -            if (sb_info->max_rate ==
> -                smap_get_int(&queue_details, "max-rate", 0)
> -                && sb_info->burst == smap_get_int(&queue_details,
> "burst", 0)) {
> -                /* This queue is consistent. */
> -                hmap_insert(&consistent_queues, &sb_info->node,
> -                            hash_int(queue_id, 0));
> -                break;
> -            }
> -        }
> -
> -        if (!is_queue_needed) {
> -            error = netdev_delete_queue(netdev_phy, queue_id);
> -            if (error) {
> -                VLOG_WARN_RL(&rl, "%s: could not delete queue %u (%s)",
> -                             egress_iface, queue_id, ovs_strerror(error));
> -            }
> -        }
> -    }
> -
> -    /* Create/Update queues. */
> -    HMAP_FOR_EACH (sb_info, node, queue_map) {
> -        if (hmap_contains(&consistent_queues, &sb_info->node)) {
> -            hmap_remove(&consistent_queues, &sb_info->node);
> -            continue;
> -        }
> -
> -        smap_clear(&queue_details);
> -        smap_add_format(&queue_details, "max-rate", "%d",
> sb_info->max_rate);
> -        smap_add_format(&queue_details, "burst", "%d", sb_info->burst);
> -        error = netdev_set_queue(netdev_phy, sb_info->queue_id,
> -                                 &queue_details);
> -        if (error) {
> -            VLOG_WARN_RL(&rl, "%s: could not configure queue %u (%s)",
> -                         egress_iface, sb_info->queue_id,
> ovs_strerror(error));
> -        }
> -    }
> -    smap_destroy(&queue_details);
> -    hmap_destroy(&consistent_queues);
> -    netdev_close(netdev_phy);
> -}
> -
> -static void
> -update_local_lport_ids(struct sset *local_lport_ids,
> -                       const struct sbrec_port_binding *binding_rec)
> -{
> -        char buf[16];
> -        snprintf(buf, sizeof(buf), "%"PRId64"_%"PRId64,
> -                 binding_rec->datapath->tunnel_key,
> -                 binding_rec->tunnel_key);
> -        sset_add(local_lport_ids, buf);
> -}
> -
> -/*
> - * Get the encap from the chassis for this port. The interface
> - * may have an external_ids:encap-ip=<encap-ip> set; if so we
> - * get the corresponding encap from the chassis.
> - * If "encap-ip" external-ids is not set, we'll not bind the port
> - * to any specific encap rec. and we'll pick up a tunnel port based on
> - * the chassis name alone for the port.
> - */
> -static struct sbrec_encap *
> -sbrec_get_port_encap(const struct sbrec_chassis *chassis_rec,
> -                     const struct ovsrec_interface *iface_rec)
> -{
> -
> -    if (!iface_rec) {
> -        return NULL;
> -    }
> -
> -    const char *encap_ip = smap_get(&iface_rec->external_ids, "encap-ip");
> -    if (!encap_ip) {
> -        return NULL;
> -    }
> -
> -    struct sbrec_encap *best_encap = NULL;
> -    uint32_t best_type = 0;
> -    for (int i = 0; i < chassis_rec->n_encaps; i++) {
> -        if (!strcmp(chassis_rec->encaps[i]->ip, encap_ip)) {
> -            uint32_t tun_type =
> get_tunnel_type(chassis_rec->encaps[i]->type);
> -            if (tun_type > best_type) {
> -                best_type = tun_type;
> -                best_encap = chassis_rec->encaps[i];
> -            }
> -        }
> -    }
> -    return best_encap;
> -}
> -
> -static bool
> -is_our_chassis(const struct sbrec_chassis *chassis_rec,
> -               const struct sbrec_port_binding *binding_rec,
> -               const struct sset *active_tunnels,
> -               const struct shash *lport_to_iface,
> -               const struct sset *local_lports)
> -{
> -    const struct ovsrec_interface *iface_rec
> -        = shash_find_data(lport_to_iface, binding_rec->logical_port);
> -
> -    bool our_chassis = false;
> -    if (iface_rec
> -        || (binding_rec->parent_port && binding_rec->parent_port[0] &&
> -            sset_contains(local_lports, binding_rec->parent_port))) {
> -        /* This port is in our chassis unless it is a localport. */
> -        our_chassis = strcmp(binding_rec->type, "localport");
> -    } else if (!strcmp(binding_rec->type, "l2gateway")) {
> -        const char *chassis_id = smap_get(&binding_rec->options,
> -                                          "l2gateway-chassis");
> -        our_chassis = chassis_id && !strcmp(chassis_id,
> chassis_rec->name);
> -    } else if (!strcmp(binding_rec->type, "chassisredirect") ||
> -               !strcmp(binding_rec->type, "external")) {
> -        our_chassis =
> ha_chassis_group_contains(binding_rec->ha_chassis_group,
> -                                                chassis_rec) &&
> -
> ha_chassis_group_is_active(binding_rec->ha_chassis_group,
> -                                                 active_tunnels,
> chassis_rec);
> -    } else if (!strcmp(binding_rec->type, "l3gateway")) {
> -        const char *chassis_id = smap_get(&binding_rec->options,
> -                                          "l3gateway-chassis");
> -        our_chassis = chassis_id && !strcmp(chassis_id,
> chassis_rec->name);
> -    }
> -
> -    return our_chassis;
> -}
> -
> -static void
> -consider_local_datapath(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                        struct ovsdb_idl_txn *ovs_idl_txn,
> -                        struct ovsdb_idl_index
> *sbrec_datapath_binding_by_key,
> -                        struct ovsdb_idl_index
> *sbrec_port_binding_by_datapath,
> -                        struct ovsdb_idl_index
> *sbrec_port_binding_by_name,
> -                        const struct sset *active_tunnels,
> -                        const struct sbrec_chassis *chassis_rec,
> -                        const struct sbrec_port_binding *binding_rec,
> -                        struct hmap *qos_map,
> -                        struct hmap *local_datapaths,
> -                        struct shash *lport_to_iface,
> -                        struct sset *local_lports,
> -                        struct sset *local_lport_ids)
> -{
> -    const struct ovsrec_interface *iface_rec
> -        = shash_find_data(lport_to_iface, binding_rec->logical_port);
> -
> -    bool our_chassis = is_our_chassis(chassis_rec, binding_rec,
> active_tunnels,
> -                                      lport_to_iface, local_lports);
> -    if (iface_rec
> -        || (binding_rec->parent_port && binding_rec->parent_port[0] &&
> -            sset_contains(local_lports, binding_rec->parent_port))) {
> -        if (binding_rec->parent_port && binding_rec->parent_port[0]) {
> -            /* Add child logical port to the set of all local ports. */
> -            sset_add(local_lports, binding_rec->logical_port);
> -        }
> -        add_local_datapath(sbrec_datapath_binding_by_key,
> -                           sbrec_port_binding_by_datapath,
> -                           sbrec_port_binding_by_name,
> -                           binding_rec->datapath, false, local_datapaths);
> -        if (iface_rec && qos_map && ovs_idl_txn) {
> -            get_qos_params(binding_rec, qos_map);
> -        }
> -    } else if (!strcmp(binding_rec->type, "l2gateway")) {
> -        if (our_chassis) {
> -            sset_add(local_lports, binding_rec->logical_port);
> -            add_local_datapath(sbrec_datapath_binding_by_key,
> -                               sbrec_port_binding_by_datapath,
> -                               sbrec_port_binding_by_name,
> -                               binding_rec->datapath, false,
> local_datapaths);
> -        }
> -    } else if (!strcmp(binding_rec->type, "chassisredirect")) {
> -        if (ha_chassis_group_contains(binding_rec->ha_chassis_group,
> -                                      chassis_rec)) {
> -            add_local_datapath(sbrec_datapath_binding_by_key,
> -                               sbrec_port_binding_by_datapath,
> -                               sbrec_port_binding_by_name,
> -                               binding_rec->datapath, false,
> local_datapaths);
> -        }
> -    } else if (!strcmp(binding_rec->type, "l3gateway")) {
> -        if (our_chassis) {
> -            add_local_datapath(sbrec_datapath_binding_by_key,
> -                               sbrec_port_binding_by_datapath,
> -                               sbrec_port_binding_by_name,
> -                               binding_rec->datapath, true,
> local_datapaths);
> -        }
> -    } else if (!strcmp(binding_rec->type, "localnet")) {
> -        /* Add all localnet ports to local_lports so that we allocate ct
> zones
> -         * for them. */
> -        sset_add(local_lports, binding_rec->logical_port);
> -    } else if (!strcmp(binding_rec->type, "external")) {
> -        if (ha_chassis_group_contains(binding_rec->ha_chassis_group,
> -                                      chassis_rec)) {
> -            add_local_datapath(sbrec_datapath_binding_by_key,
> -                               sbrec_port_binding_by_datapath,
> -                               sbrec_port_binding_by_name,
> -                               binding_rec->datapath, false,
> local_datapaths);
> -        }
> -    }
> -
> -    if (our_chassis
> -        || !strcmp(binding_rec->type, "patch")
> -        || !strcmp(binding_rec->type, "localport")
> -        || !strcmp(binding_rec->type, "vtep")
> -        || !strcmp(binding_rec->type, "localnet")) {
> -        update_local_lport_ids(local_lport_ids, binding_rec);
> -    }
> -
> -    ovs_assert(ovnsb_idl_txn);
> -    if (ovnsb_idl_txn) {
> -        const char *vif_chassis = smap_get(&binding_rec->options,
> -                                           "requested-chassis");
> -        bool can_bind = !vif_chassis || !vif_chassis[0]
> -                        || !strcmp(vif_chassis, chassis_rec->name)
> -                        || !strcmp(vif_chassis, chassis_rec->hostname);
> -
> -        if (can_bind && our_chassis) {
> -            if (binding_rec->chassis != chassis_rec) {
> -                if (binding_rec->chassis) {
> -                    VLOG_INFO("Changing chassis for lport %s from %s to
> %s.",
> -                              binding_rec->logical_port,
> -                              binding_rec->chassis->name,
> -                              chassis_rec->name);
> -                } else {
> -                    VLOG_INFO("Claiming lport %s for this chassis.",
> -                              binding_rec->logical_port);
> -                }
> -                for (int i = 0; i < binding_rec->n_mac; i++) {
> -                    VLOG_INFO("%s: Claiming %s",
> -                              binding_rec->logical_port,
> binding_rec->mac[i]);
> -                }
> -                sbrec_port_binding_set_chassis(binding_rec, chassis_rec);
> -            }
> -            /* Check if the port encap binding, if any, has changed */
> -            struct sbrec_encap *encap_rec = sbrec_get_port_encap(
> -                                            chassis_rec, iface_rec);
> -            if (encap_rec && binding_rec->encap != encap_rec) {
> -                sbrec_port_binding_set_encap(binding_rec, encap_rec);
> -            }
> -        } else if (binding_rec->chassis == chassis_rec) {
> -            VLOG_INFO("Releasing lport %s from this chassis.",
> -                      binding_rec->logical_port);
> -            if (binding_rec->encap)
> -                sbrec_port_binding_set_encap(binding_rec, NULL);
> -            sbrec_port_binding_set_chassis(binding_rec, NULL);
> -        } else if (our_chassis) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_INFO_RL(&rl,
> -                         "Not claiming lport %s, chassis %s "
> -                         "requested-chassis %s",
> -                         binding_rec->logical_port,
> -                         chassis_rec->name,
> -                         vif_chassis);
> -        }
> -    }
> -}
> -
> -static void
> -consider_localnet_port(const struct sbrec_port_binding *binding_rec,
> -                       struct hmap *local_datapaths)
> -{
> -    struct local_datapath *ld
> -        = get_local_datapath(local_datapaths,
> -                             binding_rec->datapath->tunnel_key);
> -    if (!ld) {
> -        return;
> -    }
> -
> -    if (ld->localnet_port && strcmp(ld->localnet_port->logical_port,
> -                                    binding_rec->logical_port)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "localnet port '%s' already set for datapath "
> -                     "'%"PRId64"', skipping the new port '%s'.",
> -                     ld->localnet_port->logical_port,
> -                     binding_rec->datapath->tunnel_key,
> -                     binding_rec->logical_port);
> -        return;
> -    }
> -    ld->localnet_port = binding_rec;
> -}
> -
> -void
> -binding_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -            struct ovsdb_idl_txn *ovs_idl_txn,
> -            struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -            struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -            struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -            const struct ovsrec_port_table *port_table,
> -            const struct ovsrec_qos_table *qos_table,
> -            const struct sbrec_port_binding_table *port_binding_table,
> -            const struct ovsrec_bridge *br_int,
> -            const struct sbrec_chassis *chassis_rec,
> -            const struct sset *active_tunnels,
> -            struct hmap *local_datapaths, struct sset *local_lports,
> -            struct sset *local_lport_ids)
> -{
> -    if (!chassis_rec) {
> -        return;
> -    }
> -
> -    const struct sbrec_port_binding *binding_rec;
> -    struct shash lport_to_iface = SHASH_INITIALIZER(&lport_to_iface);
> -    struct sset egress_ifaces = SSET_INITIALIZER(&egress_ifaces);
> -    struct hmap qos_map;
> -
> -    hmap_init(&qos_map);
> -    if (br_int) {
> -        get_local_iface_ids(br_int, &lport_to_iface, local_lports,
> -                            &egress_ifaces);
> -    }
> -
> -    /* Run through each binding record to see if it is resident on this
> -     * chassis and update the binding accordingly.  This includes both
> -     * directly connected logical ports and children of those ports. */
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH (binding_rec, port_binding_table) {
> -        consider_local_datapath(ovnsb_idl_txn, ovs_idl_txn,
> -                                sbrec_datapath_binding_by_key,
> -                                sbrec_port_binding_by_datapath,
> -                                sbrec_port_binding_by_name,
> -                                active_tunnels, chassis_rec, binding_rec,
> -                                sset_is_empty(&egress_ifaces) ? NULL :
> -                                &qos_map, local_datapaths,
> &lport_to_iface,
> -                                local_lports, local_lport_ids);
> -
> -    }
> -
> -    /* Run through each binding record to see if it is a localnet port
> -     * on local datapaths discovered from above loop, and update the
> -     * corresponding local datapath accordingly. */
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH (binding_rec, port_binding_table) {
> -        if (!strcmp(binding_rec->type, "localnet")) {
> -            consider_localnet_port(binding_rec, local_datapaths);
> -        }
> -    }
> -
> -    if (!sset_is_empty(&egress_ifaces)
> -        && set_noop_qos(ovs_idl_txn, port_table, qos_table,
> &egress_ifaces)) {
> -        const char *entry;
> -        SSET_FOR_EACH (entry, &egress_ifaces) {
> -            setup_qos(entry, &qos_map);
> -        }
> -    }
> -
> -    shash_destroy(&lport_to_iface);
> -    sset_destroy(&egress_ifaces);
> -    hmap_destroy(&qos_map);
> -}
> -
> -/* Returns true if port-binding changes potentially require flow changes
> on
> - * the current chassis. Returns false if we are sure there is no impact.
> */
> -bool
> -binding_evaluate_port_binding_changes(
> -        const struct sbrec_port_binding_table *pb_table,
> -        const struct ovsrec_bridge *br_int,
> -        const struct sbrec_chassis *chassis_rec,
> -        struct sset *active_tunnels,
> -        struct sset *local_lports)
> -{
> -    if (!chassis_rec) {
> -        return true;
> -    }
> -
> -    bool changed = false;
> -
> -    const struct sbrec_port_binding *binding_rec;
> -    struct shash lport_to_iface = SHASH_INITIALIZER(&lport_to_iface);
> -    struct sset egress_ifaces = SSET_INITIALIZER(&egress_ifaces);
> -    if (br_int) {
> -        get_local_iface_ids(br_int, &lport_to_iface, local_lports,
> -                            &egress_ifaces);
> -    }
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (binding_rec, pb_table) {
> -        /* XXX: currently OVSDB change tracking doesn't support getting
> old
> -         * data when the operation is update, so if a port-binding moved
> from
> -         * this chassis to another, there is no easy way to find out the
> -         * change. To workaround this problem, we just makes sure if
> -         * any port *related to* this chassis has any change, then trigger
> -         * recompute.
> -         *
> -         * - If a regular VIF is unbound from this chassis, the local
> ovsdb
> -         *   interface table will be updated, which will trigger
> recompute.
> -         *
> -         * - If the port is not a regular VIF, always trigger recompute.
> */
> -        if (binding_rec->chassis == chassis_rec
> -            || is_our_chassis(chassis_rec, binding_rec,
> -                              active_tunnels, &lport_to_iface,
> local_lports)
> -            || strcmp(binding_rec->type, "")) {
> -            changed = true;
> -            break;
> -        }
> -    }
> -
> -    shash_destroy(&lport_to_iface);
> -    sset_destroy(&egress_ifaces);
> -    return changed;
> -}
> -
> -/* Returns true if the database is all cleaned up, false if more work is
> - * required. */
> -bool
> -binding_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                const struct sbrec_port_binding_table *port_binding_table,
> -                const struct sbrec_chassis *chassis_rec)
> -{
> -    if (!ovnsb_idl_txn) {
> -        return false;
> -    }
> -    if (!chassis_rec) {
> -        return true;
> -    }
> -
> -    const struct sbrec_port_binding *binding_rec;
> -    bool any_changes = false;
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH (binding_rec, port_binding_table) {
> -        if (binding_rec->chassis == chassis_rec) {
> -            if (binding_rec->encap)
> -                sbrec_port_binding_set_encap(binding_rec, NULL);
> -            sbrec_port_binding_set_chassis(binding_rec, NULL);
> -            any_changes = true;
> -        }
> -    }
> -
> -    if (any_changes) {
> -        ovsdb_idl_txn_add_comment(
> -            ovnsb_idl_txn,
> -            "ovn-controller: removing all port bindings for '%s'",
> -            chassis_rec->name);
> -    }
> -
> -    return !any_changes;
> -}
> diff --git a/ovn/controller/binding.h b/ovn/controller/binding.h
> deleted file mode 100644
> index 8d9492630..000000000
> --- a/ovn/controller/binding.h
> +++ /dev/null
> @@ -1,57 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_BINDING_H
> -#define OVN_BINDING_H 1
> -
> -#include <stdbool.h>
> -
> -struct hmap;
> -struct ovsdb_idl;
> -struct ovsdb_idl_index;
> -struct ovsdb_idl_txn;
> -struct ovsrec_bridge;
> -struct ovsrec_port_table;
> -struct ovsrec_qos_table;
> -struct sbrec_chassis;
> -struct sbrec_port_binding_table;
> -struct sset;
> -
> -void binding_register_ovs_idl(struct ovsdb_idl *);
> -void binding_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                 struct ovsdb_idl_txn *ovs_idl_txn,
> -                 struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                 const struct ovsrec_port_table *,
> -                 const struct ovsrec_qos_table *,
> -                 const struct sbrec_port_binding_table *,
> -                 const struct ovsrec_bridge *br_int,
> -                 const struct sbrec_chassis *,
> -                 const struct sset *active_tunnels,
> -                 struct hmap *local_datapaths,
> -                 struct sset *local_lports, struct sset *local_lport_ids);
> -bool binding_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                     const struct sbrec_port_binding_table *,
> -                     const struct sbrec_chassis *);
> -bool binding_evaluate_port_binding_changes(
> -        const struct sbrec_port_binding_table *,
> -        const struct ovsrec_bridge *br_int,
> -        const struct sbrec_chassis *,
> -        struct sset *active_tunnels,
> -        struct sset *local_lports);
> -
> -#endif /* ovn/binding.h */
> diff --git a/ovn/controller/chassis.c b/ovn/controller/chassis.c
> deleted file mode 100644
> index 04b98d86c..000000000
> --- a/ovn/controller/chassis.c
> +++ /dev/null
> @@ -1,671 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include <unistd.h>
> -
> -#include "chassis.h"
> -
> -#include "lib/smap.h"
> -#include "lib/sset.h"
> -#include "lib/vswitch-idl.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/vlog.h"
> -#include "openvswitch/ofp-parse.h"
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn-controller.h"
> -#include "lib/util.h"
> -
> -VLOG_DEFINE_THIS_MODULE(chassis);
> -
> -#ifndef HOST_NAME_MAX
> -/* For windows. */
> -#define HOST_NAME_MAX 255
> -#endif /* HOST_NAME_MAX */
> -
> -/*
> - * Structure to hold chassis specific state (currently just chassis-id)
> - * to avoid database lookups when changes happen while the controller is
> - * running.
> - */
> -struct chassis_info {
> -    /* Last ID we initialized the Chassis SB record with. */
> -    struct ds id;
> -
> -    /* True if Chassis SB record is initialized, false otherwise. */
> -    uint32_t id_inited : 1;
> -};
> -
> -static struct chassis_info chassis_state = {
> -    .id = DS_EMPTY_INITIALIZER,
> -    .id_inited = false,
> -};
> -
> -static void
> -chassis_info_set_id(struct chassis_info *info, const char *id)
> -{
> -    ds_clear(&info->id);
> -    ds_put_cstr(&info->id, id);
> -    info->id_inited = true;
> -}
> -
> -static bool
> -chassis_info_id_inited(const struct chassis_info *info)
> -{
> -    return info->id_inited;
> -}
> -
> -static const char *
> -chassis_info_id(const struct chassis_info *info)
> -{
> -    return ds_cstr_ro(&info->id);
> -}
> -
> -/*
> - * Structure for storing the chassis config parsed from the ovs table.
> - */
> -struct ovs_chassis_cfg {
> -    /* Single string fields parsed from external-ids. */
> -    const char *hostname;
> -    const char *bridge_mappings;
> -    const char *datapath_type;
> -    const char *encap_csum;
> -    const char *cms_options;
> -    const char *chassis_macs;
> -
> -    /* Set of encap types parsed from the 'ovn-encap-type' external-id. */
> -    struct sset encap_type_set;
> -    /* Set of encap IPs parsed from the 'ovn-encap-type' external-id. */
> -    struct sset encap_ip_set;
> -    /* Interface type list formatted in the OVN-SB Chassis required
> format. */
> -    struct ds iface_types;
> -};
> -
> -static void
> -ovs_chassis_cfg_init(struct ovs_chassis_cfg *cfg)
> -{
> -    sset_init(&cfg->encap_type_set);
> -    sset_init(&cfg->encap_ip_set);
> -    ds_init(&cfg->iface_types);
> -}
> -
> -static void
> -ovs_chassis_cfg_destroy(struct ovs_chassis_cfg *cfg)
> -{
> -    sset_destroy(&cfg->encap_type_set);
> -    sset_destroy(&cfg->encap_ip_set);
> -    ds_destroy(&cfg->iface_types);
> -}
> -
> -void
> -chassis_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_open_vswitch);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_open_vswitch_col_external_ids);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_open_vswitch_col_iface_types);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_bridge);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_datapath_type);
> -}
> -
> -static const char *
> -get_hostname(const struct smap *ext_ids)
> -{
> -    const char *hostname = smap_get_def(ext_ids, "hostname", "");
> -
> -    if (strlen(hostname) == 0) {
> -        static char hostname_[HOST_NAME_MAX + 1];
> -
> -        if (gethostname(hostname_, sizeof(hostname_))) {
> -            hostname_[0] = 0;
> -        }
> -
> -        return &hostname_[0];
> -    }
> -
> -    return hostname;
> -}
> -
> -static const char *
> -get_bridge_mappings(const struct smap *ext_ids)
> -{
> -    return smap_get_def(ext_ids, "ovn-bridge-mappings", "");
> -}
> -
> -static const char *
> -get_chassis_mac_mappings(const struct smap *ext_ids)
> -{
> -    return smap_get_def(ext_ids, "ovn-chassis-mac-mappings", "");
> -}
> -
> -static const char *
> -get_cms_options(const struct smap *ext_ids)
> -{
> -    return smap_get_def(ext_ids, "ovn-cms-options", "");
> -}
> -
> -static const char *
> -get_encap_csum(const struct smap *ext_ids)
> -{
> -    return smap_get_def(ext_ids, "ovn-encap-csum", "true");
> -}
> -
> -static const char *
> -get_datapath_type(const struct ovsrec_bridge *br_int)
> -{
> -    if (br_int && br_int->datapath_type) {
> -        return br_int->datapath_type;
> -    }
> -
> -    return "";
> -}
> -
> -static void
> -update_chassis_transport_zones(const struct sset *transport_zones,
> -                               const struct sbrec_chassis *chassis_rec)
> -{
> -    struct sset chassis_tzones_set =
> SSET_INITIALIZER(&chassis_tzones_set);
> -    for (int i = 0; i < chassis_rec->n_transport_zones; i++) {
> -        sset_add(&chassis_tzones_set, chassis_rec->transport_zones[i]);
> -    }
> -
> -    /* Only update the transport zones if something changed */
> -    if (!sset_equals(transport_zones, &chassis_tzones_set)) {
> -        const char **ls_arr = sset_array(transport_zones);
> -        sbrec_chassis_set_transport_zones(chassis_rec, ls_arr,
> -                                          sset_count(transport_zones));
> -        free(ls_arr);
> -    }
> -
> -    sset_destroy(&chassis_tzones_set);
> -}
> -
> -/*
> - * Parse an ovs 'encap_type' string and stores the resulting types in the
> - * 'encap_type_set' string set.
> - */
> -static bool
> -chassis_parse_ovs_encap_type(const char *encap_type,
> -                             struct sset *encap_type_set)
> -{
> -    sset_from_delimited_string(encap_type_set, encap_type, ",");
> -
> -    const char *type;
> -
> -    SSET_FOR_EACH (type, encap_type_set) {
> -        if (!get_tunnel_type(type)) {
> -            VLOG_INFO("Unknown tunnel type: %s", type);
> -        }
> -    }
> -
> -    return true;
> -}
> -
> -/*
> - * Parse an ovs 'encap_ip' string and stores the resulting IP
> representations
> - * in the 'encap_ip_set' string set.
> - */
> -static bool
> -chassis_parse_ovs_encap_ip(const char *encap_ip, struct sset
> *encap_ip_set)
> -{
> -    sset_from_delimited_string(encap_ip_set, encap_ip, ",");
> -    return true;
> -}
> -
> -/*
> - * Parse the ovs 'iface_types' and store them in the format required by
> the
> - * Chassis record.
> - */
> -static bool
> -chassis_parse_ovs_iface_types(char **iface_types, size_t n_iface_types,
> -                              struct ds *iface_types_str)
> -{
> -    for (size_t i = 0; i < n_iface_types; i++) {
> -        ds_put_format(iface_types_str, "%s,", iface_types[i]);
> -    }
> -    ds_chomp(iface_types_str, ',');
> -    return true;
> -}
> -
> -/*
> - * Parse the 'ovs_table' entry and populate 'ovs_cfg'.
> - */
> -static bool
> -chassis_parse_ovs_config(const struct ovsrec_open_vswitch_table
> *ovs_table,
> -                         const struct ovsrec_bridge *br_int,
> -                         struct ovs_chassis_cfg *ovs_cfg)
> -{
> -    const struct ovsrec_open_vswitch *cfg =
> -        ovsrec_open_vswitch_table_first(ovs_table);
> -
> -    if (!cfg) {
> -        VLOG_INFO("No Open_vSwitch row defined.");
> -        return false;
> -    }
> -
> -    const char *encap_type = smap_get(&cfg->external_ids,
> "ovn-encap-type");
> -    const char *encap_ips = smap_get(&cfg->external_ids, "ovn-encap-ip");
> -    if (!encap_type || !encap_ips) {
> -        VLOG_INFO("Need to specify an encap type and ip");
> -        return false;
> -    }
> -
> -    ovs_cfg->hostname = get_hostname(&cfg->external_ids);
> -    ovs_cfg->bridge_mappings = get_bridge_mappings(&cfg->external_ids);
> -    ovs_cfg->datapath_type = get_datapath_type(br_int);
> -    ovs_cfg->encap_csum = get_encap_csum(&cfg->external_ids);
> -    ovs_cfg->cms_options = get_cms_options(&cfg->external_ids);
> -    ovs_cfg->chassis_macs = get_chassis_mac_mappings(&cfg->external_ids);
> -
> -    if (!chassis_parse_ovs_encap_type(encap_type,
> &ovs_cfg->encap_type_set)) {
> -        return false;
> -    }
> -
> -    if (!chassis_parse_ovs_encap_ip(encap_ips, &ovs_cfg->encap_ip_set)) {
> -        sset_destroy(&ovs_cfg->encap_type_set);
> -        return false;
> -    }
> -
> -    if (!chassis_parse_ovs_iface_types(cfg->iface_types,
> -                                       cfg->n_iface_types,
> -                                       &ovs_cfg->iface_types)) {
> -        sset_destroy(&ovs_cfg->encap_type_set);
> -        sset_destroy(&ovs_cfg->encap_ip_set);
> -    }
> -
> -    return true;
> -}
> -
> -static void
> -chassis_build_external_ids(struct smap *ext_ids, const char
> *bridge_mappings,
> -                           const char *datapath_type, const char
> *cms_options,
> -                           const char *chassis_macs, const char
> *iface_types)
> -{
> -    smap_replace(ext_ids, "ovn-bridge-mappings", bridge_mappings);
> -    smap_replace(ext_ids, "datapath-type", datapath_type);
> -    smap_replace(ext_ids, "ovn-cms-options", cms_options);
> -    smap_replace(ext_ids, "iface-types", iface_types);
> -    smap_replace(ext_ids, "ovn-chassis-mac-mappings", chassis_macs);
> -}
> -
> -/*
> - * Returns true if any external-id doesn't match the values in
> 'chassis-rec'.
> - */
> -static bool
> -chassis_external_ids_changed(const char *bridge_mappings,
> -                             const char *datapath_type,
> -                             const char *cms_options,
> -                             const char *chassis_macs,
> -                             const struct ds *iface_types,
> -                             const struct sbrec_chassis *chassis_rec)
> -{
> -    const char *chassis_bridge_mappings =
> -        get_bridge_mappings(&chassis_rec->external_ids);
> -
> -    if (strcmp(bridge_mappings, chassis_bridge_mappings)) {
> -        return true;
> -    }
> -
> -    const char *chassis_datapath_type =
> -        smap_get_def(&chassis_rec->external_ids, "datapath-type", "");
> -
> -    if (strcmp(datapath_type, chassis_datapath_type)) {
> -        return true;
> -    }
> -
> -    const char *chassis_cms_options =
> -        get_cms_options(&chassis_rec->external_ids);
> -
> -    if (strcmp(cms_options, chassis_cms_options)) {
> -        return true;
> -    }
> -
> -    const char *chassis_mac_mappings =
> -        get_chassis_mac_mappings(&chassis_rec->external_ids);
> -    if (strcmp(chassis_macs, chassis_mac_mappings)) {
> -        return true;
> -    }
> -
> -    const char *chassis_iface_types =
> -        smap_get_def(&chassis_rec->external_ids, "iface-types", "");
> -
> -    if (strcmp(ds_cstr_ro(iface_types), chassis_iface_types)) {
> -        return true;
> -    }
> -
> -    return false;
> -}
> -
> -/*
> - * Returns true if the tunnel config obtained by combining
> 'encap_type_set'
> - * with 'encap_ip_set' and 'encap_csum' doesn't match the values in
> - * 'chassis-rec'.
> - */
> -static bool
> -chassis_tunnels_changed(const struct sset *encap_type_set,
> -                        const struct sset *encap_ip_set,
> -                        const char *encap_csum,
> -                        const struct sbrec_chassis *chassis_rec)
> -{
> -    size_t encap_type_count = 0;
> -
> -    for (int i = 0; i < chassis_rec->n_encaps; i++) {
> -        if (strcmp(chassis_rec->name,
> chassis_rec->encaps[i]->chassis_name)) {
> -            return true;
> -        }
> -
> -        if (!sset_contains(encap_type_set, chassis_rec->encaps[i]->type))
> {
> -            return true;
> -        }
> -        encap_type_count++;
> -
> -        if (!sset_contains(encap_ip_set, chassis_rec->encaps[i]->ip)) {
> -            return true;
> -        }
> -
> -        if (strcmp(smap_get_def(&chassis_rec->encaps[i]->options, "csum",
> ""),
> -                   encap_csum)) {
> -            return true;
> -        }
> -    }
> -
> -    size_t tunnel_count =
> -        sset_count(encap_type_set) * sset_count(encap_ip_set);
> -
> -    if (tunnel_count != chassis_rec->n_encaps) {
> -        return true;
> -    }
> -
> -    if (sset_count(encap_type_set) != encap_type_count) {
> -        return true;
> -    }
> -
> -    return false;
> -}
> -
> -/*
> - * Build the new encaps config (full mesh of 'encap_type_set' and
> - * 'encap_ip_set'). Allocates and stores the new 'n_encap' Encap records
> in
> - * 'encaps'.
> - */
> -static struct sbrec_encap **
> -chassis_build_encaps(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                     const struct sset *encap_type_set,
> -                     const struct sset *encap_ip_set,
> -                     const char *chassis_id,
> -                     const char *encap_csum,
> -                     size_t *n_encap)
> -{
> -    size_t tunnel_count = 0;
> -
> -    struct sbrec_encap **encaps =
> -        xmalloc(sset_count(encap_type_set) * sset_count(encap_ip_set) *
> -                sizeof(*encaps));
> -    const struct smap options = SMAP_CONST1(&options, "csum", encap_csum);
> -
> -    const char *encap_ip;
> -    const char *encap_type;
> -
> -    SSET_FOR_EACH (encap_ip, encap_ip_set) {
> -        SSET_FOR_EACH (encap_type, encap_type_set) {
> -            struct sbrec_encap *encap = sbrec_encap_insert(ovnsb_idl_txn);
> -
> -            sbrec_encap_set_type(encap, encap_type);
> -            sbrec_encap_set_ip(encap, encap_ip);
> -            sbrec_encap_set_options(encap, &options);
> -            sbrec_encap_set_chassis_name(encap, chassis_id);
> -
> -            encaps[tunnel_count] = encap;
> -            tunnel_count++;
> -        }
> -    }
> -
> -    *n_encap = tunnel_count;
> -    return encaps;
> -}
> -
> -/*
> - * Returns a pointer to a chassis record from 'chassis_table' that
> - * matches at least one tunnel config.
> - */
> -static const struct sbrec_chassis *
> -chassis_get_stale_record(const struct sbrec_chassis_table *chassis_table,
> -                         const struct ovs_chassis_cfg *ovs_cfg,
> -                         const char *chassis_id)
> -{
> -    const struct sbrec_chassis *chassis_rec;
> -
> -    SBREC_CHASSIS_TABLE_FOR_EACH (chassis_rec, chassis_table) {
> -        for (size_t i = 0; i < chassis_rec->n_encaps; i++) {
> -            if (sset_contains(&ovs_cfg->encap_type_set,
> -                              chassis_rec->encaps[i]->type) &&
> -                    sset_contains(&ovs_cfg->encap_ip_set,
> -                                  chassis_rec->encaps[i]->ip)) {
> -                return chassis_rec;
> -            }
> -            if (strcmp(chassis_rec->name, chassis_id) == 0) {
> -                return chassis_rec;
> -            }
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -/* If this is a chassis config update after we initialized the record once
> - * then we should always be able to find it with the ID we saved in
> - * chassis_state.
> - * Otherwise (i.e., first time we create the record) then we check if
> there's
> - * a stale record from a previous controller run that didn't end
> gracefully
> - * and reuse it. If not then we create a new record.
> - */
> -static const struct sbrec_chassis *
> -chassis_get_record(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                   struct ovsdb_idl_index *sbrec_chassis_by_name,
> -                   const struct sbrec_chassis_table *chassis_table,
> -                   const struct ovs_chassis_cfg *ovs_cfg,
> -                   const char *chassis_id)
> -{
> -    const struct sbrec_chassis *chassis_rec;
> -
> -    if (chassis_info_id_inited(&chassis_state)) {
> -        chassis_rec = chassis_lookup_by_name(sbrec_chassis_by_name,
> -
>  chassis_info_id(&chassis_state));
> -        if (!chassis_rec) {
> -            VLOG_WARN("Could not find Chassis : stored (%s) ovs (%s)",
> -                      chassis_info_id(&chassis_state), chassis_id);
> -        }
> -    } else {
> -        chassis_rec =
> -            chassis_get_stale_record(chassis_table, ovs_cfg, chassis_id);
> -
> -        if (!chassis_rec && ovnsb_idl_txn) {
> -            chassis_rec = sbrec_chassis_insert(ovnsb_idl_txn);
> -        }
> -    }
> -    return chassis_rec;
> -}
> -
> -/* Update a Chassis record based on the config in the ovs config. */
> -static void
> -chassis_update(const struct sbrec_chassis *chassis_rec,
> -               struct ovsdb_idl_txn *ovnsb_idl_txn,
> -               const struct ovs_chassis_cfg *ovs_cfg,
> -               const char *chassis_id,
> -               const struct sset *transport_zones)
> -{
> -    if (strcmp(chassis_id, chassis_rec->name)) {
> -        sbrec_chassis_set_name(chassis_rec, chassis_id);
> -    }
> -
> -    if (strcmp(ovs_cfg->hostname, chassis_rec->hostname)) {
> -        sbrec_chassis_set_hostname(chassis_rec, ovs_cfg->hostname);
> -    }
> -
> -    if (chassis_external_ids_changed(ovs_cfg->bridge_mappings,
> -                                     ovs_cfg->datapath_type,
> -                                     ovs_cfg->cms_options,
> -                                     ovs_cfg->chassis_macs,
> -                                     &ovs_cfg->iface_types,
> -                                     chassis_rec)) {
> -        struct smap ext_ids;
> -
> -        smap_clone(&ext_ids, &chassis_rec->external_ids);
> -        chassis_build_external_ids(&ext_ids, ovs_cfg->bridge_mappings,
> -                                   ovs_cfg->datapath_type,
> -                                   ovs_cfg->cms_options,
> -                                   ovs_cfg->chassis_macs,
> -                                   ds_cstr_ro(&ovs_cfg->iface_types));
> -        sbrec_chassis_verify_external_ids(chassis_rec);
> -        sbrec_chassis_set_external_ids(chassis_rec, &ext_ids);
> -        smap_destroy(&ext_ids);
> -    }
> -
> -    update_chassis_transport_zones(transport_zones, chassis_rec);
> -
> -    /* If any of the encaps should change, update them. */
> -    bool tunnels_changed =
> -        chassis_tunnels_changed(&ovs_cfg->encap_type_set,
> -                                &ovs_cfg->encap_ip_set,
> ovs_cfg->encap_csum,
> -                                chassis_rec);
> -    if (!tunnels_changed) {
> -        return;
> -    }
> -
> -    struct sbrec_encap **encaps;
> -    size_t n_encap;
> -
> -    encaps =
> -        chassis_build_encaps(ovnsb_idl_txn, &ovs_cfg->encap_type_set,
> -                             &ovs_cfg->encap_ip_set, chassis_id,
> -                             ovs_cfg->encap_csum, &n_encap);
> -    sbrec_chassis_set_encaps(chassis_rec, encaps, n_encap);
> -    free(encaps);
> -}
> -
> -/* Returns this chassis's Chassis record, if it is available. */
> -const struct sbrec_chassis *
> -chassis_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -            struct ovsdb_idl_index *sbrec_chassis_by_name,
> -            const struct ovsrec_open_vswitch_table *ovs_table,
> -            const struct sbrec_chassis_table *chassis_table,
> -            const char *chassis_id,
> -            const struct ovsrec_bridge *br_int,
> -            const struct sset *transport_zones)
> -{
> -    struct ovs_chassis_cfg ovs_cfg;
> -
> -    /* Get the chassis config from the ovs table. */
> -    ovs_chassis_cfg_init(&ovs_cfg);
> -    if (!chassis_parse_ovs_config(ovs_table, br_int, &ovs_cfg)) {
> -        return NULL;
> -    }
> -
> -    const struct sbrec_chassis *chassis_rec =
> -        chassis_get_record(ovnsb_idl_txn, sbrec_chassis_by_name,
> -                           chassis_table, &ovs_cfg, chassis_id);
> -
> -    /* If we found (or created) a record, update it with the correct
> config
> -     * and store the current chassis_id for fast lookup in case it gets
> -     * modified in the ovs table.
> -     */
> -    if (chassis_rec && ovnsb_idl_txn) {
> -        chassis_update(chassis_rec, ovnsb_idl_txn, &ovs_cfg, chassis_id,
> -                       transport_zones);
> -        chassis_info_set_id(&chassis_state, chassis_id);
> -        ovsdb_idl_txn_add_comment(ovnsb_idl_txn,
> -                                  "ovn-controller: registering chassis
> '%s'",
> -                                  chassis_id);
> -    }
> -
> -    ovs_chassis_cfg_destroy(&ovs_cfg);
> -    return chassis_rec;
> -}
> -
> -bool
> -chassis_get_mac(const struct sbrec_chassis *chassis_rec,
> -                const char *bridge_mapping,
> -                struct eth_addr *chassis_mac)
> -{
> -    const char *tokens
> -        = get_chassis_mac_mappings(&chassis_rec->external_ids);
> -    if (!tokens[0]) {
> -       return false;
> -    }
> -
> -    char *save_ptr = NULL;
> -    bool ret = false;
> -    char *tokstr = xstrdup(tokens);
> -
> -    /* Format for a chassis mac configuration is:
> -     * ovn-chassis-mac-mappings="bridge-name1:MAC1,bridge-name2:MAC2"
> -     */
> -    for (char *token = strtok_r(tokstr, ",", &save_ptr);
> -         token != NULL;
> -         token = strtok_r(NULL, ",", &save_ptr)) {
> -        char *save_ptr2 = NULL;
> -        char *chassis_mac_bridge = strtok_r(token, ":", &save_ptr2);
> -        char *chassis_mac_str = strtok_r(NULL, "", &save_ptr2);
> -
> -        if (!strcmp(chassis_mac_bridge, bridge_mapping)) {
> -            struct eth_addr temp_mac;
> -
> -            /* Return the first chassis mac. */
> -            char *err_str = str_to_mac(chassis_mac_str, &temp_mac);
> -            if (err_str) {
> -                free(err_str);
> -                continue;
> -            }
> -
> -            ret = true;
> -            *chassis_mac = temp_mac;
> -            break;
> -        }
> -    }
> -
> -    free(tokstr);
> -    return ret;
> -}
> -
> -/* Returns true if the database is all cleaned up, false if more work is
> - * required. */
> -bool
> -chassis_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                const struct sbrec_chassis *chassis_rec)
> -{
> -    if (!chassis_rec) {
> -        return true;
> -    }
> -    if (ovnsb_idl_txn) {
> -        ovsdb_idl_txn_add_comment(ovnsb_idl_txn,
> -                                  "ovn-controller: unregistering chassis
> '%s'",
> -                                  chassis_rec->name);
> -        sbrec_chassis_delete(chassis_rec);
> -    }
> -    return false;
> -}
> -
> -/*
> - * Returns the last initialized chassis-id.
> - */
> -const char *
> -chassis_get_id(void)
> -{
> -    if (chassis_info_id_inited(&chassis_state)) {
> -        return chassis_info_id(&chassis_state);
> -    }
> -
> -    return NULL;
> -}
> diff --git a/ovn/controller/chassis.h b/ovn/controller/chassis.h
> deleted file mode 100644
> index 16a131a3b..000000000
> --- a/ovn/controller/chassis.h
> +++ /dev/null
> @@ -1,46 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_CHASSIS_H
> -#define OVN_CHASSIS_H 1
> -
> -#include <stdbool.h>
> -
> -struct ovsdb_idl;
> -struct ovsdb_idl_index;
> -struct ovsdb_idl_txn;
> -struct ovsrec_bridge;
> -struct ovsrec_open_vswitch_table;
> -struct sbrec_chassis;
> -struct sbrec_chassis_table;
> -struct sset;
> -struct eth_addr;
> -
> -void chassis_register_ovs_idl(struct ovsdb_idl *);
> -const struct sbrec_chassis *chassis_run(
> -    struct ovsdb_idl_txn *ovnsb_idl_txn,
> -    struct ovsdb_idl_index *sbrec_chassis_by_name,
> -    const struct ovsrec_open_vswitch_table *,
> -    const struct sbrec_chassis_table *,
> -    const char *chassis_id, const struct ovsrec_bridge *br_int,
> -    const struct sset *transport_zones);
> -bool chassis_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                     const struct sbrec_chassis *);
> -bool chassis_get_mac(const struct sbrec_chassis *chassis,
> -                     const char *bridge_mapping,
> -                     struct eth_addr *chassis_mac);
> -const char *chassis_get_id(void);
> -
> -#endif /* ovn/chassis.h */
> diff --git a/ovn/controller/encaps.c b/ovn/controller/encaps.c
> deleted file mode 100644
> index d4a436df3..000000000
> --- a/ovn/controller/encaps.c
> +++ /dev/null
> @@ -1,409 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "encaps.h"
> -
> -#include "lib/hash.h"
> -#include "lib/sset.h"
> -#include "lib/util.h"
> -#include "lib/vswitch-idl.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn-controller.h"
> -
> -VLOG_DEFINE_THIS_MODULE(encaps);
> -
> -/*
> - * Given there could be multiple tunnels with different IPs to the same
> - * chassis we annotate the ovn-chassis-id with
> - * <chassis_name>OVN_MVTEP_CHASSISID_DELIM<IP>.
> - */
> -#define        OVN_MVTEP_CHASSISID_DELIM '@'
> -
> -void
> -encaps_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_bridge);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_ports);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_port);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_interfaces);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_external_ids);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_type);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_options);
> -}
> -
> -/* Enough context to create a new tunnel, using tunnel_add(). */
> -struct tunnel_ctx {
> -    /* Maps from a chassis name to "struct chassis_node *". */
> -    struct shash chassis;
> -
> -    /* Names of all ports in the bridge, to allow checking uniqueness when
> -     * adding a new tunnel. */
> -    struct sset port_names;
> -
> -    struct ovsdb_idl_txn *ovs_txn;
> -    const struct ovsrec_bridge *br_int;
> -};
> -
> -struct chassis_node {
> -    const struct ovsrec_port *port;
> -    const struct ovsrec_bridge *bridge;
> -};
> -
> -static char *
> -tunnel_create_name(struct tunnel_ctx *tc, const char *chassis_id)
> -{
> -    int i;
> -
> -    for (i = 0; i < UINT16_MAX; i++) {
> -        char *port_name;
> -        port_name = xasprintf("ovn-%.6s-%x", chassis_id, i);
> -
> -        if (!sset_contains(&tc->port_names, port_name)) {
> -            return port_name;
> -        }
> -
> -        free(port_name);
> -    }
> -
> -    return NULL;
> -}
> -
> -/*
> - * Returns a tunnel-id of the form 'chassis_id'-delimiter-'encap_ip'.
> - */
> -char *
> -encaps_tunnel_id_create(const char *chassis_id, const char *encap_ip)
> -{
> -    return xasprintf("%s%c%s", chassis_id, OVN_MVTEP_CHASSISID_DELIM,
> -                     encap_ip);
> -}
> -
> -/*
> - * Parses a 'tunnel_id' of the form <chassis_name><delimiter><IP>.
> - * If the 'chassis_id' argument is not NULL the function will allocate
> memory
> - * and store the chassis-id part of the tunnel-id at '*chassis_id'.
> - * If the 'encap_ip' argument is not NULL the function will allocate
> memory
> - * and store the encapsulation IP part of the tunnel-id at '*encap_ip'.
> - */
> -bool
> -encaps_tunnel_id_parse(const char *tunnel_id, char **chassis_id,
> -                       char **encap_ip)
> -{
> -    /* Find the delimiter.  Fail if there is no delimiter or if
> <chassis_name>
> -     * or <IP> is the empty string.*/
> -    const char *d = strchr(tunnel_id, OVN_MVTEP_CHASSISID_DELIM);
> -    if (d == tunnel_id || !d || !d[1]) {
> -        return false;
> -    }
> -
> -    if (chassis_id) {
> -        *chassis_id = xmemdup0(tunnel_id, d - tunnel_id);
> -    }
> -    if (encap_ip) {
> -        *encap_ip = xstrdup(d + 1);
> -    }
> -    return true;
> -}
> -
> -/*
> - * Returns true if 'tunnel_id' contains 'chassis_id' and, if specified,
> the
> - * given 'encap_ip'. Returns false otherwise.
> - */
> -bool
> -encaps_tunnel_id_match(const char *tunnel_id, const char *chassis_id,
> -                       const char *encap_ip)
> -{
> -    while (*tunnel_id == *chassis_id) {
> -        if (!*tunnel_id) {
> -            /* 'tunnel_id' and 'chassis_id' are equal strings.  This is a
> -             * mismatch because 'tunnel_id' is missing the delimiter and
> IP. */
> -            return false;
> -        }
> -        tunnel_id++;
> -        chassis_id++;
> -    }
> -
> -    /* We found the first byte that disagrees between 'tunnel_id' and
> -     * 'chassis_id'.  If we consumed all of 'chassis_id' and arrived at
> the
> -     * delimiter in 'tunnel_id' (and if 'encap_ip' is correct, if it was
> -     * supplied), it's a match. */
> -    return (*tunnel_id == OVN_MVTEP_CHASSISID_DELIM
> -            && *chassis_id == '\0'
> -            && (!encap_ip || !strcmp(tunnel_id + 1, encap_ip)));
> -}
> -
> -static void
> -tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg,
> -           const char *new_chassis_id, const struct sbrec_encap *encap)
> -{
> -    struct smap options = SMAP_INITIALIZER(&options);
> -    smap_add(&options, "remote_ip", encap->ip);
> -    smap_add(&options, "key", "flow");
> -    const char *dst_port = smap_get(&encap->options, "dst_port");
> -    const char *csum = smap_get(&encap->options, "csum");
> -    char *tunnel_entry_id = NULL;
> -
> -    /*
> -     * Since a chassis may have multiple encap-ip, we can't just add the
> -     * chassis name as as the "ovn-chassis-id" for the port; we use the
> -     * combination of the chassis_name and the encap-ip to identify
> -     * a specific tunnel to the chassis.
> -     */
> -    tunnel_entry_id = encaps_tunnel_id_create(new_chassis_id, encap->ip);
> -    if (csum && (!strcmp(csum, "true") || !strcmp(csum, "false"))) {
> -        smap_add(&options, "csum", csum);
> -    }
> -    if (dst_port) {
> -        smap_add(&options, "dst_port", dst_port);
> -    }
> -
> -    /* Add auth info if ipsec is enabled. */
> -    if (sbg->ipsec) {
> -        smap_add(&options, "remote_name", new_chassis_id);
> -    }
> -
> -    /* If there's an existing chassis record that does not need any
> change,
> -     * keep it.  Otherwise, create a new record (if there was an existing
> -     * record, the new record will supplant it and encaps_run() will
> delete
> -     * it). */
> -    struct chassis_node *chassis = shash_find_data(&tc->chassis,
> -                                                   tunnel_entry_id);
> -    if (chassis
> -        && chassis->port->n_interfaces == 1
> -        && !strcmp(chassis->port->interfaces[0]->type, encap->type)
> -        && smap_equal(&chassis->port->interfaces[0]->options, &options)) {
> -        shash_find_and_delete(&tc->chassis, tunnel_entry_id);
> -        free(chassis);
> -        goto exit;
> -    }
> -
> -    /* Choose a name for the new port.  If we're replacing an old port,
> reuse
> -     * its name, otherwise generate a new, unique name. */
> -    char *port_name = (chassis
> -                       ? xstrdup(chassis->port->name)
> -                       : tunnel_create_name(tc, new_chassis_id));
> -    if (!port_name) {
> -        VLOG_WARN("Unable to allocate unique name for '%s' tunnel",
> -                  new_chassis_id);
> -        goto exit;
> -    }
> -
> -    struct ovsrec_interface *iface = ovsrec_interface_insert(tc->ovs_txn);
> -    ovsrec_interface_set_name(iface, port_name);
> -    ovsrec_interface_set_type(iface, encap->type);
> -    ovsrec_interface_set_options(iface, &options);
> -
> -    struct ovsrec_port *port = ovsrec_port_insert(tc->ovs_txn);
> -    ovsrec_port_set_name(port, port_name);
> -    ovsrec_port_set_interfaces(port, &iface, 1);
> -    const struct smap id = SMAP_CONST1(&id, "ovn-chassis-id",
> tunnel_entry_id);
> -    ovsrec_port_set_external_ids(port, &id);
> -
> -    ovsrec_bridge_update_ports_addvalue(tc->br_int, port);
> -
> -    sset_add_and_free(&tc->port_names, port_name);
> -
> -exit:
> -    free(tunnel_entry_id);
> -    smap_destroy(&options);
> -}
> -
> -struct sbrec_encap *
> -preferred_encap(const struct sbrec_chassis *chassis_rec)
> -{
> -    struct sbrec_encap *best_encap = NULL;
> -    uint32_t best_type = 0;
> -
> -    for (int i = 0; i < chassis_rec->n_encaps; i++) {
> -        uint32_t tun_type = get_tunnel_type(chassis_rec->encaps[i]->type);
> -        if (tun_type > best_type) {
> -            best_type = tun_type;
> -            best_encap = chassis_rec->encaps[i];
> -        }
> -    }
> -
> -    return best_encap;
> -}
> -
> -/*
> - * For each peer chassis, get a preferred tunnel type and create as many
> tunnels
> - * as there are VTEP of that type (differentiated by remote_ip) on that
> chassis.
> - */
> -static int
> -chassis_tunnel_add(const struct sbrec_chassis *chassis_rec, const struct
> sbrec_sb_global *sbg, struct tunnel_ctx *tc)
> -{
> -    struct sbrec_encap *encap = preferred_encap(chassis_rec);
> -    int tuncnt = 0;
> -
> -    if (!encap) {
> -        VLOG_INFO("chassis_tunnel_add: No supported encaps for '%s'",
> chassis_rec->name);
> -        return tuncnt;
> -    }
> -
> -    uint32_t pref_type = get_tunnel_type(encap->type);
> -    for (int i = 0; i < chassis_rec->n_encaps; i++) {
> -        uint32_t tun_type = get_tunnel_type(chassis_rec->encaps[i]->type);
> -        if (tun_type != pref_type) {
> -            continue;
> -        }
> -        tunnel_add(tc, sbg, chassis_rec->name, chassis_rec->encaps[i]);
> -        tuncnt++;
> -    }
> -    return tuncnt;
> -}
> -
> -/*
> -* Returns true if transport_zones and chassis_rec->transport_zones
> -* have at least one common transport zone.
> -*/
> -static bool
> -chassis_tzones_overlap(const struct sset *transport_zones,
> -                       const struct sbrec_chassis *chassis_rec)
> -{
> -    /* If neither Chassis belongs to any transport zones, return true to
> -     * form a tunnel between them */
> -    if (!chassis_rec->n_transport_zones &&
> sset_is_empty(transport_zones)) {
> -        return true;
> -    }
> -
> -    for (int i = 0; i < chassis_rec->n_transport_zones; i++) {
> -        if (sset_contains(transport_zones,
> chassis_rec->transport_zones[i])) {
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -void
> -encaps_run(struct ovsdb_idl_txn *ovs_idl_txn,
> -           const struct ovsrec_bridge_table *bridge_table,
> -           const struct ovsrec_bridge *br_int,
> -           const struct sbrec_chassis_table *chassis_table,
> -           const char *chassis_id,
> -           const struct sbrec_sb_global *sbg,
> -           const struct sset *transport_zones)
> -{
> -    if (!ovs_idl_txn || !br_int) {
> -        return;
> -    }
> -
> -    const struct sbrec_chassis *chassis_rec;
> -    const struct ovsrec_bridge *br;
> -
> -    struct tunnel_ctx tc = {
> -        .chassis = SHASH_INITIALIZER(&tc.chassis),
> -        .port_names = SSET_INITIALIZER(&tc.port_names),
> -        .br_int = br_int
> -    };
> -
> -    tc.ovs_txn = ovs_idl_txn;
> -    ovsdb_idl_txn_add_comment(tc.ovs_txn,
> -                              "ovn-controller: modifying OVS tunnels
> '%s'",
> -                              chassis_id);
> -
> -    /* Collect all port names into tc.port_names.
> -     *
> -     * Collect all the OVN-created tunnels into tc.tunnel_hmap. */
> -    OVSREC_BRIDGE_TABLE_FOR_EACH (br, bridge_table) {
> -        for (size_t i = 0; i < br->n_ports; i++) {
> -            const struct ovsrec_port *port = br->ports[i];
> -            sset_add(&tc.port_names, port->name);
> -
> -            /*
> -             * note that the id here is not just the chassis name, but the
> -             * combination of <chassis_name><delim><encap_ip>
> -             */
> -            const char *id = smap_get(&port->external_ids,
> "ovn-chassis-id");
> -            if (id) {
> -                if (!shash_find(&tc.chassis, id)) {
> -                    struct chassis_node *chassis = xzalloc(sizeof
> *chassis);
> -                    chassis->bridge = br;
> -                    chassis->port = port;
> -                    shash_add_assert(&tc.chassis, id, chassis);
> -                } else {
> -                    /* Duplicate port for ovn-chassis-id.  Arbitrarily
> choose
> -                     * to delete this one. */
> -                    ovsrec_bridge_update_ports_delvalue(br, port);
> -                }
> -            }
> -        }
> -    }
> -
> -    SBREC_CHASSIS_TABLE_FOR_EACH (chassis_rec, chassis_table) {
> -        if (strcmp(chassis_rec->name, chassis_id)) {
> -            /* Create tunnels to the other Chassis belonging to the
> -             * same transport zone */
> -            if (!chassis_tzones_overlap(transport_zones, chassis_rec)) {
> -                VLOG_DBG("Skipping encap creation for Chassis '%s'
> because "
> -                         "it belongs to different transport zones",
> -                         chassis_rec->name);
> -                continue;
> -            }
> -
> -            if (chassis_tunnel_add(chassis_rec, sbg, &tc) == 0) {
> -                VLOG_INFO("Creating encap for '%s' failed",
> chassis_rec->name);
> -                continue;
> -            }
> -        }
> -    }
> -
> -    /* Delete any existing OVN tunnels that were not still around. */
> -    struct shash_node *node, *next_node;
> -    SHASH_FOR_EACH_SAFE (node, next_node, &tc.chassis) {
> -        struct chassis_node *chassis = node->data;
> -        ovsrec_bridge_update_ports_delvalue(chassis->bridge,
> chassis->port);
> -        shash_delete(&tc.chassis, node);
> -        free(chassis);
> -    }
> -    shash_destroy(&tc.chassis);
> -    sset_destroy(&tc.port_names);
> -}
> -
> -/* Returns true if the database is all cleaned up, false if more work is
> - * required. */
> -bool
> -encaps_cleanup(struct ovsdb_idl_txn *ovs_idl_txn,
> -               const struct ovsrec_bridge *br_int)
> -{
> -    if (!br_int) {
> -        return true;
> -    }
> -
> -    /* Delete all the OVS-created tunnels from the integration bridge. */
> -    struct ovsrec_port **ports
> -        = xmalloc(sizeof *br_int->ports * br_int->n_ports);
> -    size_t n = 0;
> -    for (size_t i = 0; i < br_int->n_ports; i++) {
> -        if (!smap_get(&br_int->ports[i]->external_ids, "ovn-chassis-id"))
> {
> -            ports[n++] = br_int->ports[i];
> -        }
> -    }
> -
> -    bool any_changes = n != br_int->n_ports;
> -    if (any_changes && ovs_idl_txn) {
> -        ovsdb_idl_txn_add_comment(ovs_idl_txn,
> -                                  "ovn-controller: destroying tunnels");
> -        ovsrec_bridge_verify_ports(br_int);
> -        ovsrec_bridge_set_ports(br_int, ports, n);
> -    }
> -    free(ports);
> -
> -    return !any_changes;
> -}
> diff --git a/ovn/controller/encaps.h b/ovn/controller/encaps.h
> deleted file mode 100644
> index afa41830a..000000000
> --- a/ovn/controller/encaps.h
> +++ /dev/null
> @@ -1,48 +0,0 @@
> -/* Copyright (c) 2015 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_ENCAPS_H
> -#define OVN_ENCAPS_H 1
> -
> -#include <stdbool.h>
> -
> -struct ovsdb_idl;
> -struct ovsdb_idl_txn;
> -struct ovsrec_bridge;
> -struct ovsrec_bridge_table;
> -struct sbrec_chassis_table;
> -struct sbrec_sb_global;
> -struct ovsrec_open_vswitch_table;
> -struct sset;
> -
> -void encaps_register_ovs_idl(struct ovsdb_idl *);
> -void encaps_run(struct ovsdb_idl_txn *ovs_idl_txn,
> -                const struct ovsrec_bridge_table *,
> -                const struct ovsrec_bridge *br_int,
> -                const struct sbrec_chassis_table *,
> -                const char *chassis_id,
> -                const struct sbrec_sb_global *,
> -                const struct sset *transport_zones);
> -
> -bool encaps_cleanup(struct ovsdb_idl_txn *ovs_idl_txn,
> -                    const struct ovsrec_bridge *br_int);
> -
> -char *encaps_tunnel_id_create(const char *chassis_id, const char
> *encap_ip);
> -bool  encaps_tunnel_id_parse(const char *tunnel_id, char **chassis_id,
> -                             char **encap_ip);
> -bool  encaps_tunnel_id_match(const char *tunnel_id, const char
> *chassis_id,
> -                             const char *encap_ip);
> -
> -#endif /* ovn/encaps.h */
> diff --git a/ovn/controller/ha-chassis.c b/ovn/controller/ha-chassis.c
> deleted file mode 100644
> index 498e5ce5a..000000000
> --- a/ovn/controller/ha-chassis.c
> +++ /dev/null
> @@ -1,203 +0,0 @@
> -/* Copyright (c) 2019 Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "ha-chassis.h"
> -#include "lib/sset.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ha_chassis);
> -
> -static int
> -compare_chassis_prio_(const void *a_, const void *b_)
> -{
> -    const struct sbrec_ha_chassis *ch_a = a_;
> -    const struct sbrec_ha_chassis *ch_b = b_;
> -    int prio_diff = ch_b->priority - ch_a->priority;
> -    if (!prio_diff) {
> -        return strcmp(ch_b->chassis->name, ch_a->chassis->name);
> -    }
> -    return prio_diff;
> -}
> -
> -/* Returns the ordered HA chassis list in the HA chassis group.
> - * Eg. If an HA chassis group has 3 HA chassis
> - *   - HA1 - pri 30
> - *   - HA2 - pri 40 and
> - *   - HA3 - pri 20
> - * and the ref_chassis of HA chassis group is set to - C1 and C2.
> - *
> - * If active_tunnels is NULL, then it returns the ordered list
> - *   -  (HA2, HA1, HA3)
> - *
> - * If active_tunnels is set to - (HA1, HA2, C1, C2) and
> - * local_chassis is HA3, then it returns the ordered list
> - *  - (HA2, HA1, HA3)
> - *
> - * If active_tunnels is set to - (HA1, C1, C2) and
> - * local_chassis is HA3, then it returns the ordered list
> - *  - (HA1, HA3)
> - *
> - * If active_tunnels is set to - (C1, C2) and
> - * local_chassis is HA3, then it returns the ordered list
> - *  - (HA3)
> - *
> - * If active_tunnels is set is empty and local_chassis is HA3,
> - * then it returns NULL.
> - */
> -static struct ha_chassis_ordered *
> -get_ordered_ha_chassis_list(const struct sbrec_ha_chassis_group
> *ha_ch_grp,
> -                            const struct sset *active_tunnels,
> -                            const struct sbrec_chassis *local_chassis)
> -{
> -    struct sbrec_ha_chassis *ha_ch_order =
> -        xzalloc(sizeof *ha_ch_order * ha_ch_grp->n_ha_chassis);
> -
> -    size_t n_ha_ch = 0;
> -
> -    for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) {
> -        if (!ha_ch_grp->ha_chassis[i]->chassis) {
> -            continue;
> -        }
> -
> -        /* Don't add it to the list for ordering if it is not active. */
> -        if (ha_ch_grp->ha_chassis[i]->chassis != local_chassis &&
> -            active_tunnels &&
> -            !sset_contains(active_tunnels,
> -                           ha_ch_grp->ha_chassis[i]->chassis->name)) {
> -            continue;
> -        }
> -
> -        ha_ch_order[n_ha_ch].chassis = ha_ch_grp->ha_chassis[i]->chassis;
> -        ha_ch_order[n_ha_ch].priority =
> ha_ch_grp->ha_chassis[i]->priority;
> -        n_ha_ch++;
> -    }
> -
> -    if (!n_ha_ch) {
> -        free(ha_ch_order);
> -        return NULL;
> -    }
> -
> -    struct ha_chassis_ordered *ordered_ha_ch;
> -    if (n_ha_ch == 1) {
> -        if (active_tunnels) {
> -            /* If n_ha_ch is 1, it means only the local chassis is in the
> -            * ha_ch_order list. Check if this local chassis has active
> -            * bfd session with any of the referenced chassis. If so,
> -            * then the local chassis can be active. Otherwise it can't.
> -            * This can happen in the following scenario.
> -            * Lets say we have chassis HA1 (prioirty 20) and HA2
> (priority 10)
> -            * in the ha_chasis_group and compute chassis C1 and C2 are in
> the
> -            * reference chassis list. If HA1 chassis has lost the link and
> -            * when this function is called for HA2 we need to consider
> -            * HA2 as active since it has active BFD sessions with C1 and
> C2.
> -            * On HA1 chassis, this function won't be called since
> -            * active_tunnels set will be empty.
> -            * */
> -            bool can_local_chassis_be_active = false;
> -            for (size_t i = 0; i < ha_ch_grp->n_ref_chassis; i++) {
> -                if (sset_contains(active_tunnels,
> -                                ha_ch_grp->ref_chassis[i]->name)) {
> -                    can_local_chassis_be_active = true;
> -                    break;
> -                }
> -            }
> -            if (!can_local_chassis_be_active) {
> -                free(ha_ch_order);
> -                return NULL;
> -            }
> -        }
> -    } else {
> -        qsort(ha_ch_order, n_ha_ch, sizeof *ha_ch_order,
> -              compare_chassis_prio_);
> -    }
> -
> -    ordered_ha_ch = xmalloc(sizeof *ordered_ha_ch);
> -    ordered_ha_ch->ha_ch = ha_ch_order;
> -    ordered_ha_ch->n_ha_ch = n_ha_ch;
> -
> -    return ordered_ha_ch;
> -}
> -
> -void
> -ha_chassis_destroy_ordered(struct ha_chassis_ordered *ordered_ha_ch)
> -{
> -    if (ordered_ha_ch) {
> -        free(ordered_ha_ch->ha_ch);
> -        free(ordered_ha_ch);
> -    }
> -}
> -
> -
> -/* Returns true if the local_chassis is the master of
> - * the HA chassis group, false otherwise. */
> -bool
> -ha_chassis_group_is_active(
> -    const struct sbrec_ha_chassis_group *ha_ch_grp,
> -    const struct sset *active_tunnels,
> -    const struct sbrec_chassis *local_chassis)
> -{
> -    if (!ha_ch_grp || !ha_ch_grp->n_ha_chassis) {
> -        return false;
> -    }
> -
> -    if (ha_ch_grp->n_ha_chassis == 1) {
> -        return (ha_ch_grp->ha_chassis[0]->chassis == local_chassis);
> -    }
> -
> -    if (sset_is_empty(active_tunnels)) {
> -        /* If active tunnel sset is empty, it means it has lost
> -         * connectivity with other chassis. */
> -        return false;
> -    }
> -
> -    struct ha_chassis_ordered *ordered_ha_ch =
> -        get_ordered_ha_chassis_list(ha_ch_grp, active_tunnels,
> local_chassis);
> -    if (!ordered_ha_ch) {
> -        return false;
> -    }
> -
> -    struct sbrec_chassis *active_ch = ordered_ha_ch->ha_ch[0].chassis;
> -    ha_chassis_destroy_ordered(ordered_ha_ch);
> -
> -    return (active_ch == local_chassis);
> -}
> -
> -bool
> -ha_chassis_group_contains(
> -    const struct sbrec_ha_chassis_group *ha_chassis_grp,
> -    const struct sbrec_chassis *chassis)
> -{
> -    if (ha_chassis_grp && chassis) {
> -        for (size_t i = 0; i < ha_chassis_grp->n_ha_chassis; i++) {
> -            if (ha_chassis_grp->ha_chassis[i]->chassis == chassis) {
> -                return true;
> -            }
> -        }
> -    }
> -    return false;
> -}
> -
> -struct ha_chassis_ordered *
> -ha_chassis_get_ordered(const struct sbrec_ha_chassis_group
> *ha_chassis_grp)
> -{
> -    if (!ha_chassis_grp || !ha_chassis_grp->n_ha_chassis) {
> -        return NULL;
> -    }
> -
> -    return get_ordered_ha_chassis_list(ha_chassis_grp, NULL, NULL);
> -}
> diff --git a/ovn/controller/ha-chassis.h b/ovn/controller/ha-chassis.h
> deleted file mode 100644
> index 3768c2a5c..000000000
> --- a/ovn/controller/ha-chassis.h
> +++ /dev/null
> @@ -1,50 +0,0 @@
> -/* Copyright (c) 2019 Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_HA_CHASSIS_H
> -#define OVN_HA_CHASSIS_H 1
> -
> -#include <stdint.h>
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/list.h"
> -
> -struct sbrec_chassis;
> -struct sbrec_ha_chassis_group;
> -struct sset;
> -
> -struct ha_chassis_ordered {
> -    struct sbrec_ha_chassis *ha_ch;
> -    size_t n_ha_ch;
> -};
> -
> -/* Returns true if the local chassis is the active gateway among a set
> - * of gateway_chassis.  Return false if the local chassis is currently a
> - * backup in a set of multiple gateway_chassis. */
> -bool ha_chassis_group_is_active(
> -    const struct sbrec_ha_chassis_group *ha_chassis_grp,
> -    const struct sset *active_tunnels,
> -    const struct sbrec_chassis *local_chassis);
> -
> -bool ha_chassis_group_contains(
> -    const struct sbrec_ha_chassis_group *ha_chassis_grp,
> -    const struct sbrec_chassis *chassis);
> -
> -struct ha_chassis_ordered *ha_chassis_get_ordered(
> -    const struct sbrec_ha_chassis_group *ha_chassis_grp);
> -
> -void ha_chassis_destroy_ordered(
> -    struct ha_chassis_ordered *ordered_ha_ch);
> -
> -#endif /* OVN_HA_CHASSIS_H */
> diff --git a/ovn/controller/ip-mcast.c b/ovn/controller/ip-mcast.c
> deleted file mode 100644
> index ef36be2ca..000000000
> --- a/ovn/controller/ip-mcast.c
> +++ /dev/null
> @@ -1,164 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "ip-mcast.h"
> -#include "lport.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -/*
> - * Used for (faster) updating of IGMP_Group ports.
> - */
> -struct igmp_group_port {
> -    struct hmap_node hmap_node;
> -    const struct sbrec_port_binding *port;
> -};
> -
> -struct ovsdb_idl_index *
> -igmp_group_index_create(struct ovsdb_idl *idl)
> -{
> -    const struct ovsdb_idl_index_column cols[] = {
> -        { .column = &sbrec_igmp_group_col_address },
> -        { .column = &sbrec_igmp_group_col_datapath },
> -        { .column = &sbrec_igmp_group_col_chassis },
> -    };
> -
> -    return ovsdb_idl_index_create(idl, cols, ARRAY_SIZE(cols));
> -}
> -
> -/* Looks up an IGMP group based on an IPv4 (mapped in IPv6) or IPv6
> 'address'
> - * and 'datapath'.
> - */
> -const struct sbrec_igmp_group *
> -igmp_group_lookup(struct ovsdb_idl_index *igmp_groups,
> -                  const struct in6_addr *address,
> -                  const struct sbrec_datapath_binding *datapath,
> -                  const struct sbrec_chassis *chassis)
> -{
> -    char addr_str[INET6_ADDRSTRLEN];
> -
> -    if (!ipv6_string_mapped(addr_str, address)) {
> -        return NULL;
> -    }
> -
> -    struct sbrec_igmp_group *target =
> -        sbrec_igmp_group_index_init_row(igmp_groups);
> -
> -    sbrec_igmp_group_index_set_address(target, addr_str);
> -    sbrec_igmp_group_index_set_datapath(target, datapath);
> -    sbrec_igmp_group_index_set_chassis(target, chassis);
> -
> -    const struct sbrec_igmp_group *g =
> -        sbrec_igmp_group_index_find(igmp_groups, target);
> -    sbrec_igmp_group_index_destroy_row(target);
> -    return g;
> -}
> -
> -/* Creates and returns a new IGMP group based on an IPv4 (mapped in IPv6)
> or
> - * IPv6 'address', 'datapath' and 'chassis'.
> - */
> -struct sbrec_igmp_group *
> -igmp_group_create(struct ovsdb_idl_txn *idl_txn,
> -                  const struct in6_addr *address,
> -                  const struct sbrec_datapath_binding *datapath,
> -                  const struct sbrec_chassis *chassis)
> -{
> -    char addr_str[INET6_ADDRSTRLEN];
> -
> -    if (!ipv6_string_mapped(addr_str, address)) {
> -        return NULL;
> -    }
> -
> -    struct sbrec_igmp_group *g = sbrec_igmp_group_insert(idl_txn);
> -
> -    sbrec_igmp_group_set_address(g, addr_str);
> -    sbrec_igmp_group_set_datapath(g, datapath);
> -    sbrec_igmp_group_set_chassis(g, chassis);
> -
> -    return g;
> -}
> -
> -void
> -igmp_group_update_ports(const struct sbrec_igmp_group *g,
> -                        struct ovsdb_idl_index *datapaths,
> -                        struct ovsdb_idl_index *port_bindings,
> -                        const struct mcast_snooping *ms OVS_UNUSED,
> -                        const struct mcast_group *mc_group)
> -    OVS_REQ_RDLOCK(ms->rwlock)
> -{
> -    struct igmp_group_port *old_ports_storage =
> -        (g->n_ports ? xmalloc(g->n_ports * sizeof *old_ports_storage) :
> NULL);
> -
> -    struct hmap old_ports = HMAP_INITIALIZER(&old_ports);
> -
> -    for (size_t i = 0; i < g->n_ports; i++) {
> -        struct igmp_group_port *old_port = &old_ports_storage[i];
> -
> -        old_port->port = g->ports[i];
> -        hmap_insert(&old_ports, &old_port->hmap_node,
> -                    old_port->port->tunnel_key);
> -    }
> -
> -    struct mcast_group_bundle *bundle;
> -    uint64_t dp_key = g->datapath->tunnel_key;
> -
> -    LIST_FOR_EACH (bundle, bundle_node, &mc_group->bundle_lru) {
> -        uint32_t port_key = (uintptr_t)bundle->port;
> -        const struct sbrec_port_binding *sbrec_port =
> -            lport_lookup_by_key(datapaths, port_bindings, dp_key,
> port_key);
> -        if (!sbrec_port) {
> -            continue;
> -        }
> -
> -        struct hmap_node *node = hmap_first_with_hash(&old_ports,
> port_key);
> -        if (!node) {
> -            sbrec_igmp_group_update_ports_addvalue(g, sbrec_port);
> -        } else {
> -            hmap_remove(&old_ports, node);
> -        }
> -    }
> -
> -    struct igmp_group_port *igmp_port;
> -    HMAP_FOR_EACH_POP (igmp_port, hmap_node, &old_ports) {
> -        sbrec_igmp_group_update_ports_delvalue(g, igmp_port->port);
> -    }
> -
> -    free(old_ports_storage);
> -    hmap_destroy(&old_ports);
> -}
> -
> -void
> -igmp_group_delete(const struct sbrec_igmp_group *g)
> -{
> -    sbrec_igmp_group_delete(g);
> -}
> -
> -bool
> -igmp_group_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                   struct ovsdb_idl_index *igmp_groups)
> -{
> -    const struct sbrec_igmp_group *g;
> -
> -    if (!ovnsb_idl_txn) {
> -        return true;
> -    }
> -
> -    SBREC_IGMP_GROUP_FOR_EACH_BYINDEX (g, igmp_groups) {
> -        igmp_group_delete(g);
> -    }
> -
> -    return true;
> -}
> diff --git a/ovn/controller/ip-mcast.h b/ovn/controller/ip-mcast.h
> deleted file mode 100644
> index 6014f43d5..000000000
> --- a/ovn/controller/ip-mcast.h
> +++ /dev/null
> @@ -1,52 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_IP_MCAST_H
> -#define OVN_IP_MCAST_H 1
> -
> -#include "mcast-snooping.h"
> -
> -struct ovsdb_idl;
> -struct ovsdb_idl_txn;
> -
> -struct sbrec_chassis;
> -struct sbrec_datapath_binding;
> -
> -struct ovsdb_idl_index *igmp_group_index_create(struct ovsdb_idl *);
> -const struct sbrec_igmp_group *igmp_group_lookup(
> -    struct ovsdb_idl_index *igmp_groups,
> -    const struct in6_addr *address,
> -    const struct sbrec_datapath_binding *datapath,
> -    const struct sbrec_chassis *chassis);
> -
> -struct sbrec_igmp_group *igmp_group_create(
> -    struct ovsdb_idl_txn *idl_txn,
> -    const struct in6_addr *address,
> -    const struct sbrec_datapath_binding *datapath,
> -    const struct sbrec_chassis *chassis);
> -
> -void igmp_group_update_ports(const struct sbrec_igmp_group *g,
> -                             struct ovsdb_idl_index *datapaths,
> -                             struct ovsdb_idl_index *port_bindings,
> -                             const struct mcast_snooping *ms,
> -                             const struct mcast_group *mc_group)
> -    OVS_REQ_RDLOCK(ms->rwlock);
> -
> -void igmp_group_delete(const struct sbrec_igmp_group *g);
> -
> -bool igmp_group_cleanup(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                        struct ovsdb_idl_index *igmp_groups);
> -
> -#endif /* ovn/controller/ip-mcast.h */
> diff --git a/ovn/controller/lflow.c b/ovn/controller/lflow.c
> deleted file mode 100644
> index 1aafafb33..000000000
> --- a/ovn/controller/lflow.c
> +++ /dev/null
> @@ -1,898 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "lflow.h"
> -#include "coverage.h"
> -#include "ha-chassis.h"
> -#include "lport.h"
> -#include "ofctrl.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn-controller.h"
> -#include "ovn/actions.h"
> -#include "ovn/expr.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/extend-table.h"
> -#include "packets.h"
> -#include "physical.h"
> -#include "simap.h"
> -#include "sset.h"
> -
> -VLOG_DEFINE_THIS_MODULE(lflow);
> -
> -COVERAGE_DEFINE(lflow_run);
> -
> -/* Symbol table. */
> -
> -/* Contains "struct expr_symbol"s for fields supported by OVN lflows. */
> -static struct shash symtab;
> -
> -void
> -lflow_init(void)
> -{
> -    ovn_init_symtab(&symtab);
> -}
> -
> -struct lookup_port_aux {
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath;
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name;
> -    const struct sbrec_datapath_binding *dp;
> -};
> -
> -struct condition_aux {
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name;
> -    const struct sbrec_chassis *chassis;
> -    const struct sset *active_tunnels;
> -};
> -
> -static bool consider_logical_flow(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_logical_flow *,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *,
> -    struct hmap *dhcp_opts,
> -    struct hmap *dhcpv6_opts,
> -    struct hmap *nd_ra_opts,
> -    struct controller_event_options *controller_event_opts,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *lfrr,
> -    uint32_t *conj_id_ofs);
> -
> -static bool
> -lookup_port_cb(const void *aux_, const char *port_name, unsigned int
> *portp)
> -{
> -    const struct lookup_port_aux *aux = aux_;
> -
> -    const struct sbrec_port_binding *pb
> -        = lport_lookup_by_name(aux->sbrec_port_binding_by_name,
> port_name);
> -    if (pb && pb->datapath == aux->dp) {
> -        *portp = pb->tunnel_key;
> -        return true;
> -    }
> -
> -    const struct sbrec_multicast_group *mg = mcgroup_lookup_by_dp_name(
> -        aux->sbrec_multicast_group_by_name_datapath, aux->dp, port_name);
> -    if (mg) {
> -        *portp = mg->tunnel_key;
> -        return true;
> -    }
> -
> -    return false;
> -}
> -
> -static bool
> -is_chassis_resident_cb(const void *c_aux_, const char *port_name)
> -{
> -    const struct condition_aux *c_aux = c_aux_;
> -
> -    const struct sbrec_port_binding *pb
> -        = lport_lookup_by_name(c_aux->sbrec_port_binding_by_name,
> port_name);
> -    if (!pb) {
> -        return false;
> -    }
> -    if (strcmp(pb->type, "chassisredirect")) {
> -        /* for non-chassisredirect ports */
> -        return pb->chassis && pb->chassis == c_aux->chassis;
> -    } else {
> -        if (ha_chassis_group_contains(pb->ha_chassis_group,
> -                                      c_aux->chassis)) {
> -            bool active = ha_chassis_group_is_active(pb->ha_chassis_group,
> -
>  c_aux->active_tunnels,
> -                                                     c_aux->chassis);
> -            return active;
> -        }
> -        return false;
> -    }
> -}
> -
> -static bool
> -is_switch(const struct sbrec_datapath_binding *ldp)
> -{
> -    return smap_get(&ldp->external_ids, "logical-switch") != NULL;
> -
> -}
> -
> -void
> -lflow_resource_init(struct lflow_resource_ref *lfrr)
> -{
> -    hmap_init(&lfrr->ref_lflow_table);
> -    hmap_init(&lfrr->lflow_ref_table);
> -}
> -
> -void
> -lflow_resource_destroy(struct lflow_resource_ref *lfrr)
> -{
> -    struct ref_lflow_node *rlfn, *rlfn_next;
> -    HMAP_FOR_EACH_SAFE (rlfn, rlfn_next, node, &lfrr->ref_lflow_table) {
> -        free(rlfn->ref_name);
> -        struct lflow_ref_list_node *lrln, *next;
> -        LIST_FOR_EACH_SAFE (lrln, next, ref_list, &rlfn->ref_lflow_head) {
> -            ovs_list_remove(&lrln->ref_list);
> -            ovs_list_remove(&lrln->lflow_list);
> -            free(lrln);
> -        }
> -        hmap_remove(&lfrr->ref_lflow_table, &rlfn->node);
> -        free(rlfn);
> -    }
> -    hmap_destroy(&lfrr->ref_lflow_table);
> -
> -    struct lflow_ref_node *lfrn, *lfrn_next;
> -    HMAP_FOR_EACH_SAFE (lfrn, lfrn_next, node, &lfrr->lflow_ref_table) {
> -        hmap_remove(&lfrr->lflow_ref_table, &lfrn->node);
> -        free(lfrn);
> -    }
> -    hmap_destroy(&lfrr->lflow_ref_table);
> -}
> -
> -void
> -lflow_resource_clear(struct lflow_resource_ref *lfrr)
> -{
> -    lflow_resource_destroy(lfrr);
> -    lflow_resource_init(lfrr);
> -}
> -
> -static struct ref_lflow_node*
> -ref_lflow_lookup(struct hmap *ref_lflow_table,
> -                 enum ref_type type, const char *ref_name)
> -{
> -    struct ref_lflow_node *rlfn;
> -
> -    HMAP_FOR_EACH_WITH_HASH (rlfn, node, hash_string(ref_name, type),
> -                             ref_lflow_table) {
> -        if (rlfn->type == type && !strcmp(rlfn->ref_name, ref_name)) {
> -            return rlfn;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static struct lflow_ref_node*
> -lflow_ref_lookup(struct hmap *lflow_ref_table,
> -                 const struct uuid *lflow_uuid)
> -{
> -    struct lflow_ref_node *lfrn;
> -
> -    HMAP_FOR_EACH_WITH_HASH (lfrn, node, uuid_hash(lflow_uuid),
> -                             lflow_ref_table) {
> -        if (uuid_equals(&lfrn->lflow_uuid, lflow_uuid)) {
> -            return lfrn;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -lflow_resource_add(struct lflow_resource_ref *lfrr, enum ref_type type,
> -                   const char *ref_name, const struct uuid *lflow_uuid)
> -{
> -    struct ref_lflow_node *rlfn = ref_lflow_lookup(&lfrr->ref_lflow_table,
> -                                                   type, ref_name);
> -    if (!rlfn) {
> -        rlfn = xzalloc(sizeof *rlfn);
> -        rlfn->node.hash = hash_string(ref_name, type);
> -        rlfn->type = type;
> -        rlfn->ref_name = xstrdup(ref_name);
> -        ovs_list_init(&rlfn->ref_lflow_head);
> -        hmap_insert(&lfrr->ref_lflow_table, &rlfn->node, rlfn->node.hash);
> -    }
> -
> -    struct lflow_ref_node *lfrn = lflow_ref_lookup(&lfrr->lflow_ref_table,
> -                                                   lflow_uuid);
> -    if (!lfrn) {
> -        lfrn = xzalloc(sizeof *lfrn);
> -        lfrn->node.hash = uuid_hash(lflow_uuid);
> -        lfrn->lflow_uuid = *lflow_uuid;
> -        ovs_list_init(&lfrn->lflow_ref_head);
> -        hmap_insert(&lfrr->lflow_ref_table, &lfrn->node, lfrn->node.hash);
> -    }
> -
> -    struct lflow_ref_list_node *lrln = xzalloc(sizeof *lrln);
> -    lrln->type = type;
> -    lrln->ref_name = xstrdup(ref_name);
> -    lrln->lflow_uuid = *lflow_uuid;
> -    ovs_list_push_back(&rlfn->ref_lflow_head, &lrln->ref_list);
> -    ovs_list_push_back(&lfrn->lflow_ref_head, &lrln->lflow_list);
> -}
> -
> -static void
> -lflow_resource_destroy_lflow(struct lflow_resource_ref *lfrr,
> -                            const struct uuid *lflow_uuid)
> -{
> -    struct lflow_ref_node *lfrn = lflow_ref_lookup(&lfrr->lflow_ref_table,
> -                                                   lflow_uuid);
> -    if (!lfrn) {
> -        return;
> -    }
> -
> -    hmap_remove(&lfrr->lflow_ref_table, &lfrn->node);
> -    struct lflow_ref_list_node *lrln, *next;
> -    LIST_FOR_EACH_SAFE (lrln, next, lflow_list, &lfrn->lflow_ref_head) {
> -        ovs_list_remove(&lrln->ref_list);
> -        ovs_list_remove(&lrln->lflow_list);
> -        free(lrln);
> -    }
> -    free(lfrn);
> -}
> -
> -/* Adds the logical flows from the Logical_Flow table to flow tables. */
> -static void
> -add_logical_flows(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_dhcp_options_table *dhcp_options_table,
> -    const struct sbrec_dhcpv6_options_table *dhcpv6_options_table,
> -    const struct sbrec_logical_flow_table *logical_flow_table,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *chassis,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *flow_table,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *lfrr,
> -    uint32_t *conj_id_ofs)
> -{
> -    const struct sbrec_logical_flow *lflow;
> -
> -    struct hmap dhcp_opts = HMAP_INITIALIZER(&dhcp_opts);
> -    struct hmap dhcpv6_opts = HMAP_INITIALIZER(&dhcpv6_opts);
> -    const struct sbrec_dhcp_options *dhcp_opt_row;
> -    SBREC_DHCP_OPTIONS_TABLE_FOR_EACH (dhcp_opt_row, dhcp_options_table) {
> -        dhcp_opt_add(&dhcp_opts, dhcp_opt_row->name, dhcp_opt_row->code,
> -                     dhcp_opt_row->type);
> -    }
> -
> -
> -    const struct sbrec_dhcpv6_options *dhcpv6_opt_row;
> -    SBREC_DHCPV6_OPTIONS_TABLE_FOR_EACH (dhcpv6_opt_row,
> -                                         dhcpv6_options_table) {
> -       dhcp_opt_add(&dhcpv6_opts, dhcpv6_opt_row->name,
> dhcpv6_opt_row->code,
> -                    dhcpv6_opt_row->type);
> -    }
> -
> -    struct hmap nd_ra_opts = HMAP_INITIALIZER(&nd_ra_opts);
> -    nd_ra_opts_init(&nd_ra_opts);
> -
> -    struct controller_event_options controller_event_opts;
> -    controller_event_opts_init(&controller_event_opts);
> -
> -    SBREC_LOGICAL_FLOW_TABLE_FOR_EACH (lflow, logical_flow_table) {
> -        if (!consider_logical_flow(sbrec_multicast_group_by_name_datapath,
> -                                   sbrec_port_binding_by_name,
> -                                   lflow, local_datapaths,
> -                                   chassis, &dhcp_opts, &dhcpv6_opts,
> -                                   &nd_ra_opts, &controller_event_opts,
> -                                   addr_sets, port_groups,
> -                                   active_tunnels, local_lport_ids,
> -                                   flow_table, group_table, meter_table,
> -                                   lfrr, conj_id_ofs)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5);
> -            VLOG_ERR_RL(&rl, "Conjunction id overflow when processing
> lflow "
> -                        UUID_FMT, UUID_ARGS(&lflow->header_.uuid));
> -        }
> -    }
> -
> -    dhcp_opts_destroy(&dhcp_opts);
> -    dhcp_opts_destroy(&dhcpv6_opts);
> -    nd_ra_opts_destroy(&nd_ra_opts);
> -    controller_event_opts_destroy(&controller_event_opts);
> -}
> -
> -bool
> -lflow_handle_changed_flows(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_dhcp_options_table *dhcp_options_table,
> -    const struct sbrec_dhcpv6_options_table *dhcpv6_options_table,
> -    const struct sbrec_logical_flow_table *logical_flow_table,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *chassis,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *flow_table,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *lfrr,
> -    uint32_t *conj_id_ofs)
> -{
> -    bool ret = true;
> -    const struct sbrec_logical_flow *lflow;
> -
> -    struct hmap dhcp_opts = HMAP_INITIALIZER(&dhcp_opts);
> -    struct hmap dhcpv6_opts = HMAP_INITIALIZER(&dhcpv6_opts);
> -    const struct sbrec_dhcp_options *dhcp_opt_row;
> -    SBREC_DHCP_OPTIONS_TABLE_FOR_EACH (dhcp_opt_row, dhcp_options_table) {
> -        dhcp_opt_add(&dhcp_opts, dhcp_opt_row->name, dhcp_opt_row->code,
> -                     dhcp_opt_row->type);
> -    }
> -
> -
> -    const struct sbrec_dhcpv6_options *dhcpv6_opt_row;
> -    SBREC_DHCPV6_OPTIONS_TABLE_FOR_EACH (dhcpv6_opt_row,
> -                                         dhcpv6_options_table) {
> -       dhcp_opt_add(&dhcpv6_opts, dhcpv6_opt_row->name,
> dhcpv6_opt_row->code,
> -                    dhcpv6_opt_row->type);
> -    }
> -
> -    struct hmap nd_ra_opts = HMAP_INITIALIZER(&nd_ra_opts);
> -    nd_ra_opts_init(&nd_ra_opts);
> -
> -    /* Handle removed flows first, and then other flows, so that when
> -     * the flows being added and removed have same match conditions
> -     * can be processed in the proper order */
> -    SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow, logical_flow_table)
> {
> -        /* Remove any flows that should be removed. */
> -        if (sbrec_logical_flow_is_deleted(lflow)) {
> -            VLOG_DBG("handle deleted lflow "UUID_FMT,
> -                     UUID_ARGS(&lflow->header_.uuid));
> -            ofctrl_remove_flows(flow_table, &lflow->header_.uuid);
> -            /* Delete entries from lflow resource reference. */
> -            lflow_resource_destroy_lflow(lfrr, &lflow->header_.uuid);
> -        }
> -    }
> -
> -    struct controller_event_options controller_event_opts;
> -    controller_event_opts_init(&controller_event_opts);
> -
> -    SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow, logical_flow_table)
> {
> -        if (!sbrec_logical_flow_is_deleted(lflow)) {
> -            /* Now, add/modify existing flows. If the logical
> -             * flow is a modification, just remove the flows
> -             * for this row, and then add new flows. */
> -            if (!sbrec_logical_flow_is_new(lflow)) {
> -                VLOG_DBG("handle updated lflow "UUID_FMT,
> -                         UUID_ARGS(&lflow->header_.uuid));
> -                ofctrl_remove_flows(flow_table, &lflow->header_.uuid);
> -                /* Delete entries from lflow resource reference. */
> -                lflow_resource_destroy_lflow(lfrr, &lflow->header_.uuid);
> -            }
> -            VLOG_DBG("handle new lflow "UUID_FMT,
> -                     UUID_ARGS(&lflow->header_.uuid));
> -            if
> (!consider_logical_flow(sbrec_multicast_group_by_name_datapath,
> -                                       sbrec_port_binding_by_name,
> -                                       lflow, local_datapaths,
> -                                       chassis, &dhcp_opts, &dhcpv6_opts,
> -                                       &nd_ra_opts,
> &controller_event_opts,
> -                                       addr_sets, port_groups,
> -                                       active_tunnels, local_lport_ids,
> -                                       flow_table, group_table,
> meter_table,
> -                                       lfrr, conj_id_ofs)) {
> -                ret = false;
> -                break;
> -            }
> -        }
> -    }
> -    dhcp_opts_destroy(&dhcp_opts);
> -    dhcp_opts_destroy(&dhcpv6_opts);
> -    nd_ra_opts_destroy(&nd_ra_opts);
> -    controller_event_opts_destroy(&controller_event_opts);
> -    return ret;
> -}
> -
> -bool
> -lflow_handle_changed_ref(
> -    enum ref_type ref_type,
> -    const char *ref_name,
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_dhcp_options_table *dhcp_options_table,
> -    const struct sbrec_dhcpv6_options_table *dhcpv6_options_table,
> -    const struct sbrec_logical_flow_table *logical_flow_table,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *chassis,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *flow_table,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *lfrr,
> -    uint32_t *conj_id_ofs,
> -    bool *changed)
> -{
> -    struct ref_lflow_node *rlfn = ref_lflow_lookup(&lfrr->ref_lflow_table,
> -                                                   ref_type, ref_name);
> -    if (!rlfn) {
> -        *changed = false;
> -        return true;
> -    }
> -    VLOG_DBG("Handle changed lflow reference for resource type: %d,"
> -             " name: %s.", ref_type, ref_name);
> -    *changed = false;
> -    bool ret = true;
> -
> -    hmap_remove(&lfrr->ref_lflow_table, &rlfn->node);
> -
> -    struct lflow_ref_list_node *lrln, *next;
> -    /* Detach the rlfn->ref_lflow_head nodes from the lfrr table and clean
> -     * up all other nodes related to the lflows that uses the resource,
> -     * so that the old nodes won't interfere with updating the lfrr table
> -     * when reparsing the lflows. */
> -    LIST_FOR_EACH (lrln, ref_list, &rlfn->ref_lflow_head) {
> -        ovs_list_remove(&lrln->lflow_list);
> -        lflow_resource_destroy_lflow(lfrr, &lrln->lflow_uuid);
> -    }
> -
> -    struct hmap dhcp_opts = HMAP_INITIALIZER(&dhcp_opts);
> -    struct hmap dhcpv6_opts = HMAP_INITIALIZER(&dhcpv6_opts);
> -    const struct sbrec_dhcp_options *dhcp_opt_row;
> -    SBREC_DHCP_OPTIONS_TABLE_FOR_EACH (dhcp_opt_row, dhcp_options_table) {
> -        dhcp_opt_add(&dhcp_opts, dhcp_opt_row->name, dhcp_opt_row->code,
> -                     dhcp_opt_row->type);
> -    }
> -
> -    const struct sbrec_dhcpv6_options *dhcpv6_opt_row;
> -    SBREC_DHCPV6_OPTIONS_TABLE_FOR_EACH(dhcpv6_opt_row,
> dhcpv6_options_table) {
> -       dhcp_opt_add(&dhcpv6_opts, dhcpv6_opt_row->name,
> dhcpv6_opt_row->code,
> -                    dhcpv6_opt_row->type);
> -    }
> -
> -    struct hmap nd_ra_opts = HMAP_INITIALIZER(&nd_ra_opts);
> -    nd_ra_opts_init(&nd_ra_opts);
> -
> -    struct controller_event_options controller_event_opts;
> -    controller_event_opts_init(&controller_event_opts);
> -
> -    /* Re-parse the related lflows. */
> -    LIST_FOR_EACH (lrln, ref_list, &rlfn->ref_lflow_head) {
> -        const struct sbrec_logical_flow *lflow =
> -            sbrec_logical_flow_table_get_for_uuid(logical_flow_table,
> -                                                  &lrln->lflow_uuid);
> -        if (!lflow) {
> -            VLOG_DBG("Reprocess lflow "UUID_FMT" for resource type: %d,"
> -                     " name: %s - not found.",
> -                     UUID_ARGS(&lrln->lflow_uuid),
> -                     ref_type, ref_name);
> -            continue;
> -        }
> -        VLOG_DBG("Reprocess lflow "UUID_FMT" for resource type: %d,"
> -                 " name: %s.",
> -                 UUID_ARGS(&lrln->lflow_uuid),
> -                 ref_type, ref_name);
> -        ofctrl_remove_flows(flow_table, &lrln->lflow_uuid);
> -
> -        if (!consider_logical_flow(sbrec_multicast_group_by_name_datapath,
> -                                   sbrec_port_binding_by_name,
> -                                   lflow, local_datapaths,
> -                                   chassis, &dhcp_opts, &dhcpv6_opts,
> -                                   &nd_ra_opts, &controller_event_opts,
> -                                   addr_sets, port_groups,
> -                                   active_tunnels, local_lport_ids,
> -                                   flow_table, group_table, meter_table,
> -                                   lfrr, conj_id_ofs)) {
> -            ret = false;
> -            break;
> -        }
> -        *changed = true;
> -    }
> -
> -    LIST_FOR_EACH_SAFE (lrln, next, ref_list, &rlfn->ref_lflow_head) {
> -        ovs_list_remove(&lrln->ref_list);
> -        free(lrln);
> -    }
> -    free(rlfn);
> -
> -    dhcp_opts_destroy(&dhcp_opts);
> -    dhcp_opts_destroy(&dhcpv6_opts);
> -    nd_ra_opts_destroy(&nd_ra_opts);
> -    controller_event_opts_destroy(&controller_event_opts);
> -    return ret;
> -}
> -
> -static bool
> -update_conj_id_ofs(uint32_t *conj_id_ofs, uint32_t n_conjs)
> -{
> -    if (*conj_id_ofs + n_conjs < *conj_id_ofs) {
> -        /* overflow */
> -        return false;
> -    }
> -    *conj_id_ofs += n_conjs;
> -    return true;
> -}
> -
> -static bool
> -consider_logical_flow(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_logical_flow *lflow,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *chassis,
> -    struct hmap *dhcp_opts,
> -    struct hmap *dhcpv6_opts,
> -    struct hmap *nd_ra_opts,
> -    struct controller_event_options *controller_event_opts,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *flow_table,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *lfrr,
> -    uint32_t *conj_id_ofs)
> -{
> -    /* Determine translation of logical table IDs to physical table IDs.
> */
> -    bool ingress = !strcmp(lflow->pipeline, "ingress");
> -
> -    const struct sbrec_datapath_binding *ldp = lflow->logical_datapath;
> -    if (!ldp) {
> -        VLOG_DBG("lflow "UUID_FMT" has no datapath binding, skip",
> -                 UUID_ARGS(&lflow->header_.uuid));
> -        return true;
> -    }
> -    if (!get_local_datapath(local_datapaths, ldp->tunnel_key)) {
> -        VLOG_DBG("lflow "UUID_FMT" is not for local datapath, skip",
> -                 UUID_ARGS(&lflow->header_.uuid));
> -        return true;
> -    }
> -
> -    /* Determine translation of logical table IDs to physical table IDs.
> */
> -    uint8_t first_ptable = (ingress
> -                            ? OFTABLE_LOG_INGRESS_PIPELINE
> -                            : OFTABLE_LOG_EGRESS_PIPELINE);
> -    uint8_t ptable = first_ptable + lflow->table_id;
> -    uint8_t output_ptable = (ingress
> -                             ? OFTABLE_REMOTE_OUTPUT
> -                             : OFTABLE_SAVE_INPORT);
> -
> -    /* Parse OVN logical actions.
> -     *
> -     * XXX Deny changes to 'outport' in egress pipeline. */
> -    uint64_t ovnacts_stub[1024 / 8];
> -    struct ofpbuf ovnacts = OFPBUF_STUB_INITIALIZER(ovnacts_stub);
> -    struct ovnact_parse_params pp = {
> -        .symtab = &symtab,
> -        .dhcp_opts = dhcp_opts,
> -        .dhcpv6_opts = dhcpv6_opts,
> -        .nd_ra_opts = nd_ra_opts,
> -        .controller_event_opts = controller_event_opts,
> -
> -        .pipeline = ingress ? OVNACT_P_INGRESS : OVNACT_P_EGRESS,
> -        .n_tables = LOG_PIPELINE_LEN,
> -        .cur_ltable = lflow->table_id,
> -    };
> -    struct expr *prereqs;
> -    char *error;
> -
> -    error = ovnacts_parse_string(lflow->actions, &pp, &ovnacts, &prereqs);
> -    if (error) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "error parsing actions \"%s\": %s",
> -                     lflow->actions, error);
> -        free(error);
> -        ovnacts_free(ovnacts.data, ovnacts.size);
> -        ofpbuf_uninit(&ovnacts);
> -        return true;
> -    }
> -
> -    /* Translate OVN match into table of OpenFlow matches. */
> -    struct hmap matches;
> -    struct expr *expr;
> -
> -    struct sset addr_sets_ref = SSET_INITIALIZER(&addr_sets_ref);
> -    expr = expr_parse_string(lflow->match, &symtab, addr_sets,
> port_groups,
> -                             &addr_sets_ref, &error);
> -    const char *addr_set_name;
> -    SSET_FOR_EACH (addr_set_name, &addr_sets_ref) {
> -        lflow_resource_add(lfrr, REF_TYPE_ADDRSET, addr_set_name,
> -                           &lflow->header_.uuid);
> -    }
> -    sset_destroy(&addr_sets_ref);
> -
> -    if (!error) {
> -        if (prereqs) {
> -            expr = expr_combine(EXPR_T_AND, expr, prereqs);
> -            prereqs = NULL;
> -        }
> -        expr = expr_annotate(expr, &symtab, &error);
> -    }
> -    if (error) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "error parsing match \"%s\": %s",
> -                     lflow->match, error);
> -        expr_destroy(prereqs);
> -        free(error);
> -        ovnacts_free(ovnacts.data, ovnacts.size);
> -        ofpbuf_uninit(&ovnacts);
> -        return true;
> -    }
> -
> -    struct lookup_port_aux aux = {
> -        .sbrec_multicast_group_by_name_datapath
> -            = sbrec_multicast_group_by_name_datapath,
> -        .sbrec_port_binding_by_name = sbrec_port_binding_by_name,
> -        .dp = lflow->logical_datapath
> -    };
> -    struct condition_aux cond_aux = {
> -        .sbrec_port_binding_by_name = sbrec_port_binding_by_name,
> -        .chassis = chassis,
> -        .active_tunnels = active_tunnels,
> -    };
> -    expr = expr_simplify(expr, is_chassis_resident_cb, &cond_aux);
> -    expr = expr_normalize(expr);
> -    uint32_t n_conjs = expr_to_matches(expr, lookup_port_cb, &aux,
> -                                       &matches);
> -    expr_destroy(expr);
> -
> -    if (hmap_is_empty(&matches)) {
> -        VLOG_DBG("lflow "UUID_FMT" matches are empty, skip",
> -                 UUID_ARGS(&lflow->header_.uuid));
> -        ovnacts_free(ovnacts.data, ovnacts.size);
> -        ofpbuf_uninit(&ovnacts);
> -        expr_matches_destroy(&matches);
> -        return true;
> -    }
> -
> -    /* Encode OVN logical actions into OpenFlow. */
> -    uint64_t ofpacts_stub[1024 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -    struct ovnact_encode_params ep = {
> -        .lookup_port = lookup_port_cb,
> -        .aux = &aux,
> -        .is_switch = is_switch(ldp),
> -        .group_table = group_table,
> -        .meter_table = meter_table,
> -        .lflow_uuid = lflow->header_.uuid,
> -
> -        .pipeline = ingress ? OVNACT_P_INGRESS : OVNACT_P_EGRESS,
> -        .ingress_ptable = OFTABLE_LOG_INGRESS_PIPELINE,
> -        .egress_ptable = OFTABLE_LOG_EGRESS_PIPELINE,
> -        .output_ptable = output_ptable,
> -        .mac_bind_ptable = OFTABLE_MAC_BINDING,
> -    };
> -    ovnacts_encode(ovnacts.data, ovnacts.size, &ep, &ofpacts);
> -    ovnacts_free(ovnacts.data, ovnacts.size);
> -    ofpbuf_uninit(&ovnacts);
> -
> -    /* Prepare the OpenFlow matches for adding to the flow table. */
> -    struct expr_match *m;
> -    HMAP_FOR_EACH (m, hmap_node, &matches) {
> -        match_set_metadata(&m->match,
> -                           htonll(lflow->logical_datapath->tunnel_key));
> -        if (m->match.wc.masks.conj_id) {
> -            m->match.flow.conj_id += *conj_id_ofs;
> -        }
> -        if (is_switch(ldp)) {
> -            unsigned int reg_index
> -                = (ingress ? MFF_LOG_INPORT : MFF_LOG_OUTPORT) - MFF_REG0;
> -            int64_t port_id = m->match.flow.regs[reg_index];
> -            if (port_id) {
> -                int64_t dp_id = lflow->logical_datapath->tunnel_key;
> -                char buf[16];
> -                snprintf(buf, sizeof(buf), "%"PRId64"_%"PRId64, dp_id,
> port_id);
> -                if (!sset_contains(local_lport_ids, buf)) {
> -                    VLOG_DBG("lflow "UUID_FMT
> -                             " port %s in match is not local, skip",
> -                             UUID_ARGS(&lflow->header_.uuid),
> -                             buf);
> -                    continue;
> -                }
> -            }
> -        }
> -        if (!m->n) {
> -            ofctrl_add_flow(flow_table, ptable, lflow->priority,
> -                            lflow->header_.uuid.parts[0], &m->match,
> &ofpacts,
> -                            &lflow->header_.uuid);
> -        } else {
> -            uint64_t conj_stubs[64 / 8];
> -            struct ofpbuf conj;
> -
> -            ofpbuf_use_stub(&conj, conj_stubs, sizeof conj_stubs);
> -            for (int i = 0; i < m->n; i++) {
> -                const struct cls_conjunction *src = &m->conjunctions[i];
> -                struct ofpact_conjunction *dst;
> -
> -                dst = ofpact_put_CONJUNCTION(&conj);
> -                dst->id = src->id + *conj_id_ofs;
> -                dst->clause = src->clause;
> -                dst->n_clauses = src->n_clauses;
> -            }
> -            ofctrl_add_flow(flow_table, ptable, lflow->priority, 0,
> &m->match,
> -                            &conj, &lflow->header_.uuid);
> -            ofpbuf_uninit(&conj);
> -        }
> -    }
> -
> -    /* Clean up. */
> -    expr_matches_destroy(&matches);
> -    ofpbuf_uninit(&ofpacts);
> -    return update_conj_id_ofs(conj_id_ofs, n_conjs);
> -}
> -
> -static void
> -put_load(const uint8_t *data, size_t len,
> -         enum mf_field_id dst, int ofs, int n_bits,
> -         struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts,
> -                                                       mf_from_id(dst),
> NULL,
> -                                                       NULL);
> -    bitwise_copy(data, len, 0, sf->value, sf->field->n_bytes, ofs,
> n_bits);
> -    bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs,
> n_bits);
> -}
> -
> -static void
> -consider_neighbor_flow(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                       const struct sbrec_mac_binding *b,
> -                       struct ovn_desired_flow_table *flow_table)
> -{
> -    const struct sbrec_port_binding *pb
> -        = lport_lookup_by_name(sbrec_port_binding_by_name,
> b->logical_port);
> -    if (!pb) {
> -        return;
> -    }
> -
> -    struct eth_addr mac;
> -    if (!eth_addr_from_string(b->mac, &mac)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "bad 'mac' %s", b->mac);
> -        return;
> -    }
> -
> -    struct match match = MATCH_CATCHALL_INITIALIZER;
> -    if (strchr(b->ip, '.')) {
> -        ovs_be32 ip;
> -        if (!ip_parse(b->ip, &ip)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad 'ip' %s", b->ip);
> -            return;
> -        }
> -        match_set_reg(&match, 0, ntohl(ip));
> -    } else {
> -        struct in6_addr ip6;
> -        if (!ipv6_parse(b->ip, &ip6)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad 'ip' %s", b->ip);
> -            return;
> -        }
> -        ovs_be128 value;
> -        memcpy(&value, &ip6, sizeof(value));
> -        match_set_xxreg(&match, 0, ntoh128(value));
> -    }
> -
> -    match_set_metadata(&match, htonll(pb->datapath->tunnel_key));
> -    match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, pb->tunnel_key);
> -
> -    uint64_t stub[1024 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(stub);
> -    put_load(mac.ea, sizeof mac.ea, MFF_ETH_DST, 0, 48, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_MAC_BINDING, 100, 0, &match,
> &ofpacts,
> -                    &b->header_.uuid);
> -    ofpbuf_uninit(&ofpacts);
> -}
> -
> -/* Adds an OpenFlow flow to flow tables for each MAC binding in the OVN
> - * southbound database. */
> -static void
> -add_neighbor_flows(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                   const struct sbrec_mac_binding_table
> *mac_binding_table,
> -                   struct ovn_desired_flow_table *flow_table)
> -{
> -    const struct sbrec_mac_binding *b;
> -    SBREC_MAC_BINDING_TABLE_FOR_EACH (b, mac_binding_table) {
> -        consider_neighbor_flow(sbrec_port_binding_by_name, b, flow_table);
> -    }
> -}
> -
> -/* Handles neighbor changes in mac_binding table. */
> -void
> -lflow_handle_changed_neighbors(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_mac_binding_table *mac_binding_table,
> -    struct ovn_desired_flow_table *flow_table)
> -{
> -
> -    const struct sbrec_mac_binding *mb;
> -    /* Handle deleted mac_bindings first, to avoid *duplicated flow*
> problem
> -     * when same flow needs to be added. */
> -    SBREC_MAC_BINDING_TABLE_FOR_EACH_TRACKED (mb, mac_binding_table) {
> -        /* Remove any flows that should be removed. */
> -        if (sbrec_mac_binding_is_deleted(mb)) {
> -            VLOG_DBG("handle deleted mac_binding "UUID_FMT,
> -                     UUID_ARGS(&mb->header_.uuid));
> -            ofctrl_remove_flows(flow_table, &mb->header_.uuid);
> -        }
> -    }
> -    SBREC_MAC_BINDING_TABLE_FOR_EACH_TRACKED (mb, mac_binding_table) {
> -        if (!sbrec_mac_binding_is_deleted(mb)) {
> -            if (!sbrec_mac_binding_is_new(mb)) {
> -                VLOG_DBG("handle updated mac_binding "UUID_FMT,
> -                         UUID_ARGS(&mb->header_.uuid));
> -                ofctrl_remove_flows(flow_table, &mb->header_.uuid);
> -            }
> -            VLOG_DBG("handle new mac_binding "UUID_FMT,
> -                     UUID_ARGS(&mb->header_.uuid));
> -            consider_neighbor_flow(sbrec_port_binding_by_name, mb,
> flow_table);
> -        }
> -    }
> -}
> -
> -
> -/* Translates logical flows in the Logical_Flow table in the OVN_SB
> database
> - * into OpenFlow flows.  See ovn-architecture(7) for more information. */
> -void
> -lflow_run(struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -          struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -          const struct sbrec_dhcp_options_table *dhcp_options_table,
> -          const struct sbrec_dhcpv6_options_table *dhcpv6_options_table,
> -          const struct sbrec_logical_flow_table *logical_flow_table,
> -          const struct sbrec_mac_binding_table *mac_binding_table,
> -          const struct sbrec_chassis *chassis,
> -          const struct hmap *local_datapaths,
> -          const struct shash *addr_sets,
> -          const struct shash *port_groups,
> -          const struct sset *active_tunnels,
> -          const struct sset *local_lport_ids,
> -          struct ovn_desired_flow_table *flow_table,
> -          struct ovn_extend_table *group_table,
> -          struct ovn_extend_table *meter_table,
> -          struct lflow_resource_ref *lfrr,
> -          uint32_t *conj_id_ofs)
> -{
> -    COVERAGE_INC(lflow_run);
> -
> -    add_logical_flows(sbrec_multicast_group_by_name_datapath,
> -                      sbrec_port_binding_by_name, dhcp_options_table,
> -                      dhcpv6_options_table, logical_flow_table,
> -                      local_datapaths, chassis, addr_sets, port_groups,
> -                      active_tunnels, local_lport_ids, flow_table,
> group_table,
> -                      meter_table, lfrr, conj_id_ofs);
> -    add_neighbor_flows(sbrec_port_binding_by_name, mac_binding_table,
> -                       flow_table);
> -}
> -
> -void
> -lflow_destroy(void)
> -{
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -    ovn_destroy_ovnfields();
> -}
> diff --git a/ovn/controller/lflow.h b/ovn/controller/lflow.h
> deleted file mode 100644
> index 4e1086eb6..000000000
> --- a/ovn/controller/lflow.h
> +++ /dev/null
> @@ -1,184 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_LFLOW_H
> -#define OVN_LFLOW_H 1
> -
> -#include "ovn/logical-fields.h"
> -
> -/* Logical_Flow table translation to OpenFlow
> - * ==========================================
> - *
> - * The Logical_Flow table obtained from the OVN_Southbound database works
> in
> - * terms of logical entities, that is, logical flows among logical
> datapaths
> - * and logical ports.  This code translates these logical flows into
> OpenFlow
> - * flows that, again, work in terms of logical entities implemented
> through
> - * OpenFlow extensions (e.g. registers represent the logical input and
> output
> - * ports).
> - *
> - * Physical-to-logical and logical-to-physical translation are
> implemented in
> - * physical.[ch] as separate OpenFlow tables that run before and after,
> - * respectively, the logical pipeline OpenFlow tables.
> - */
> -
> -#include <stdint.h>
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/uuid.h"
> -#include "openvswitch/list.h"
> -
> -struct ovn_extend_table;
> -struct ovsdb_idl_index;
> -struct ovn_desired_flow_table;
> -struct hmap;
> -struct hmap_node;
> -struct sbrec_chassis;
> -struct sbrec_dhcp_options_table;
> -struct sbrec_dhcpv6_options_table;
> -struct sbrec_logical_flow_table;
> -struct sbrec_mac_binding_table;
> -struct simap;
> -struct sset;
> -struct uuid;
> -
> -/* OpenFlow table numbers.
> - *
> - * These are heavily documented in ovn-architecture(7), please update it
> if
> - * you make any changes. */
> -#define OFTABLE_PHY_TO_LOG            0
> -#define OFTABLE_LOG_INGRESS_PIPELINE  8 /* First of LOG_PIPELINE_LEN
> tables. */
> -#define OFTABLE_REMOTE_OUTPUT        32
> -#define OFTABLE_LOCAL_OUTPUT         33
> -#define OFTABLE_CHECK_LOOPBACK       34
> -#define OFTABLE_LOG_EGRESS_PIPELINE  40 /* First of LOG_PIPELINE_LEN
> tables. */
> -#define OFTABLE_SAVE_INPORT          64
> -#define OFTABLE_LOG_TO_PHY           65
> -#define OFTABLE_MAC_BINDING          66
> -
> -/* The number of tables for the ingress and egress pipelines. */
> -#define LOG_PIPELINE_LEN 24
> -
> -enum ref_type {
> -    REF_TYPE_ADDRSET,
> -    REF_TYPE_PORTGROUP
> -};
> -
> -/* Maintains the relationship for a pair of named resource and
> - * a lflow, indexed by both ref_lflow_table and lflow_ref_table. */
> -struct lflow_ref_list_node {
> -    struct ovs_list lflow_list; /* list for same lflow */
> -    struct ovs_list ref_list; /* list for same ref */
> -    enum ref_type type;
> -    char *ref_name;
> -    struct uuid lflow_uuid;
> -};
> -
> -struct ref_lflow_node {
> -    struct hmap_node node;
> -    enum ref_type type; /* key */
> -    char *ref_name; /* key */
> -    struct ovs_list ref_lflow_head;
> -};
> -
> -struct lflow_ref_node {
> -    struct hmap_node node;
> -    struct uuid lflow_uuid; /* key */
> -    struct ovs_list lflow_ref_head;
> -};
> -
> -struct lflow_resource_ref {
> -    /* A map from a referenced resource type & name (e.g. address_set AS1)
> -     * to a list of lflows that are referencing the named resource. Data
> -     * type of each node in this hmap is struct ref_lflow_node. The
> -     * ref_lflow_head in each node points to a list of
> -     * lflow_ref_list_node.ref_list. */
> -    struct hmap ref_lflow_table;
> -
> -    /* A map from a lflow uuid to a list of named resources that are
> -     * referenced by the lflow. Data type of each node in this hmap is
> -     * struct lflow_ref_node. The lflow_ref_head in each node points to
> -     * a list of lflow_ref_list_node.lflow_list. */
> -    struct hmap lflow_ref_table;
> -};
> -
> -void lflow_resource_init(struct lflow_resource_ref *);
> -void lflow_resource_destroy(struct lflow_resource_ref *);
> -void lflow_resource_clear(struct lflow_resource_ref *);
> -
> -void lflow_init(void);
> -void lflow_run(struct ovsdb_idl_index
> *sbrec_multicast_group_by_name_datapath,
> -               struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -               const struct sbrec_dhcp_options_table *,
> -               const struct sbrec_dhcpv6_options_table *,
> -               const struct sbrec_logical_flow_table *,
> -               const struct sbrec_mac_binding_table *,
> -               const struct sbrec_chassis *chassis,
> -               const struct hmap *local_datapaths,
> -               const struct shash *addr_sets,
> -               const struct shash *port_groups,
> -               const struct sset *active_tunnels,
> -               const struct sset *local_lport_ids,
> -               struct ovn_desired_flow_table *,
> -               struct ovn_extend_table *group_table,
> -               struct ovn_extend_table *meter_table,
> -               struct lflow_resource_ref *,
> -               uint32_t *conj_id_ofs);
> -
> -bool lflow_handle_changed_flows(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_dhcp_options_table *,
> -    const struct sbrec_dhcpv6_options_table *,
> -    const struct sbrec_logical_flow_table *,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *,
> -    uint32_t *conj_id_ofs);
> -
> -bool lflow_handle_changed_ref(
> -    enum ref_type,
> -    const char *ref_name,
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_dhcp_options_table *,
> -    const struct sbrec_dhcpv6_options_table *,
> -    const struct sbrec_logical_flow_table *,
> -    const struct hmap *local_datapaths,
> -    const struct sbrec_chassis *,
> -    const struct shash *addr_sets,
> -    const struct shash *port_groups,
> -    const struct sset *active_tunnels,
> -    const struct sset *local_lport_ids,
> -    struct ovn_desired_flow_table *,
> -    struct ovn_extend_table *group_table,
> -    struct ovn_extend_table *meter_table,
> -    struct lflow_resource_ref *,
> -    uint32_t *conj_id_ofs,
> -    bool *changed);
> -
> -void lflow_handle_changed_neighbors(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct sbrec_mac_binding_table *,
> -    struct ovn_desired_flow_table *);
> -
> -void lflow_destroy(void);
> -
> -#endif /* ovn/lflow.h */
> diff --git a/ovn/controller/lport.c b/ovn/controller/lport.c
> deleted file mode 100644
> index cc5c5fbb2..000000000
> --- a/ovn/controller/lport.c
> +++ /dev/null
> @@ -1,102 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "lib/sset.h"
> -#include "lport.h"
> -#include "hash.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -VLOG_DEFINE_THIS_MODULE(lport);
> -
> -const struct sbrec_port_binding *
> -lport_lookup_by_name(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                     const char *name)
> -{
> -    struct sbrec_port_binding *pb = sbrec_port_binding_index_init_row(
> -        sbrec_port_binding_by_name);
> -    sbrec_port_binding_index_set_logical_port(pb, name);
> -
> -    const struct sbrec_port_binding *retval =
> sbrec_port_binding_index_find(
> -        sbrec_port_binding_by_name, pb);
> -
> -    sbrec_port_binding_index_destroy_row(pb);
> -
> -    return retval;
> -}
> -
> -const struct sbrec_port_binding *
> -lport_lookup_by_key(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -                    struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -                    uint64_t dp_key, uint64_t port_key)
> -{
> -    /* Lookup datapath corresponding to dp_key. */
> -    const struct sbrec_datapath_binding *db = datapath_lookup_by_key(
> -        sbrec_datapath_binding_by_key, dp_key);
> -    if (!db) {
> -        return NULL;
> -    }
> -
> -    /* Build key for an indexed lookup. */
> -    struct sbrec_port_binding *pb = sbrec_port_binding_index_init_row(
> -        sbrec_port_binding_by_key);
> -    sbrec_port_binding_index_set_datapath(pb, db);
> -    sbrec_port_binding_index_set_tunnel_key(pb, port_key);
> -
> -    const struct sbrec_port_binding *retval =
> sbrec_port_binding_index_find(
> -        sbrec_port_binding_by_key, pb);
> -
> -    sbrec_port_binding_index_destroy_row(pb);
> -
> -    return retval;
> -}
> -
> -const struct sbrec_datapath_binding *
> -datapath_lookup_by_key(struct ovsdb_idl_index
> *sbrec_datapath_binding_by_key,
> -                       uint64_t dp_key)
> -{
> -    struct sbrec_datapath_binding *db =
> sbrec_datapath_binding_index_init_row(
> -        sbrec_datapath_binding_by_key);
> -    sbrec_datapath_binding_index_set_tunnel_key(db, dp_key);
> -
> -    const struct sbrec_datapath_binding *retval
> -        = sbrec_datapath_binding_index_find(sbrec_datapath_binding_by_key,
> -                                            db);
> -
> -    sbrec_datapath_binding_index_destroy_row(db);
> -
> -    return retval;
> -}
> -
> -const struct sbrec_multicast_group *
> -mcgroup_lookup_by_dp_name(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    const struct sbrec_datapath_binding *db, const char *name)
> -{
> -    /* Build key for an indexed lookup. */
> -    struct sbrec_multicast_group *mc =
> sbrec_multicast_group_index_init_row(
> -        sbrec_multicast_group_by_name_datapath);
> -    sbrec_multicast_group_index_set_name(mc, name);
> -    sbrec_multicast_group_index_set_datapath(mc, db);
> -
> -    const struct sbrec_multicast_group *retval
> -        = sbrec_multicast_group_index_find(
> -            sbrec_multicast_group_by_name_datapath, mc);
> -
> -    sbrec_multicast_group_index_destroy_row(mc);
> -
> -    return retval;
> -}
> diff --git a/ovn/controller/lport.h b/ovn/controller/lport.h
> deleted file mode 100644
> index 7dcd5bee0..000000000
> --- a/ovn/controller/lport.h
> +++ /dev/null
> @@ -1,52 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_LPORT_H
> -#define OVN_LPORT_H 1
> -
> -#include <stdint.h>
> -
> -struct ovsdb_idl_index;
> -struct sbrec_chassis;
> -struct sbrec_datapath_binding;
> -struct sbrec_multicast_group;
> -struct sbrec_port_binding;
> -
> -
> -/* Database indexes.
> - * =================
> - *
> - * If the database IDL were a little smarter, it would allow us to
> directly
> - * look up data based on values of its fields.  It's not that smart
> (yet), so
> - * instead we define our own indexes.
> - */
> -
> -const struct sbrec_port_binding *lport_lookup_by_name(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const char *name);
> -
> -const struct sbrec_port_binding *lport_lookup_by_key(
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -    uint64_t dp_key, uint64_t port_key);
> -
> -const struct sbrec_datapath_binding *datapath_lookup_by_key(
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key, uint64_t
> dp_key);
> -
> -const struct sbrec_multicast_group *mcgroup_lookup_by_dp_name(
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath,
> -    const struct sbrec_datapath_binding *, const char *name);
> -
> -#endif /* ovn/lport.h */
> diff --git a/ovn/controller/ofctrl.c b/ovn/controller/ofctrl.c
> deleted file mode 100644
> index 043abd69d..000000000
> --- a/ovn/controller/ofctrl.c
> +++ /dev/null
> @@ -1,1393 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "bitmap.h"
> -#include "byte-order.h"
> -#include "dirs.h"
> -#include "dp-packet.h"
> -#include "flow.h"
> -#include "hash.h"
> -#include "hindex.h"
> -#include "lflow.h"
> -#include "ofctrl.h"
> -#include "openflow/openflow.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/list.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofp-flow.h"
> -#include "openvswitch/ofp-group.h"
> -#include "openvswitch/ofp-match.h"
> -#include "openvswitch/ofp-msgs.h"
> -#include "openvswitch/ofp-meter.h"
> -#include "openvswitch/ofp-packet.h"
> -#include "openvswitch/ofp-print.h"
> -#include "openvswitch/ofp-util.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn-controller.h"
> -#include "ovn/actions.h"
> -#include "ovn/lib/extend-table.h"
> -#include "openvswitch/poll-loop.h"
> -#include "physical.h"
> -#include "openvswitch/rconn.h"
> -#include "socket-util.h"
> -#include "util.h"
> -#include "vswitch-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ofctrl);
> -
> -/* An OpenFlow flow. */
> -struct ovn_flow {
> -    struct hmap_node match_hmap_node; /* For match based hashing. */
> -    struct hindex_node uuid_hindex_node; /* For uuid based hashing. */
> -    struct ovs_list list_node; /* For handling lists of flows. */
> -
> -    /* Key. */
> -    uint8_t table_id;
> -    uint16_t priority;
> -    struct minimatch match;
> -
> -    /* Data. */
> -    struct uuid sb_uuid;
> -    struct ofpact *ofpacts;
> -    size_t ofpacts_len;
> -    uint64_t cookie;
> -};
> -
> -static uint32_t ovn_flow_match_hash(const struct ovn_flow *);
> -static struct ovn_flow *ovn_flow_lookup(struct hmap *flow_table,
> -                                        const struct ovn_flow *target,
> -                                        bool cmp_sb_uuid);
> -static char *ovn_flow_to_string(const struct ovn_flow *);
> -static void ovn_flow_log(const struct ovn_flow *, const char *action);
> -static void ovn_flow_destroy(struct ovn_flow *);
> -
> -/* OpenFlow connection to the switch. */
> -static struct rconn *swconn;
> -
> -/* Symbol table for OVN expressions. */
> -static struct shash symtab;
> -
> -/* Last seen sequence number for 'swconn'.  When this differs from
> - * rconn_get_connection_seqno(rconn), 'swconn' has reconnected. */
> -static unsigned int seqno;
> -
> -/* Connection state machine. */
> -#define STATES                                  \
> -    STATE(S_NEW)                                \
> -    STATE(S_TLV_TABLE_REQUESTED)                \
> -    STATE(S_TLV_TABLE_MOD_SENT)                 \
> -    STATE(S_CLEAR_FLOWS)                        \
> -    STATE(S_UPDATE_FLOWS)
> -enum ofctrl_state {
> -#define STATE(NAME) NAME,
> -    STATES
> -#undef STATE
> -};
> -
> -/* An in-flight update to the switch's flow table.
> - *
> - * When we receive a barrier reply from the switch with the given 'xid',
> we
> - * know that the switch is caught up to northbound database sequence
> number
> - * 'nb_cfg' (and make that available to the client via
> ofctrl_get_cur_cfg(), so
> - * that it can store it into our Chassis record's nb_cfg column). */
> -struct ofctrl_flow_update {
> -    struct ovs_list list_node;  /* In 'flow_updates'. */
> -    ovs_be32 xid;               /* OpenFlow transaction ID for barrier. */
> -    int64_t nb_cfg;             /* Northbound database sequence number. */
> -};
> -
> -static struct ofctrl_flow_update *
> -ofctrl_flow_update_from_list_node(const struct ovs_list *list_node)
> -{
> -    return CONTAINER_OF(list_node, struct ofctrl_flow_update, list_node);
> -}
> -
> -/* Currently in-flight updates. */
> -static struct ovs_list flow_updates;
> -
> -/* nb_cfg of latest committed flow update. */
> -static int64_t cur_cfg;
> -
> -/* Current state. */
> -static enum ofctrl_state state;
> -
> -/* Transaction IDs for messages in flight to the switch. */
> -static ovs_be32 xid, xid2;
> -
> -/* Counter for in-flight OpenFlow messages on 'swconn'.  We only send a
> new
> - * round of flow table modifications to the switch when the counter falls
> to
> - * zero, to avoid unbounded buffering. */
> -static struct rconn_packet_counter *tx_counter;
> -
> -/* Flow table of "struct ovn_flow"s, that holds the flow table currently
> - * installed in the switch. */
> -static struct hmap installed_flows;
> -
> -/* A reference to the group_table. */
> -static struct ovn_extend_table *groups;
> -
> -/* A reference to the meter_table. */
> -static struct ovn_extend_table *meters;
> -
> -/* MFF_* field ID for our Geneve option.  In S_TLV_TABLE_MOD_SENT, this is
> - * the option we requested (we don't know whether we obtained it yet).  In
> - * S_CLEAR_FLOWS or S_UPDATE_FLOWS, this is really the option we have. */
> -static enum mf_field_id mff_ovn_geneve;
> -
> -/* Indicates if flows need to be reinstalled for scenarios when ovs
> - * is restarted, even if there is no change in the desired flow table. */
> -static bool need_reinstall_flows;
> -
> -static ovs_be32 queue_msg(struct ofpbuf *);
> -
> -static struct ofpbuf *encode_flow_mod(struct ofputil_flow_mod *);
> -
> -static struct ofpbuf *encode_group_mod(const struct ofputil_group_mod *);
> -
> -static struct ofpbuf *encode_meter_mod(const struct ofputil_meter_mod *);
> -
> -static void ovn_installed_flow_table_clear(void);
> -static void ovn_installed_flow_table_destroy(void);
> -
> -static void ofctrl_recv(const struct ofp_header *, enum ofptype);
> -
> -void
> -ofctrl_init(struct ovn_extend_table *group_table,
> -            struct ovn_extend_table *meter_table,
> -            int inactivity_probe_interval)
> -{
> -    swconn = rconn_create(inactivity_probe_interval, 0,
> -                          DSCP_DEFAULT, 1 << OFP13_VERSION);
> -    tx_counter = rconn_packet_counter_create();
> -    hmap_init(&installed_flows);
> -    ovs_list_init(&flow_updates);
> -    ovn_init_symtab(&symtab);
> -    groups = group_table;
> -    meters = meter_table;
> -}
> -
> -/* S_NEW, for a new connection.
> - *
> - * Sends NXT_TLV_TABLE_REQUEST and transitions to
> - * S_TLV_TABLE_REQUESTED. */
> -
> -static void
> -run_S_NEW(void)
> -{
> -    struct ofpbuf *buf = ofpraw_alloc(OFPRAW_NXT_TLV_TABLE_REQUEST,
> -                                      rconn_get_version(swconn), 0);
> -    xid = queue_msg(buf);
> -    state = S_TLV_TABLE_REQUESTED;
> -}
> -
> -static void
> -recv_S_NEW(const struct ofp_header *oh OVS_UNUSED,
> -           enum ofptype type OVS_UNUSED,
> -           struct shash *pending_ct_zones OVS_UNUSED)
> -{
> -    OVS_NOT_REACHED();
> -}
> -
> -/* S_TLV_TABLE_REQUESTED, when NXT_TLV_TABLE_REQUEST has been sent
> - * and we're waiting for a reply.
> - *
> - * If we receive an NXT_TLV_TABLE_REPLY:
> - *
> - *     - If it contains our tunnel metadata option, assign its field ID to
> - *       mff_ovn_geneve and transition to S_CLEAR_FLOWS.
> - *
> - *     - Otherwise, if there is an unused tunnel metadata field ID, send
> - *       NXT_TLV_TABLE_MOD and OFPT_BARRIER_REQUEST, and transition to
> - *       S_TLV_TABLE_MOD_SENT.
> - *
> - *     - Otherwise, log an error, disable Geneve, and transition to
> - *       S_CLEAR_FLOWS.
> - *
> - * If we receive an OFPT_ERROR:
> - *
> - *     - Log an error, disable Geneve, and transition to S_CLEAR_FLOWS. */
> -
> -static void
> -run_S_TLV_TABLE_REQUESTED(void)
> -{
> -}
> -
> -static bool
> -process_tlv_table_reply(const struct ofputil_tlv_table_reply *reply)
> -{
> -    const struct ofputil_tlv_map *map;
> -    uint64_t md_free = UINT64_MAX;
> -    BUILD_ASSERT(TUN_METADATA_NUM_OPTS == 64);
> -
> -    LIST_FOR_EACH (map, list_node, &reply->mappings) {
> -        if (map->option_class == OVN_GENEVE_CLASS
> -            && map->option_type == OVN_GENEVE_TYPE
> -            && map->option_len == OVN_GENEVE_LEN) {
> -            if (map->index >= TUN_METADATA_NUM_OPTS) {
> -                VLOG_ERR("desired Geneve tunnel option 0x%"PRIx16","
> -                         "%"PRIu8",%"PRIu8" already in use with "
> -                         "unsupported index %"PRIu16,
> -                         map->option_class, map->option_type,
> -                         map->option_len, map->index);
> -                return false;
> -            } else {
> -                mff_ovn_geneve = MFF_TUN_METADATA0 + map->index;
> -                state = S_CLEAR_FLOWS;
> -                return true;
> -            }
> -        }
> -
> -        if (map->index < TUN_METADATA_NUM_OPTS) {
> -            md_free &= ~(UINT64_C(1) << map->index);
> -        }
> -    }
> -
> -    VLOG_DBG("OVN Geneve option not found");
> -    if (!md_free) {
> -        VLOG_ERR("no Geneve options free for use by OVN");
> -        return false;
> -    }
> -
> -    unsigned int index = rightmost_1bit_idx(md_free);
> -    mff_ovn_geneve = MFF_TUN_METADATA0 + index;
> -    struct ofputil_tlv_map tm;
> -    tm.option_class = OVN_GENEVE_CLASS;
> -    tm.option_type = OVN_GENEVE_TYPE;
> -    tm.option_len = OVN_GENEVE_LEN;
> -    tm.index = index;
> -
> -    struct ofputil_tlv_table_mod ttm;
> -    ttm.command = NXTTMC_ADD;
> -    ovs_list_init(&ttm.mappings);
> -    ovs_list_push_back(&ttm.mappings, &tm.list_node);
> -
> -    xid = queue_msg(ofputil_encode_tlv_table_mod(OFP13_VERSION, &ttm));
> -    xid2 = queue_msg(ofputil_encode_barrier_request(OFP13_VERSION));
> -    state = S_TLV_TABLE_MOD_SENT;
> -
> -    return true;
> -}
> -
> -static void
> -recv_S_TLV_TABLE_REQUESTED(const struct ofp_header *oh, enum ofptype type,
> -                           struct shash *pending_ct_zones OVS_UNUSED)
> -{
> -    if (oh->xid != xid) {
> -        ofctrl_recv(oh, type);
> -        return;
> -    } else if (type == OFPTYPE_NXT_TLV_TABLE_REPLY) {
> -        struct ofputil_tlv_table_reply reply;
> -        enum ofperr error = ofputil_decode_tlv_table_reply(oh, &reply);
> -        if (!error) {
> -            bool ok = process_tlv_table_reply(&reply);
> -            ofputil_uninit_tlv_table(&reply.mappings);
> -            if (ok) {
> -                return;
> -            }
> -        } else {
> -            VLOG_ERR("failed to decode TLV table request (%s)",
> -                     ofperr_to_string(error));
> -        }
> -    } else if (type == OFPTYPE_ERROR) {
> -        VLOG_ERR("switch refused to allocate Geneve option (%s)",
> -                 ofperr_to_string(ofperr_decode_msg(oh, NULL)));
> -    } else {
> -        char *s = ofp_to_string(oh, ntohs(oh->length), NULL, NULL, 1);
> -        VLOG_ERR("unexpected reply to TLV table request (%s)", s);
> -        free(s);
> -    }
> -
> -    /* Error path. */
> -    mff_ovn_geneve = 0;
> -    state = S_CLEAR_FLOWS;
> -}
> -
> -/* S_TLV_TABLE_MOD_SENT, when NXT_TLV_TABLE_MOD and OFPT_BARRIER_REQUEST
> - * have been sent and we're waiting for a reply to one or the other.
> - *
> - * If we receive an OFPT_ERROR:
> - *
> - *     - If the error is NXTTMFC_ALREADY_MAPPED or NXTTMFC_DUP_ENTRY, we
> - *       raced with some other controller.  Transition to S_NEW.
> - *
> - *     - Otherwise, log an error, disable Geneve, and transition to
> - *       S_CLEAR_FLOWS.
> - *
> - * If we receive OFPT_BARRIER_REPLY:
> - *
> - *     - Set the tunnel metadata field ID to the one that we requested.
> - *       Transition to S_CLEAR_FLOWS.
> - */
> -
> -static void
> -run_S_TLV_TABLE_MOD_SENT(void)
> -{
> -}
> -
> -static void
> -recv_S_TLV_TABLE_MOD_SENT(const struct ofp_header *oh, enum ofptype type,
> -                          struct shash *pending_ct_zones OVS_UNUSED)
> -{
> -    if (oh->xid != xid && oh->xid != xid2) {
> -        ofctrl_recv(oh, type);
> -    } else if (oh->xid == xid2 && type == OFPTYPE_BARRIER_REPLY) {
> -        state = S_CLEAR_FLOWS;
> -    } else if (oh->xid == xid && type == OFPTYPE_ERROR) {
> -        enum ofperr error = ofperr_decode_msg(oh, NULL);
> -        if (error == OFPERR_NXTTMFC_ALREADY_MAPPED ||
> -            error == OFPERR_NXTTMFC_DUP_ENTRY) {
> -            VLOG_INFO("raced with another controller adding "
> -                      "Geneve option (%s); trying again",
> -                      ofperr_to_string(error));
> -            state = S_NEW;
> -        } else {
> -            VLOG_ERR("error adding Geneve option (%s)",
> -                     ofperr_to_string(error));
> -            goto error;
> -        }
> -    } else {
> -        char *s = ofp_to_string(oh, ntohs(oh->length), NULL, NULL, 1);
> -        VLOG_ERR("unexpected reply to Geneve option allocation request
> (%s)",
> -                 s);
> -        free(s);
> -        goto error;
> -    }
> -    return;
> -
> -error:
> -    state = S_CLEAR_FLOWS;
> -}
> -
> -/* S_CLEAR_FLOWS, after we've established a Geneve metadata field ID and
> it's
> - * time to set up some flows.
> - *
> - * Sends an OFPT_TABLE_MOD to clear all flows, then transitions to
> - * S_UPDATE_FLOWS. */
> -
> -static void
> -run_S_CLEAR_FLOWS(void)
> -{
> -    VLOG_DBG("clearing all flows");
> -
> -    need_reinstall_flows = true;
> -    /* Send a flow_mod to delete all flows. */
> -    struct ofputil_flow_mod fm = {
> -        .table_id = OFPTT_ALL,
> -        .command = OFPFC_DELETE,
> -    };
> -    minimatch_init_catchall(&fm.match);
> -    queue_msg(encode_flow_mod(&fm));
> -    minimatch_destroy(&fm.match);
> -
> -    /* Send a group_mod to delete all groups. */
> -    struct ofputil_group_mod gm;
> -    memset(&gm, 0, sizeof gm);
> -    gm.command = OFPGC11_DELETE;
> -    gm.group_id = OFPG_ALL;
> -    gm.command_bucket_id = OFPG15_BUCKET_ALL;
> -    ovs_list_init(&gm.buckets);
> -    queue_msg(encode_group_mod(&gm));
> -    ofputil_uninit_group_mod(&gm);
> -
> -    /* Clear installed_flows, to match the state of the switch. */
> -    ovn_installed_flow_table_clear();
> -
> -    /* Clear existing groups, to match the state of the switch. */
> -    if (groups) {
> -        ovn_extend_table_clear(groups, true);
> -    }
> -
> -    /* Send a meter_mod to delete all meters. */
> -    struct ofputil_meter_mod mm;
> -    memset(&mm, 0, sizeof mm);
> -    mm.command = OFPMC13_DELETE;
> -    mm.meter.meter_id = OFPM13_ALL;
> -    queue_msg(encode_meter_mod(&mm));
> -
> -    /* Clear existing meters, to match the state of the switch. */
> -    if (meters) {
> -        ovn_extend_table_clear(meters, true);
> -    }
> -
> -    /* All flow updates are irrelevant now. */
> -    struct ofctrl_flow_update *fup, *next;
> -    LIST_FOR_EACH_SAFE (fup, next, list_node, &flow_updates) {
> -        ovs_list_remove(&fup->list_node);
> -        free(fup);
> -    }
> -
> -    state = S_UPDATE_FLOWS;
> -}
> -
> -static void
> -recv_S_CLEAR_FLOWS(const struct ofp_header *oh, enum ofptype type,
> -                   struct shash *pending_ct_zones OVS_UNUSED)
> -{
> -    ofctrl_recv(oh, type);
> -}
> -
> -/* S_UPDATE_FLOWS, for maintaining the flow table over time.
> - *
> - * Compare the installed flows to the ones we want.  Send OFPT_FLOW_MOD as
> - * necessary.
> - *
> - * This is a terminal state.  We only transition out of it if the
> connection
> - * drops. */
> -
> -static void
> -run_S_UPDATE_FLOWS(void)
> -{
> -    /* Nothing to do here.
> -     *
> -     * Being in this state enables ofctrl_put() to work, however. */
> -}
> -
> -static void
> -recv_S_UPDATE_FLOWS(const struct ofp_header *oh, enum ofptype type,
> -                    struct shash *pending_ct_zones)
> -{
> -    if (type == OFPTYPE_BARRIER_REPLY &&
> !ovs_list_is_empty(&flow_updates)) {
> -        struct ofctrl_flow_update *fup =
> ofctrl_flow_update_from_list_node(
> -            ovs_list_front(&flow_updates));
> -        if (fup->xid == oh->xid) {
> -            if (fup->nb_cfg >= cur_cfg) {
> -                cur_cfg = fup->nb_cfg;
> -            }
> -            ovs_list_remove(&fup->list_node);
> -            free(fup);
> -        }
> -
> -        /* If the barrier xid is associated with an outstanding conntrack
> -         * flush, the flush succeeded.  Move the pending ct zone entry
> -         * to the next stage. */
> -        struct shash_node *iter;
> -        SHASH_FOR_EACH(iter, pending_ct_zones) {
> -            struct ct_zone_pending_entry *ctzpe = iter->data;
> -            if (ctzpe->state == CT_ZONE_OF_SENT && ctzpe->of_xid ==
> oh->xid) {
> -                ctzpe->state = CT_ZONE_DB_QUEUED;
> -            }
> -        }
> -    } else {
> -        ofctrl_recv(oh, type);
> -    }
> -}
> -
> -
> -enum mf_field_id
> -ofctrl_get_mf_field_id(void)
> -{
> -    if (!rconn_is_connected(swconn)) {
> -        return 0;
> -    }
> -    return (state == S_CLEAR_FLOWS || state == S_UPDATE_FLOWS
> -            ? mff_ovn_geneve : 0);
> -}
> -
> -/* Runs the OpenFlow state machine against 'br_int', which is local to the
> - * hypervisor on which we are running.  Attempts to negotiate a Geneve
> option
> - * field for class OVN_GENEVE_CLASS, type OVN_GENEVE_TYPE. */
> -void
> -ofctrl_run(const struct ovsrec_bridge *br_int, struct shash
> *pending_ct_zones)
> -{
> -    char *target = xasprintf("unix:%s/%s.mgmt", ovs_rundir(),
> br_int->name);
> -    if (strcmp(target, rconn_get_target(swconn))) {
> -        VLOG_INFO("%s: connecting to switch", target);
> -        rconn_connect(swconn, target, target);
> -    }
> -    free(target);
> -
> -    rconn_run(swconn);
> -
> -    if (!rconn_is_connected(swconn)) {
> -        return;
> -    }
> -    if (seqno != rconn_get_connection_seqno(swconn)) {
> -        seqno = rconn_get_connection_seqno(swconn);
> -        state = S_NEW;
> -
> -        /* Reset the state of any outstanding ct flushes to resend them.
> */
> -        struct shash_node *iter;
> -        SHASH_FOR_EACH(iter, pending_ct_zones) {
> -            struct ct_zone_pending_entry *ctzpe = iter->data;
> -            if (ctzpe->state == CT_ZONE_OF_SENT) {
> -                ctzpe->state = CT_ZONE_OF_QUEUED;
> -            }
> -        }
> -    }
> -
> -    bool progress = true;
> -    for (int i = 0; progress && i < 50; i++) {
> -        /* Allow the state machine to run. */
> -        enum ofctrl_state old_state = state;
> -        switch (state) {
> -#define STATE(NAME) case NAME: run_##NAME(); break;
> -            STATES
> -#undef STATE
> -        default:
> -            OVS_NOT_REACHED();
> -        }
> -
> -        /* Try to process a received packet. */
> -        struct ofpbuf *msg = rconn_recv(swconn);
> -        if (msg) {
> -            const struct ofp_header *oh = msg->data;
> -            enum ofptype type;
> -            enum ofperr error;
> -
> -            error = ofptype_decode(&type, oh);
> -            if (!error) {
> -                switch (state) {
> -#define STATE(NAME) case NAME: recv_##NAME(oh, type, pending_ct_zones);
> break;
> -                    STATES
> -#undef STATE
> -                default:
> -                    OVS_NOT_REACHED();
> -                }
> -            } else {
> -                char *s = ofp_to_string(oh, ntohs(oh->length), NULL,
> NULL, 1);
> -                VLOG_WARN("could not decode OpenFlow message (%s): %s",
> -                          ofperr_to_string(error), s);
> -                free(s);
> -            }
> -
> -            ofpbuf_delete(msg);
> -        }
> -
> -        /* If we did some work, plan to go around again. */
> -        progress = old_state != state || msg;
> -    }
> -    if (progress) {
> -        /* We bailed out to limit the amount of work we do in one go, to
> allow
> -         * other code a chance to run.  We were still making progress at
> that
> -         * point, so ensure that we come back again without waiting. */
> -        poll_immediate_wake();
> -    }
> -}
> -
> -void
> -ofctrl_wait(void)
> -{
> -    rconn_run_wait(swconn);
> -    rconn_recv_wait(swconn);
> -}
> -
> -void
> -ofctrl_destroy(void)
> -{
> -    rconn_destroy(swconn);
> -    ovn_installed_flow_table_destroy();
> -    rconn_packet_counter_destroy(tx_counter);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -int64_t
> -ofctrl_get_cur_cfg(void)
> -{
> -    return cur_cfg;
> -}
> -
> -static ovs_be32
> -queue_msg(struct ofpbuf *msg)
> -{
> -    const struct ofp_header *oh = msg->data;
> -    ovs_be32 xid_ = oh->xid;
> -    rconn_send(swconn, msg, tx_counter);
> -    return xid_;
> -}
> -
> -static void
> -log_openflow_rl(struct vlog_rate_limit *rl, enum vlog_level level,
> -                const struct ofp_header *oh, const char *title)
> -{
> -    if (!vlog_should_drop(&this_module, level, rl)) {
> -        char *s = ofp_to_string(oh, ntohs(oh->length), NULL, NULL, 2);
> -        vlog(&this_module, level, "%s: %s", title, s);
> -        free(s);
> -    }
> -}
> -
> -static void
> -ofctrl_recv(const struct ofp_header *oh, enum ofptype type)
> -{
> -    if (type == OFPTYPE_ECHO_REQUEST) {
> -        queue_msg(ofputil_encode_echo_reply(oh));
> -    } else if (type == OFPTYPE_ERROR) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(30, 300);
> -        log_openflow_rl(&rl, VLL_INFO, oh, "OpenFlow error");
> -    } else {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(30, 300);
> -        log_openflow_rl(&rl, VLL_DBG, oh, "OpenFlow packet ignored");
> -    }
> -}
> -
> -/* Flow table interfaces to the rest of ovn-controller. */
> -
> -/* Adds a flow to 'desired_flows' with the specified 'match' and
> 'actions' to
> - * the OpenFlow table numbered 'table_id' with the given 'priority' and
> - * OpenFlow 'cookie'.  The caller retains ownership of 'match' and
> 'actions'.
> - *
> - * The flow is also added to a hash index based on sb_uuid.
> - *
> - * This just assembles the desired flow table in memory.  Nothing is
> actually
> - * sent to the switch until a later call to ofctrl_put().
> - *
> - * The caller should initialize its own hmap to hold the flows. */
> -void
> -ofctrl_check_and_add_flow(struct ovn_desired_flow_table *flow_table,
> -                          uint8_t table_id, uint16_t priority,
> -                          uint64_t cookie, const struct match *match,
> -                          const struct ofpbuf *actions,
> -                          const struct uuid *sb_uuid,
> -                          bool log_duplicate_flow)
> -{
> -    struct ovn_flow *f = xmalloc(sizeof *f);
> -    f->table_id = table_id;
> -    f->priority = priority;
> -    minimatch_init(&f->match, match);
> -    f->ofpacts = xmemdup(actions->data, actions->size);
> -    f->ofpacts_len = actions->size;
> -    f->sb_uuid = *sb_uuid;
> -    f->match_hmap_node.hash = ovn_flow_match_hash(f);
> -    f->uuid_hindex_node.hash = uuid_hash(&f->sb_uuid);
> -    f->cookie = cookie;
> -
> -    ovn_flow_log(f, "ofctrl_add_flow");
> -
> -    if (ovn_flow_lookup(&flow_table->match_flow_table, f, true)) {
> -        if (log_duplicate_flow) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5);
> -            if (!VLOG_DROP_DBG(&rl)) {
> -                char *s = ovn_flow_to_string(f);
> -                VLOG_DBG("dropping duplicate flow: %s", s);
> -                free(s);
> -            }
> -        }
> -        ovn_flow_destroy(f);
> -        return;
> -    }
> -
> -    hmap_insert(&flow_table->match_flow_table, &f->match_hmap_node,
> -                f->match_hmap_node.hash);
> -    hindex_insert(&flow_table->uuid_flow_table, &f->uuid_hindex_node,
> -                  f->uuid_hindex_node.hash);
> -}
> -
> -/* Removes a bundles of flows from the flow table. */
> -void
> -ofctrl_remove_flows(struct ovn_desired_flow_table *flow_table,
> -                    const struct uuid *sb_uuid)
> -{
> -    struct ovn_flow *f, *next;
> -    HINDEX_FOR_EACH_WITH_HASH_SAFE (f, next, uuid_hindex_node,
> -                                    uuid_hash(sb_uuid),
> -                                    &flow_table->uuid_flow_table) {
> -        if (uuid_equals(&f->sb_uuid, sb_uuid)) {
> -            ovn_flow_log(f, "ofctrl_remove_flow");
> -            hmap_remove(&flow_table->match_flow_table,
> -                        &f->match_hmap_node);
> -            hindex_remove(&flow_table->uuid_flow_table,
> &f->uuid_hindex_node);
> -            ovn_flow_destroy(f);
> -        }
> -    }
> -
> -    /* remove any related group and meter info */
> -    ovn_extend_table_remove_desired(groups, sb_uuid);
> -    ovn_extend_table_remove_desired(meters, sb_uuid);
> -}
> -
> -void
> -ofctrl_add_flow(struct ovn_desired_flow_table *desired_flows,
> -                uint8_t table_id, uint16_t priority, uint64_t cookie,
> -                const struct match *match, const struct ofpbuf *actions,
> -                const struct uuid *sb_uuid)
> -{
> -    ofctrl_check_and_add_flow(desired_flows, table_id, priority, cookie,
> -                              match, actions, sb_uuid, true);
> -}
> -
> -/* ovn_flow. */
> -
> -/* Returns a hash of the match key in 'f'. */
> -static uint32_t
> -ovn_flow_match_hash(const struct ovn_flow *f)
> -{
> -    return hash_2words((f->table_id << 16) | f->priority,
> -                       minimatch_hash(&f->match, 0));
> -}
> -
> -/* Duplicate an ovn_flow structure. */
> -struct ovn_flow *
> -ofctrl_dup_flow(struct ovn_flow *src)
> -{
> -    struct ovn_flow *dst = xmalloc(sizeof *dst);
> -    dst->table_id = src->table_id;
> -    dst->priority = src->priority;
> -    minimatch_clone(&dst->match, &src->match);
> -    dst->ofpacts = xmemdup(src->ofpacts, src->ofpacts_len);
> -    dst->ofpacts_len = src->ofpacts_len;
> -    dst->sb_uuid = src->sb_uuid;
> -    dst->match_hmap_node.hash = ovn_flow_match_hash(dst);
> -    dst->uuid_hindex_node.hash = uuid_hash(&src->sb_uuid);
> -    return dst;
> -}
> -
> -/* Finds and returns an ovn_flow in 'flow_table' whose key is identical to
> - * 'target''s key, or NULL if there is none. */
> -static struct ovn_flow *
> -ovn_flow_lookup(struct hmap *flow_table, const struct ovn_flow *target,
> -                bool cmp_sb_uuid)
> -{
> -    struct ovn_flow *f;
> -
> -    HMAP_FOR_EACH_WITH_HASH (f, match_hmap_node,
> target->match_hmap_node.hash,
> -                             flow_table) {
> -        if (f->table_id == target->table_id
> -            && f->priority == target->priority
> -            && minimatch_equal(&f->match, &target->match)) {
> -            if (!cmp_sb_uuid || uuid_equals(&target->sb_uuid,
> &f->sb_uuid)) {
> -                return f;
> -            }
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static char *
> -ovn_flow_to_string(const struct ovn_flow *f)
> -{
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&s, "sb_uuid="UUID_FMT", ", UUID_ARGS(&f->sb_uuid));
> -    ds_put_format(&s, "table_id=%"PRIu8", ", f->table_id);
> -    ds_put_format(&s, "priority=%"PRIu16", ", f->priority);
> -    minimatch_format(&f->match, NULL, NULL, &s, OFP_DEFAULT_PRIORITY);
> -    ds_put_cstr(&s, ", actions=");
> -    struct ofpact_format_params fp = { .s = &s };
> -    ofpacts_format(f->ofpacts, f->ofpacts_len, &fp);
> -    return ds_steal_cstr(&s);
> -}
> -
> -static void
> -ovn_flow_log(const struct ovn_flow *f, const char *action)
> -{
> -    if (VLOG_IS_DBG_ENABLED()) {
> -        char *s = ovn_flow_to_string(f);
> -        VLOG_DBG("%s flow: %s", action, s);
> -        free(s);
> -    }
> -}
> -
> -static void
> -ovn_flow_destroy(struct ovn_flow *f)
> -{
> -    if (f) {
> -        minimatch_destroy(&f->match);
> -        free(f->ofpacts);
> -        free(f);
> -    }
> -}
> -
> -/* Flow tables of struct ovn_flow. */
> -void
> -ovn_desired_flow_table_init(struct ovn_desired_flow_table *flow_table)
> -{
> -    hmap_init(&flow_table->match_flow_table);
> -    hindex_init(&flow_table->uuid_flow_table);
> -}
> -
> -void
> -ovn_desired_flow_table_clear(struct ovn_desired_flow_table *flow_table)
> -{
> -    struct ovn_flow *f, *next;
> -    HMAP_FOR_EACH_SAFE (f, next, match_hmap_node,
> -                        &flow_table->match_flow_table) {
> -        hmap_remove(&flow_table->match_flow_table, &f->match_hmap_node);
> -        hindex_remove(&flow_table->uuid_flow_table, &f->uuid_hindex_node);
> -        ovn_flow_destroy(f);
> -    }
> -}
> -
> -void
> -ovn_desired_flow_table_destroy(struct ovn_desired_flow_table *flow_table)
> -{
> -    ovn_desired_flow_table_clear(flow_table);
> -    hmap_destroy(&flow_table->match_flow_table);
> -    hindex_destroy(&flow_table->uuid_flow_table);
> -}
> -
> -static void
> -ovn_installed_flow_table_clear(void)
> -{
> -    struct ovn_flow *f, *next;
> -    HMAP_FOR_EACH_SAFE (f, next, match_hmap_node, &installed_flows) {
> -        hmap_remove(&installed_flows, &f->match_hmap_node);
> -        ovn_flow_destroy(f);
> -    }
> -}
> -
> -static void
> -ovn_installed_flow_table_destroy(void)
> -{
> -    ovn_installed_flow_table_clear();
> -    hmap_destroy(&installed_flows);
> -}
> -
> -/* Flow table update. */
> -
> -static struct ofpbuf *
> -encode_flow_mod(struct ofputil_flow_mod *fm)
> -{
> -    fm->buffer_id = UINT32_MAX;
> -    fm->out_port = OFPP_ANY;
> -    fm->out_group = OFPG_ANY;
> -    return ofputil_encode_flow_mod(fm, OFPUTIL_P_OF13_OXM);
> -}
> -
> -static void
> -add_flow_mod(struct ofputil_flow_mod *fm, struct ovs_list *msgs)
> -{
> -    struct ofpbuf *msg = encode_flow_mod(fm);
> -    ovs_list_push_back(msgs, &msg->list_node);
> -}
> -
> -/* group_table. */
> -
> -static struct ofpbuf *
> -encode_group_mod(const struct ofputil_group_mod *gm)
> -{
> -    return ofputil_encode_group_mod(OFP13_VERSION, gm, NULL, -1);
> -}
> -
> -static void
> -add_group_mod(const struct ofputil_group_mod *gm, struct ovs_list *msgs)
> -{
> -    struct ofpbuf *msg = encode_group_mod(gm);
> -    ovs_list_push_back(msgs, &msg->list_node);
> -}
> -
> -
> -static struct ofpbuf *
> -encode_meter_mod(const struct ofputil_meter_mod *mm)
> -{
> -    return ofputil_encode_meter_mod(OFP13_VERSION, mm);
> -}
> -
> -static void
> -add_meter_mod(const struct ofputil_meter_mod *mm, struct ovs_list *msgs)
> -{
> -    struct ofpbuf *msg = encode_meter_mod(mm);
> -    ovs_list_push_back(msgs, &msg->list_node);
> -}
> -
> -static void
> -add_ct_flush_zone(uint16_t zone_id, struct ovs_list *msgs)
> -{
> -    struct ofpbuf *msg = ofpraw_alloc(OFPRAW_NXT_CT_FLUSH_ZONE,
> -                                      rconn_get_version(swconn), 0);
> -    struct nx_zone_id *nzi = ofpbuf_put_zeros(msg, sizeof *nzi);
> -    nzi->zone_id = htons(zone_id);
> -
> -    ovs_list_push_back(msgs, &msg->list_node);
> -}
> -
> -static void
> -add_meter_string(struct ovn_extend_table_info *m_desired,
> -                 struct ovs_list *msgs)
> -{
> -    /* Create and install new meter. */
> -    struct ofputil_meter_mod mm;
> -    enum ofputil_protocol usable_protocols;
> -    char *meter_string = xasprintf("meter=%"PRIu32",%s",
> -                                   m_desired->table_id,
> -                                   &m_desired->name[9]);
> -    char *error = parse_ofp_meter_mod_str(&mm, meter_string, OFPMC13_ADD,
> -                                          &usable_protocols);
> -    if (!error) {
> -        add_meter_mod(&mm, msgs);
> -    } else {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_ERR_RL(&rl, "new meter %s %s", error, meter_string);
> -        free(error);
> -    }
> -    free(meter_string);
> -}
> -
> -static void
> -add_meter(struct ovn_extend_table_info *m_desired,
> -          const struct sbrec_meter_table *meter_table,
> -          struct ovs_list *msgs)
> -{
> -    const struct sbrec_meter *sb_meter;
> -    SBREC_METER_TABLE_FOR_EACH (sb_meter, meter_table) {
> -        if (!strcmp(m_desired->name, sb_meter->name)) {
> -            break;
> -        }
> -    }
> -
> -    if (!sb_meter) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_ERR_RL(&rl, "could not find meter named \"%s\"",
> m_desired->name);
> -        return;
> -    }
> -
> -    struct ofputil_meter_mod mm;
> -    mm.command = OFPMC13_ADD;
> -    mm.meter.meter_id = m_desired->table_id;
> -    mm.meter.flags = OFPMF13_STATS;
> -
> -    if (!strcmp(sb_meter->unit, "pktps")) {
> -        mm.meter.flags |= OFPMF13_PKTPS;
> -    } else {
> -        mm.meter.flags |= OFPMF13_KBPS;
> -    }
> -
> -    mm.meter.n_bands = sb_meter->n_bands;
> -    mm.meter.bands = xcalloc(mm.meter.n_bands, sizeof *mm.meter.bands);
> -
> -    for (size_t i = 0; i < sb_meter->n_bands; i++) {
> -        struct sbrec_meter_band *sb_band = sb_meter->bands[i];
> -        struct ofputil_meter_band *mm_band = &mm.meter.bands[i];
> -
> -        if (!strcmp(sb_band->action, "drop")) {
> -            mm_band->type = OFPMBT13_DROP;
> -        }
> -
> -        mm_band->prec_level = 0;
> -        mm_band->rate = sb_band->rate;
> -        mm_band->burst_size = sb_band->burst_size;
> -
> -        if (mm_band->burst_size) {
> -            mm.meter.flags |= OFPMF13_BURST;
> -        }
> -    }
> -
> -    add_meter_mod(&mm, msgs);
> -    free(mm.meter.bands);
> -}
> -
> -/* The flow table can be updated if the connection to the switch is up and
> - * in the correct state and not backlogged with existing flow_mods.  (Our
> - * criteria for being backlogged appear very conservative, but the socket
> - * between ovn-controller and OVS provides some buffering.) */
> -static bool
> -ofctrl_can_put(void)
> -{
> -    if (state != S_UPDATE_FLOWS
> -        || rconn_packet_counter_n_packets(tx_counter)
> -        || rconn_get_version(swconn) < 0) {
> -        return false;
> -    }
> -    return true;
> -}
> -
> -/* Replaces the flow table on the switch, if possible, by the flows added
> - * with ofctrl_add_flow().
> - *
> - * Replaces the group table and meter table on the switch, if possible,
> - * by the contents of '->desired'.
> - *
> - * Sends conntrack flush messages to each zone in 'pending_ct_zones' that
> - * is in the CT_ZONE_OF_QUEUED state and then moves the zone into the
> - * CT_ZONE_OF_SENT state.
> - *
> - * This should be called after ofctrl_run() within the main loop. */
> -void
> -ofctrl_put(struct ovn_desired_flow_table *flow_table,
> -           struct shash *pending_ct_zones,
> -           const struct sbrec_meter_table *meter_table,
> -           int64_t nb_cfg,
> -           bool flow_changed)
> -{
> -    static bool skipped_last_time = false;
> -    static int64_t old_nb_cfg = 0;
> -    bool need_put = false;
> -    if (flow_changed || skipped_last_time || need_reinstall_flows) {
> -        need_put = true;
> -    } else if (nb_cfg != old_nb_cfg) {
> -        /* nb_cfg changed since last ofctrl_put() call */
> -        if (cur_cfg == old_nb_cfg) {
> -            /* we were up-to-date already, so just update with the
> -             * new nb_cfg */
> -            cur_cfg = nb_cfg;
> -        } else {
> -            need_put = true;
> -        }
> -    }
> -
> -    old_nb_cfg = nb_cfg;
> -
> -    if (!need_put) {
> -        VLOG_DBG("ofctrl_put not needed");
> -        return;
> -    }
> -    if (!ofctrl_can_put()) {
> -        VLOG_DBG("ofctrl_put can't be performed");
> -        skipped_last_time = true;
> -        return;
> -    }
> -
> -    skipped_last_time = false;
> -    need_reinstall_flows = false;
> -
> -    /* OpenFlow messages to send to the switch to bring it up-to-date. */
> -    struct ovs_list msgs = OVS_LIST_INITIALIZER(&msgs);
> -
> -    /* Iterate through ct zones that need to be flushed. */
> -    struct shash_node *iter;
> -    SHASH_FOR_EACH(iter, pending_ct_zones) {
> -        struct ct_zone_pending_entry *ctzpe = iter->data;
> -        if (ctzpe->state == CT_ZONE_OF_QUEUED) {
> -            add_ct_flush_zone(ctzpe->zone, &msgs);
> -            ctzpe->state = CT_ZONE_OF_SENT;
> -            ctzpe->of_xid = 0;
> -        }
> -    }
> -
> -    /* Iterate through all the desired groups. If there are new ones,
> -     * add them to the switch. */
> -    struct ovn_extend_table_info *desired;
> -    EXTEND_TABLE_FOR_EACH_UNINSTALLED (desired, groups) {
> -        /* Create and install new group. */
> -        struct ofputil_group_mod gm;
> -        enum ofputil_protocol usable_protocols;
> -        char *group_string = xasprintf("group_id=%"PRIu32",%s",
> -                                       desired->table_id,
> -                                       desired->name);
> -        char *error = parse_ofp_group_mod_str(&gm, OFPGC11_ADD,
> group_string,
> -                                              NULL, NULL,
> &usable_protocols);
> -        if (!error) {
> -            add_group_mod(&gm, &msgs);
> -        } else {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_ERR_RL(&rl, "new group %s %s", error, group_string);
> -            free(error);
> -        }
> -        free(group_string);
> -        ofputil_uninit_group_mod(&gm);
> -    }
> -
> -    /* Iterate through all the desired meters. If there are new ones,
> -     * add them to the switch. */
> -    struct ovn_extend_table_info *m_desired;
> -    EXTEND_TABLE_FOR_EACH_UNINSTALLED (m_desired, meters) {
> -        if (!strncmp(m_desired->name, "__string: ", 10)) {
> -            /* The "set-meter" action creates a meter entry name that
> -             * describes the meter itself. */
> -            add_meter_string(m_desired, &msgs);
> -        } else {
> -            add_meter(m_desired, meter_table, &msgs);
> -        }
> -    }
> -
> -    /* Iterate through all of the installed flows.  If any of them are no
> -     * longer desired, delete them; if any of them should have different
> -     * actions, update them. */
> -    struct ovn_flow *i, *next;
> -    HMAP_FOR_EACH_SAFE (i, next, match_hmap_node, &installed_flows) {
> -        struct ovn_flow *d =
> ovn_flow_lookup(&flow_table->match_flow_table,
> -                                             i, false);
> -        if (!d) {
> -            /* Installed flow is no longer desirable.  Delete it from the
> -             * switch and from installed_flows. */
> -            struct ofputil_flow_mod fm = {
> -                .match = i->match,
> -                .priority = i->priority,
> -                .table_id = i->table_id,
> -                .command = OFPFC_DELETE_STRICT,
> -            };
> -            add_flow_mod(&fm, &msgs);
> -            ovn_flow_log(i, "removing installed");
> -
> -            hmap_remove(&installed_flows, &i->match_hmap_node);
> -            ovn_flow_destroy(i);
> -        } else {
> -            if (!uuid_equals(&i->sb_uuid, &d->sb_uuid)) {
> -                /* Update installed flow's UUID. */
> -                i->sb_uuid = d->sb_uuid;
> -            }
> -            if (!ofpacts_equal(i->ofpacts, i->ofpacts_len,
> -                               d->ofpacts, d->ofpacts_len)) {
> -                /* Update actions in installed flow. */
> -                struct ofputil_flow_mod fm = {
> -                    .match = i->match,
> -                    .priority = i->priority,
> -                    .table_id = i->table_id,
> -                    .ofpacts = d->ofpacts,
> -                    .ofpacts_len = d->ofpacts_len,
> -                    .command = OFPFC_MODIFY_STRICT,
> -                };
> -                add_flow_mod(&fm, &msgs);
> -                ovn_flow_log(i, "updating installed");
> -
> -                /* Replace 'i''s actions by 'd''s. */
> -                free(i->ofpacts);
> -                i->ofpacts = xmemdup(d->ofpacts, d->ofpacts_len);
> -                i->ofpacts_len = d->ofpacts_len;
> -            }
> -
> -        }
> -    }
> -
> -    /* Iterate through the desired flows and add those that aren't found
> -     * in the installed flow table. */
> -    struct ovn_flow *d;
> -    HMAP_FOR_EACH (d, match_hmap_node, &flow_table->match_flow_table) {
> -        i = ovn_flow_lookup(&installed_flows, d, false);
> -        if (!i) {
> -            /* Send flow_mod to add flow. */
> -            struct ofputil_flow_mod fm = {
> -                .match = d->match,
> -                .priority = d->priority,
> -                .table_id = d->table_id,
> -                .ofpacts = d->ofpacts,
> -                .ofpacts_len = d->ofpacts_len,
> -                .new_cookie = htonll(d->cookie),
> -                .command = OFPFC_ADD,
> -            };
> -            add_flow_mod(&fm, &msgs);
> -            ovn_flow_log(d, "adding installed");
> -
> -            /* Copy 'd' from 'flow_table' to installed_flows. */
> -            struct ovn_flow *new_node = ofctrl_dup_flow(d);
> -            hmap_insert(&installed_flows, &new_node->match_hmap_node,
> -                        new_node->match_hmap_node.hash);
> -        }
> -    }
> -
> -    /* Iterate through the installed groups from previous runs. If they
> -     * are not needed delete them. */
> -    struct ovn_extend_table_info *installed, *next_group;
> -    EXTEND_TABLE_FOR_EACH_INSTALLED (installed, next_group, groups) {
> -        /* Delete the group. */
> -        struct ofputil_group_mod gm;
> -        enum ofputil_protocol usable_protocols;
> -        char *group_string = xasprintf("group_id=%"PRIu32"",
> -                                       installed->table_id);
> -        char *error = parse_ofp_group_mod_str(&gm, OFPGC11_DELETE,
> -                                              group_string, NULL, NULL,
> -                                              &usable_protocols);
> -        if (!error) {
> -            add_group_mod(&gm, &msgs);
> -        } else {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_ERR_RL(&rl, "Error deleting group %d: %s",
> -                        installed->table_id, error);
> -            free(error);
> -        }
> -        free(group_string);
> -        ofputil_uninit_group_mod(&gm);
> -        ovn_extend_table_remove_existing(groups, installed);
> -    }
> -
> -    /* Sync the contents of groups->desired to groups->existing. */
> -    ovn_extend_table_sync(groups);
> -
> -    /* Iterate through the installed meters from previous runs. If they
> -     * are not needed delete them. */
> -    struct ovn_extend_table_info *m_installed, *next_meter;
> -    EXTEND_TABLE_FOR_EACH_INSTALLED (m_installed, next_meter, meters) {
> -        /* Delete the meter. */
> -        struct ofputil_meter_mod mm = {
> -            .command = OFPMC13_DELETE,
> -            .meter = { .meter_id = m_installed->table_id },
> -        };
> -        add_meter_mod(&mm, &msgs);
> -
> -        ovn_extend_table_remove_existing(meters, m_installed);
> -    }
> -
> -    /* Sync the contents of meters->desired to meters->existing. */
> -    ovn_extend_table_sync(meters);
> -
> -    if (!ovs_list_is_empty(&msgs)) {
> -        /* Add a barrier to the list of messages. */
> -        struct ofpbuf *barrier =
> ofputil_encode_barrier_request(OFP13_VERSION);
> -        const struct ofp_header *oh = barrier->data;
> -        ovs_be32 xid_ = oh->xid;
> -        ovs_list_push_back(&msgs, &barrier->list_node);
> -
> -        /* Queue the messages. */
> -        struct ofpbuf *msg;
> -        LIST_FOR_EACH_POP (msg, list_node, &msgs) {
> -            queue_msg(msg);
> -        }
> -
> -        /* Store the barrier's xid with any newly sent ct flushes. */
> -        SHASH_FOR_EACH(iter, pending_ct_zones) {
> -            struct ct_zone_pending_entry *ctzpe = iter->data;
> -            if (ctzpe->state == CT_ZONE_OF_SENT && !ctzpe->of_xid) {
> -                ctzpe->of_xid = xid_;
> -            }
> -        }
> -
> -        /* Track the flow update. */
> -        struct ofctrl_flow_update *fup, *prev;
> -        LIST_FOR_EACH_REVERSE_SAFE (fup, prev, list_node, &flow_updates) {
> -            if (nb_cfg < fup->nb_cfg) {
> -                /* This ofctrl_flow_update is for a configuration later
> than
> -                 * 'nb_cfg'.  This should not normally happen, because it
> means
> -                 * that 'nb_cfg' in the SB_Global table of the southbound
> -                 * database decreased, and it should normally be
> monotonically
> -                 * increasing. */
> -                VLOG_WARN("nb_cfg regressed from %"PRId64" to %"PRId64,
> -                          fup->nb_cfg, nb_cfg);
> -                ovs_list_remove(&fup->list_node);
> -                free(fup);
> -            } else if (nb_cfg == fup->nb_cfg) {
> -                /* This ofctrl_flow_update is for the same configuration
> as
> -                 * 'nb_cfg'.  Probably, some change to the physical
> topology
> -                 * means that we had to revise the OpenFlow flow table
> even
> -                 * though the logical topology did not change.  Update
> fp->xid,
> -                 * so that we don't send a notification that we're
> up-to-date
> -                 * until we're really caught up. */
> -                VLOG_DBG("advanced xid target for nb_cfg=%"PRId64,
> nb_cfg);
> -                fup->xid = xid_;
> -                goto done;
> -            } else {
> -                break;
> -            }
> -        }
> -
> -        /* Add a flow update. */
> -        fup = xmalloc(sizeof *fup);
> -        ovs_list_push_back(&flow_updates, &fup->list_node);
> -        fup->xid = xid_;
> -        fup->nb_cfg = nb_cfg;
> -    done:;
> -    } else if (!ovs_list_is_empty(&flow_updates)) {
> -        /* Getting up-to-date with 'nb_cfg' didn't require any extra flow
> table
> -         * changes, so whenever we get up-to-date with the most recent
> flow
> -         * table update, we're also up-to-date with 'nb_cfg'. */
> -        struct ofctrl_flow_update *fup =
> ofctrl_flow_update_from_list_node(
> -            ovs_list_back(&flow_updates));
> -        fup->nb_cfg = nb_cfg;
> -    } else {
> -        /* We were completely up-to-date before and still are. */
> -        cur_cfg = nb_cfg;
> -    }
> -}
> -
> -/* Looks up the logical port with the name 'port_name' in 'br_int_'.  If
> - * found, returns true and sets '*portp' to the OpenFlow port number
> - * assigned to the port.  Otherwise, returns false. */
> -static bool
> -ofctrl_lookup_port(const void *br_int_, const char *port_name,
> -                   unsigned int *portp)
> -{
> -    const struct ovsrec_bridge *br_int = br_int_;
> -
> -    for (int i = 0; i < br_int->n_ports; i++) {
> -        const struct ovsrec_port *port_rec = br_int->ports[i];
> -        for (int j = 0; j < port_rec->n_interfaces; j++) {
> -            const struct ovsrec_interface *iface_rec =
> port_rec->interfaces[j];
> -            const char *iface_id = smap_get(&iface_rec->external_ids,
> -                                            "iface-id");
> -
> -            if (iface_id && !strcmp(iface_id, port_name)) {
> -                if (!iface_rec->n_ofport) {
> -                    continue;
> -                }
> -
> -                int64_t ofport = iface_rec->ofport[0];
> -                if (ofport < 1 || ofport > ofp_to_u16(OFPP_MAX)) {
> -                    continue;
> -                }
> -                *portp = ofport;
> -                return true;
> -            }
> -        }
> -    }
> -
> -    return false;
> -}
> -
> -/* Generates a packet described by 'flow_s' in the syntax of an OVN
> - * logical expression and injects it into 'br_int'.  The flow
> - * description must contain an ingress logical port that is present on
> - * 'br_int'.
> - *
> - * Returns NULL if successful, otherwise an error message that the caller
> - * must free(). */
> -char *
> -ofctrl_inject_pkt(const struct ovsrec_bridge *br_int, const char *flow_s,
> -                  const struct shash *addr_sets,
> -                  const struct shash *port_groups)
> -{
> -    int version = rconn_get_version(swconn);
> -    if (version < 0) {
> -        return xstrdup("OpenFlow channel not ready.");
> -    }
> -
> -    struct flow uflow;
> -    char *error = expr_parse_microflow(flow_s, &symtab, addr_sets,
> -                                       port_groups, ofctrl_lookup_port,
> -                                       br_int, &uflow);
> -    if (error) {
> -        return error;
> -    }
> -
> -    /* The physical OpenFlow port was stored in the logical ingress
> -     * port, so put it in the correct location for a flow structure. */
> -    uflow.in_port.ofp_port = u16_to_ofp(uflow.regs[MFF_LOG_INPORT -
> MFF_REG0]);
> -    uflow.regs[MFF_LOG_INPORT - MFF_REG0] = 0;
> -
> -    if (!uflow.in_port.ofp_port) {
> -        return xstrdup("ingress port not found on hypervisor.");
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    flow_compose(&packet, &uflow, NULL, 64);
> -
> -    uint64_t ofpacts_stub[1024 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(&ofpacts);
> -    resubmit->in_port = OFPP_IN_PORT;
> -    resubmit->table_id = 0;
> -
> -    struct ofputil_packet_out po = {
> -        .packet = dp_packet_data(&packet),
> -        .packet_len = dp_packet_size(&packet),
> -        .buffer_id = UINT32_MAX,
> -        .ofpacts = ofpacts.data,
> -        .ofpacts_len = ofpacts.size,
> -    };
> -    match_set_in_port(&po.flow_metadata, uflow.in_port.ofp_port);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    queue_msg(ofputil_encode_packet_out(&po, proto));
> -    dp_packet_uninit(&packet);
> -    ofpbuf_uninit(&ofpacts);
> -
> -    return NULL;
> -}
> -
> -bool
> -ofctrl_is_connected(void)
> -{
> -    return rconn_is_connected(swconn);
> -}
> -
> -void
> -ofctrl_set_probe_interval(int probe_interval)
> -{
> -    if (swconn) {
> -        rconn_set_probe_interval(swconn, probe_interval);
> -    }
> -}
> diff --git a/ovn/controller/ofctrl.h b/ovn/controller/ofctrl.h
> deleted file mode 100644
> index ed8918aae..000000000
> --- a/ovn/controller/ofctrl.h
> +++ /dev/null
> @@ -1,87 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OFCTRL_H
> -#define OFCTRL_H 1
> -
> -#include <stdint.h>
> -
> -#include "openvswitch/meta-flow.h"
> -#include "ovsdb-idl.h"
> -#include "hindex.h"
> -
> -struct ovn_extend_table;
> -struct hmap;
> -struct match;
> -struct ofpbuf;
> -struct ovsrec_bridge;
> -struct sbrec_meter_table;
> -struct shash;
> -
> -struct ovn_desired_flow_table {
> -    /* Hash map flow table using flow match conditions as hash key.*/
> -    struct hmap match_flow_table;
> -
> -    /* SB uuid index for the nodes in match_flow_table.*/
> -    struct hindex uuid_flow_table;
> -};
> -
> -/* Interface for OVN main loop. */
> -void ofctrl_init(struct ovn_extend_table *group_table,
> -                 struct ovn_extend_table *meter_table,
> -                 int inactivity_probe_interval);
> -void ofctrl_run(const struct ovsrec_bridge *br_int,
> -                struct shash *pending_ct_zones);
> -enum mf_field_id ofctrl_get_mf_field_id(void);
> -void ofctrl_put(struct ovn_desired_flow_table *,
> -                struct shash *pending_ct_zones,
> -                const struct sbrec_meter_table *,
> -                int64_t nb_cfg,
> -                bool flow_changed);
> -void ofctrl_wait(void);
> -void ofctrl_destroy(void);
> -int64_t ofctrl_get_cur_cfg(void);
> -
> -struct ovn_flow *ofctrl_dup_flow(struct ovn_flow *source);
> -
> -void ofctrl_ct_flush_zone(uint16_t zone_id);
> -
> -char *ofctrl_inject_pkt(const struct ovsrec_bridge *br_int,
> -                        const char *flow_s, const struct shash *addr_sets,
> -                        const struct shash *port_groups);
> -
> -/* Flow table interfaces to the rest of ovn-controller. */
> -void ofctrl_add_flow(struct ovn_desired_flow_table *, uint8_t table_id,
> -                     uint16_t priority, uint64_t cookie,
> -                     const struct match *, const struct ofpbuf *ofpacts,
> -                     const struct uuid *);
> -
> -void ofctrl_remove_flows(struct ovn_desired_flow_table *, const struct
> uuid *);
> -
> -void ovn_desired_flow_table_init(struct ovn_desired_flow_table *);
> -void ovn_desired_flow_table_clear(struct ovn_desired_flow_table *);
> -void ovn_desired_flow_table_destroy(struct ovn_desired_flow_table *);
> -
> -void ofctrl_check_and_add_flow(struct ovn_desired_flow_table *,
> -                               uint8_t table_id, uint16_t priority,
> -                               uint64_t cookie, const struct match *,
> -                               const struct ofpbuf *ofpacts,
> -                               const struct uuid *, bool
> log_duplicate_flow);
> -
> -bool ofctrl_is_connected(void);
> -void ofctrl_set_probe_interval(int probe_interval);
> -
> -#endif /* ovn/ofctrl.h */
> diff --git a/ovn/controller/ovn-controller.8.xml
> b/ovn/controller/ovn-controller.8.xml
> deleted file mode 100644
> index 780625ff8..000000000
> --- a/ovn/controller/ovn-controller.8.xml
> +++ /dev/null
> @@ -1,456 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-controller" section="8" title="ovn-controller">
> -    <h1>Name</h1>
> -    <p>ovn-controller -- Open Virtual Network local controller</p>
> -
> -    <h1>Synopsis</h1>
> -    <p><code>ovn-controller</code> [<var>options</var>]
> [<var>ovs-database</var>]</p>
> -
> -    <h1>Description</h1>
> -    <p>
> -      <code>ovn-controller</code> is the local controller daemon for
> -      OVN, the Open Virtual Network.  It connects up to the OVN
> -      Southbound database (see <code>ovn-sb</code>(5)) over the OVSDB
> -      protocol, and down to the Open vSwitch database (see
> -      <code>ovs-vswitchd.conf.db</code>(5)) over the OVSDB protocol and
> -      to <code>ovs-vswitchd</code>(8) via OpenFlow.  Each hypervisor and
> -      software gateway in an OVN deployment runs its own independent
> -      copy of <code>ovn-controller</code>; thus,
> -      <code>ovn-controller</code>'s downward connections are
> -      machine-local and do not run over a physical network.
> -    </p>
> -
> -    <h1>ACL Logging</h1>
> -    <p>
> -      ACL log messages are logged through <code>ovn-controller</code>'s
> -      logging mechanism.  ACL log entries have the module
> -      <code>acl_log</code> at log level <code>info</code>.  Configuring
> -      logging is described below in the <code>Logging Options</code>
> -      section.
> -    </p>
> -
> -    <h1>Options</h1>
> -
> -    <h2>Daemon Options</h2>
> -    <xi:include href="lib/daemon.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>Logging Options</h2>
> -    <xi:include href="lib/vlog.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>PKI Options</h2>
> -    <p>
> -      PKI configuration is required in order to use SSL for the
> connections to
> -      the Northbound and Southbound databases.
> -    </p>
> -    <xi:include href="lib/ssl.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -    <xi:include href="lib/ssl-bootstrap.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -    <xi:include href="lib/ssl-peer-ca-cert.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>Other Options</h2>
> -
> -    <xi:include href="lib/common.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -
> -    <h1>Configuration</h1>
> -    <p>
> -      <code>ovn-controller</code> retrieves most of its configuration
> -      information from the local Open vSwitch's ovsdb-server instance.
> -      The default location is <code>db.sock</code> in the local Open
> -      vSwitch's "run" directory.  It may be overridden by specifying the
> -      <var>ovs-database</var> argument as an OVSDB active or passive
> -      connection method, as described in <code>ovsdb</code>(7).
> -    </p>
> -
> -    <p>
> -      <code>ovn-controller</code> assumes it gets configuration
> -      information from the following keys in the <code>Open_vSwitch</code>
> -      table of the local OVS instance:
> -    </p>
> -    <dl>
> -      <dt><code>external_ids:system-id</code></dt>
> -      <dd>The chassis name to use in the Chassis table.</dd>
> -
> -      <dt><code>external_ids:hostname</code></dt>
> -      <dd>The hostname to use in the Chassis table.</dd>
> -
> -      <dt><code>external_ids:ovn-bridge</code></dt>
> -      <dd>
> -        The integration bridge to which logical ports are attached.  The
> -        default is <code>br-int</code>.  If this bridge does not exist
> when
> -        ovn-controller starts, it will be created automatically with the
> -        default configuration suggested in
> <code>ovn-architecture</code>(7).
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-bridge-datapath-type</code></dt>
> -      <dd>
> -        This configuration is optional. If set, then the datapath type of
> -        the integration bridge will be set to the configured value. If
> this
> -        option is not set, then <code>ovn-controller</code> will not
> modify
> -        the existing <code>datapath-type</code> of the integration bridge.
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-remote</code></dt>
> -      <dd>
> -        <p>
> -          The OVN database that this system should connect to for its
> -          configuration, in one of the same forms documented above for the
> -          <var>ovs-database</var>.
> -        </p>
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-remote-probe-interval</code></dt>
> -      <dd>
> -        <p>
> -          The inactivity probe interval of the connection to the OVN
> database,
> -          in milliseconds.
> -          If the value is zero, it disables the connection keepalive
> feature.
> -        </p>
> -
> -        <p>
> -          If the value is nonzero, then it will be forced to a value of
> -          at least 1000 ms.
> -        </p>
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-openflow-probe-interval</code></dt>
> -      <dd>
> -        <p>
> -          The inactivity probe interval of the OpenFlow connection to the
> -          OpenvSwitch integration bridge, in seconds.
> -          If the value is zero, it disables the connection keepalive
> feature.
> -        </p>
> -
> -        <p>
> -          If the value is nonzero, then it will be forced to a value of
> -          at least 5s.
> -        </p>
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-encap-type</code></dt>
> -      <dd>
> -        <p>
> -          The encapsulation type that a chassis should use to connect to
> -          this node.  Multiple encapsulation types may be specified with
> -          a comma-separated list.  Each listed encapsulation type will
> -          be paired with <code>ovn-encap-ip</code>.
> -        </p>
> -
> -        <p>
> -          Supported tunnel types for connecting hypervisors
> -          are <code>geneve</code> and <code>stt</code>.  Gateways may
> -          use <code>geneve</code>, <code>vxlan</code>, or
> -          <code>stt</code>.
> -        </p>
> -
> -        <p>
> -          Due to the limited amount of metadata in <code>vxlan</code>,
> -          the capabilities and performance of connected gateways will be
> -          reduced versus other tunnel formats.
> -        </p>
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-encap-ip</code></dt>
> -      <dd>
> -        The IP address that a chassis should use to connect to this node
> -        using encapsulation types specified by
> -        <code>external_ids:ovn-encap-type</code>.
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-bridge-mappings</code></dt>
> -      <dd>
> -        A list of key-value pairs that map a physical network name to a
> local
> -        ovs bridge that provides connectivity to that network.  An example
> -        value mapping two physical network names to two ovs bridges would
> be:
> -        <code>physnet1:br-eth0,physnet2:br-eth1</code>.
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-encap-csum</code></dt>
> -      <dd>
> -        <code>ovn-encap-csum</code> indicates that encapsulation
> checksums can
> -        be transmitted and received with reasonable performance. It is a
> hint
> -        to senders transmitting data to this chassis that they should use
> -        checksums to protect OVN metadata. Set to <code>true</code> to
> enable
> -        or <code>false</code> to disable. Depending on the capabilities
> of the
> -        network interface card, enabling encapsulation checksum may incur
> -        performance loss. In such cases, encapsulation checksums can be
> disabled.
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-cms-options</code></dt>
> -      <dd>
> -        A list of options that will be consumed by the CMS Plugin and
> which
> -        specific to this particular chassis. An example would be:
> -        <code>cms_option1,cms_option2:foo</code>.
> -      </dd>
> -
> -      <dt><code>external_ids:ovn-transport-zones</code></dt>
> -      <dd>
> -        <p>
> -          The transport zone(s) that this chassis belongs to. Transport
> -          zones is a way to group different chassis so that tunnels are
> only
> -          formed between members of the same group(s). Multiple transport
> -          zones may be specified with a comma-separated list. For example:
> -          tz1,tz2,tz3.
> -        </p>
> -        <p>
> -          If not set, the Chassis will be considered part of a default
> -          transport zone.
> -        </p>
> -      </dd>
> -      <dt><code>external_ids:ovn-chassis-mac-mappings</code></dt>
> -      <dd>
> -        A list of key-value pairs that map a chassis specific mac to
> -        a physical network name. An example
> -        value mapping two chassis macs to two physical network names
> would be:
> -
> <code>physnet1:aa:bb:cc:dd:ee:ff,physnet2:a1:b2:c3:d4:e5:f6</code>.
> -        These are the macs that ovn-controller will replace a router port
> -        mac with, if packet is going from a distributed router port on
> -        vlan type logical switch.
> -      </dd>
> -    </dl>
> -
> -    <p>
> -      <code>ovn-controller</code> reads the following values from the
> -      <code>Open_vSwitch</code> database of the local OVS instance:
> -    </p>
> -
> -    <dl>
> -      <dt><code>datapath-type</code> from <ref table="Bridge"
> db="Open_vSwitch"/> table</dt>
> -      <dd>
> -        This value is read from local OVS integration bridge row of
> -        <ref table="Bridge" db="Open_vSwitch"/> table and populated in
> -        <ref key="datapath-type" table="Chassis" column="external_ids"
> -        db="OVN_Southbound"/> of the <ref table="Chassis"
> db="OVN_Southbound"/>
> -        table in the OVN_Southbound database.
> -      </dd>
> -
> -      <dt><code>iface-types</code> from <ref table="Open_vSwitch"
> db="Open_vSwitch"/> table</dt>
> -      <dd>
> -        This value is populated in <ref key="iface-types" table="Chassis"
> -        column="external_ids" db="OVN_Southbound"/> of the
> -        <ref table="Chassis" db="OVN_Southbound"/> table in the
> OVN_Southbound
> -        database.
> -      </dd>
> -
> -      <dt><code>private_key</code>, <code>certificate</code>,
> -          <code>ca_cert</code>, and <code>bootstrap_ca_cert</code>
> -          from <ref table="SSL" db="Open_vSwitch"/> table</dt>
> -      <dd>
> -        These values provide the SSL configuration used for connecting
> -        to the OVN southbound database server when an SSL connection type
> -        is configured via <code>external_ids:ovn-remote</code>.  Note that
> -        this SSL configuration can also be provided via command-line
> options,
> -        the configuration in the database takes precedence if both are
> present.
> -      </dd>
> -    </dl>
> -
> -    <h1>Open vSwitch Database Usage</h1>
> -
> -    <p>
> -      <code>ovn-controller</code> uses a number of
> <code>external_ids</code>
> -      keys in the Open vSwitch database to keep track of ports and
> interfaces.
> -      For proper operation, users should not change or clear these keys:
> -    </p>
> -
> -    <dl>
> -      <dt>
> -        <code>external_ids:ovn-chassis-id</code> in the <code>Port</code>
> table
> -      </dt>
> -      <dd>
> -        The presence of this key identifies a tunnel port within the
> -        integration bridge as one created by <code>ovn-controller</code>
> to
> -        reach a remote chassis.  Its value is the chassis ID of the remote
> -        chassis.
> -      </dd>
> -
> -      <dt>
> -        <code>external_ids:ct-zone-*</code> in the <code>Bridge</code>
> table
> -      </dt>
> -      <dd>
> -        Logical ports and gateway routers are assigned a connection
> -        tracking zone by <code>ovn-controller</code> for stateful
> -        services.  To keep state across restarts of
> -        <code>ovn-controller</code>, these keys are stored in the
> -        integration bridge's Bridge table.  The name contains a prefix
> -        of <code>ct-zone-</code> followed by the name of the logical
> -        port or gateway router's zone key.  The value for this key
> -        identifies the zone used for this port.
> -      </dd>
> -
> -      <dt>
> -        <code>external_ids:ovn-localnet-port</code> in the
> <code>Port</code>
> -        table
> -      </dt>
> -      <dd>
> -        <p>
> -          The presence of this key identifies a patch port as one created
> by
> -          <code>ovn-controller</code> to connect the integration bridge
> and
> -          another bridge to implement a <code>localnet</code> logical
> port.
> -          Its value is the name of the logical port with <code>type</code>
> -          set to <code>localnet</code> that the port implements. See
> -          <code>external_ids:ovn-bridge-mappings</code>, above, for more
> -          information.
> -        </p>
> -
> -        <p>
> -          Each <code>localnet</code> logical port is implemented as a
> pair of
> -          patch ports, one in the integration bridge, one in a different
> -          bridge, with the same
> <code>external_ids:ovn-localnet-port</code>
> -          value.
> -        </p>
> -      </dd>
> -
> -      <dt>
> -        <code>external_ids:ovn-l2gateway-port</code> in the
> <code>Port</code>
> -        table
> -      </dt>
> -      <dd>
> -        <p>
> -          The presence of this key identifies a patch port as one created
> by
> -          <code>ovn-controller</code> to connect the integration bridge
> and
> -          another bridge to implement a <code>l2gateway</code> logical
> port.
> -          Its value is the name of the logical port with <code>type</code>
> -          set to <code>l2gateway</code> that the port implements. See
> -          <code>external_ids:ovn-bridge-mappings</code>, above, for more
> -          information.
> -        </p>
> -
> -        <p>
> -          Each <code>l2gateway</code> logical port is implemented as a
> pair
> -          of patch ports, one in the integration bridge, one in a
> different
> -          bridge, with the same
> <code>external_ids:ovn-l2gateway-port</code>
> -          value.
> -        </p>
> -      </dd>
> -
> -      <dt>
> -        <code>external-ids:ovn-l3gateway-port</code> in the
> <code>Port</code>
> -        table
> -      </dt>
> -
> -      <dd>
> -        <p>
> -          This key identifies a patch port as one created by
> -          <code>ovn-controller</code> to implement a
> <code>l3gateway</code>
> -          logical port. Its value is the name of the logical port with
> type
> -          set to <code>l3gateway</code>. This patch port is similar to
> -          the OVN logical patch port, except that <code>l3gateway</code>
> -          port can only be bound to a paticular chassis.
> -        </p>
> -      </dd>
> -
> -      <dt>
> -        <code>external-ids:ovn-logical-patch-port</code> in the
> -        <code>Port</code> table
> -      </dt>
> -
> -      <dd>
> -        <p>
> -          This key identifies a patch port as one created by
> -          <code>ovn-controller</code> to implement an OVN logical patch
> port
> -          within the integration bridge.  Its value is the name of the OVN
> -          logical patch port that it implements.
> -        </p>
> -      </dd>
> -    </dl>
> -
> -    <h1>OVN Southbound Database Usage</h1>
> -
> -    <p>
> -      <code>ovn-controller</code> reads from much of the
> -      <code>OVN_Southbound</code> database to guide its operation.
> -      <code>ovn-controller</code> also writes to the following tables:
> -    </p>
> -
> -    <dl>
> -      <dt><code>Chassis</code></dt>
> -      <dd>
> -        Upon startup, <code>ovn-controller</code> creates a row in this
> table
> -        to represent its own chassis.  Upon graceful termination, e.g.
> with
> -        <code>ovs-appctl -t ovn-controller exit</code> (but not
> -        <code>SIGTERM</code>), <code>ovn-controller</code> removes its
> row.
> -      </dd>
> -
> -      <dt><code>Encap</code></dt>
> -      <dd>
> -        Upon startup, <code>ovn-controller</code> creates a row or rows
> in this
> -        table that represent the tunnel encapsulations by which its
> chassis can
> -        be reached, and points its <code>Chassis</code> row to them.  Upon
> -        graceful termination, <code>ovn-controller</code> removes these
> rows.
> -      </dd>
> -
> -      <dt><code>Port_Binding</code></dt>
> -      <dd>
> -        At runtime, <code>ovn-controller</code> sets the
> <code>chassis</code>
> -        columns of ports that are resident on its chassis to point to its
> -        <code>Chassis</code> row, and, conversely, clears the
> -        <code>chassis</code> column of ports that point to its
> -        <code>Chassis</code> row but are no longer resident on its
> chassis.
> -        The <code>chassis</code> column has a weak reference type, so when
> -        <code>ovn-controller</code> gracefully exits and removes its
> -        <code>Chassis</code> row, the database server automatically
> clears any
> -        remaining references to that row.
> -      </dd>
> -
> -      <dt><code>MAC_Binding</code></dt>
> -      <dd>
> -        At runtime, <code>ovn-controller</code> updates the
> -        <code>MAC_Binding</code> table as instructed by
> <code>put_arp</code>
> -        and <code>put_nd</code> logical actions.  These changes persist
> beyond
> -        the lifetime of <code>ovn-controller</code>.
> -      </dd>
> -    </dl>
> -
> -    <h1>Runtime Management Commands</h1>
> -    <p>
> -      <code>ovs-appctl</code> can send commands to a running
> -      <code>ovn-controller</code> process.  The currently supported
> -      commands are described below.
> -      <dl>
> -      <dt><code>exit</code></dt>
> -      <dd>
> -        Causes <code>ovn-controller</code> to gracefully terminate.
> -      </dd>
> -
> -      <dt><code>ct-zone-list</code></dt>
> -      <dd>
> -        Lists each local logical port and its connection tracking zone.
> -      </dd>
> -
> -      <dt><code>meter-table-list</code></dt>
> -      <dd>
> -        Lists each meter table entry and its local meter id.
> -      </dd>
> -
> -      <dt><code>group-table-list</code></dt>
> -      <dd>
> -        Lists each group table entry and its local group id.
> -      </dd>
> -
> -      <dt><code>inject-pkt</code> <var>microflow</var></dt>
> -      <dd>
> -      <p>
> -        Injects <var>microflow</var> into the connected Open vSwitch
> -        instance.  <var>microflow</var> must contain an ingress logical
> -        port (<code>inport</code> argument) that is present on the Open
> -        vSwitch instance.
> -      </p>
> -
> -      <p>
> -        The <var>microflow</var> argument describes the packet whose
> -        forwarding is to be simulated, in the syntax of an OVN logical
> -        expression, as described in <code>ovn-sb</code>(5), to express
> -        constraints.  The parser understands prerequisites; for example,
> -        if the expression refers to <code>ip4.src</code>, there is no
> -        need to explicitly state <code>ip4</code> or <code>eth.type ==
> -        0x800</code>.
> -      </p>
> -      </dd>
> -
> -      <dt><code>connection-status</code></dt>
> -      <dd>
> -        Show OVN SBDB connection status for the chassis.
> -      </dd>
> -      </dl>
> -    </p>
> -
> -</manpage>
> diff --git a/ovn/controller/ovn-controller.c
> b/ovn/controller/ovn-controller.c
> deleted file mode 100644
> index cf6c8ae79..000000000
> --- a/ovn/controller/ovn-controller.c
> +++ /dev/null
> @@ -1,2366 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "ovn-controller.h"
> -
> -#include <errno.h>
> -#include <getopt.h>
> -#include <signal.h>
> -#include <stdlib.h>
> -#include <string.h>
> -
> -#include "bfd.h"
> -#include "binding.h"
> -#include "chassis.h"
> -#include "command-line.h"
> -#include "compiler.h"
> -#include "daemon.h"
> -#include "dirs.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "encaps.h"
> -#include "fatal-signal.h"
> -#include "ip-mcast.h"
> -#include "openvswitch/hmap.h"
> -#include "lflow.h"
> -#include "lib/vswitch-idl.h"
> -#include "lport.h"
> -#include "ofctrl.h"
> -#include "openvswitch/vconn.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/actions.h"
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/extend-table.h"
> -#include "ovn/lib/ip-mcast-index.h"
> -#include "ovn/lib/mcast-group-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "patch.h"
> -#include "physical.h"
> -#include "pinctrl.h"
> -#include "openvswitch/poll-loop.h"
> -#include "lib/bitmap.h"
> -#include "lib/hash.h"
> -#include "smap.h"
> -#include "sset.h"
> -#include "stream-ssl.h"
> -#include "stream.h"
> -#include "unixctl.h"
> -#include "util.h"
> -#include "timeval.h"
> -#include "timer.h"
> -#include "stopwatch.h"
> -#include "ovn/lib/inc-proc-eng.h"
> -
> -VLOG_DEFINE_THIS_MODULE(main);
> -
> -static unixctl_cb_func ovn_controller_exit;
> -static unixctl_cb_func ct_zone_list;
> -static unixctl_cb_func meter_table_list;
> -static unixctl_cb_func group_table_list;
> -static unixctl_cb_func inject_pkt;
> -static unixctl_cb_func ovn_controller_conn_show;
> -
> -#define DEFAULT_BRIDGE_NAME "br-int"
> -#define DEFAULT_PROBE_INTERVAL_MSEC 5000
> -#define OFCTRL_DEFAULT_PROBE_INTERVAL_SEC 5
> -
> -#define CONTROLLER_LOOP_STOPWATCH_NAME "ovn-controller-flow-generation"
> -
> -static char *parse_options(int argc, char *argv[]);
> -OVS_NO_RETURN static void usage(void);
> -
> -/* Pending packet to be injected into connected OVS. */
> -struct pending_pkt {
> -    /* Setting 'conn' indicates that a request is pending. */
> -    struct unixctl_conn *conn;
> -    char *flow_s;
> -};
> -
> -struct local_datapath *
> -get_local_datapath(const struct hmap *local_datapaths, uint32_t
> tunnel_key)
> -{
> -    struct hmap_node *node = hmap_first_with_hash(local_datapaths,
> tunnel_key);
> -    return (node
> -            ? CONTAINER_OF(node, struct local_datapath, hmap_node)
> -            : NULL);
> -}
> -
> -uint32_t
> -get_tunnel_type(const char *name)
> -{
> -    if (!strcmp(name, "geneve")) {
> -        return GENEVE;
> -    } else if (!strcmp(name, "stt")) {
> -        return STT;
> -    } else if (!strcmp(name, "vxlan")) {
> -        return VXLAN;
> -    }
> -
> -    return 0;
> -}
> -
> -const struct ovsrec_bridge *
> -get_bridge(const struct ovsrec_bridge_table *bridge_table, const char
> *br_name)
> -{
> -    const struct ovsrec_bridge *br;
> -    OVSREC_BRIDGE_TABLE_FOR_EACH (br, bridge_table) {
> -        if (!strcmp(br->name, br_name)) {
> -            return br;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -update_sb_monitors(struct ovsdb_idl *ovnsb_idl,
> -                   const struct sbrec_chassis *chassis,
> -                   const struct sset *local_ifaces,
> -                   struct hmap *local_datapaths)
> -{
> -    /* Monitor Port_Bindings rows for local interfaces and local
> datapaths.
> -     *
> -     * Monitor Logical_Flow, MAC_Binding, Multicast_Group, and DNS tables
> for
> -     * local datapaths.
> -     *
> -     * Monitor Controller_Event rows for local chassis.
> -     *
> -     * Monitor IP_Multicast for local datapaths.
> -     *
> -     * Monitor IGMP_Groups for local chassis.
> -     *
> -     * We always monitor patch ports because they allow us to see the
> linkages
> -     * between related logical datapaths.  That way, when we know that we
> have
> -     * a VIF on a particular logical switch, we immediately know to
> monitor all
> -     * the connected logical routers and logical switches. */
> -    struct ovsdb_idl_condition pb = OVSDB_IDL_CONDITION_INIT(&pb);
> -    struct ovsdb_idl_condition lf = OVSDB_IDL_CONDITION_INIT(&lf);
> -    struct ovsdb_idl_condition mb = OVSDB_IDL_CONDITION_INIT(&mb);
> -    struct ovsdb_idl_condition mg = OVSDB_IDL_CONDITION_INIT(&mg);
> -    struct ovsdb_idl_condition dns = OVSDB_IDL_CONDITION_INIT(&dns);
> -    struct ovsdb_idl_condition ce =  OVSDB_IDL_CONDITION_INIT(&ce);
> -    struct ovsdb_idl_condition ip_mcast =
> OVSDB_IDL_CONDITION_INIT(&ip_mcast);
> -    struct ovsdb_idl_condition igmp = OVSDB_IDL_CONDITION_INIT(&igmp);
> -    sbrec_port_binding_add_clause_type(&pb, OVSDB_F_EQ, "patch");
> -    /* XXX: We can optimize this, if we find a way to only monitor
> -     * ports that have a Gateway_Chassis that point's to our own
> -     * chassis */
> -    sbrec_port_binding_add_clause_type(&pb, OVSDB_F_EQ,
> "chassisredirect");
> -    sbrec_port_binding_add_clause_type(&pb, OVSDB_F_EQ, "external");
> -    if (chassis) {
> -        /* This should be mostly redundant with the other clauses for port
> -         * bindings, but it allows us to catch any ports that are
> assigned to
> -         * us but should not be.  That way, we can clear their chassis
> -         * assignments. */
> -        sbrec_port_binding_add_clause_chassis(&pb, OVSDB_F_EQ,
> -                                              &chassis->header_.uuid);
> -
> -        /* Ensure that we find out about l2gateway and l3gateway ports
> that
> -         * should be present on this chassis.  Otherwise, we might never
> find
> -         * out about those ports, if their datapaths don't otherwise have
> a VIF
> -         * in this chassis. */
> -        const char *id = chassis->name;
> -        const struct smap l2 = SMAP_CONST1(&l2, "l2gateway-chassis", id);
> -        sbrec_port_binding_add_clause_options(&pb, OVSDB_F_INCLUDES, &l2);
> -        const struct smap l3 = SMAP_CONST1(&l3, "l3gateway-chassis", id);
> -        sbrec_port_binding_add_clause_options(&pb, OVSDB_F_INCLUDES, &l3);
> -
> -        sbrec_controller_event_add_clause_chassis(&ce, OVSDB_F_EQ,
> -                                                  &chassis->header_.uuid);
> -        sbrec_igmp_group_add_clause_chassis(&igmp, OVSDB_F_EQ,
> -                                            &chassis->header_.uuid);
> -    }
> -    if (local_ifaces) {
> -        const char *name;
> -        SSET_FOR_EACH (name, local_ifaces) {
> -            sbrec_port_binding_add_clause_logical_port(&pb, OVSDB_F_EQ,
> name);
> -            sbrec_port_binding_add_clause_parent_port(&pb, OVSDB_F_EQ,
> name);
> -        }
> -    }
> -    if (local_datapaths) {
> -        const struct local_datapath *ld;
> -        HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> -            struct uuid *uuid = CONST_CAST(struct uuid *,
> -                                           &ld->datapath->header_.uuid);
> -            sbrec_port_binding_add_clause_datapath(&pb, OVSDB_F_EQ, uuid);
> -            sbrec_logical_flow_add_clause_logical_datapath(&lf,
> OVSDB_F_EQ,
> -                                                           uuid);
> -            sbrec_mac_binding_add_clause_datapath(&mb, OVSDB_F_EQ, uuid);
> -            sbrec_multicast_group_add_clause_datapath(&mg, OVSDB_F_EQ,
> uuid);
> -            sbrec_dns_add_clause_datapaths(&dns, OVSDB_F_INCLUDES, &uuid,
> 1);
> -            sbrec_ip_multicast_add_clause_datapath(&ip_mcast, OVSDB_F_EQ,
> -                                                   uuid);
> -        }
> -    }
> -    sbrec_port_binding_set_condition(ovnsb_idl, &pb);
> -    sbrec_logical_flow_set_condition(ovnsb_idl, &lf);
> -    sbrec_mac_binding_set_condition(ovnsb_idl, &mb);
> -    sbrec_multicast_group_set_condition(ovnsb_idl, &mg);
> -    sbrec_dns_set_condition(ovnsb_idl, &dns);
> -    sbrec_controller_event_set_condition(ovnsb_idl, &ce);
> -    sbrec_ip_multicast_set_condition(ovnsb_idl, &ip_mcast);
> -    sbrec_igmp_group_set_condition(ovnsb_idl, &igmp);
> -    ovsdb_idl_condition_destroy(&pb);
> -    ovsdb_idl_condition_destroy(&lf);
> -    ovsdb_idl_condition_destroy(&mb);
> -    ovsdb_idl_condition_destroy(&mg);
> -    ovsdb_idl_condition_destroy(&dns);
> -    ovsdb_idl_condition_destroy(&ce);
> -    ovsdb_idl_condition_destroy(&ip_mcast);
> -    ovsdb_idl_condition_destroy(&igmp);
> -}
> -
> -static const char *
> -br_int_name(const struct ovsrec_open_vswitch *cfg)
> -{
> -    return smap_get_def(&cfg->external_ids, "ovn-bridge",
> DEFAULT_BRIDGE_NAME);
> -}
> -
> -static const struct ovsrec_bridge *
> -create_br_int(struct ovsdb_idl_txn *ovs_idl_txn,
> -              const struct ovsrec_open_vswitch_table *ovs_table)
> -{
> -    if (!ovs_idl_txn) {
> -        return NULL;
> -    }
> -
> -    const struct ovsrec_open_vswitch *cfg;
> -    cfg = ovsrec_open_vswitch_table_first(ovs_table);
> -    if (!cfg) {
> -        return NULL;
> -    }
> -    const char *bridge_name = br_int_name(cfg);
> -
> -    ovsdb_idl_txn_add_comment(ovs_idl_txn,
> -            "ovn-controller: creating integration bridge '%s'",
> bridge_name);
> -
> -    struct ovsrec_interface *iface;
> -    iface = ovsrec_interface_insert(ovs_idl_txn);
> -    ovsrec_interface_set_name(iface, bridge_name);
> -    ovsrec_interface_set_type(iface, "internal");
> -
> -    struct ovsrec_port *port;
> -    port = ovsrec_port_insert(ovs_idl_txn);
> -    ovsrec_port_set_name(port, bridge_name);
> -    ovsrec_port_set_interfaces(port, &iface, 1);
> -
> -    struct ovsrec_bridge *bridge;
> -    bridge = ovsrec_bridge_insert(ovs_idl_txn);
> -    ovsrec_bridge_set_name(bridge, bridge_name);
> -    ovsrec_bridge_set_fail_mode(bridge, "secure");
> -    const struct smap oc = SMAP_CONST1(&oc, "disable-in-band", "true");
> -    ovsrec_bridge_set_other_config(bridge, &oc);
> -    ovsrec_bridge_set_ports(bridge, &port, 1);
> -
> -    struct ovsrec_bridge **bridges;
> -    size_t bytes = sizeof *bridges * cfg->n_bridges;
> -    bridges = xmalloc(bytes + sizeof *bridges);
> -    memcpy(bridges, cfg->bridges, bytes);
> -    bridges[cfg->n_bridges] = bridge;
> -    ovsrec_open_vswitch_verify_bridges(cfg);
> -    ovsrec_open_vswitch_set_bridges(cfg, bridges, cfg->n_bridges + 1);
> -    free(bridges);
> -
> -    return bridge;
> -}
> -
> -static const struct ovsrec_bridge *
> -get_br_int(const struct ovsrec_bridge_table *bridge_table,
> -           const struct ovsrec_open_vswitch_table *ovs_table)
> -{
> -    const struct ovsrec_open_vswitch *cfg;
> -    cfg = ovsrec_open_vswitch_table_first(ovs_table);
> -    if (!cfg) {
> -        return NULL;
> -    }
> -
> -    return get_bridge(bridge_table, br_int_name(cfg));
> -}
> -
> -static const struct ovsrec_bridge *
> -process_br_int(struct ovsdb_idl_txn *ovs_idl_txn,
> -               const struct ovsrec_bridge_table *bridge_table,
> -               const struct ovsrec_open_vswitch_table *ovs_table)
> -{
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> -                                                    ovs_table);
> -    if (!br_int) {
> -        br_int = create_br_int(ovs_idl_txn, ovs_table);
> -    }
> -    if (br_int && ovs_idl_txn) {
> -        const struct ovsrec_open_vswitch *cfg;
> -        cfg = ovsrec_open_vswitch_table_first(ovs_table);
> -        ovs_assert(cfg);
> -        const char *datapath_type = smap_get(&cfg->external_ids,
> -                                             "ovn-bridge-datapath-type");
> -        /* Check for the datapath_type and set it only if it is defined in
> -         * cfg. */
> -        if (datapath_type && strcmp(br_int->datapath_type,
> datapath_type)) {
> -            ovsrec_bridge_set_datapath_type(br_int, datapath_type);
> -        }
> -    }
> -    return br_int;
> -}
> -
> -static const char *
> -get_ovs_chassis_id(const struct ovsrec_open_vswitch_table *ovs_table)
> -{
> -    const struct ovsrec_open_vswitch *cfg
> -        = ovsrec_open_vswitch_table_first(ovs_table);
> -    const char *chassis_id = cfg ? smap_get(&cfg->external_ids,
> "system-id")
> -                                 : NULL;
> -
> -    if (!chassis_id) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "'system-id' in Open_vSwitch database is
> missing.");
> -    }
> -
> -    return chassis_id;
> -}
> -
> -/* Iterate address sets in the southbound database.  Create and update the
> - * corresponding symtab entries as necessary. */
> -static void
> -addr_sets_init(const struct sbrec_address_set_table *address_set_table,
> -               struct shash *addr_sets)
> -{
> -    const struct sbrec_address_set *as;
> -    SBREC_ADDRESS_SET_TABLE_FOR_EACH (as, address_set_table) {
> -        expr_const_sets_add(addr_sets, as->name,
> -                            (const char *const *) as->addresses,
> -                            as->n_addresses, true);
> -    }
> -}
> -
> -static void
> -addr_sets_update(const struct sbrec_address_set_table *address_set_table,
> -                 struct shash *addr_sets, struct sset *new,
> -                 struct sset *deleted, struct sset *updated)
> -{
> -    const struct sbrec_address_set *as;
> -    SBREC_ADDRESS_SET_TABLE_FOR_EACH_TRACKED (as, address_set_table) {
> -        if (sbrec_address_set_is_deleted(as)) {
> -            expr_const_sets_remove(addr_sets, as->name);
> -            sset_add(deleted, as->name);
> -        } else {
> -            expr_const_sets_add(addr_sets, as->name,
> -                                (const char *const *) as->addresses,
> -                                as->n_addresses, true);
> -            if (sbrec_address_set_is_new(as)) {
> -                sset_add(new, as->name);
> -            } else {
> -                sset_add(updated, as->name);
> -            }
> -        }
> -    }
> -}
> -
> -/* Iterate port groups in the southbound database.  Create and update the
> - * corresponding symtab entries as necessary. */
> - static void
> -port_groups_init(const struct sbrec_port_group_table *port_group_table,
> -                 struct shash *port_groups)
> -{
> -    const struct sbrec_port_group *pg;
> -    SBREC_PORT_GROUP_TABLE_FOR_EACH (pg, port_group_table) {
> -        expr_const_sets_add(port_groups, pg->name,
> -                            (const char *const *) pg->ports,
> -                            pg->n_ports, false);
> -    }
> -}
> -
> -static void
> -port_groups_update(const struct sbrec_port_group_table *port_group_table,
> -                   struct shash *port_groups, struct sset *new,
> -                   struct sset *deleted, struct sset *updated)
> -{
> -    const struct sbrec_port_group *pg;
> -    SBREC_PORT_GROUP_TABLE_FOR_EACH_TRACKED (pg, port_group_table) {
> -        if (sbrec_port_group_is_deleted(pg)) {
> -            expr_const_sets_remove(port_groups, pg->name);
> -            sset_add(deleted, pg->name);
> -        } else {
> -            expr_const_sets_add(port_groups, pg->name,
> -                                (const char *const *) pg->ports,
> -                                pg->n_ports, false);
> -            if (sbrec_port_group_is_new(pg)) {
> -                sset_add(new, pg->name);
> -            } else {
> -                sset_add(updated, pg->name);
> -            }
> -        }
> -    }
> -}
> -
> -static void
> -update_ssl_config(const struct ovsrec_ssl_table *ssl_table)
> -{
> -    const struct ovsrec_ssl *ssl = ovsrec_ssl_table_first(ssl_table);
> -
> -    if (ssl) {
> -        stream_ssl_set_key_and_cert(ssl->private_key, ssl->certificate);
> -        stream_ssl_set_ca_cert_file(ssl->ca_cert, ssl->bootstrap_ca_cert);
> -    }
> -}
> -
> -static int
> -get_ofctrl_probe_interval(struct ovsdb_idl *ovs_idl)
> -{
> -    const struct ovsrec_open_vswitch *cfg =
> ovsrec_open_vswitch_first(ovs_idl);
> -    return smap_get_int(&cfg->external_ids,
> -                        "ovn-openflow-probe-interval",
> -                        OFCTRL_DEFAULT_PROBE_INTERVAL_SEC);
> -}
> -
> -/* Retrieves the pointer to the OVN Southbound database from 'ovs_idl' and
> - * updates 'sbdb_idl' with that pointer. */
> -static void
> -update_sb_db(struct ovsdb_idl *ovs_idl, struct ovsdb_idl *ovnsb_idl)
> -{
> -    const struct ovsrec_open_vswitch *cfg =
> ovsrec_open_vswitch_first(ovs_idl);
> -
> -    /* Set remote based on user configuration. */
> -    const char *remote = NULL;
> -    if (cfg) {
> -        remote = smap_get(&cfg->external_ids, "ovn-remote");
> -    }
> -    ovsdb_idl_set_remote(ovnsb_idl, remote, true);
> -
> -    /* Set probe interval, based on user configuration and the remote. */
> -    int default_interval = (remote &&
> !stream_or_pstream_needs_probes(remote)
> -                            ? 0 : DEFAULT_PROBE_INTERVAL_MSEC);
> -    int interval = smap_get_int(&cfg->external_ids,
> -                                "ovn-remote-probe-interval",
> default_interval);
> -    ovsdb_idl_set_probe_interval(ovnsb_idl, interval);
> -}
> -
> -static void
> -update_ct_zones(const struct sset *lports, const struct hmap
> *local_datapaths,
> -                struct simap *ct_zones, unsigned long *ct_zone_bitmap,
> -                struct shash *pending_ct_zones)
> -{
> -    struct simap_node *ct_zone, *ct_zone_next;
> -    int scan_start = 1;
> -    const char *user;
> -    struct sset all_users = SSET_INITIALIZER(&all_users);
> -
> -    SSET_FOR_EACH(user, lports) {
> -        sset_add(&all_users, user);
> -    }
> -
> -    /* Local patched datapath (gateway routers) need zones assigned. */
> -    const struct local_datapath *ld;
> -    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> -        /* XXX Add method to limit zone assignment to logical router
> -         * datapaths with NAT */
> -        char *dnat = alloc_nat_zone_key(&ld->datapath->header_.uuid,
> "dnat");
> -        char *snat = alloc_nat_zone_key(&ld->datapath->header_.uuid,
> "snat");
> -        sset_add(&all_users, dnat);
> -        sset_add(&all_users, snat);
> -        free(dnat);
> -        free(snat);
> -    }
> -
> -    /* Delete zones that do not exist in above sset. */
> -    SIMAP_FOR_EACH_SAFE(ct_zone, ct_zone_next, ct_zones) {
> -        if (!sset_contains(&all_users, ct_zone->name)) {
> -            VLOG_DBG("removing ct zone %"PRId32" for '%s'",
> -                     ct_zone->data, ct_zone->name);
> -
> -            struct ct_zone_pending_entry *pending = xmalloc(sizeof
> *pending);
> -            pending->state = CT_ZONE_DB_QUEUED; /* Skip flushing zone. */
> -            pending->zone = ct_zone->data;
> -            pending->add = false;
> -            shash_add(pending_ct_zones, ct_zone->name, pending);
> -
> -            bitmap_set0(ct_zone_bitmap, ct_zone->data);
> -            simap_delete(ct_zones, ct_zone);
> -        }
> -    }
> -
> -    /* xxx This is wasteful to assign a zone to each port--even if no
> -     * xxx security policy is applied. */
> -
> -    /* Assign a unique zone id for each logical port and two zones
> -     * to a gateway router. */
> -    SSET_FOR_EACH(user, &all_users) {
> -        int zone;
> -
> -        if (simap_contains(ct_zones, user)) {
> -            continue;
> -        }
> -
> -        /* We assume that there are 64K zones and that we own them all. */
> -        zone = bitmap_scan(ct_zone_bitmap, 0, scan_start, MAX_CT_ZONES +
> 1);
> -        if (zone == MAX_CT_ZONES + 1) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_WARN_RL(&rl, "exhausted all ct zones");
> -            return;
> -        }
> -        scan_start = zone + 1;
> -
> -        VLOG_DBG("assigning ct zone %"PRId32" to '%s'", zone, user);
> -
> -        struct ct_zone_pending_entry *pending = xmalloc(sizeof *pending);
> -        pending->state = CT_ZONE_OF_QUEUED;
> -        pending->zone = zone;
> -        pending->add = true;
> -        shash_add(pending_ct_zones, user, pending);
> -
> -        bitmap_set1(ct_zone_bitmap, zone);
> -        simap_put(ct_zones, user, zone);
> -    }
> -
> -    sset_destroy(&all_users);
> -}
> -
> -static void
> -commit_ct_zones(const struct ovsrec_bridge *br_int,
> -                struct shash *pending_ct_zones)
> -{
> -    struct smap new_ids;
> -    smap_clone(&new_ids, &br_int->external_ids);
> -
> -    bool updated = false;
> -    struct shash_node *iter;
> -    SHASH_FOR_EACH(iter, pending_ct_zones) {
> -        struct ct_zone_pending_entry *ctzpe = iter->data;
> -
> -        /* The transaction is open, so any pending entries in the
> -         * CT_ZONE_DB_QUEUED must be sent and any in CT_ZONE_DB_QUEUED
> -         * need to be retried. */
> -        if (ctzpe->state != CT_ZONE_DB_QUEUED
> -            && ctzpe->state != CT_ZONE_DB_SENT) {
> -            continue;
> -        }
> -
> -        char *user_str = xasprintf("ct-zone-%s", iter->name);
> -        if (ctzpe->add) {
> -            char *zone_str = xasprintf("%"PRId32, ctzpe->zone);
> -            smap_replace(&new_ids, user_str, zone_str);
> -            free(zone_str);
> -        } else {
> -            smap_remove(&new_ids, user_str);
> -        }
> -        free(user_str);
> -
> -        ctzpe->state = CT_ZONE_DB_SENT;
> -        updated = true;
> -    }
> -
> -    if (updated) {
> -        ovsrec_bridge_verify_external_ids(br_int);
> -        ovsrec_bridge_set_external_ids(br_int, &new_ids);
> -    }
> -    smap_destroy(&new_ids);
> -}
> -
> -static void
> -restore_ct_zones(const struct ovsrec_bridge_table *bridge_table,
> -                 const struct ovsrec_open_vswitch_table *ovs_table,
> -                 struct simap *ct_zones, unsigned long *ct_zone_bitmap)
> -{
> -    const struct ovsrec_open_vswitch *cfg;
> -    cfg = ovsrec_open_vswitch_table_first(ovs_table);
> -    if (!cfg) {
> -        return;
> -    }
> -
> -    const struct ovsrec_bridge *br_int;
> -    br_int = get_bridge(bridge_table, br_int_name(cfg));
> -    if (!br_int) {
> -        /* If the integration bridge hasn't been defined, assume that
> -         * any existing ct-zone definitions aren't valid. */
> -        return;
> -    }
> -
> -    struct smap_node *node;
> -    SMAP_FOR_EACH(node, &br_int->external_ids) {
> -        if (strncmp(node->key, "ct-zone-", 8)) {
> -            continue;
> -        }
> -
> -        const char *user = node->key + 8;
> -        int zone = atoi(node->value);
> -
> -        if (user[0] && zone) {
> -            VLOG_DBG("restoring ct zone %"PRId32" for '%s'", zone, user);
> -            bitmap_set1(ct_zone_bitmap, zone);
> -            simap_put(ct_zones, user, zone);
> -        }
> -    }
> -}
> -
> -static int64_t
> -get_nb_cfg(const struct sbrec_sb_global_table *sb_global_table)
> -{
> -    const struct sbrec_sb_global *sb
> -        = sbrec_sb_global_table_first(sb_global_table);
> -    return sb ? sb->nb_cfg : 0;
> -}
> -
> -static const char *
> -get_transport_zones(const struct ovsrec_open_vswitch_table *ovs_table)
> -{
> -    const struct ovsrec_open_vswitch *cfg
> -        = ovsrec_open_vswitch_table_first(ovs_table);
> -    return smap_get_def(&cfg->external_ids, "ovn-transport-zones", "");
> -}
> -
> -static void
> -ctrl_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    /* We do not monitor all tables by default, so modules must register
> -     * their interest explicitly. */
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_open_vswitch);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_open_vswitch_col_external_ids);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_open_vswitch_col_bridges);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_bfd_status);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_type);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_options);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_ofport);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_port);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_interfaces);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_external_ids);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_bridge);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_ports);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_name);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_fail_mode);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_other_config);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_external_ids);
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_ssl);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_ssl_col_bootstrap_ca_cert);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_ssl_col_ca_cert);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_ssl_col_certificate);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_ssl_col_private_key);
> -    chassis_register_ovs_idl(ovs_idl);
> -    encaps_register_ovs_idl(ovs_idl);
> -    binding_register_ovs_idl(ovs_idl);
> -    bfd_register_ovs_idl(ovs_idl);
> -    physical_register_ovs_idl(ovs_idl);
> -}
> -
> -#define SB_NODES \
> -    SB_NODE(chassis, "chassis") \
> -    SB_NODE(encap, "encap") \
> -    SB_NODE(address_set, "address_set") \
> -    SB_NODE(port_group, "port_group") \
> -    SB_NODE(multicast_group, "multicast_group") \
> -    SB_NODE(datapath_binding, "datapath_binding") \
> -    SB_NODE(port_binding, "port_binding") \
> -    SB_NODE(mac_binding, "mac_binding") \
> -    SB_NODE(logical_flow, "logical_flow") \
> -    SB_NODE(dhcp_options, "dhcp_options") \
> -    SB_NODE(dhcpv6_options, "dhcpv6_options") \
> -    SB_NODE(dns, "dns")
> -
> -enum sb_engine_node {
> -#define SB_NODE(NAME, NAME_STR) SB_##NAME,
> -    SB_NODES
> -#undef SB_NODE
> -};
> -
> -#define SB_NODE(NAME, NAME_STR) ENGINE_FUNC_SB(NAME);
> -    SB_NODES
> -#undef SB_NODE
> -
> -#define OVS_NODES \
> -    OVS_NODE(open_vswitch, "open_vswitch") \
> -    OVS_NODE(bridge, "bridge") \
> -    OVS_NODE(port, "port") \
> -    OVS_NODE(qos, "qos")
> -
> -enum ovs_engine_node {
> -#define OVS_NODE(NAME, NAME_STR) OVS_##NAME,
> -    OVS_NODES
> -#undef OVS_NODE
> -};
> -
> -#define OVS_NODE(NAME, NAME_STR) ENGINE_FUNC_OVS(NAME);
> -    OVS_NODES
> -#undef OVS_NODE
> -
> -struct ed_type_ofctrl_is_connected {
> -    bool connected;
> -};
> -
> -static void
> -en_ofctrl_is_connected_init(struct engine_node *node)
> -{
> -    struct ed_type_ofctrl_is_connected *data =
> -        (struct ed_type_ofctrl_is_connected *)node->data;
> -    data->connected = false;
> -}
> -
> -static void
> -en_ofctrl_is_connected_cleanup(struct engine_node *node OVS_UNUSED)
> -{
> -}
> -
> -static void
> -en_ofctrl_is_connected_run(struct engine_node *node)
> -{
> -    struct ed_type_ofctrl_is_connected *data =
> -        (struct ed_type_ofctrl_is_connected *)node->data;
> -    if (data->connected != ofctrl_is_connected()) {
> -        data->connected = !data->connected;
> -        node->changed = true;
> -        return;
> -    }
> -    node->changed = false;
> -}
> -
> -struct ed_type_addr_sets {
> -    struct shash addr_sets;
> -    bool change_tracked;
> -    struct sset new;
> -    struct sset deleted;
> -    struct sset updated;
> -};
> -
> -static void
> -en_addr_sets_init(struct engine_node *node)
> -{
> -    struct ed_type_addr_sets *as = (struct ed_type_addr_sets *)node->data;
> -    shash_init(&as->addr_sets);
> -    as->change_tracked = false;
> -    sset_init(&as->new);
> -    sset_init(&as->deleted);
> -    sset_init(&as->updated);
> -}
> -
> -static void
> -en_addr_sets_cleanup(struct engine_node *node)
> -{
> -    struct ed_type_addr_sets *as = (struct ed_type_addr_sets *)node->data;
> -    expr_const_sets_destroy(&as->addr_sets);
> -    shash_destroy(&as->addr_sets);
> -    sset_destroy(&as->new);
> -    sset_destroy(&as->deleted);
> -    sset_destroy(&as->updated);
> -}
> -
> -static void
> -en_addr_sets_run(struct engine_node *node)
> -{
> -    struct ed_type_addr_sets *as = (struct ed_type_addr_sets *)node->data;
> -
> -    sset_clear(&as->new);
> -    sset_clear(&as->deleted);
> -    sset_clear(&as->updated);
> -    expr_const_sets_destroy(&as->addr_sets);
> -
> -    struct sbrec_address_set_table *as_table =
> -        (struct sbrec_address_set_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_address_set", node));
> -
> -    addr_sets_init(as_table, &as->addr_sets);
> -
> -    as->change_tracked = false;
> -    node->changed = true;
> -}
> -
> -static bool
> -addr_sets_sb_address_set_handler(struct engine_node *node)
> -{
> -    struct ed_type_addr_sets *as = (struct ed_type_addr_sets *)node->data;
> -
> -    sset_clear(&as->new);
> -    sset_clear(&as->deleted);
> -    sset_clear(&as->updated);
> -
> -    struct sbrec_address_set_table *as_table =
> -        (struct sbrec_address_set_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_address_set", node));
> -
> -    addr_sets_update(as_table, &as->addr_sets, &as->new,
> -                     &as->deleted, &as->updated);
> -
> -    node->changed = !sset_is_empty(&as->new) ||
> !sset_is_empty(&as->deleted)
> -                    || !sset_is_empty(&as->updated);
> -
> -    as->change_tracked = true;
> -    node->changed = true;
> -    return true;
> -}
> -
> -struct ed_type_port_groups{
> -    struct shash port_groups;
> -    bool change_tracked;
> -    struct sset new;
> -    struct sset deleted;
> -    struct sset updated;
> -};
> -
> -static void
> -en_port_groups_init(struct engine_node *node)
> -{
> -    struct ed_type_port_groups *pg = (struct ed_type_port_groups
> *)node->data;
> -    shash_init(&pg->port_groups);
> -    pg->change_tracked = false;
> -    sset_init(&pg->new);
> -    sset_init(&pg->deleted);
> -    sset_init(&pg->updated);
> -}
> -
> -static void
> -en_port_groups_cleanup(struct engine_node *node)
> -{
> -    struct ed_type_port_groups *pg = (struct ed_type_port_groups
> *)node->data;
> -    expr_const_sets_destroy(&pg->port_groups);
> -    shash_destroy(&pg->port_groups);
> -    sset_destroy(&pg->new);
> -    sset_destroy(&pg->deleted);
> -    sset_destroy(&pg->updated);
> -}
> -
> -static void
> -en_port_groups_run(struct engine_node *node)
> -{
> -    struct ed_type_port_groups *pg = (struct ed_type_port_groups
> *)node->data;
> -
> -    sset_clear(&pg->new);
> -    sset_clear(&pg->deleted);
> -    sset_clear(&pg->updated);
> -    expr_const_sets_destroy(&pg->port_groups);
> -
> -    struct sbrec_port_group_table *pg_table =
> -        (struct sbrec_port_group_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_group", node));
> -
> -    port_groups_init(pg_table, &pg->port_groups);
> -
> -    pg->change_tracked = false;
> -    node->changed = true;
> -}
> -
> -static bool
> -port_groups_sb_port_group_handler(struct engine_node *node)
> -{
> -    struct ed_type_port_groups *pg = (struct ed_type_port_groups
> *)node->data;
> -
> -    sset_clear(&pg->new);
> -    sset_clear(&pg->deleted);
> -    sset_clear(&pg->updated);
> -
> -    struct sbrec_port_group_table *pg_table =
> -        (struct sbrec_port_group_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_group", node));
> -
> -    port_groups_update(pg_table, &pg->port_groups, &pg->new,
> -                     &pg->deleted, &pg->updated);
> -
> -    node->changed = !sset_is_empty(&pg->new) ||
> !sset_is_empty(&pg->deleted)
> -                    || !sset_is_empty(&pg->updated);
> -
> -    pg->change_tracked = true;
> -    node->changed = true;
> -    return true;
> -}
> -
> -struct ed_type_runtime_data {
> -    /* Contains "struct local_datapath" nodes. */
> -    struct hmap local_datapaths;
> -
> -    /* Contains the name of each logical port resident on the local
> -     * hypervisor.  These logical ports include the VIFs (and their child
> -     * logical ports, if any) that belong to VMs running on the
> hypervisor,
> -     * l2gateway ports for which options:l2gateway-chassis designates the
> -     * local hypervisor, and localnet ports. */
> -    struct sset local_lports;
> -
> -    /* Contains the same ports as local_lports, but in the format:
> -     * <datapath-tunnel-key>_<port-tunnel-key> */
> -    struct sset local_lport_ids;
> -    struct sset active_tunnels;
> -
> -    /* connection tracking zones. */
> -    unsigned long ct_zone_bitmap[BITMAP_N_LONGS(MAX_CT_ZONES)];
> -    struct shash pending_ct_zones;
> -    struct simap ct_zones;
> -};
> -
> -static void
> -en_runtime_data_init(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)node->data;
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    hmap_init(&data->local_datapaths);
> -    sset_init(&data->local_lports);
> -    sset_init(&data->local_lport_ids);
> -    sset_init(&data->active_tunnels);
> -    shash_init(&data->pending_ct_zones);
> -    simap_init(&data->ct_zones);
> -
> -    /* Initialize connection tracking zones. */
> -    memset(data->ct_zone_bitmap, 0, sizeof data->ct_zone_bitmap);
> -    bitmap_set1(data->ct_zone_bitmap, 0); /* Zone 0 is reserved. */
> -    restore_ct_zones(bridge_table, ovs_table,
> -                     &data->ct_zones, data->ct_zone_bitmap);
> -}
> -
> -static void
> -en_runtime_data_cleanup(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)node->data;
> -
> -    sset_destroy(&data->local_lports);
> -    sset_destroy(&data->local_lport_ids);
> -    sset_destroy(&data->active_tunnels);
> -    struct local_datapath *cur_node, *next_node;
> -    HMAP_FOR_EACH_SAFE (cur_node, next_node, hmap_node,
> -                        &data->local_datapaths) {
> -        free(cur_node->peer_ports);
> -        hmap_remove(&data->local_datapaths, &cur_node->hmap_node);
> -        free(cur_node);
> -    }
> -    hmap_destroy(&data->local_datapaths);
> -
> -    simap_destroy(&data->ct_zones);
> -    shash_destroy(&data->pending_ct_zones);
> -}
> -
> -static void
> -en_runtime_data_run(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)node->data;
> -    struct hmap *local_datapaths = &data->local_datapaths;
> -    struct sset *local_lports = &data->local_lports;
> -    struct sset *local_lport_ids = &data->local_lport_ids;
> -    struct sset *active_tunnels = &data->active_tunnels;
> -    unsigned long *ct_zone_bitmap = data->ct_zone_bitmap;
> -    struct shash *pending_ct_zones = &data->pending_ct_zones;
> -    struct simap *ct_zones = &data->ct_zones;
> -
> -    static bool first_run = true;
> -    if (first_run) {
> -        /* don't cleanup since there is no data yet */
> -        first_run = false;
> -    } else {
> -        struct local_datapath *cur_node, *next_node;
> -        HMAP_FOR_EACH_SAFE (cur_node, next_node, hmap_node,
> local_datapaths) {
> -            free(cur_node->peer_ports);
> -            hmap_remove(local_datapaths, &cur_node->hmap_node);
> -            free(cur_node);
> -        }
> -        hmap_clear(local_datapaths);
> -        sset_destroy(local_lports);
> -        sset_destroy(local_lport_ids);
> -        sset_destroy(active_tunnels);
> -        sset_init(local_lports);
> -        sset_init(local_lport_ids);
> -        sset_init(active_tunnels);
> -    }
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const char *chassis_id = get_ovs_chassis_id(ovs_table);
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -
> -    ovs_assert(br_int && chassis_id);
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -
> -    const struct sbrec_chassis *chassis
> -        = chassis_lookup_by_name(sbrec_chassis_by_name, chassis_id);
> -    ovs_assert(chassis);
> -
> -    struct ed_type_ofctrl_is_connected *ed_ofctrl_is_connected =
> -        (struct ed_type_ofctrl_is_connected *)engine_get_input(
> -            "ofctrl_is_connected", node)->data;
> -    if (ed_ofctrl_is_connected->connected) {
> -        /* Calculate the active tunnels only if have an an active
> -         * OpenFlow connection to br-int.
> -         * If we don't have a connection to br-int, it could mean
> -         * ovs-vswitchd is down for some reason and the BFD status
> -         * in the Interface rows could be stale. So its better to
> -         * consider 'active_tunnels' set to be empty if it's not
> -         * connected. */
> -        bfd_calculate_active_tunnels(br_int, active_tunnels);
> -    }
> -
> -    struct ovsrec_port_table *port_table =
> -        (struct ovsrec_port_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_port", node));
> -
> -    struct ovsrec_qos_table *qos_table =
> -        (struct ovsrec_qos_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_qos", node));
> -
> -    struct sbrec_port_binding_table *pb_table =
> -        (struct sbrec_port_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_binding", node));
> -
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_datapath_binding", node),
> -                "key");
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_datapath =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "datapath");
> -
> -    binding_run(engine_get_context()->ovnsb_idl_txn,
> -                engine_get_context()->ovs_idl_txn,
> -                sbrec_datapath_binding_by_key,
> -                sbrec_port_binding_by_datapath,
> -                sbrec_port_binding_by_name,
> -                port_table, qos_table, pb_table,
> -                br_int, chassis,
> -                active_tunnels, local_datapaths,
> -                local_lports, local_lport_ids);
> -
> -    update_ct_zones(local_lports, local_datapaths, ct_zones,
> -                    ct_zone_bitmap, pending_ct_zones);
> -
> -    node->changed = true;
> -}
> -
> -static bool
> -runtime_data_sb_port_binding_handler(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)node->data;
> -    struct sset *local_lports = &data->local_lports;
> -    struct sset *active_tunnels = &data->active_tunnels;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const char *chassis_id = chassis_get_id();
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -
> -    ovs_assert(br_int && chassis_id);
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -
> -    const struct sbrec_chassis *chassis
> -        = chassis_lookup_by_name(sbrec_chassis_by_name, chassis_id);
> -    ovs_assert(chassis);
> -
> -    struct sbrec_port_binding_table *pb_table =
> -        (struct sbrec_port_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_binding", node));
> -
> -    bool changed = binding_evaluate_port_binding_changes(
> -        pb_table, br_int, chassis, active_tunnels, local_lports);
> -
> -    return !changed;
> -}
> -
> -struct ed_type_mff_ovn_geneve {
> -    enum mf_field_id mff_ovn_geneve;
> -};
> -
> -static void
> -en_mff_ovn_geneve_init(struct engine_node *node)
> -{
> -    struct ed_type_mff_ovn_geneve *data =
> -        (struct ed_type_mff_ovn_geneve *)node->data;
> -    data->mff_ovn_geneve = 0;
> -}
> -
> -static void
> -en_mff_ovn_geneve_cleanup(struct engine_node *node OVS_UNUSED)
> -{
> -}
> -
> -static void
> -en_mff_ovn_geneve_run(struct engine_node *node)
> -{
> -    struct ed_type_mff_ovn_geneve *data =
> -        (struct ed_type_mff_ovn_geneve *)node->data;
> -    enum mf_field_id mff_ovn_geneve = ofctrl_get_mf_field_id();
> -    if (data->mff_ovn_geneve != mff_ovn_geneve) {
> -        data->mff_ovn_geneve = mff_ovn_geneve;
> -        node->changed = true;
> -        return;
> -    }
> -    node->changed = false;
> -}
> -
> -struct ed_type_flow_output {
> -    /* desired flows */
> -    struct ovn_desired_flow_table flow_table;
> -    /* group ids for load balancing */
> -    struct ovn_extend_table group_table;
> -    /* meter ids for QoS */
> -    struct ovn_extend_table meter_table;
> -    /* conjunction id offset */
> -    uint32_t conj_id_ofs;
> -    /* lflow resource cross reference */
> -    struct lflow_resource_ref lflow_resource_ref;
> -};
> -
> -static void
> -en_flow_output_init(struct engine_node *node)
> -{
> -    struct ed_type_flow_output *data =
> -        (struct ed_type_flow_output *)node->data;
> -    ovn_desired_flow_table_init(&data->flow_table);
> -    ovn_extend_table_init(&data->group_table);
> -    ovn_extend_table_init(&data->meter_table);
> -    data->conj_id_ofs = 1;
> -    lflow_resource_init(&data->lflow_resource_ref);
> -}
> -
> -static void
> -en_flow_output_cleanup(struct engine_node *node)
> -{
> -    struct ed_type_flow_output *data =
> -        (struct ed_type_flow_output *)node->data;
> -    ovn_desired_flow_table_destroy(&data->flow_table);
> -    ovn_extend_table_destroy(&data->group_table);
> -    ovn_extend_table_destroy(&data->meter_table);
> -    lflow_resource_destroy(&data->lflow_resource_ref);
> -}
> -
> -static void
> -en_flow_output_run(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *rt_data =
> -        (struct ed_type_runtime_data *)engine_get_input(
> -            "runtime_data", node)->data;
> -    struct hmap *local_datapaths = &rt_data->local_datapaths;
> -    struct sset *local_lports = &rt_data->local_lports;
> -    struct sset *local_lport_ids = &rt_data->local_lport_ids;
> -    struct sset *active_tunnels = &rt_data->active_tunnels;
> -    struct simap *ct_zones = &rt_data->ct_zones;
> -
> -    struct ed_type_mff_ovn_geneve *ed_mff_ovn_geneve =
> -        (struct ed_type_mff_ovn_geneve *)engine_get_input(
> -            "mff_ovn_geneve", node)->data;
> -    enum mf_field_id mff_ovn_geneve = ed_mff_ovn_geneve->mff_ovn_geneve;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -    const char *chassis_id = chassis_get_id();
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -    struct ed_type_addr_sets *as_data =
> -        (struct ed_type_addr_sets *)engine_get_input("addr_sets",
> node)->data;
> -    struct shash *addr_sets = &as_data->addr_sets;
> -
> -    struct ed_type_port_groups *pg_data =
> -        (struct ed_type_port_groups *)engine_get_input(
> -            "port_groups", node)->data;
> -    struct shash *port_groups = &pg_data->port_groups;
> -
> -    const struct sbrec_chassis *chassis = NULL;
> -    if (chassis_id) {
> -        chassis = chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id);
> -    }
> -
> -    ovs_assert(br_int && chassis);
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -    struct ovn_extend_table *group_table = &fo->group_table;
> -    struct ovn_extend_table *meter_table = &fo->meter_table;
> -    uint32_t *conj_id_ofs = &fo->conj_id_ofs;
> -    struct lflow_resource_ref *lfrr = &fo->lflow_resource_ref;
> -
> -    static bool first_run = true;
> -    if (first_run) {
> -        first_run = false;
> -    } else {
> -        ovn_desired_flow_table_clear(flow_table);
> -        ovn_extend_table_clear(group_table, false /* desired */);
> -        ovn_extend_table_clear(meter_table, false /* desired */);
> -        lflow_resource_clear(lfrr);
> -    }
> -
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_multicast_group", node),
> -                "name_datapath");
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct sbrec_dhcp_options_table *dhcp_table =
> -        (struct sbrec_dhcp_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcp_options", node));
> -
> -    struct sbrec_dhcpv6_options_table *dhcpv6_table =
> -        (struct sbrec_dhcpv6_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcpv6_options", node));
> -
> -    struct sbrec_logical_flow_table *logical_flow_table =
> -        (struct sbrec_logical_flow_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_logical_flow", node));
> -
> -    struct sbrec_mac_binding_table *mac_binding_table =
> -        (struct sbrec_mac_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_mac_binding", node));
> -
> -    *conj_id_ofs = 1;
> -    lflow_run(sbrec_multicast_group_by_name_datapath,
> -              sbrec_port_binding_by_name,
> -              dhcp_table, dhcpv6_table,
> -              logical_flow_table,
> -              mac_binding_table,
> -              chassis, local_datapaths, addr_sets,
> -              port_groups, active_tunnels, local_lport_ids,
> -              flow_table, group_table, meter_table, lfrr,
> -              conj_id_ofs);
> -
> -    struct sbrec_multicast_group_table *multicast_group_table =
> -        (struct sbrec_multicast_group_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_multicast_group", node));
> -
> -    struct sbrec_port_binding_table *port_binding_table =
> -        (struct sbrec_port_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_binding", node));
> -
> -    physical_run(sbrec_port_binding_by_name,
> -                 multicast_group_table,
> -                 port_binding_table,
> -                 mff_ovn_geneve,
> -                 br_int, chassis, ct_zones,
> -                 local_datapaths, local_lports,
> -                 active_tunnels,
> -                 flow_table);
> -
> -    node->changed = true;
> -}
> -
> -static bool
> -flow_output_sb_logical_flow_handler(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)engine_get_input(
> -                "runtime_data", node)->data;
> -    struct hmap *local_datapaths = &data->local_datapaths;
> -    struct sset *local_lport_ids = &data->local_lport_ids;
> -    struct sset *active_tunnels = &data->active_tunnels;
> -    struct ed_type_addr_sets *as_data =
> -        (struct ed_type_addr_sets *)engine_get_input("addr_sets",
> node)->data;
> -    struct shash *addr_sets = &as_data->addr_sets;
> -
> -    struct ed_type_port_groups *pg_data =
> -        (struct ed_type_port_groups *)engine_get_input(
> -            "port_groups", node)->data;
> -    struct shash *port_groups = &pg_data->port_groups;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -    const char *chassis_id = chassis_get_id();
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -
> -    const struct sbrec_chassis *chassis = NULL;
> -    if (chassis_id) {
> -        chassis = chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id);
> -    }
> -
> -    ovs_assert(br_int && chassis);
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -    struct ovn_extend_table *group_table = &fo->group_table;
> -    struct ovn_extend_table *meter_table = &fo->meter_table;
> -    uint32_t *conj_id_ofs = &fo->conj_id_ofs;
> -    struct lflow_resource_ref *lfrr = &fo->lflow_resource_ref;
> -
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_multicast_group", node),
> -                "name_datapath");
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct sbrec_dhcp_options_table *dhcp_table =
> -        (struct sbrec_dhcp_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcp_options", node));
> -
> -    struct sbrec_dhcpv6_options_table *dhcpv6_table =
> -        (struct sbrec_dhcpv6_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcpv6_options", node));
> -
> -    struct sbrec_logical_flow_table *logical_flow_table =
> -        (struct sbrec_logical_flow_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_logical_flow", node));
> -
> -    bool handled = lflow_handle_changed_flows(
> -              sbrec_multicast_group_by_name_datapath,
> -              sbrec_port_binding_by_name,
> -              dhcp_table, dhcpv6_table,
> -              logical_flow_table,
> -              local_datapaths, chassis, addr_sets,
> -              port_groups, active_tunnels, local_lport_ids,
> -              flow_table, group_table, meter_table, lfrr,
> -              conj_id_ofs);
> -
> -    node->changed = true;
> -    return handled;
> -}
> -
> -static bool
> -flow_output_sb_mac_binding_handler(struct engine_node *node)
> -{
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct sbrec_mac_binding_table *mac_binding_table =
> -        (struct sbrec_mac_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_mac_binding", node));
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -
> -    lflow_handle_changed_neighbors(sbrec_port_binding_by_name,
> -            mac_binding_table, flow_table);
> -
> -    node->changed = true;
> -    return true;
> -}
> -
> -static bool
> -flow_output_sb_port_binding_handler(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)engine_get_input(
> -                "runtime_data", node)->data;
> -    struct hmap *local_datapaths = &data->local_datapaths;
> -    struct sset *active_tunnels = &data->active_tunnels;
> -    struct simap *ct_zones = &data->ct_zones;
> -
> -    struct ed_type_mff_ovn_geneve *ed_mff_ovn_geneve =
> -        (struct ed_type_mff_ovn_geneve *)engine_get_input(
> -            "mff_ovn_geneve", node)->data;
> -    enum mf_field_id mff_ovn_geneve = ed_mff_ovn_geneve->mff_ovn_geneve;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -    const char *chassis_id = chassis_get_id();
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -    const struct sbrec_chassis *chassis = NULL;
> -    if (chassis_id) {
> -        chassis = chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id);
> -    }
> -    ovs_assert(br_int && chassis);
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct sbrec_port_binding_table *port_binding_table =
> -        (struct sbrec_port_binding_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_port_binding", node));
> -
> -    /* XXX: now we handle port-binding changes for physical flow
> processing
> -     * only, but port-binding change can have impact to logical flow
> -     * processing, too, in below circumstances:
> -     *
> -     *  - When a port-binding for a lport is inserted/deleted but the
> lflow
> -     *    using that lport doesn't change.
> -     *
> -     *    This can happen only when the lport name is used by ACL match
> -     *    condition, which is specified by user. Even in that case, if
> the port
> -     *    is actually bound on the current chassis it will trigger
> recompute on
> -     *    that chassis since ovs interface would be updated. So the only
> -     *    situation this would have real impact is when user defines an
> ACL
> -     *    that includes lport that is not on current chassis, and there
> is a
> -     *    port-binding creation/deletion related to that lport.e.g.: an
> ACL is
> -     *    defined:
> -     *
> -     *    to-lport 1000 'outport=="A" && inport=="B"' allow-related
> -     *
> -     *    If "A" is on current chassis, but "B" is lport that hasn't been
> -     *    created yet. When a lport "B" is created and bound on another
> -     *    chassis, the ACL will not take effect on the current chassis
> until a
> -     *    recompute is triggered later. This case doesn't seem to be a
> problem
> -     *    for real world use cases because usually lport is created before
> -     *    being referenced by name in ACLs.
> -     *
> -     *  - When is_chassis_resident(<lport>) is used in lflow. In this
> case the
> -     *    port binding is not a regular VIF. It can be either "patch" or
> -     *    "external", with ha-chassis-group assigned.  In current
> -     *    "runtime_data" handling, port-binding changes for these types
> always
> -     *    trigger recomputing. So it is fine even if we do not handle it
> here.
> -     *    (due to the ovsdb tracking support for referenced table changes,
> -     *    ha-chassis-group changes will appear as port-binding change).
> -     *
> -     *  - When a mac-binding doesn't change but the port-binding related
> to
> -     *    that mac-binding is deleted. In this case the neighbor flow
> generated
> -     *    for the mac-binding should be deleted. This would not cause any
> real
> -     *    issue for now, since the port-binding related to mac-binding is
> -     *    always logical router port, and any change to logical router
> port
> -     *    would just trigger recompute.
> -     *
> -     * Although there is no correctness issue so far (except the unusual
> ACL
> -     * use case, which doesn't seem to be a real problem), it might be
> better
> -     * to handle this more gracefully, without the need to consider these
> -     * tricky scenarios.  One approach is to maintain a mapping between
> lport
> -     * names and the lflows that uses them, and reprocess the related
> lflows
> -     * when related port-bindings change.
> -     */
> -    physical_handle_port_binding_changes(
> -            sbrec_port_binding_by_name,
> -            port_binding_table, mff_ovn_geneve,
> -            chassis, ct_zones, local_datapaths,
> -            active_tunnels, flow_table);
> -
> -    node->changed = true;
> -    return true;
> -}
> -
> -static bool
> -flow_output_sb_multicast_group_handler(struct engine_node *node)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)engine_get_input(
> -                "runtime_data", node)->data;
> -    struct hmap *local_datapaths = &data->local_datapaths;
> -    struct simap *ct_zones = &data->ct_zones;
> -
> -    struct ed_type_mff_ovn_geneve *ed_mff_ovn_geneve =
> -        (struct ed_type_mff_ovn_geneve *)engine_get_input(
> -            "mff_ovn_geneve", node)->data;
> -    enum mf_field_id mff_ovn_geneve = ed_mff_ovn_geneve->mff_ovn_geneve;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -    const char *chassis_id = chassis_get_id();
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -    const struct sbrec_chassis *chassis = NULL;
> -    if (chassis_id) {
> -        chassis = chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id);
> -    }
> -    ovs_assert(br_int && chassis);
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -
> -    struct sbrec_multicast_group_table *multicast_group_table =
> -        (struct sbrec_multicast_group_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_multicast_group", node));
> -
> -    physical_handle_mc_group_changes(multicast_group_table,
> -            mff_ovn_geneve, chassis, ct_zones, local_datapaths,
> -            flow_table);
> -
> -    node->changed = true;
> -    return true;
> -
> -}
> -
> -static bool
> -_flow_output_resource_ref_handler(struct engine_node *node,
> -                                 enum ref_type ref_type)
> -{
> -    struct ed_type_runtime_data *data =
> -        (struct ed_type_runtime_data *)engine_get_input(
> -                "runtime_data", node)->data;
> -    struct hmap *local_datapaths = &data->local_datapaths;
> -    struct sset *local_lport_ids = &data->local_lport_ids;
> -    struct sset *active_tunnels = &data->active_tunnels;
> -
> -    struct ed_type_addr_sets *as_data =
> -        (struct ed_type_addr_sets *)engine_get_input("addr_sets",
> node)->data;
> -    struct shash *addr_sets = &as_data->addr_sets;
> -
> -    struct ed_type_port_groups *pg_data =
> -        (struct ed_type_port_groups *)engine_get_input(
> -            "port_groups", node)->data;
> -    struct shash *port_groups = &pg_data->port_groups;
> -
> -    struct ovsrec_open_vswitch_table *ovs_table =
> -        (struct ovsrec_open_vswitch_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_open_vswitch", node));
> -    struct ovsrec_bridge_table *bridge_table =
> -        (struct ovsrec_bridge_table *)EN_OVSDB_GET(
> -            engine_get_input("OVS_bridge", node));
> -    const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> ovs_table);
> -    const char *chassis_id = chassis_get_id();
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_chassis", node),
> -                "name");
> -    const struct sbrec_chassis *chassis = NULL;
> -    if (chassis_id) {
> -        chassis = chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id);
> -    }
> -
> -    ovs_assert(br_int && chassis);
> -
> -    struct ed_type_flow_output *fo =
> -        (struct ed_type_flow_output *)node->data;
> -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> -    struct ovn_extend_table *group_table = &fo->group_table;
> -    struct ovn_extend_table *meter_table = &fo->meter_table;
> -    uint32_t *conj_id_ofs = &fo->conj_id_ofs;
> -    struct lflow_resource_ref *lfrr = &fo->lflow_resource_ref;
> -
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_multicast_group", node),
> -                "name_datapath");
> -
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name =
> -        engine_ovsdb_node_get_index(
> -                engine_get_input("SB_port_binding", node),
> -                "name");
> -
> -    struct sbrec_dhcp_options_table *dhcp_table =
> -        (struct sbrec_dhcp_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcp_options", node));
> -
> -    struct sbrec_dhcpv6_options_table *dhcpv6_table =
> -        (struct sbrec_dhcpv6_options_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_dhcpv6_options", node));
> -
> -    struct sbrec_logical_flow_table *logical_flow_table =
> -        (struct sbrec_logical_flow_table *)EN_OVSDB_GET(
> -            engine_get_input("SB_logical_flow", node));
> -
> -    bool changed;
> -    const char *ref_name;
> -    struct sset *new, *updated, *deleted;
> -
> -    switch (ref_type) {
> -        case REF_TYPE_ADDRSET:
> -            /* XXX: The change_tracked check may be added to inc-proc
> -             * framework. */
> -            if (!as_data->change_tracked) {
> -                return false;
> -            }
> -            new = &as_data->new;
> -            updated = &as_data->updated;
> -            deleted = &as_data->deleted;
> -            break;
> -        case REF_TYPE_PORTGROUP:
> -            if (!pg_data->change_tracked) {
> -                return false;
> -            }
> -            new = &pg_data->new;
> -            updated = &pg_data->updated;
> -            deleted = &pg_data->deleted;
> -            break;
> -        default:
> -            OVS_NOT_REACHED();
> -    }
> -
> -
> -    SSET_FOR_EACH (ref_name, deleted) {
> -        if (!lflow_handle_changed_ref(ref_type, ref_name,
> -                    sbrec_multicast_group_by_name_datapath,
> -                    sbrec_port_binding_by_name,dhcp_table,
> -                    dhcpv6_table, logical_flow_table,
> -                    local_datapaths, chassis, addr_sets,
> -                    port_groups, active_tunnels, local_lport_ids,
> -                    flow_table, group_table, meter_table, lfrr,
> -                    conj_id_ofs, &changed)) {
> -            return false;
> -        }
> -        node->changed = changed || node->changed;
> -    }
> -    SSET_FOR_EACH (ref_name, updated) {
> -        if (!lflow_handle_changed_ref(ref_type, ref_name,
> -                    sbrec_multicast_group_by_name_datapath,
> -                    sbrec_port_binding_by_name,dhcp_table,
> -                    dhcpv6_table, logical_flow_table,
> -                    local_datapaths, chassis, addr_sets,
> -                    port_groups, active_tunnels, local_lport_ids,
> -                    flow_table, group_table, meter_table, lfrr,
> -                    conj_id_ofs, &changed)) {
> -            return false;
> -        }
> -        node->changed = changed || node->changed;
> -    }
> -    SSET_FOR_EACH (ref_name, new) {
> -        if (!lflow_handle_changed_ref(ref_type, ref_name,
> -                    sbrec_multicast_group_by_name_datapath,
> -                    sbrec_port_binding_by_name,dhcp_table,
> -                    dhcpv6_table, logical_flow_table,
> -                    local_datapaths, chassis, addr_sets,
> -                    port_groups, active_tunnels, local_lport_ids,
> -                    flow_table, group_table, meter_table, lfrr,
> -                    conj_id_ofs, &changed)) {
> -            return false;
> -        }
> -        node->changed = changed || node->changed;
> -    }
> -
> -    return true;
> -}
> -
> -static bool
> -flow_output_addr_sets_handler(struct engine_node *node)
> -{
> -    return _flow_output_resource_ref_handler(node, REF_TYPE_ADDRSET);
> -}
> -
> -static bool
> -flow_output_port_groups_handler(struct engine_node *node)
> -{
> -    return _flow_output_resource_ref_handler(node, REF_TYPE_PORTGROUP);
> -}
> -
> -struct ovn_controller_exit_args {
> -    bool *exiting;
> -    bool *restart;
> -};
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    struct unixctl_server *unixctl;
> -    bool exiting;
> -    bool restart;
> -    struct ovn_controller_exit_args exit_args = {&exiting, &restart};
> -    int retval;
> -
> -    ovs_cmdl_proctitle_init(argc, argv);
> -    set_program_name(argv[0]);
> -    service_start(&argc, &argv);
> -    char *ovs_remote = parse_options(argc, argv);
> -    fatal_ignore_sigpipe();
> -
> -    daemonize_start(false);
> -
> -    retval = unixctl_server_create(NULL, &unixctl);
> -    if (retval) {
> -        exit(EXIT_FAILURE);
> -    }
> -    unixctl_command_register("exit", "", 0, 1, ovn_controller_exit,
> -                             &exit_args);
> -
> -    daemonize_complete();
> -
> -    pinctrl_init();
> -    lflow_init();
> -
> -    /* Connect to OVS OVSDB instance. */
> -    struct ovsdb_idl_loop ovs_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create(ovs_remote, &ovsrec_idl_class, false, true));
> -    ctrl_register_ovs_idl(ovs_idl_loop.idl);
> -    ovsdb_idl_get_initial_snapshot(ovs_idl_loop.idl);
> -
> -    /* Configure OVN SB database. */
> -    struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create_unconnected(&sbrec_idl_class, true));
> -    ovsdb_idl_set_leader_only(ovnsb_idl_loop.idl, false);
> -
> -    unixctl_command_register("connection-status", "", 0, 0,
> -                             ovn_controller_conn_show,
> ovnsb_idl_loop.idl);
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name
> -        = chassis_index_create(ovnsb_idl_loop.idl);
> -    struct ovsdb_idl_index *sbrec_multicast_group_by_name_datapath
> -        = mcast_group_index_create(ovnsb_idl_loop.idl);
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name
> -        = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
> -                                  &sbrec_port_binding_col_logical_port);
> -    struct ovsdb_idl_index *sbrec_port_binding_by_key
> -        = ovsdb_idl_index_create2(ovnsb_idl_loop.idl,
> -                                  &sbrec_port_binding_col_tunnel_key,
> -                                  &sbrec_port_binding_col_datapath);
> -    struct ovsdb_idl_index *sbrec_port_binding_by_datapath
> -        = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
> -                                  &sbrec_port_binding_col_datapath);
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key
> -        = ovsdb_idl_index_create1(ovnsb_idl_loop.idl,
> -                                  &sbrec_datapath_binding_col_tunnel_key);
> -    struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip
> -        = ovsdb_idl_index_create2(ovnsb_idl_loop.idl,
> -                                  &sbrec_mac_binding_col_logical_port,
> -                                  &sbrec_mac_binding_col_ip);
> -    struct ovsdb_idl_index *sbrec_ip_multicast
> -        = ip_mcast_index_create(ovnsb_idl_loop.idl);
> -    struct ovsdb_idl_index *sbrec_igmp_group
> -        = igmp_group_index_create(ovnsb_idl_loop.idl);
> -
> -    ovsdb_idl_track_add_all(ovnsb_idl_loop.idl);
> -    ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_chassis_col_nb_cfg);
> -
> -    /* Omit the external_ids column of all the tables except for -
> -     *  - DNS. pinctrl.c uses the external_ids column of DNS,
> -     *    which it shouldn't. This should be removed.
> -     *
> -     *  - Chassis - chassis.c copies the chassis configuration from
> -     *              local open_vswitch table to the external_ids of
> -     *              chassis.
> -     *
> -     *  - Datapath_binding - lflow.c is using this to check if the
> datapath
> -     *                       is switch or not. This should be removed.
> -     * */
> -
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl, &sbrec_sb_global_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> &sbrec_logical_flow_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> &sbrec_port_binding_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> &sbrec_connection_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl, &sbrec_ssl_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> -                   &sbrec_gateway_chassis_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> &sbrec_ha_chassis_col_external_ids);
> -    ovsdb_idl_omit(ovnsb_idl_loop.idl,
> -                   &sbrec_ha_chassis_group_col_external_ids);
> -
> -    update_sb_monitors(ovnsb_idl_loop.idl, NULL, NULL, NULL);
> -
> -    stopwatch_create(CONTROLLER_LOOP_STOPWATCH_NAME, SW_MS);
> -
> -    /* Define inc-proc-engine nodes. */
> -    struct ed_type_runtime_data ed_runtime_data;
> -    struct ed_type_mff_ovn_geneve ed_mff_ovn_geneve;
> -    struct ed_type_ofctrl_is_connected ed_ofctrl_is_connected;
> -    struct ed_type_flow_output ed_flow_output;
> -    struct ed_type_addr_sets ed_addr_sets;
> -    struct ed_type_port_groups ed_port_groups;
> -
> -    ENGINE_NODE(runtime_data, "runtime_data");
> -    ENGINE_NODE(mff_ovn_geneve, "mff_ovn_geneve");
> -    ENGINE_NODE(ofctrl_is_connected, "ofctrl_is_connected");
> -    ENGINE_NODE(flow_output, "flow_output");
> -    ENGINE_NODE(addr_sets, "addr_sets");
> -    ENGINE_NODE(port_groups, "port_groups");
> -
> -#define SB_NODE(NAME, NAME_STR) ENGINE_NODE_SB(NAME, NAME_STR);
> -    SB_NODES
> -#undef SB_NODE
> -
> -#define OVS_NODE(NAME, NAME_STR) ENGINE_NODE_OVS(NAME, NAME_STR);
> -    OVS_NODES
> -#undef OVS_NODE
> -
> -    engine_ovsdb_node_add_index(&en_sb_chassis, "name",
> sbrec_chassis_by_name);
> -    engine_ovsdb_node_add_index(&en_sb_multicast_group, "name_datapath",
> -                                sbrec_multicast_group_by_name_datapath);
> -    engine_ovsdb_node_add_index(&en_sb_port_binding, "name",
> -                                sbrec_port_binding_by_name);
> -    engine_ovsdb_node_add_index(&en_sb_port_binding, "key",
> -                                sbrec_port_binding_by_key);
> -    engine_ovsdb_node_add_index(&en_sb_port_binding, "datapath",
> -                                sbrec_port_binding_by_datapath);
> -    engine_ovsdb_node_add_index(&en_sb_datapath_binding, "key",
> -                                sbrec_datapath_binding_by_key);
> -
> -    /* Add dependencies between inc-proc-engine nodes. */
> -
> -    engine_add_input(&en_addr_sets, &en_sb_address_set,
> -                     addr_sets_sb_address_set_handler);
> -    engine_add_input(&en_port_groups, &en_sb_port_group,
> -                     port_groups_sb_port_group_handler);
> -
> -    engine_add_input(&en_flow_output, &en_addr_sets,
> -                     flow_output_addr_sets_handler);
> -    engine_add_input(&en_flow_output, &en_port_groups,
> -                     flow_output_port_groups_handler);
> -    engine_add_input(&en_flow_output, &en_runtime_data, NULL);
> -    engine_add_input(&en_flow_output, &en_mff_ovn_geneve, NULL);
> -
> -    engine_add_input(&en_flow_output, &en_ovs_open_vswitch, NULL);
> -    engine_add_input(&en_flow_output, &en_ovs_bridge, NULL);
> -
> -    engine_add_input(&en_flow_output, &en_sb_chassis, NULL);
> -    engine_add_input(&en_flow_output, &en_sb_encap, NULL);
> -    engine_add_input(&en_flow_output, &en_sb_multicast_group,
> -                     flow_output_sb_multicast_group_handler);
> -    engine_add_input(&en_flow_output, &en_sb_port_binding,
> -                     flow_output_sb_port_binding_handler);
> -    engine_add_input(&en_flow_output, &en_sb_mac_binding,
> -                     flow_output_sb_mac_binding_handler);
> -    engine_add_input(&en_flow_output, &en_sb_logical_flow,
> -                     flow_output_sb_logical_flow_handler);
> -    engine_add_input(&en_flow_output, &en_sb_dhcp_options, NULL);
> -    engine_add_input(&en_flow_output, &en_sb_dhcpv6_options, NULL);
> -    engine_add_input(&en_flow_output, &en_sb_dns, NULL);
> -
> -    engine_add_input(&en_runtime_data, &en_ofctrl_is_connected, NULL);
> -
> -    engine_add_input(&en_runtime_data, &en_ovs_open_vswitch, NULL);
> -    engine_add_input(&en_runtime_data, &en_ovs_bridge, NULL);
> -    engine_add_input(&en_runtime_data, &en_ovs_port, NULL);
> -    engine_add_input(&en_runtime_data, &en_ovs_qos, NULL);
> -
> -    engine_add_input(&en_runtime_data, &en_sb_chassis, NULL);
> -    engine_add_input(&en_runtime_data, &en_sb_datapath_binding, NULL);
> -    engine_add_input(&en_runtime_data, &en_sb_port_binding,
> -                     runtime_data_sb_port_binding_handler);
> -
> -    engine_init(&en_flow_output);
> -
> -    ofctrl_init(&ed_flow_output.group_table,
> -                &ed_flow_output.meter_table,
> -                get_ofctrl_probe_interval(ovs_idl_loop.idl));
> -
> -    unixctl_command_register("group-table-list", "", 0, 0,
> -                             group_table_list,
> &ed_flow_output.group_table);
> -
> -    unixctl_command_register("meter-table-list", "", 0, 0,
> -                             meter_table_list,
> &ed_flow_output.meter_table);
> -
> -    unixctl_command_register("ct-zone-list", "", 0, 0,
> -                             ct_zone_list, &ed_runtime_data.ct_zones);
> -
> -    struct pending_pkt pending_pkt = { .conn = NULL };
> -    unixctl_command_register("inject-pkt", "MICROFLOW", 1, 1, inject_pkt,
> -                             &pending_pkt);
> -
> -    uint64_t engine_run_id = 0;
> -    uint64_t old_engine_run_id = 0;
> -
> -    unsigned int ovs_cond_seqno = UINT_MAX;
> -    unsigned int ovnsb_cond_seqno = UINT_MAX;
> -
> -    /* Main loop. */
> -    exiting = false;
> -    restart = false;
> -    while (!exiting) {
> -        update_sb_db(ovs_idl_loop.idl, ovnsb_idl_loop.idl);
> -        update_ssl_config(ovsrec_ssl_table_get(ovs_idl_loop.idl));
> -
> ofctrl_set_probe_interval(get_ofctrl_probe_interval(ovs_idl_loop.idl));
> -        old_engine_run_id = engine_run_id;
> -
> -        struct ovsdb_idl_txn *ovs_idl_txn =
> ovsdb_idl_loop_run(&ovs_idl_loop);
> -        unsigned int new_ovs_cond_seqno
> -            = ovsdb_idl_get_condition_seqno(ovs_idl_loop.idl);
> -        if (new_ovs_cond_seqno != ovs_cond_seqno) {
> -            if (!new_ovs_cond_seqno) {
> -                VLOG_INFO("OVS IDL reconnected, force recompute.");
> -                engine_set_force_recompute(true);
> -            }
> -            ovs_cond_seqno = new_ovs_cond_seqno;
> -        }
> -
> -        struct ovsdb_idl_txn *ovnsb_idl_txn
> -            = ovsdb_idl_loop_run(&ovnsb_idl_loop);
> -        unsigned int new_ovnsb_cond_seqno
> -            = ovsdb_idl_get_condition_seqno(ovnsb_idl_loop.idl);
> -        if (new_ovnsb_cond_seqno != ovnsb_cond_seqno) {
> -            if (!new_ovnsb_cond_seqno) {
> -                VLOG_INFO("OVNSB IDL reconnected, force recompute.");
> -                engine_set_force_recompute(true);
> -            }
> -            ovnsb_cond_seqno = new_ovnsb_cond_seqno;
> -        }
> -
> -        struct engine_context eng_ctx = {
> -            .ovs_idl_txn = ovs_idl_txn,
> -            .ovnsb_idl_txn = ovnsb_idl_txn
> -        };
> -
> -        engine_set_context(&eng_ctx);
> -
> -        if (ovsdb_idl_has_ever_connected(ovnsb_idl_loop.idl)) {
> -            /* Contains the transport zones that this Chassis belongs to
> */
> -            struct sset transport_zones =
> SSET_INITIALIZER(&transport_zones);
> -            sset_from_delimited_string(&transport_zones,
> -                get_transport_zones(ovsrec_open_vswitch_table_get(
> -                                    ovs_idl_loop.idl)), ",");
> -
> -            const struct ovsrec_bridge_table *bridge_table =
> -                ovsrec_bridge_table_get(ovs_idl_loop.idl);
> -            const struct ovsrec_open_vswitch_table *ovs_table =
> -                ovsrec_open_vswitch_table_get(ovs_idl_loop.idl);
> -            const struct sbrec_chassis_table *chassis_table =
> -                sbrec_chassis_table_get(ovnsb_idl_loop.idl);
> -            const struct ovsrec_bridge *br_int =
> -                process_br_int(ovs_idl_txn, bridge_table, ovs_table);
> -            const char *chassis_id = get_ovs_chassis_id(ovs_table);
> -            const struct sbrec_chassis *chassis = NULL;
> -            if (chassis_id) {
> -                chassis = chassis_run(ovnsb_idl_txn,
> sbrec_chassis_by_name,
> -                                      ovs_table, chassis_table,
> chassis_id,
> -                                      br_int, &transport_zones);
> -            }
> -
> -            if (br_int) {
> -                ofctrl_run(br_int, &ed_runtime_data.pending_ct_zones);
> -
> -                if (chassis) {
> -                    patch_run(ovs_idl_txn,
> -                              ovsrec_bridge_table_get(ovs_idl_loop.idl),
> -
> ovsrec_open_vswitch_table_get(ovs_idl_loop.idl),
> -                              ovsrec_port_table_get(ovs_idl_loop.idl),
> -
> sbrec_port_binding_table_get(ovnsb_idl_loop.idl),
> -                              br_int, chassis);
> -                    encaps_run(ovs_idl_txn,
> -                               bridge_table, br_int,
> -
>  sbrec_chassis_table_get(ovnsb_idl_loop.idl),
> -                               chassis_id,
> -                               sbrec_sb_global_first(ovnsb_idl_loop.idl),
> -                               &transport_zones);
> -
> -                    stopwatch_start(CONTROLLER_LOOP_STOPWATCH_NAME,
> -                                    time_msec());
> -                    if (ovnsb_idl_txn) {
> -                        engine_run(&en_flow_output, ++engine_run_id);
> -                    }
> -                    stopwatch_stop(CONTROLLER_LOOP_STOPWATCH_NAME,
> -                                   time_msec());
> -                    if (ovs_idl_txn) {
> -                        commit_ct_zones(br_int,
> -
> &ed_runtime_data.pending_ct_zones);
> -
> bfd_run(ovsrec_interface_table_get(ovs_idl_loop.idl),
> -                                br_int, chassis,
> -                                sbrec_ha_chassis_group_table_get(
> -                                    ovnsb_idl_loop.idl),
> -
> sbrec_sb_global_table_get(ovnsb_idl_loop.idl));
> -                    }
> -                    ofctrl_put(&ed_flow_output.flow_table,
> -                               &ed_runtime_data.pending_ct_zones,
> -                               sbrec_meter_table_get(ovnsb_idl_loop.idl),
> -                               get_nb_cfg(sbrec_sb_global_table_get(
> -                                              ovnsb_idl_loop.idl)),
> -                               en_flow_output.changed);
> -                    pinctrl_run(ovnsb_idl_txn,
> -                                sbrec_datapath_binding_by_key,
> -                                sbrec_port_binding_by_datapath,
> -                                sbrec_port_binding_by_key,
> -                                sbrec_port_binding_by_name,
> -                                sbrec_mac_binding_by_lport_ip,
> -                                sbrec_igmp_group,
> -                                sbrec_ip_multicast,
> -                                sbrec_dns_table_get(ovnsb_idl_loop.idl),
> -                                sbrec_controller_event_table_get(
> -                                    ovnsb_idl_loop.idl),
> -                                br_int, chassis,
> -                                &ed_runtime_data.local_datapaths,
> -                                &ed_runtime_data.active_tunnels);
> -
> -                    if (en_runtime_data.changed) {
> -                        update_sb_monitors(ovnsb_idl_loop.idl, chassis,
> -                                           &ed_runtime_data.local_lports,
> -
>  &ed_runtime_data.local_datapaths);
> -                    }
> -                }
> -
> -            }
> -            if (old_engine_run_id == engine_run_id) {
> -                if (engine_need_run(&en_flow_output)) {
> -                    VLOG_DBG("engine did not run, force recompute next
> time: "
> -                             "br_int %p, chassis %p", br_int, chassis);
> -                    engine_set_force_recompute(true);
> -                    poll_immediate_wake();
> -                } else {
> -                    VLOG_DBG("engine did not run, and it was not needed"
> -                             " either: br_int %p, chassis %p",
> -                             br_int, chassis);
> -                }
> -            } else {
> -                engine_set_force_recompute(false);
> -            }
> -
> -            if (ovnsb_idl_txn && chassis) {
> -                int64_t cur_cfg = ofctrl_get_cur_cfg();
> -                if (cur_cfg && cur_cfg != chassis->nb_cfg) {
> -                    sbrec_chassis_set_nb_cfg(chassis, cur_cfg);
> -                }
> -            }
> -
> -
> -            if (pending_pkt.conn) {
> -                if (br_int && chassis) {
> -                    char *error = ofctrl_inject_pkt(br_int,
> pending_pkt.flow_s,
> -                        &ed_addr_sets.addr_sets,
> &ed_port_groups.port_groups);
> -                    if (error) {
> -                        unixctl_command_reply_error(pending_pkt.conn,
> error);
> -                        free(error);
> -                    } else {
> -                        VLOG_DBG("Pending_pkt conn but br_int %p or
> chassis "
> -                                 "%p not ready. run-id: %"PRIu64, br_int,
> -                                 chassis, engine_run_id);
> -                        unixctl_command_reply_error(pending_pkt.conn,
> -                            "ovn-controller not ready.");
> -                    }
> -                }
> -                pending_pkt.conn = NULL;
> -                free(pending_pkt.flow_s);
> -            }
> -
> -            sset_destroy(&transport_zones);
> -
> -            if (br_int) {
> -                ofctrl_wait();
> -                pinctrl_wait(ovnsb_idl_txn);
> -            }
> -        }
> -
> -        unixctl_server_run(unixctl);
> -
> -        unixctl_server_wait(unixctl);
> -        if (exiting || pending_pkt.conn) {
> -            poll_immediate_wake();
> -        }
> -
> -        if (!ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop)) {
> -            VLOG_INFO("OVNSB commit failed, force recompute next time.");
> -            engine_set_force_recompute(true);
> -        }
> -
> -        if (ovsdb_idl_loop_commit_and_wait(&ovs_idl_loop) == 1) {
> -            struct shash_node *iter, *iter_next;
> -            SHASH_FOR_EACH_SAFE (iter, iter_next,
> -                                 &ed_runtime_data.pending_ct_zones) {
> -                struct ct_zone_pending_entry *ctzpe = iter->data;
> -                if (ctzpe->state == CT_ZONE_DB_SENT) {
> -                    shash_delete(&ed_runtime_data.pending_ct_zones, iter);
> -                    free(ctzpe);
> -                }
> -            }
> -        }
> -
> -        ovsdb_idl_track_clear(ovnsb_idl_loop.idl);
> -        ovsdb_idl_track_clear(ovs_idl_loop.idl);
> -        poll_block();
> -        if (should_service_stop()) {
> -            exiting = true;
> -        }
> -    }
> -
> -    engine_set_context(NULL);
> -    engine_cleanup(&en_flow_output);
> -
> -    /* It's time to exit.  Clean up the databases if we are not
> restarting */
> -    if (!restart) {
> -        bool done = !ovsdb_idl_has_ever_connected(ovnsb_idl_loop.idl);
> -        while (!done) {
> -            update_sb_db(ovs_idl_loop.idl, ovnsb_idl_loop.idl);
> -            update_ssl_config(ovsrec_ssl_table_get(ovs_idl_loop.idl));
> -
> -            struct ovsdb_idl_txn *ovs_idl_txn
> -                = ovsdb_idl_loop_run(&ovs_idl_loop);
> -            struct ovsdb_idl_txn *ovnsb_idl_txn
> -                = ovsdb_idl_loop_run(&ovnsb_idl_loop);
> -
> -            const struct ovsrec_bridge_table *bridge_table
> -                = ovsrec_bridge_table_get(ovs_idl_loop.idl);
> -            const struct ovsrec_open_vswitch_table *ovs_table
> -                = ovsrec_open_vswitch_table_get(ovs_idl_loop.idl);
> -
> -            const struct sbrec_port_binding_table *port_binding_table
> -                = sbrec_port_binding_table_get(ovnsb_idl_loop.idl);
> -
> -            const struct ovsrec_bridge *br_int = get_br_int(bridge_table,
> -                                                            ovs_table);
> -            const char *chassis_id = chassis_get_id();
> -            const struct sbrec_chassis *chassis
> -                = (chassis_id
> -                   ? chassis_lookup_by_name(sbrec_chassis_by_name,
> chassis_id)
> -                   : NULL);
> -
> -            /* Run all of the cleanup functions, even if one of them
> returns
> -             * false. We're done if all of them return true. */
> -            done = binding_cleanup(ovnsb_idl_txn, port_binding_table,
> chassis);
> -            done = chassis_cleanup(ovnsb_idl_txn, chassis) && done;
> -            done = encaps_cleanup(ovs_idl_txn, br_int) && done;
> -            done = igmp_group_cleanup(ovnsb_idl_txn, sbrec_igmp_group) &&
> done;
> -            if (done) {
> -                poll_immediate_wake();
> -            }
> -
> -            ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop);
> -            ovsdb_idl_loop_commit_and_wait(&ovs_idl_loop);
> -            poll_block();
> -        }
> -    }
> -
> -    unixctl_server_destroy(unixctl);
> -    lflow_destroy();
> -    ofctrl_destroy();
> -    pinctrl_destroy();
> -
> -    ovsdb_idl_loop_destroy(&ovs_idl_loop);
> -    ovsdb_idl_loop_destroy(&ovnsb_idl_loop);
> -
> -    free(ovs_remote);
> -    service_stop();
> -
> -    exit(retval);
> -}
> -
> -static char *
> -parse_options(int argc, char *argv[])
> -{
> -    enum {
> -        OPT_PEER_CA_CERT = UCHAR_MAX + 1,
> -        OPT_BOOTSTRAP_CA_CERT,
> -        VLOG_OPTION_ENUMS,
> -        DAEMON_OPTION_ENUMS,
> -        SSL_OPTION_ENUMS,
> -    };
> -
> -    static struct option long_options[] = {
> -        {"help", no_argument, NULL, 'h'},
> -        {"version", no_argument, NULL, 'V'},
> -        VLOG_LONG_OPTIONS,
> -        DAEMON_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {"peer-ca-cert", required_argument, NULL, OPT_PEER_CA_CERT},
> -        {"bootstrap-ca-cert", required_argument, NULL,
> OPT_BOOTSTRAP_CA_CERT},
> -        {NULL, 0, NULL, 0}
> -    };
> -    char *short_options =
> ovs_cmdl_long_options_to_short_options(long_options);
> -
> -    for (;;) {
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, long_options, NULL);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        switch (c) {
> -        case 'h':
> -            usage();
> -
> -        case 'V':
> -            ovs_print_version(OFP13_VERSION, OFP13_VERSION);
> -            exit(EXIT_SUCCESS);
> -
> -        VLOG_OPTION_HANDLERS
> -        DAEMON_OPTION_HANDLERS
> -        STREAM_SSL_OPTION_HANDLERS
> -
> -        case OPT_PEER_CA_CERT:
> -            stream_ssl_set_peer_ca_cert_file(optarg);
> -            break;
> -
> -        case OPT_BOOTSTRAP_CA_CERT:
> -            stream_ssl_set_ca_cert_file(optarg, true);
> -            break;
> -
> -        case '?':
> -            exit(EXIT_FAILURE);
> -
> -        default:
> -            abort();
> -        }
> -    }
> -    free(short_options);
> -
> -    argc -= optind;
> -    argv += optind;
> -
> -    char *ovs_remote;
> -    if (argc == 0) {
> -        ovs_remote = xasprintf("unix:%s/db.sock", ovs_rundir());
> -    } else if (argc == 1) {
> -        ovs_remote = xstrdup(argv[0]);
> -    } else {
> -        VLOG_FATAL("exactly zero or one non-option argument required; "
> -                   "use --help for usage");
> -    }
> -    return ovs_remote;
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("%s: OVN controller\n"
> -           "usage %s [OPTIONS] [OVS-DATABASE]\n"
> -           "where OVS-DATABASE is a socket on which the OVS OVSDB server
> is listening.\n",
> -               program_name, program_name);
> -    stream_usage("OVS-DATABASE", true, false, true);
> -    daemon_usage();
> -    vlog_usage();
> -    printf("\nOther options:\n"
> -           "  -h, --help              display this help message\n"
> -           "  -V, --version           display version information\n");
> -    exit(EXIT_SUCCESS);
> -}
> -
> -static void
> -ovn_controller_exit(struct unixctl_conn *conn, int argc,
> -             const char *argv[], void *exit_args_)
> -{
> -    struct ovn_controller_exit_args *exit_args = exit_args_;
> -    *exit_args->exiting = true;
> -    *exit_args->restart = argc == 2 && !strcmp(argv[1], "--restart");
> -    unixctl_command_reply(conn, NULL);
> -}
> -
> -static void
> -ct_zone_list(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -             const char *argv[] OVS_UNUSED, void *ct_zones_)
> -{
> -    struct simap *ct_zones = ct_zones_;
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -    struct simap_node *zone;
> -
> -    SIMAP_FOR_EACH(zone, ct_zones) {
> -        ds_put_format(&ds, "%s %d\n", zone->name, zone->data);
> -    }
> -
> -    unixctl_command_reply(conn, ds_cstr(&ds));
> -    ds_destroy(&ds);
> -}
> -
> -static void
> -meter_table_list(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                 const char *argv[] OVS_UNUSED, void *meter_table_)
> -{
> -    struct ovn_extend_table *meter_table = meter_table_;
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -    struct simap meters = SIMAP_INITIALIZER(&meters);
> -
> -    struct ovn_extend_table_info *m_installed, *next_meter;
> -    EXTEND_TABLE_FOR_EACH_INSTALLED (m_installed, next_meter,
> meter_table) {
> -        simap_put(&meters, m_installed->name, m_installed->table_id);
> -    }
> -
> -    const struct simap_node **nodes = simap_sort(&meters);
> -    size_t n_nodes = simap_count(&meters);
> -    for (size_t i = 0; i < n_nodes; i++) {
> -        const struct simap_node *node = nodes[i];
> -        ds_put_format(&ds, "%s: %d\n", node->name, node->data);
> -    }
> -
> -    free(nodes);
> -    simap_destroy(&meters);
> -
> -    unixctl_command_reply(conn, ds_cstr(&ds));
> -    ds_destroy(&ds);
> -}
> -
> -static void
> -group_table_list(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                 const char *argv[] OVS_UNUSED, void *group_table_)
> -{
> -    struct ovn_extend_table *group_table = group_table_;
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -    struct simap groups = SIMAP_INITIALIZER(&groups);
> -
> -    struct ovn_extend_table_info *m_installed, *next_group;
> -    EXTEND_TABLE_FOR_EACH_INSTALLED (m_installed, next_group,
> group_table) {
> -        simap_put(&groups, m_installed->name, m_installed->table_id);
> -    }
> -
> -    const struct simap_node **nodes = simap_sort(&groups);
> -    size_t n_nodes = simap_count(&groups);
> -    for (size_t i = 0; i < n_nodes; i++) {
> -        const struct simap_node *node = nodes[i];
> -        ds_put_format(&ds, "%s: %d\n", node->name, node->data);
> -    }
> -
> -    free(nodes);
> -    simap_destroy(&groups);
> -
> -    unixctl_command_reply(conn, ds_cstr(&ds));
> -    ds_destroy(&ds);
> -}
> -
> -static void
> -inject_pkt(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -           const char *argv[], void *pending_pkt_)
> -{
> -    struct pending_pkt *pending_pkt = pending_pkt_;
> -
> -    if (pending_pkt->conn) {
> -        unixctl_command_reply_error(conn, "already pending packet
> injection");
> -        return;
> -    }
> -    pending_pkt->conn = conn;
> -    pending_pkt->flow_s = xstrdup(argv[1]);
> -}
> -
> -static void
> -ovn_controller_conn_show(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                         const char *argv[] OVS_UNUSED, void *idl_)
> -{
> -    const char *result = "not connected";
> -    const struct ovsdb_idl *idl = idl_;
> -
> -    if (ovsdb_idl_is_connected(idl)) {
> -       result = "connected";
> -    }
> -    unixctl_command_reply(conn, result);
> -}
> diff --git a/ovn/controller/ovn-controller.h
> b/ovn/controller/ovn-controller.h
> deleted file mode 100644
> index 078c9eabe..000000000
> --- a/ovn/controller/ovn-controller.h
> +++ /dev/null
> @@ -1,85 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_CONTROLLER_H
> -#define OVN_CONTROLLER_H 1
> -
> -#include "simap.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -struct ovsrec_bridge_table;
> -
> -/* Linux supports a maximum of 64K zones, which seems like a fine
> default. */
> -#define MAX_CT_ZONES 65535
> -
> -/* States to move through when a new conntrack zone has been allocated. */
> -enum ct_zone_pending_state {
> -    CT_ZONE_OF_QUEUED,    /* Waiting to send conntrack flush command. */
> -    CT_ZONE_OF_SENT,      /* Sent and waiting for confirmation on flush.
> */
> -    CT_ZONE_DB_QUEUED,    /* Waiting for DB transaction to open. */
> -    CT_ZONE_DB_SENT,      /* Sent and waiting for confirmation from DB. */
> -};
> -
> -struct ct_zone_pending_entry {
> -    int zone;
> -    bool add;             /* Is the entry being added? */
> -    ovs_be32 of_xid;      /* Transaction id for barrier. */
> -    enum ct_zone_pending_state state;
> -};
> -
> -/* A logical datapath that has some relevance to this hypervisor.  A
> logical
> - * datapath D is relevant to hypervisor H if:
> - *
> - *     - Some VIF or l2gateway or l3gateway port in D is located on H.
> - *
> - *     - D is reachable over a series of hops across patch ports,
> starting from
> - *       a datapath relevant to H.
> - *
> - * The 'hmap_node''s hash value is 'datapath->tunnel_key'. */
> -struct local_datapath {
> -    struct hmap_node hmap_node;
> -    const struct sbrec_datapath_binding *datapath;
> -
> -    /* The localnet port in this datapath, if any (at most one is
> allowed). */
> -    const struct sbrec_port_binding *localnet_port;
> -
> -    /* True if this datapath contains an l3gateway port located on this
> -     * hypervisor. */
> -    bool has_local_l3gateway;
> -
> -    const struct sbrec_port_binding **peer_ports;
> -    size_t n_peer_ports;
> -};
> -
> -struct local_datapath *get_local_datapath(const struct hmap *,
> -                                          uint32_t tunnel_key);
> -
> -const struct ovsrec_bridge *get_bridge(const struct ovsrec_bridge_table *,
> -                                       const char *br_name);
> -
> -struct sbrec_encap *preferred_encap(const struct sbrec_chassis *);
> -
> -/* Must be a bit-field ordered from most-preferred (higher number) to
> - * least-preferred (lower number). */
> -enum chassis_tunnel_type {
> -    GENEVE = 1 << 2,
> -    STT    = 1 << 1,
> -    VXLAN  = 1 << 0
> -};
> -
> -uint32_t get_tunnel_type(const char *name);
> -
> -#endif /* ovn/ovn-controller.h */
> diff --git a/ovn/controller/patch.c b/ovn/controller/patch.c
> deleted file mode 100644
> index a6770c6d5..000000000
> --- a/ovn/controller/patch.c
> +++ /dev/null
> @@ -1,273 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "patch.h"
> -
> -#include "hash.h"
> -#include "lflow.h"
> -#include "lib/vswitch-idl.h"
> -#include "lport.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn-controller.h"
> -
> -VLOG_DEFINE_THIS_MODULE(patch);
> -
> -static char *
> -patch_port_name(const char *src, const char *dst)
> -{
> -    return xasprintf("patch-%s-to-%s", src, dst);
> -}
> -
> -/* Return true if 'port' is a patch port with the specified 'peer'. */
> -static bool
> -match_patch_port(const struct ovsrec_port *port, const char *peer)
> -{
> -    for (size_t i = 0; i < port->n_interfaces; i++) {
> -        struct ovsrec_interface *iface = port->interfaces[i];
> -        if (strcmp(iface->type, "patch")) {
> -            continue;
> -        }
> -        const char *iface_peer = smap_get(&iface->options, "peer");
> -        if (iface_peer && !strcmp(iface_peer, peer)) {
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -/* Creates a patch port in bridge 'src' named 'src_name', whose peer is
> - * 'dst_name' in bridge 'dst'.  Initializes the patch port's
> external-ids:'key'
> - * to 'key'.
> - *
> - * If such a patch port already exists, removes it from 'existing_ports'.
> */
> -static void
> -create_patch_port(struct ovsdb_idl_txn *ovs_idl_txn,
> -                  const char *key, const char *value,
> -                  const struct ovsrec_bridge *src, const char *src_name,
> -                  const struct ovsrec_bridge *dst, const char *dst_name,
> -                  struct shash *existing_ports)
> -{
> -    for (size_t i = 0; i < src->n_ports; i++) {
> -        if (match_patch_port(src->ports[i], dst_name)) {
> -            /* Patch port already exists on 'src'. */
> -            shash_find_and_delete(existing_ports, src->ports[i]->name);
> -            return;
> -        }
> -    }
> -
> -    ovsdb_idl_txn_add_comment(ovs_idl_txn,
> -            "ovn-controller: creating patch port '%s' from '%s' to '%s'",
> -            src_name, src->name, dst->name);
> -
> -    struct ovsrec_interface *iface;
> -    iface = ovsrec_interface_insert(ovs_idl_txn);
> -    ovsrec_interface_set_name(iface, src_name);
> -    ovsrec_interface_set_type(iface, "patch");
> -    const struct smap options = SMAP_CONST1(&options, "peer", dst_name);
> -    ovsrec_interface_set_options(iface, &options);
> -
> -    struct ovsrec_port *port;
> -    port = ovsrec_port_insert(ovs_idl_txn);
> -    ovsrec_port_set_name(port, src_name);
> -    ovsrec_port_set_interfaces(port, &iface, 1);
> -    const struct smap ids = SMAP_CONST1(&ids, key, value);
> -    ovsrec_port_set_external_ids(port, &ids);
> -
> -    struct ovsrec_port **ports;
> -    ports = xmalloc(sizeof *ports * (src->n_ports + 1));
> -    memcpy(ports, src->ports, sizeof *ports * src->n_ports);
> -    ports[src->n_ports] = port;
> -    ovsrec_bridge_verify_ports(src);
> -    ovsrec_bridge_set_ports(src, ports, src->n_ports + 1);
> -
> -    free(ports);
> -}
> -
> -static void
> -remove_port(const struct ovsrec_bridge_table *bridge_table,
> -            const struct ovsrec_port *port)
> -{
> -    const struct ovsrec_bridge *bridge;
> -
> -    /* We know the port we want to delete, but we have to find the bridge
> its
> -     * on to do so.  Note this only runs on a config change that should be
> -     * pretty rare. */
> -    OVSREC_BRIDGE_TABLE_FOR_EACH (bridge, bridge_table) {
> -        size_t i;
> -        for (i = 0; i < bridge->n_ports; i++) {
> -            if (bridge->ports[i] != port) {
> -                continue;
> -            }
> -            struct ovsrec_port **new_ports;
> -            new_ports = xmemdup(bridge->ports,
> -                    sizeof *new_ports * (bridge->n_ports - 1));
> -            if (i != bridge->n_ports - 1) {
> -                /* Removed port was not last */
> -                new_ports[i] = bridge->ports[bridge->n_ports - 1];
> -            }
> -            ovsrec_bridge_verify_ports(bridge);
> -            ovsrec_bridge_set_ports(bridge, new_ports, bridge->n_ports -
> 1);
> -            free(new_ports);
> -            ovsrec_port_delete(port);
> -            return;
> -        }
> -    }
> -}
> -
> -/* Obtains external-ids:ovn-bridge-mappings from OVSDB and adds patch
> ports for
> - * the local bridge mappings.  Removes any patch ports for bridge
> mappings that
> - * already existed from 'existing_ports'. */
> -static void
> -add_bridge_mappings(struct ovsdb_idl_txn *ovs_idl_txn,
> -                    const struct ovsrec_bridge_table *bridge_table,
> -                    const struct ovsrec_open_vswitch_table *ovs_table,
> -                    const struct sbrec_port_binding_table
> *port_binding_table,
> -                    const struct ovsrec_bridge *br_int,
> -                    struct shash *existing_ports,
> -                    const struct sbrec_chassis *chassis)
> -{
> -    /* Get ovn-bridge-mappings. */
> -    const char *mappings_cfg = "";
> -    const struct ovsrec_open_vswitch *cfg;
> -    cfg = ovsrec_open_vswitch_table_first(ovs_table);
> -    if (cfg) {
> -        mappings_cfg = smap_get(&cfg->external_ids,
> "ovn-bridge-mappings");
> -        if (!mappings_cfg || !mappings_cfg[0]) {
> -            return;
> -        }
> -    }
> -
> -    /* Parse bridge mappings. */
> -    struct shash bridge_mappings = SHASH_INITIALIZER(&bridge_mappings);
> -    char *cur, *next, *start;
> -    next = start = xstrdup(mappings_cfg);
> -    while ((cur = strsep(&next, ",")) && *cur) {
> -        char *network, *bridge = cur;
> -        const struct ovsrec_bridge *ovs_bridge;
> -
> -        network = strsep(&bridge, ":");
> -        if (!bridge || !*network || !*bridge) {
> -            VLOG_ERR("Invalid ovn-bridge-mappings configuration: '%s'",
> -                    mappings_cfg);
> -            break;
> -        }
> -
> -        ovs_bridge = get_bridge(bridge_table, bridge);
> -        if (!ovs_bridge) {
> -            VLOG_WARN("Bridge '%s' not found for network '%s'",
> -                    bridge, network);
> -            continue;
> -        }
> -
> -        shash_add(&bridge_mappings, network, ovs_bridge);
> -    }
> -    free(start);
> -
> -    const struct sbrec_port_binding *binding;
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH (binding, port_binding_table) {
> -        const char *patch_port_id;
> -        if (!strcmp(binding->type, "localnet")) {
> -            patch_port_id = "ovn-localnet-port";
> -        } else if (!strcmp(binding->type, "l2gateway")) {
> -            if (!binding->chassis
> -                || strcmp(chassis->name, binding->chassis->name)) {
> -                /* This L2 gateway port is not bound to this chassis,
> -                 * so we should not create any patch ports for it. */
> -                continue;
> -            }
> -            patch_port_id = "ovn-l2gateway-port";
> -        } else {
> -            /* not a localnet or L2 gateway port. */
> -            continue;
> -        }
> -
> -        const char *network = smap_get(&binding->options, "network_name");
> -        if (!network) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_ERR_RL(&rl, "%s port '%s' has no network name.",
> -                         binding->type, binding->logical_port);
> -            continue;
> -        }
> -        struct ovsrec_bridge *br_ln = shash_find_data(&bridge_mappings,
> network);
> -        if (!br_ln) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_ERR_RL(&rl, "bridge not found for %s port '%s' "
> -                    "with network name '%s'",
> -                    binding->type, binding->logical_port, network);
> -            continue;
> -        }
> -
> -        char *name1 = patch_port_name(br_int->name,
> binding->logical_port);
> -        char *name2 = patch_port_name(binding->logical_port,
> br_int->name);
> -        create_patch_port(ovs_idl_txn, patch_port_id,
> binding->logical_port,
> -                          br_int, name1, br_ln, name2, existing_ports);
> -        create_patch_port(ovs_idl_txn, patch_port_id,
> binding->logical_port,
> -                          br_ln, name2, br_int, name1, existing_ports);
> -        free(name1);
> -        free(name2);
> -    }
> -
> -    shash_destroy(&bridge_mappings);
> -}
> -
> -void
> -patch_run(struct ovsdb_idl_txn *ovs_idl_txn,
> -          const struct ovsrec_bridge_table *bridge_table,
> -          const struct ovsrec_open_vswitch_table *ovs_table,
> -          const struct ovsrec_port_table *port_table,
> -          const struct sbrec_port_binding_table *port_binding_table,
> -          const struct ovsrec_bridge *br_int,
> -          const struct sbrec_chassis *chassis)
> -{
> -    if (!ovs_idl_txn) {
> -        return;
> -    }
> -
> -    /* Figure out what patch ports already exist.
> -     *
> -     * ovn-controller does not create or use ports of type
> "ovn-l3gateway-port"
> -     * or "ovn-logical-patch-port", but older version did.  We still
> recognize
> -     * them here, so that we delete them at the end of this function, to
> avoid
> -     * leaving useless ports on upgrade. */
> -    struct shash existing_ports = SHASH_INITIALIZER(&existing_ports);
> -    const struct ovsrec_port *port;
> -    OVSREC_PORT_TABLE_FOR_EACH (port, port_table) {
> -        if (smap_get(&port->external_ids, "ovn-localnet-port")
> -            || smap_get(&port->external_ids, "ovn-l2gateway-port")
> -            || smap_get(&port->external_ids, "ovn-l3gateway-port")
> -            || smap_get(&port->external_ids, "ovn-logical-patch-port")) {
> -            shash_add(&existing_ports, port->name, port);
> -        }
> -    }
> -
> -    /* Create in the database any patch ports that should exist.  Remove
> from
> -     * 'existing_ports' any patch ports that do exist in the database and
> -     * should be there. */
> -    add_bridge_mappings(ovs_idl_txn, bridge_table, ovs_table,
> -                        port_binding_table, br_int, &existing_ports,
> chassis);
> -
> -    /* Now 'existing_ports' only still contains patch ports that exist in
> the
> -     * database but shouldn't.  Delete them from the database. */
> -    struct shash_node *port_node, *port_next_node;
> -    SHASH_FOR_EACH_SAFE (port_node, port_next_node, &existing_ports) {
> -        port = port_node->data;
> -        shash_delete(&existing_ports, port_node);
> -        remove_port(bridge_table, port);
> -    }
> -    shash_destroy(&existing_ports);
> -}
> diff --git a/ovn/controller/patch.h b/ovn/controller/patch.h
> deleted file mode 100644
> index dd052cfd8..000000000
> --- a/ovn/controller/patch.h
> +++ /dev/null
> @@ -1,42 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_PATCH_H
> -#define OVN_PATCH_H 1
> -
> -/* Patch Ports
> - * ===========
> - *
> - * This module adds and removes patch ports between the integration
> bridge and
> - * physical bridges, as directed by other-config:ovn-bridge-mappings. */
> -
> -struct hmap;
> -struct ovsdb_idl_txn;
> -struct ovsrec_bridge;
> -struct ovsrec_bridge_table;
> -struct ovsrec_open_vswitch_table;
> -struct ovsrec_port_table;
> -struct sbrec_port_binding_table;
> -struct sbrec_chassis;
> -
> -void patch_run(struct ovsdb_idl_txn *ovs_idl_txn,
> -               const struct ovsrec_bridge_table *,
> -               const struct ovsrec_open_vswitch_table *,
> -               const struct ovsrec_port_table *,
> -               const struct sbrec_port_binding_table *,
> -               const struct ovsrec_bridge *br_int,
> -               const struct sbrec_chassis *);
> -
> -#endif /* ovn/patch.h */
> diff --git a/ovn/controller/physical.c b/ovn/controller/physical.c
> deleted file mode 100644
> index 316d3738c..000000000
> --- a/ovn/controller/physical.c
> +++ /dev/null
> @@ -1,1459 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "binding.h"
> -#include "byte-order.h"
> -#include "encaps.h"
> -#include "flow.h"
> -#include "ha-chassis.h"
> -#include "lflow.h"
> -#include "lport.h"
> -#include "chassis.h"
> -#include "lib/bundle.h"
> -#include "openvswitch/poll-loop.h"
> -#include "lib/uuid.h"
> -#include "ofctrl.h"
> -#include "openvswitch/list.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "openvswitch/ofp-parse.h"
> -#include "ovn-controller.h"
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "physical.h"
> -#include "openvswitch/shash.h"
> -#include "simap.h"
> -#include "smap.h"
> -#include "sset.h"
> -#include "util.h"
> -#include "vswitch-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(physical);
> -
> -/* UUID to identify OF flows not associated with ovsdb rows. */
> -static struct uuid *hc_uuid = NULL;
> -
> -void
> -physical_register_ovs_idl(struct ovsdb_idl *ovs_idl)
> -{
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_bridge);
> -    ovsdb_idl_add_column(ovs_idl, &ovsrec_bridge_col_ports);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_port);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_interfaces);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_port_col_external_ids);
> -
> -    ovsdb_idl_add_table(ovs_idl, &ovsrec_table_interface);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_name);
> -    ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_ofport);
> -    ovsdb_idl_track_add_column(ovs_idl,
> &ovsrec_interface_col_external_ids);
> -}
> -
> -static struct simap localvif_to_ofport =
> -    SIMAP_INITIALIZER(&localvif_to_ofport);
> -static struct hmap tunnels = HMAP_INITIALIZER(&tunnels);
> -
> -/* Maps from a chassis to the OpenFlow port number of the tunnel that can
> be
> - * used to reach that chassis. */
> -struct chassis_tunnel {
> -    struct hmap_node hmap_node;
> -    char *chassis_id;
> -    ofp_port_t ofport;
> -    enum chassis_tunnel_type type;
> -};
> -
> -/*
> - * This function looks up the list of tunnel ports (provided by
> - * ovn-chassis-id ports) and returns the tunnel for the given chassid-id
> and
> - * encap-ip. The ovn-chassis-id is formed using the chassis-id and
> encap-ip.
> - * The list is hashed using the chassis-id. If the encap-ip is not
> specified,
> - * it means we'll just return a tunnel for that chassis-id, i.e. we just
> check
> - * for chassis-id and if there is a match, we'll return the tunnel.
> - * If encap-ip is also provided we use both chassis-id and encap-ip to do
> - * a more specific lookup.
> - */
> -static struct chassis_tunnel *
> -chassis_tunnel_find(const char *chassis_id, char *encap_ip)
> -{
> -    /*
> -     * If the specific encap_ip is given, look for the chassisid_ip entry,
> -     * else return the 1st found entry for the chassis.
> -     */
> -    struct chassis_tunnel *tun = NULL;
> -    HMAP_FOR_EACH_WITH_HASH (tun, hmap_node, hash_string(chassis_id, 0),
> -                             &tunnels) {
> -        if (encaps_tunnel_id_match(tun->chassis_id, chassis_id,
> encap_ip)) {
> -            return tun;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -put_load(uint64_t value, enum mf_field_id dst, int ofs, int n_bits,
> -         struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts,
> -                                                       mf_from_id(dst),
> NULL,
> -                                                       NULL);
> -    ovs_be64 n_value = htonll(value);
> -    bitwise_copy(&n_value, 8, 0, sf->value, sf->field->n_bytes, ofs,
> n_bits);
> -    bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs,
> n_bits);
> -}
> -
> -static void
> -put_move(enum mf_field_id src, int src_ofs,
> -         enum mf_field_id dst, int dst_ofs,
> -         int n_bits,
> -         struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_reg_move *move = ofpact_put_REG_MOVE(ofpacts);
> -    move->src.field = mf_from_id(src);
> -    move->src.ofs = src_ofs;
> -    move->src.n_bits = n_bits;
> -    move->dst.field = mf_from_id(dst);
> -    move->dst.ofs = dst_ofs;
> -    move->dst.n_bits = n_bits;
> -}
> -
> -static void
> -put_resubmit(uint8_t table_id, struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(ofpacts);
> -    resubmit->in_port = OFPP_IN_PORT;
> -    resubmit->table_id = table_id;
> -}
> -
> -/*
> - * For a port binding, get the corresponding ovn-chassis-id tunnel port
> - * from the associated encap.
> - */
> -static struct chassis_tunnel *
> -get_port_binding_tun(const struct sbrec_port_binding *binding)
> -{
> -    struct sbrec_encap *encap = binding->encap;
> -    struct sbrec_chassis *chassis = binding->chassis;
> -    struct chassis_tunnel *tun = NULL;
> -
> -    if (encap) {
> -        tun = chassis_tunnel_find(chassis->name, encap->ip);
> -    }
> -    if (!tun) {
> -        tun = chassis_tunnel_find(chassis->name, NULL);
> -    }
> -    return tun;
> -}
> -
> -static void
> -put_encapsulation(enum mf_field_id mff_ovn_geneve,
> -                  const struct chassis_tunnel *tun,
> -                  const struct sbrec_datapath_binding *datapath,
> -                  uint16_t outport, struct ofpbuf *ofpacts)
> -{
> -    if (tun->type == GENEVE) {
> -        put_load(datapath->tunnel_key, MFF_TUN_ID, 0, 24, ofpacts);
> -        put_load(outport, mff_ovn_geneve, 0, 32, ofpacts);
> -        put_move(MFF_LOG_INPORT, 0, mff_ovn_geneve, 16, 15, ofpacts);
> -    } else if (tun->type == STT) {
> -        put_load(datapath->tunnel_key | ((uint64_t) outport << 24),
> -                 MFF_TUN_ID, 0, 64, ofpacts);
> -        put_move(MFF_LOG_INPORT, 0, MFF_TUN_ID, 40, 15, ofpacts);
> -    } else if (tun->type == VXLAN) {
> -        put_load(datapath->tunnel_key, MFF_TUN_ID, 0, 24, ofpacts);
> -    } else {
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -put_stack(enum mf_field_id field, struct ofpact_stack *stack)
> -{
> -    stack->subfield.field = mf_from_id(field);
> -    stack->subfield.ofs = 0;
> -    stack->subfield.n_bits = stack->subfield.field->n_bits;
> -}
> -
> -static const struct sbrec_port_binding *
> -get_localnet_port(const struct hmap *local_datapaths, int64_t tunnel_key)
> -{
> -    const struct local_datapath *ld = get_local_datapath(local_datapaths,
> -                                                         tunnel_key);
> -    return ld ? ld->localnet_port : NULL;
> -}
> -
> -/* Datapath zone IDs for connection tracking and NAT */
> -struct zone_ids {
> -    int ct;                     /* MFF_LOG_CT_ZONE. */
> -    int dnat;                   /* MFF_LOG_DNAT_ZONE. */
> -    int snat;                   /* MFF_LOG_SNAT_ZONE. */
> -};
> -
> -static struct zone_ids
> -get_zone_ids(const struct sbrec_port_binding *binding,
> -             const struct simap *ct_zones)
> -{
> -    struct zone_ids zone_ids;
> -
> -    zone_ids.ct = simap_get(ct_zones, binding->logical_port);
> -
> -    const struct uuid *key = &binding->datapath->header_.uuid;
> -
> -    char *dnat = alloc_nat_zone_key(key, "dnat");
> -    zone_ids.dnat = simap_get(ct_zones, dnat);
> -    free(dnat);
> -
> -    char *snat = alloc_nat_zone_key(key, "snat");
> -    zone_ids.snat = simap_get(ct_zones, snat);
> -    free(snat);
> -
> -    return zone_ids;
> -}
> -
> -static void
> -put_replace_router_port_mac_flows(const struct
> -                                  sbrec_port_binding *localnet_port,
> -                                  const struct sbrec_chassis *chassis,
> -                                  const struct hmap *local_datapaths,
> -                                  struct ofpbuf *ofpacts_p,
> -                                  ofp_port_t ofport,
> -                                  struct ovn_desired_flow_table
> *flow_table)
> -{
> -    struct local_datapath *ld = get_local_datapath(local_datapaths,
> -
>  localnet_port->datapath->
> -                                                   tunnel_key);
> -    ovs_assert(ld);
> -
> -    uint32_t dp_key = localnet_port->datapath->tunnel_key;
> -    uint32_t port_key = localnet_port->tunnel_key;
> -    int tag = localnet_port->tag ? *localnet_port->tag : 0;
> -    const char *network = smap_get(&localnet_port->options,
> "network_name");
> -    struct eth_addr chassis_mac;
> -
> -    if (!network) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "Physical network not configured for datapath:"
> -                     "%"PRId64" with localnet port",
> -                     localnet_port->datapath->tunnel_key);
> -        return;
> -    }
> -
> -    /* Get chassis mac */
> -    if (!chassis_get_mac(chassis, network, &chassis_mac)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        /* Keeping the log level low for backward compatibility.
> -         * Chassis mac is a new configuration.
> -         */
> -        VLOG_DBG_RL(&rl, "Could not get chassis mac for network: %s",
> network);
> -        return;
> -    }
> -
> -    for (int i = 0; i < ld->n_peer_ports; i++) {
> -        const struct sbrec_port_binding *rport_binding =
> ld->peer_ports[i];
> -        struct eth_addr router_port_mac;
> -        struct match match;
> -        struct ofpact_mac *replace_mac;
> -
> -        /* Table 65, priority 150.
> -         * =======================
> -         *
> -         * Implements output to localnet port.
> -         * a. Flow replaces ingress router port mac with a chassis mac.
> -         * b. Flow appends the vlan id localnet port is configured with.
> -         */
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -
> -        ovs_assert(rport_binding->n_mac == 1);
> -        char *err_str = str_to_mac(rport_binding->mac[0],
> &router_port_mac);
> -        if (err_str) {
> -            /* Parsing of mac failed. */
> -            VLOG_WARN("Parsing or router port mac failed for router port:
> %s, "
> -                      "with error: %s", rport_binding->logical_port,
> err_str);
> -            free(err_str);
> -            return;
> -        }
> -
> -        /* Replace Router mac flow */
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -        match_set_dl_src(&match, router_port_mac);
> -
> -        replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p);
> -        replace_mac->mac = chassis_mac;
> -
> -        if (tag) {
> -            struct ofpact_vlan_vid *vlan_vid;
> -            vlan_vid = ofpact_put_SET_VLAN_VID(ofpacts_p);
> -            vlan_vid->vlan_vid = tag;
> -            vlan_vid->push_vlan_if_needed = true;
> -        }
> -
> -        ofpact_put_OUTPUT(ofpacts_p)->port = ofport;
> -
> -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 150, 0,
> -                        &match, ofpacts_p, &localnet_port->header_.uuid);
> -    }
> -}
> -
> -static void
> -put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> -                       uint32_t parent_port_key,
> -                       const struct zone_ids *zone_ids,
> -                       struct ofpbuf *ofpacts_p,
> -                       struct ovn_desired_flow_table *flow_table)
> -{
> -    struct match match;
> -
> -    /* Table 33, priority 100.
> -     * =======================
> -     *
> -     * Implements output to local hypervisor.  Each flow matches a
> -     * logical output port on the local hypervisor, and resubmits to
> -     * table 34.
> -     */
> -
> -    match_init_catchall(&match);
> -    ofpbuf_clear(ofpacts_p);
> -
> -    /* Match MFF_LOG_DATAPATH, MFF_LOG_OUTPORT. */
> -    match_set_metadata(&match, htonll(dp_key));
> -    match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -
> -    if (zone_ids) {
> -        if (zone_ids->ct) {
> -            put_load(zone_ids->ct, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p);
> -        }
> -        if (zone_ids->dnat) {
> -            put_load(zone_ids->dnat, MFF_LOG_DNAT_ZONE, 0, 32, ofpacts_p);
> -        }
> -        if (zone_ids->snat) {
> -            put_load(zone_ids->snat, MFF_LOG_SNAT_ZONE, 0, 32, ofpacts_p);
> -        }
> -    }
> -
> -    /* Resubmit to table 34. */
> -    put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> -    ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100, 0,
> -                    &match, ofpacts_p, hc_uuid);
> -
> -    /* Table 34, Priority 100.
> -     * =======================
> -     *
> -     * Drop packets whose logical inport and outport are the same
> -     * and the MLF_ALLOW_LOOPBACK flag is not set. */
> -    match_init_catchall(&match);
> -    ofpbuf_clear(ofpacts_p);
> -    match_set_metadata(&match, htonll(dp_key));
> -    match_set_reg_masked(&match, MFF_LOG_FLAGS - MFF_REG0,
> -                         0, MLF_ALLOW_LOOPBACK);
> -    match_set_reg(&match, MFF_LOG_INPORT - MFF_REG0, port_key);
> -    match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -    ofctrl_add_flow(flow_table, OFTABLE_CHECK_LOOPBACK, 100, 0,
> -                    &match, ofpacts_p, hc_uuid);
> -
> -    /* Table 64, Priority 100.
> -     * =======================
> -     *
> -     * If the packet is supposed to hair-pin because the
> -     *   - "loopback" flag is set
> -     *   - or if the destination is a nested container
> -     *   - or if "nested_container" flag is set and the destination is the
> -     *     parent port,
> -     * temporarily set the in_port to zero, resubmit to
> -     * table 65 for logical-to-physical translation, then restore
> -     * the port number.
> -     *
> -     * If 'parent_port_key' is set, then the 'port_key' represents a
> nested
> -     * container. */
> -
> -    bool nested_container = parent_port_key ? true: false;
> -    match_init_catchall(&match);
> -    ofpbuf_clear(ofpacts_p);
> -    match_set_metadata(&match, htonll(dp_key));
> -    match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -    if (!nested_container) {
> -        match_set_reg_masked(&match, MFF_LOG_FLAGS - MFF_REG0,
> -                             MLF_ALLOW_LOOPBACK, MLF_ALLOW_LOOPBACK);
> -    }
> -
> -    put_stack(MFF_IN_PORT, ofpact_put_STACK_PUSH(ofpacts_p));
> -    put_load(0, MFF_IN_PORT, 0, 16, ofpacts_p);
> -    put_resubmit(OFTABLE_LOG_TO_PHY, ofpacts_p);
> -    put_stack(MFF_IN_PORT, ofpact_put_STACK_POP(ofpacts_p));
> -    ofctrl_add_flow(flow_table, OFTABLE_SAVE_INPORT, 100, 0,
> -                    &match, ofpacts_p, hc_uuid);
> -
> -    if (nested_container) {
> -        /* It's a nested container and when the packet from the nested
> -         * container is to be sent to the parent port, "nested_container"
> -         * flag will be set. We need to temporarily set the in_port to
> zero
> -         * as mentioned in the comment above.
> -         *
> -         * If a parent port has multiple child ports, then this if
> condition
> -         * will be hit multiple times, but we want to add only one flow.
> -         * ofctrl_add_flow() logs a warning message for duplicate flows.
> -         * So use the function 'ofctrl_check_and_add_flow' which doesn't
> -         * log a warning.
> -         *
> -         * Other option is to add this flow for all the ports which are
> not
> -         * nested containers. In which case we will add this flow for all
> the
> -         * ports even if they don't have any child ports which is
> -         * unnecessary.
> -         */
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0,
> parent_port_key);
> -        match_set_reg_masked(&match, MFF_LOG_FLAGS - MFF_REG0,
> -                             MLF_NESTED_CONTAINER, MLF_NESTED_CONTAINER);
> -
> -        put_stack(MFF_IN_PORT, ofpact_put_STACK_PUSH(ofpacts_p));
> -        put_load(0, MFF_IN_PORT, 0, 16, ofpacts_p);
> -        put_resubmit(OFTABLE_LOG_TO_PHY, ofpacts_p);
> -        put_stack(MFF_IN_PORT, ofpact_put_STACK_POP(ofpacts_p));
> -        ofctrl_check_and_add_flow(flow_table, OFTABLE_SAVE_INPORT, 100, 0,
> -                                  &match, ofpacts_p, hc_uuid, false);
> -    }
> -}
> -
> -static void
> -load_logical_ingress_metadata(const struct sbrec_port_binding *binding,
> -                              const struct zone_ids *zone_ids,
> -                              struct ofpbuf *ofpacts_p)
> -{
> -    if (zone_ids) {
> -        if (zone_ids->ct) {
> -            put_load(zone_ids->ct, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p);
> -        }
> -        if (zone_ids->dnat) {
> -            put_load(zone_ids->dnat, MFF_LOG_DNAT_ZONE, 0, 32, ofpacts_p);
> -        }
> -        if (zone_ids->snat) {
> -            put_load(zone_ids->snat, MFF_LOG_SNAT_ZONE, 0, 32, ofpacts_p);
> -        }
> -    }
> -
> -    /* Set MFF_LOG_DATAPATH and MFF_LOG_INPORT. */
> -    uint32_t dp_key = binding->datapath->tunnel_key;
> -    uint32_t port_key = binding->tunnel_key;
> -    put_load(dp_key, MFF_LOG_DATAPATH, 0, 64, ofpacts_p);
> -    put_load(port_key, MFF_LOG_INPORT, 0, 32, ofpacts_p);
> -}
> -
> -static void
> -consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                      enum mf_field_id mff_ovn_geneve,
> -                      const struct simap *ct_zones,
> -                      const struct sset *active_tunnels,
> -                      const struct hmap *local_datapaths,
> -                      const struct sbrec_port_binding *binding,
> -                      const struct sbrec_chassis *chassis,
> -                      struct ovn_desired_flow_table *flow_table,
> -                      struct ofpbuf *ofpacts_p)
> -{
> -    uint32_t dp_key = binding->datapath->tunnel_key;
> -    uint32_t port_key = binding->tunnel_key;
> -    if (!get_local_datapath(local_datapaths, dp_key)) {
> -        return;
> -    }
> -
> -    struct match match;
> -    if (!strcmp(binding->type, "patch")
> -        || (!strcmp(binding->type, "l3gateway")
> -            && binding->chassis == chassis)) {
> -        const char *peer_name = smap_get(&binding->options, "peer");
> -        if (!peer_name) {
> -            return;
> -        }
> -
> -        const struct sbrec_port_binding *peer = lport_lookup_by_name(
> -            sbrec_port_binding_by_name, peer_name);
> -        if (!peer || strcmp(peer->type, binding->type)) {
> -            return;
> -        }
> -        const char *peer_peer_name = smap_get(&peer->options, "peer");
> -        if (!peer_peer_name || strcmp(peer_peer_name,
> binding->logical_port)) {
> -            return;
> -        }
> -
> -        struct zone_ids binding_zones = get_zone_ids(binding, ct_zones);
> -        put_local_common_flows(dp_key, port_key, 0, &binding_zones,
> -                               ofpacts_p, flow_table);
> -
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -
> -        size_t clone_ofs = ofpacts_p->size;
> -        struct ofpact_nest *clone = ofpact_put_CLONE(ofpacts_p);
> -        ofpact_put_CT_CLEAR(ofpacts_p);
> -        put_load(0, MFF_LOG_DNAT_ZONE, 0, 32, ofpacts_p);
> -        put_load(0, MFF_LOG_SNAT_ZONE, 0, 32, ofpacts_p);
> -        put_load(0, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p);
> -        struct zone_ids peer_zones = get_zone_ids(peer, ct_zones);
> -        load_logical_ingress_metadata(peer, &peer_zones, ofpacts_p);
> -        put_load(0, MFF_LOG_FLAGS, 0, 32, ofpacts_p);
> -        put_load(0, MFF_LOG_OUTPORT, 0, 32, ofpacts_p);
> -        for (int i = 0; i < MFF_N_LOG_REGS; i++) {
> -            put_load(0, MFF_LOG_REG0 + i, 0, 32, ofpacts_p);
> -        }
> -        put_load(0, MFF_IN_PORT, 0, 16, ofpacts_p);
> -        put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
> -        clone = ofpbuf_at_assert(ofpacts_p, clone_ofs, sizeof *clone);
> -        ofpacts_p->header = clone;
> -        ofpact_finish_CLONE(ofpacts_p, &clone);
> -
> -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 100, 0,
> -                        &match, ofpacts_p, &binding->header_.uuid);
> -        return;
> -    }
> -
> -    struct ha_chassis_ordered *ha_ch_ordered
> -        = ha_chassis_get_ordered(binding->ha_chassis_group);
> -
> -    if (!strcmp(binding->type, "chassisredirect")
> -        && (binding->chassis == chassis
> -            || ha_chassis_group_is_active(binding->ha_chassis_group,
> -                                          active_tunnels, chassis))) {
> -
> -        /* Table 33, priority 100.
> -         * =======================
> -         *
> -         * Implements output to local hypervisor.  Each flow matches a
> -         * logical output port on the local hypervisor, and resubmits to
> -         * table 34.  For ports of type "chassisredirect", the logical
> -         * output port is changed from the "chassisredirect" port to the
> -         * underlying distributed port. */
> -
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -
> -        const char *distributed_port = smap_get_def(&binding->options,
> -                                                    "distributed-port",
> "");
> -        const struct sbrec_port_binding *distributed_binding
> -            = lport_lookup_by_name(sbrec_port_binding_by_name,
> -                                   distributed_port);
> -
> -        if (!distributed_binding) {
> -            /* Packet will be dropped. */
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_WARN_RL(&rl, "No port binding record for distributed "
> -                         "port %s referred by chassisredirect port %s",
> -                         distributed_port,
> -                         binding->logical_port);
> -        } else if (binding->datapath !=
> -                   distributed_binding->datapath) {
> -            /* Packet will be dropped. */
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_WARN_RL(&rl,
> -                         "chassisredirect port %s refers to "
> -                         "distributed port %s in wrong datapath",
> -                         binding->logical_port,
> -                         distributed_port);
> -        } else {
> -            put_load(distributed_binding->tunnel_key,
> -                     MFF_LOG_OUTPORT, 0, 32, ofpacts_p);
> -
> -            struct zone_ids zone_ids = get_zone_ids(distributed_binding,
> -                                                    ct_zones);
> -            if (zone_ids.ct) {
> -                put_load(zone_ids.ct, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p);
> -            }
> -            if (zone_ids.dnat) {
> -                put_load(zone_ids.dnat, MFF_LOG_DNAT_ZONE, 0, 32,
> ofpacts_p);
> -            }
> -            if (zone_ids.snat) {
> -                put_load(zone_ids.snat, MFF_LOG_SNAT_ZONE, 0, 32,
> ofpacts_p);
> -            }
> -
> -            /* Resubmit to table 34. */
> -            put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> -        }
> -
> -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100, 0,
> -                        &match, ofpacts_p, &binding->header_.uuid);
> -
> -        goto out;
> -    }
> -
> -    /* Find the OpenFlow port for the logical port, as 'ofport'.  This is
> -     * one of:
> -     *
> -     *     - If the port is a VIF on the chassis we're managing, the
> -     *       OpenFlow port for the VIF.  'tun' will be NULL.
> -     *
> -     *       The same logic handles ports that OVN implements as Open
> vSwitch
> -     *       patch ports, that is, "localnet" and "l2gateway" ports.
> -     *
> -     *       For a container nested inside a VM and accessible via a VLAN,
> -     *       'tag' is the VLAN ID; otherwise 'tag' is 0.
> -     *
> -     *       For a localnet or l2gateway patch port, if a VLAN ID was
> -     *       configured, 'tag' is set to that VLAN ID; otherwise 'tag' is
> 0.
> -     *
> -     *     - If the port is on a remote chassis, the OpenFlow port for a
> -     *       tunnel to the VIF's remote chassis.  'tun' identifies that
> -     *       tunnel.
> -     */
> -
> -    int tag = 0;
> -    bool nested_container = false;
> -    const struct sbrec_port_binding *parent_port = NULL;
> -    ofp_port_t ofport;
> -    bool is_remote = false;
> -    if (binding->parent_port && *binding->parent_port) {
> -        if (!binding->tag) {
> -            goto out;
> -        }
> -        ofport = u16_to_ofp(simap_get(&localvif_to_ofport,
> -                                      binding->parent_port));
> -        if (ofport) {
> -            tag = *binding->tag;
> -            nested_container = true;
> -            parent_port = lport_lookup_by_name(
> -                sbrec_port_binding_by_name, binding->parent_port);
> -        }
> -    } else {
> -        ofport = u16_to_ofp(simap_get(&localvif_to_ofport,
> -                                      binding->logical_port));
> -        const char *requested_chassis = smap_get(&binding->options,
> -                                                 "requested-chassis");
> -        if (ofport && requested_chassis && requested_chassis[0] &&
> -            strcmp(requested_chassis, chassis->name) &&
> -            strcmp(requested_chassis, chassis->hostname)) {
> -            /* Even though there is an ofport for this port_binding, it is
> -             * requested on a different chassis. So ignore this ofport.
> -             */
> -            ofport = 0;
> -        }
> -
> -        if ((!strcmp(binding->type, "localnet")
> -            || !strcmp(binding->type, "l2gateway"))
> -            && ofport && binding->tag) {
> -            tag = *binding->tag;
> -        }
> -    }
> -
> -    bool is_ha_remote = false;
> -    const struct chassis_tunnel *tun = NULL;
> -    const struct sbrec_port_binding *localnet_port =
> -        get_localnet_port(local_datapaths, dp_key);
> -    if (!ofport) {
> -        /* It is remote port, may be reached by tunnel or localnet port */
> -        is_remote = true;
> -        if (localnet_port) {
> -            ofport = u16_to_ofp(simap_get(&localvif_to_ofport,
> -                                          localnet_port->logical_port));
> -            if (!ofport) {
> -                goto out;
> -            }
> -        } else {
> -            if (!ha_ch_ordered || ha_ch_ordered->n_ha_ch < 2) {
> -                /* It's on a single remote chassis */
> -                if (!binding->chassis) {
> -                    goto out;
> -                }
> -                tun = chassis_tunnel_find(binding->chassis->name, NULL);
> -                if (!tun) {
> -                    goto out;
> -                }
> -                ofport = tun->ofport;
> -            } else {
> -                /* It's distributed across the chassis belonging to
> -                 * an HA chassis group. */
> -                is_ha_remote = true;
> -            }
> -        }
> -    }
> -
> -    if (!is_remote) {
> -        /* Packets that arrive from a vif can belong to a VM or
> -         * to a container located inside that VM. Packets that
> -         * arrive from containers have a tag (vlan) associated with them.
> -         */
> -
> -        struct zone_ids zone_ids = get_zone_ids(binding, ct_zones);
> -        uint32_t parent_port_key = parent_port ? parent_port->tunnel_key
> : 0;
> -        /* Pass the parent port tunnel key if the port is a nested
> -         * container. */
> -        put_local_common_flows(dp_key, port_key, parent_port_key,
> &zone_ids,
> -                               ofpacts_p, flow_table);
> -
> -        /* Table 0, Priority 150 and 100.
> -         * ==============================
> -         *
> -         * Priority 150 is for tagged traffic. This may be containers in a
> -         * VM or a VLAN on a local network. For such traffic, match on the
> -         * tags and then strip the tag.
> -         *
> -         * Priority 100 is for traffic belonging to VMs or untagged
> locally
> -         * connected networks.
> -         *
> -         * For both types of traffic: set MFF_LOG_INPORT to the logical
> -         * input port, MFF_LOG_DATAPATH to the logical datapath, and
> -         * resubmit into the logical ingress pipeline starting at table
> -         * 16. */
> -        ofpbuf_clear(ofpacts_p);
> -        match_init_catchall(&match);
> -        match_set_in_port(&match, ofport);
> -
> -        /* Match a VLAN tag and strip it, including stripping priority
> tags
> -         * (e.g. VLAN ID 0).  In the latter case we'll add a second flow
> -         * for frames that lack any 802.1Q header later. */
> -        if (tag || !strcmp(binding->type, "localnet")
> -            || !strcmp(binding->type, "l2gateway")) {
> -            match_set_dl_vlan(&match, htons(tag), 0);
> -            if (nested_container) {
> -                /* When a packet comes from a container sitting behind a
> -                 * parent_port, we should let it loopback to other
> containers
> -                 * or the parent_port itself. Indicate this by setting the
> -                 * MLF_NESTED_CONTAINER_BIT in MFF_LOG_FLAGS.*/
> -                put_load(1, MFF_LOG_FLAGS, MLF_NESTED_CONTAINER_BIT, 1,
> -                         ofpacts_p);
> -            }
> -            ofpact_put_STRIP_VLAN(ofpacts_p);
> -        }
> -
> -        /* Remember the size with just strip vlan added so far,
> -         * as we're going to remove this with ofpbuf_pull() later. */
> -        uint32_t ofpacts_orig_size = ofpacts_p->size;
> -
> -        load_logical_ingress_metadata(binding, &zone_ids, ofpacts_p);
> -
> -        /* Resubmit to first logical ingress pipeline table. */
> -        put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
> -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG,
> -                        tag ? 150 : 100, 0, &match, ofpacts_p,
> -                        &binding->header_.uuid);
> -
> -        if (!tag && (!strcmp(binding->type, "localnet")
> -                     || !strcmp(binding->type, "l2gateway"))) {
> -
> -            /* Add a second flow for frames that lack any 802.1Q
> -             * header.  For these, drop the OFPACT_STRIP_VLAN
> -             * action. */
> -            ofpbuf_pull(ofpacts_p, ofpacts_orig_size);
> -            match_set_dl_tci_masked(&match, 0, htons(VLAN_CFI));
> -            ofctrl_add_flow(flow_table, 0, 100, 0, &match, ofpacts_p,
> -                            &binding->header_.uuid);
> -        }
> -
> -        /* Table 65, Priority 100.
> -         * =======================
> -         *
> -         * Deliver the packet to the local vif. */
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -        if (tag) {
> -            /* For containers sitting behind a local vif, tag the packets
> -             * before delivering them. */
> -            struct ofpact_vlan_vid *vlan_vid;
> -            vlan_vid = ofpact_put_SET_VLAN_VID(ofpacts_p);
> -            vlan_vid->vlan_vid = tag;
> -            vlan_vid->push_vlan_if_needed = true;
> -        }
> -        ofpact_put_OUTPUT(ofpacts_p)->port = ofport;
> -        if (tag) {
> -            /* Revert the tag added to the packets headed to containers
> -             * in the previous step. If we don't do this, the packets
> -             * that are to be broadcasted to a VM in the same logical
> -             * switch will also contain the tag. */
> -            ofpact_put_STRIP_VLAN(ofpacts_p);
> -        }
> -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 100, 0,
> -                        &match, ofpacts_p, &binding->header_.uuid);
> -
> -        if (!strcmp(binding->type, "localnet")) {
> -            put_replace_router_port_mac_flows(binding, chassis,
> -                                              local_datapaths, ofpacts_p,
> -                                              ofport, flow_table);
> -        }
> -
> -    } else if (!tun && !is_ha_remote) {
> -        /* Remote port connected by localnet port */
> -        /* Table 33, priority 100.
> -         * =======================
> -         *
> -         * Implements switching to localnet port. Each flow matches a
> -         * logical output port on remote hypervisor, switch the output
> port
> -         * to connected localnet port and resubmits to same table.
> -         */
> -
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -
> -        /* Match MFF_LOG_DATAPATH, MFF_LOG_OUTPORT. */
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -
> -        put_load(localnet_port->tunnel_key, MFF_LOG_OUTPORT, 0, 32,
> ofpacts_p);
> -
> -        /* Resubmit to table 33. */
> -        put_resubmit(OFTABLE_LOCAL_OUTPUT, ofpacts_p);
> -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100, 0,
> -                        &match, ofpacts_p, &binding->header_.uuid);
> -    } else {
> -        /* Remote port connected by tunnel */
> -
> -        /* Table 32, priority 100.
> -         * =======================
> -         *
> -         * Handles traffic that needs to be sent to a remote hypervisor.
> Each
> -         * flow matches an output port that includes a logical port on a
> remote
> -         * hypervisor, and tunnels the packet to that hypervisor.
> -         */
> -        match_init_catchall(&match);
> -        ofpbuf_clear(ofpacts_p);
> -
> -        /* Match MFF_LOG_DATAPATH, MFF_LOG_OUTPORT. */
> -        match_set_metadata(&match, htonll(dp_key));
> -        match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> -
> -        if (!is_ha_remote) {
> -            /* Setup encapsulation */
> -            const struct chassis_tunnel *rem_tun =
> -                get_port_binding_tun(binding);
> -            if (!rem_tun) {
> -                goto out;
> -            }
> -            put_encapsulation(mff_ovn_geneve, tun, binding->datapath,
> -                              port_key, ofpacts_p);
> -            /* Output to tunnel. */
> -            ofpact_put_OUTPUT(ofpacts_p)->port = rem_tun->ofport;
> -        } else {
> -            /* Make sure all tunnel endpoints use the same encapsulation,
> -             * and set it up */
> -            for (size_t i = 0; i < ha_ch_ordered->n_ha_ch; i++) {
> -                const struct sbrec_chassis *ch =
> -                    ha_ch_ordered->ha_ch[i].chassis;
> -                if (!ch) {
> -                    continue;
> -                }
> -                if (!tun) {
> -                    tun = chassis_tunnel_find(ch->name, NULL);
> -                } else {
> -                    struct chassis_tunnel *chassis_tunnel =
> -                        chassis_tunnel_find(ch->name, NULL);
> -                    if (chassis_tunnel &&
> -                        tun->type != chassis_tunnel->type) {
> -                        static struct vlog_rate_limit rl =
> -                            VLOG_RATE_LIMIT_INIT(1, 1);
> -                        VLOG_ERR_RL(&rl, "Port %s has Gateway_Chassis "
> -                                            "with mixed encapsulations,
> only "
> -                                            "uniform encapsulations are "
> -                                            "supported.",
> -                                    binding->logical_port);
> -                        goto out;
> -                    }
> -                }
> -            }
> -            if (!tun) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_ERR_RL(&rl, "No tunnel endpoint found for HA chassis
> in "
> -                                 "HA chassis group of port %s",
> -                            binding->logical_port);
> -                goto out;
> -            }
> -
> -            put_encapsulation(mff_ovn_geneve, tun, binding->datapath,
> -                              port_key, ofpacts_p);
> -
> -            /* Output to tunnels with active/backup */
> -            struct ofpact_bundle *bundle = ofpact_put_BUNDLE(ofpacts_p);
> -
> -            for (size_t i = 0; i < ha_ch_ordered->n_ha_ch; i++) {
> -                const struct sbrec_chassis *ch =
> -                    ha_ch_ordered->ha_ch[i].chassis;
> -                if (!ch) {
> -                    continue;
> -                }
> -                tun = chassis_tunnel_find(ch->name, NULL);
> -                if (!tun) {
> -                    continue;
> -                }
> -                if (bundle->n_slaves >= BUNDLE_MAX_SLAVES) {
> -                    static struct vlog_rate_limit rl =
> -                            VLOG_RATE_LIMIT_INIT(1, 1);
> -                    VLOG_WARN_RL(&rl, "Remote endpoints for port beyond "
> -                                        "BUNDLE_MAX_SLAVES");
> -                    break;
> -                }
> -                ofpbuf_put(ofpacts_p, &tun->ofport,
> -                            sizeof tun->ofport);
> -                bundle = ofpacts_p->header;
> -                bundle->n_slaves++;
> -            }
> -
> -            bundle->algorithm = NX_BD_ALG_ACTIVE_BACKUP;
> -            /* Although ACTIVE_BACKUP bundle algorithm seems to ignore
> -             * the next two fields, those are always set */
> -            bundle->basis = 0;
> -            bundle->fields = NX_HASH_FIELDS_ETH_SRC;
> -            ofpact_finish_BUNDLE(ofpacts_p, &bundle);
> -        }
> -        ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100, 0,
> -                        &match, ofpacts_p, &binding->header_.uuid);
> -    }
> -out:
> -    if (ha_ch_ordered) {
> -        ha_chassis_destroy_ordered(ha_ch_ordered);
> -    }
> -}
> -
> -static void
> -consider_mc_group(enum mf_field_id mff_ovn_geneve,
> -                  const struct simap *ct_zones,
> -                  const struct hmap *local_datapaths,
> -                  const struct sbrec_chassis *chassis,
> -                  const struct sbrec_multicast_group *mc,
> -                  struct ovn_desired_flow_table *flow_table)
> -{
> -    uint32_t dp_key = mc->datapath->tunnel_key;
> -    if (!get_local_datapath(local_datapaths, dp_key)) {
> -        return;
> -    }
> -
> -    struct sset remote_chassis = SSET_INITIALIZER(&remote_chassis);
> -    struct match match;
> -
> -    match_init_catchall(&match);
> -    match_set_metadata(&match, htonll(dp_key));
> -    match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, mc->tunnel_key);
> -
> -    /* Go through all of the ports in the multicast group:
> -     *
> -     *    - For remote ports, add the chassis to 'remote_chassis'.
> -     *
> -     *    - For local ports (other than logical patch ports), add actions
> -     *      to 'ofpacts' to set the output port and resubmit.
> -     *
> -     *    - For logical patch ports, add actions to 'remote_ofpacts'
> -     *      instead.  (If we put them in 'ofpacts', then the output
> -     *      would happen on every hypervisor in the multicast group,
> -     *      effectively duplicating the packet.)
> -     */
> -    struct ofpbuf ofpacts;
> -    ofpbuf_init(&ofpacts, 0);
> -    struct ofpbuf remote_ofpacts;
> -    ofpbuf_init(&remote_ofpacts, 0);
> -    for (size_t i = 0; i < mc->n_ports; i++) {
> -        struct sbrec_port_binding *port = mc->ports[i];
> -
> -        if (port->datapath != mc->datapath) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, UUID_FMT": multicast group contains ports "
> -                         "in wrong datapath",
> -                         UUID_ARGS(&mc->header_.uuid));
> -            continue;
> -        }
> -
> -        int zone_id = simap_get(ct_zones, port->logical_port);
> -        if (zone_id) {
> -            put_load(zone_id, MFF_LOG_CT_ZONE, 0, 32, &ofpacts);
> -        }
> -
> -        if (!strcmp(port->type, "patch")) {
> -            put_load(port->tunnel_key, MFF_LOG_OUTPORT, 0, 32,
> -                     &remote_ofpacts);
> -            put_resubmit(OFTABLE_CHECK_LOOPBACK, &remote_ofpacts);
> -        } else if (simap_contains(&localvif_to_ofport,
> -                           (port->parent_port && *port->parent_port)
> -                           ? port->parent_port : port->logical_port)
> -                   || (!strcmp(port->type, "l3gateway")
> -                       && port->chassis == chassis)) {
> -            put_load(port->tunnel_key, MFF_LOG_OUTPORT, 0, 32, &ofpacts);
> -            put_resubmit(OFTABLE_CHECK_LOOPBACK, &ofpacts);
> -        } else if (port->chassis && !get_localnet_port(local_datapaths,
> -                                         mc->datapath->tunnel_key)) {
> -            /* Add remote chassis only when localnet port not exist,
> -             * otherwise multicast will reach remote ports through
> localnet
> -             * port. */
> -            sset_add(&remote_chassis, port->chassis->name);
> -        }
> -    }
> -
> -    /* Table 33, priority 100.
> -     * =======================
> -     *
> -     * Handle output to the local logical ports in the multicast group, if
> -     * any. */
> -    bool local_ports = ofpacts.size > 0;
> -    if (local_ports) {
> -        /* Following delivery to local logical ports, restore the
> multicast
> -         * group as the logical output port. */
> -        put_load(mc->tunnel_key, MFF_LOG_OUTPORT, 0, 32, &ofpacts);
> -
> -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100, 0,
> -                        &match, &ofpacts, &mc->header_.uuid);
> -    }
> -
> -    /* Table 32, priority 100.
> -     * =======================
> -     *
> -     * Handle output to the remote chassis in the multicast group, if
> -     * any. */
> -    if (!sset_is_empty(&remote_chassis) || remote_ofpacts.size > 0) {
> -        if (remote_ofpacts.size > 0) {
> -            /* Following delivery to logical patch ports, restore the
> -             * multicast group as the logical output port. */
> -            put_load(mc->tunnel_key, MFF_LOG_OUTPORT, 0, 32,
> -                     &remote_ofpacts);
> -        }
> -
> -        const char *chassis_name;
> -        const struct chassis_tunnel *prev = NULL;
> -        SSET_FOR_EACH (chassis_name, &remote_chassis) {
> -            const struct chassis_tunnel *tun
> -                = chassis_tunnel_find(chassis_name, NULL);
> -            if (!tun) {
> -                continue;
> -            }
> -
> -            if (!prev || tun->type != prev->type) {
> -                put_encapsulation(mff_ovn_geneve, tun, mc->datapath,
> -                                  mc->tunnel_key, &remote_ofpacts);
> -                prev = tun;
> -            }
> -            ofpact_put_OUTPUT(&remote_ofpacts)->port = tun->ofport;
> -        }
> -
> -        if (remote_ofpacts.size) {
> -            if (local_ports) {
> -                put_resubmit(OFTABLE_LOCAL_OUTPUT, &remote_ofpacts);
> -            }
> -            ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100, 0,
> -                            &match, &remote_ofpacts, &mc->header_.uuid);
> -        }
> -    }
> -    ofpbuf_uninit(&ofpacts);
> -    ofpbuf_uninit(&remote_ofpacts);
> -    sset_destroy(&remote_chassis);
> -}
> -
> -/* Replaces 'old' by 'new' (destroying 'new').  Returns true if 'old' and
> 'new'
> - * contained different data, false if they were the same. */
> -static bool
> -update_ofports(struct simap *old, struct simap *new)
> -{
> -    bool changed = !simap_equal(old, new);
> -    simap_swap(old, new);
> -    simap_destroy(new);
> -    return changed;
> -}
> -
> -void physical_handle_port_binding_changes(
> -        struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -        const struct sbrec_port_binding_table *pb_table,
> -        enum mf_field_id mff_ovn_geneve,
> -        const struct sbrec_chassis *chassis,
> -        const struct simap *ct_zones,
> -        struct hmap *local_datapaths,
> -        struct sset *active_tunnels,
> -        struct ovn_desired_flow_table *flow_table)
> -{
> -    const struct sbrec_port_binding *binding;
> -    struct ofpbuf ofpacts;
> -    ofpbuf_init(&ofpacts, 0);
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (binding, pb_table) {
> -        if (sbrec_port_binding_is_deleted(binding)) {
> -            ofctrl_remove_flows(flow_table, &binding->header_.uuid);
> -        } else {
> -            if (!sbrec_port_binding_is_new(binding)) {
> -                ofctrl_remove_flows(flow_table, &binding->header_.uuid);
> -            }
> -            consider_port_binding(sbrec_port_binding_by_name,
> -                                  mff_ovn_geneve, ct_zones,
> -                                  active_tunnels, local_datapaths,
> -                                  binding, chassis,
> -                                  flow_table, &ofpacts);
> -        }
> -    }
> -
> -}
> -
> -void
> -physical_handle_mc_group_changes(
> -        const struct sbrec_multicast_group_table *multicast_group_table,
> -        enum mf_field_id mff_ovn_geneve,
> -        const struct sbrec_chassis *chassis,
> -        const struct simap *ct_zones,
> -        const struct hmap *local_datapaths,
> -        struct ovn_desired_flow_table *flow_table)
> -{
> -    const struct sbrec_multicast_group *mc;
> -    SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_TRACKED (mc,
> multicast_group_table) {
> -        if (sbrec_multicast_group_is_deleted(mc)) {
> -            ofctrl_remove_flows(flow_table, &mc->header_.uuid);
> -        } else {
> -            if (!sbrec_multicast_group_is_new(mc)) {
> -                ofctrl_remove_flows(flow_table, &mc->header_.uuid);
> -            }
> -            consider_mc_group(mff_ovn_geneve, ct_zones, local_datapaths,
> -                              chassis, mc, flow_table);
> -        }
> -    }
> -}
> -
> -void
> -physical_run(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -             const struct sbrec_multicast_group_table
> *multicast_group_table,
> -             const struct sbrec_port_binding_table *port_binding_table,
> -             enum mf_field_id mff_ovn_geneve,
> -             const struct ovsrec_bridge *br_int,
> -             const struct sbrec_chassis *chassis,
> -             const struct simap *ct_zones,
> -             const struct hmap *local_datapaths,
> -             const struct sset *local_lports,
> -             const struct sset *active_tunnels,
> -             struct ovn_desired_flow_table *flow_table)
> -{
> -    if (!hc_uuid) {
> -        hc_uuid = xmalloc(sizeof(struct uuid));
> -        uuid_generate(hc_uuid);
> -    }
> -
> -    /* This bool tracks physical mapping changes. */
> -    bool physical_map_changed = false;
> -
> -    struct simap new_localvif_to_ofport =
> -        SIMAP_INITIALIZER(&new_localvif_to_ofport);
> -    struct simap new_tunnel_to_ofport =
> -        SIMAP_INITIALIZER(&new_tunnel_to_ofport);
> -    for (int i = 0; i < br_int->n_ports; i++) {
> -        const struct ovsrec_port *port_rec = br_int->ports[i];
> -        if (!strcmp(port_rec->name, br_int->name)) {
> -            continue;
> -        }
> -
> -        const char *tunnel_id = smap_get(&port_rec->external_ids,
> -                                         "ovn-chassis-id");
> -        if (tunnel_id &&
> -                encaps_tunnel_id_match(tunnel_id, chassis->name, NULL)) {
> -            continue;
> -        }
> -
> -        const char *localnet = smap_get(&port_rec->external_ids,
> -                                        "ovn-localnet-port");
> -        const char *l2gateway = smap_get(&port_rec->external_ids,
> -                                        "ovn-l2gateway-port");
> -
> -        for (int j = 0; j < port_rec->n_interfaces; j++) {
> -            const struct ovsrec_interface *iface_rec =
> port_rec->interfaces[j];
> -
> -            /* Get OpenFlow port number. */
> -            if (!iface_rec->n_ofport) {
> -                continue;
> -            }
> -            int64_t ofport = iface_rec->ofport[0];
> -            if (ofport < 1 || ofport > ofp_to_u16(OFPP_MAX)) {
> -                continue;
> -            }
> -
> -            /* Record as patch to local net, logical patch port, chassis,
> or
> -             * local logical port. */
> -            bool is_patch = !strcmp(iface_rec->type, "patch");
> -            if (is_patch && localnet) {
> -                /* localnet patch ports can be handled just like VIFs. */
> -                simap_put(&new_localvif_to_ofport, localnet, ofport);
> -                break;
> -            } else if (is_patch && l2gateway) {
> -                /* L2 gateway patch ports can be handled just like VIFs.
> */
> -                simap_put(&new_localvif_to_ofport, l2gateway, ofport);
> -                break;
> -            } else if (tunnel_id) {
> -                enum chassis_tunnel_type tunnel_type;
> -                if (!strcmp(iface_rec->type, "geneve")) {
> -                    tunnel_type = GENEVE;
> -                    if (!mff_ovn_geneve) {
> -                        continue;
> -                    }
> -                } else if (!strcmp(iface_rec->type, "stt")) {
> -                    tunnel_type = STT;
> -                } else if (!strcmp(iface_rec->type, "vxlan")) {
> -                    tunnel_type = VXLAN;
> -                } else {
> -                    continue;
> -                }
> -
> -                simap_put(&new_tunnel_to_ofport, tunnel_id, ofport);
> -                /*
> -                 * We split the tunnel_id to get the chassis-id
> -                 * and hash the tunnel list on the chassis-id. The
> -                 * reason to use the chassis-id alone is because
> -                 * there might be cases (multicast, gateway chassis)
> -                 * where we need to tunnel to the chassis, but won't
> -                 * have the encap-ip specifically.
> -                 */
> -                char *hash_id = NULL;
> -                char *ip = NULL;
> -
> -                if (!encaps_tunnel_id_parse(tunnel_id, &hash_id, &ip)) {
> -                    continue;
> -                }
> -                struct chassis_tunnel *tun = chassis_tunnel_find(hash_id,
> ip);
> -                if (tun) {
> -                    /* If the tunnel's ofport has changed, update. */
> -                    if (tun->ofport != u16_to_ofp(ofport) ||
> -                        tun->type != tunnel_type) {
> -                        tun->ofport = u16_to_ofp(ofport);
> -                        tun->type = tunnel_type;
> -                        physical_map_changed = true;
> -                    }
> -                } else {
> -                    tun = xmalloc(sizeof *tun);
> -                    hmap_insert(&tunnels, &tun->hmap_node,
> -                                hash_string(hash_id, 0));
> -                    tun->chassis_id = xstrdup(tunnel_id);
> -                    tun->ofport = u16_to_ofp(ofport);
> -                    tun->type = tunnel_type;
> -                    physical_map_changed = true;
> -                }
> -                free(hash_id);
> -                free(ip);
> -                break;
> -            } else {
> -                const char *iface_id = smap_get(&iface_rec->external_ids,
> -                                                "iface-id");
> -                if (iface_id) {
> -                    simap_put(&new_localvif_to_ofport, iface_id, ofport);
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Remove tunnels that are no longer here. */
> -    struct chassis_tunnel *tun, *tun_next;
> -    HMAP_FOR_EACH_SAFE (tun, tun_next, hmap_node, &tunnels) {
> -        if (!simap_find(&new_tunnel_to_ofport, tun->chassis_id)) {
> -            hmap_remove(&tunnels, &tun->hmap_node);
> -            physical_map_changed = true;
> -            free(tun->chassis_id);
> -            free(tun);
> -        }
> -    }
> -
> -    /* Capture changed or removed openflow ports. */
> -    physical_map_changed |= update_ofports(&localvif_to_ofport,
> -                                           &new_localvif_to_ofport);
> -    if (physical_map_changed) {
> -        /* Reprocess logical flow table immediately. */
> -        poll_immediate_wake();
> -    }
> -
> -    struct ofpbuf ofpacts;
> -    ofpbuf_init(&ofpacts, 0);
> -
> -    /* Set up flows in table 0 for physical-to-logical translation and in
> table
> -     * 64 for logical-to-physical translation. */
> -    const struct sbrec_port_binding *binding;
> -    SBREC_PORT_BINDING_TABLE_FOR_EACH (binding, port_binding_table) {
> -        consider_port_binding(sbrec_port_binding_by_name,
> -                              mff_ovn_geneve, ct_zones,
> -                              active_tunnels, local_datapaths,
> -                              binding, chassis,
> -                              flow_table, &ofpacts);
> -    }
> -
> -    /* Handle output to multicast groups, in tables 32 and 33. */
> -    const struct sbrec_multicast_group *mc;
> -    SBREC_MULTICAST_GROUP_TABLE_FOR_EACH (mc, multicast_group_table) {
> -        consider_mc_group(mff_ovn_geneve, ct_zones, local_datapaths,
> -                          chassis, mc, flow_table);
> -    }
> -
> -    /* Table 0, priority 100.
> -     * ======================
> -     *
> -     * Process packets that arrive from a remote hypervisor (by matching
> -     * on tunnel in_port). */
> -
> -    /* Add flows for Geneve and STT encapsulations.  These
> -     * encapsulations have metadata about the ingress and egress logical
> -     * ports.  We set MFF_LOG_DATAPATH, MFF_LOG_INPORT, and
> -     * MFF_LOG_OUTPORT from the tunnel key data, then resubmit to table
> -     * 33 to handle packets to the local hypervisor. */
> -    HMAP_FOR_EACH (tun, hmap_node, &tunnels) {
> -        struct match match = MATCH_CATCHALL_INITIALIZER;
> -        match_set_in_port(&match, tun->ofport);
> -
> -        ofpbuf_clear(&ofpacts);
> -        if (tun->type == GENEVE) {
> -            put_move(MFF_TUN_ID, 0,  MFF_LOG_DATAPATH, 0, 24, &ofpacts);
> -            put_move(mff_ovn_geneve, 16, MFF_LOG_INPORT, 0, 15,
> -                     &ofpacts);
> -            put_move(mff_ovn_geneve, 0, MFF_LOG_OUTPORT, 0, 16,
> -                     &ofpacts);
> -        } else if (tun->type == STT) {
> -            put_move(MFF_TUN_ID, 40, MFF_LOG_INPORT,   0, 15, &ofpacts);
> -            put_move(MFF_TUN_ID, 24, MFF_LOG_OUTPORT,  0, 16, &ofpacts);
> -            put_move(MFF_TUN_ID,  0, MFF_LOG_DATAPATH, 0, 24, &ofpacts);
> -        } else if (tun->type == VXLAN) {
> -            /* We'll handle VXLAN later. */
> -            continue;
> -        } else {
> -            OVS_NOT_REACHED();
> -        }
> -
> -        put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> -
> -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 100, 0, &match,
> -                        &ofpacts, hc_uuid);
> -    }
> -
> -    /* Add flows for VXLAN encapsulations.  Due to the limited amount of
> -     * metadata, we only support VXLAN for connections to gateways.  The
> -     * VNI is used to populate MFF_LOG_DATAPATH.  The gateway's logical
> -     * port is set to MFF_LOG_INPORT.  Then the packet is resubmitted to
> -     * table 16 to determine the logical egress port. */
> -    HMAP_FOR_EACH (tun, hmap_node, &tunnels) {
> -        if (tun->type != VXLAN) {
> -            continue;
> -        }
> -
> -        SBREC_PORT_BINDING_TABLE_FOR_EACH (binding, port_binding_table) {
> -            struct match match = MATCH_CATCHALL_INITIALIZER;
> -
> -            if (!binding->chassis ||
> -                !encaps_tunnel_id_match(tun->chassis_id,
> -                                        binding->chassis->name, NULL)) {
> -                continue;
> -            }
> -
> -            match_set_in_port(&match, tun->ofport);
> -            match_set_tun_id(&match,
> htonll(binding->datapath->tunnel_key));
> -
> -            ofpbuf_clear(&ofpacts);
> -            put_move(MFF_TUN_ID, 0,  MFF_LOG_DATAPATH, 0, 24, &ofpacts);
> -            put_load(binding->tunnel_key, MFF_LOG_INPORT, 0, 15,
> &ofpacts);
> -            /* For packets received from a vxlan tunnel, set a flag to
> that
> -             * effect. */
> -            put_load(1, MFF_LOG_FLAGS, MLF_RCV_FROM_VXLAN_BIT, 1,
> &ofpacts);
> -            put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, &ofpacts);
> -
> -            ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 100, 0,
> &match,
> -                            &ofpacts, hc_uuid);
> -        }
> -    }
> -
> -    /* Table 32, priority 150.
> -     * =======================
> -     *
> -     * Handles packets received from a VXLAN tunnel which get resubmitted
> to
> -     * OFTABLE_LOG_INGRESS_PIPELINE due to lack of needed metadata in
> VXLAN,
> -     * explicitly skip sending back out any tunnels and resubmit to table
> 33
> -     * for local delivery.
> -     */
> -    struct match match;
> -    match_init_catchall(&match);
> -    match_set_reg_masked(&match, MFF_LOG_FLAGS - MFF_REG0,
> -                         MLF_RCV_FROM_VXLAN, MLF_RCV_FROM_VXLAN);
> -
> -    /* Resubmit to table 33. */
> -    ofpbuf_clear(&ofpacts);
> -    put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150, 0,
> -                    &match, &ofpacts, hc_uuid);
> -
> -    /* Table 32, priority 150.
> -     * =======================
> -     *
> -     * Packets that should not be sent to other hypervisors.
> -     */
> -    match_init_catchall(&match);
> -    match_set_reg_masked(&match, MFF_LOG_FLAGS - MFF_REG0,
> -                         MLF_LOCAL_ONLY, MLF_LOCAL_ONLY);
> -    /* Resubmit to table 33. */
> -    ofpbuf_clear(&ofpacts);
> -    put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150, 0,
> -                    &match, &ofpacts, hc_uuid);
> -
> -    /* Table 32, priority 150.
> -     * =======================
> -     *
> -     * Handles packets received from ports of type "localport".  These
> ports
> -     * are present on every hypervisor.  Traffic that originates at one
> should
> -     * never go over a tunnel to a remote hypervisor, so resubmit them to
> table
> -     * 33 for local delivery. */
> -    match_init_catchall(&match);
> -    ofpbuf_clear(&ofpacts);
> -    put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> -    const char *localport;
> -    SSET_FOR_EACH (localport, local_lports) {
> -        /* Iterate over all local logical ports and insert a drop
> -         * rule with higher priority for every localport in this
> -         * datapath. */
> -        const struct sbrec_port_binding *pb = lport_lookup_by_name(
> -            sbrec_port_binding_by_name, localport);
> -        if (pb && !strcmp(pb->type, "localport")) {
> -            match_set_reg(&match, MFF_LOG_INPORT - MFF_REG0,
> pb->tunnel_key);
> -            match_set_metadata(&match, htonll(pb->datapath->tunnel_key));
> -            ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150, 0,
> -                            &match, &ofpacts, hc_uuid);
> -        }
> -    }
> -
> -    /* Table 32, Priority 0.
> -     * =======================
> -     *
> -     * Resubmit packets that are not directed at tunnels or part of a
> -     * multicast group to the local output table. */
> -    match_init_catchall(&match);
> -    ofpbuf_clear(&ofpacts);
> -    put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 0, 0, &match,
> &ofpacts,
> -                    hc_uuid);
> -
> -    /* Table 34, Priority 0.
> -     * =======================
> -     *
> -     * Resubmit packets that don't output to the ingress port (already
> checked
> -     * in table 33) to the logical egress pipeline, clearing the logical
> -     * registers (for consistent behavior with packets that get
> tunneled). */
> -    match_init_catchall(&match);
> -    ofpbuf_clear(&ofpacts);
> -    for (int i = 0; i < MFF_N_LOG_REGS; i++) {
> -        put_load(0, MFF_REG0 + i, 0, 32, &ofpacts);
> -    }
> -    put_resubmit(OFTABLE_LOG_EGRESS_PIPELINE, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_CHECK_LOOPBACK, 0, 0, &match,
> -                    &ofpacts, hc_uuid);
> -
> -    /* Table 64, Priority 0.
> -     * =======================
> -     *
> -     * Resubmit packets that do not have the MLF_ALLOW_LOOPBACK flag set
> -     * to table 65 for logical-to-physical translation. */
> -    match_init_catchall(&match);
> -    ofpbuf_clear(&ofpacts);
> -    put_resubmit(OFTABLE_LOG_TO_PHY, &ofpacts);
> -    ofctrl_add_flow(flow_table, OFTABLE_SAVE_INPORT, 0, 0, &match,
> &ofpacts,
> -                    hc_uuid);
> -
> -    ofpbuf_uninit(&ofpacts);
> -
> -    simap_destroy(&new_tunnel_to_ofport);
> -}
> diff --git a/ovn/controller/physical.h b/ovn/controller/physical.h
> deleted file mode 100644
> index c5544e8de..000000000
> --- a/ovn/controller/physical.h
> +++ /dev/null
> @@ -1,74 +0,0 @@
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_PHYSICAL_H
> -#define OVN_PHYSICAL_H 1
> -
> -/* Logical/Physical Translation
> - * ============================
> - *
> - * This module implements physical-to-logical and logical-to-physical
> - * translation as separate OpenFlow tables that run before the ingress
> pipeline
> - * and after the egress pipeline, respectively, as well as to connect the
> - * two pipelines.
> - */
> -
> -#include "openvswitch/meta-flow.h"
> -
> -struct hmap;
> -struct ovsdb_idl_index;
> -struct ovsrec_bridge;
> -struct simap;
> -struct sbrec_multicast_group_table;
> -struct sbrec_port_binding_table;
> -struct sset;
> -
> -/* OVN Geneve option information.
> - *
> - * Keep these in sync with the documentation in ovn-architecture(7). */
> -#define OVN_GENEVE_CLASS 0x0102  /* Assigned Geneve class for OVN. */
> -#define OVN_GENEVE_TYPE 0x80     /* Critical option. */
> -#define OVN_GENEVE_LEN 4
> -
> -void physical_register_ovs_idl(struct ovsdb_idl *);
> -void physical_run(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                  const struct sbrec_multicast_group_table *,
> -                  const struct sbrec_port_binding_table *,
> -                  enum mf_field_id mff_ovn_geneve,
> -                  const struct ovsrec_bridge *br_int,
> -                  const struct sbrec_chassis *chassis,
> -                  const struct simap *ct_zones,
> -                  const struct hmap *local_datapaths,
> -                  const struct sset *local_lports,
> -                  const struct sset *active_tunnels,
> -                  struct ovn_desired_flow_table *);
> -void physical_handle_port_binding_changes(
> -        struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -        const struct sbrec_port_binding_table *,
> -        enum mf_field_id mff_ovn_geneve,
> -        const struct sbrec_chassis *,
> -        const struct simap *ct_zones,
> -        struct hmap *local_datapaths,
> -        struct sset *active_tunnels,
> -        struct ovn_desired_flow_table *);
> -
> -void physical_handle_mc_group_changes(
> -        const struct sbrec_multicast_group_table *,
> -        enum mf_field_id mff_ovn_geneve,
> -        const struct sbrec_chassis *,
> -        const struct simap *ct_zones,
> -        const struct hmap *local_datapaths,
> -        struct ovn_desired_flow_table *);
> -#endif /* ovn/physical.h */
> diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c
> deleted file mode 100644
> index d857067a5..000000000
> --- a/ovn/controller/pinctrl.c
> +++ /dev/null
> @@ -1,4343 +0,0 @@
> -/* Copyright (c) 2015, 2016, 2017 Red Hat, Inc.
> - * Copyright (c) 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "pinctrl.h"
> -
> -#include "coverage.h"
> -#include "csum.h"
> -#include "dirs.h"
> -#include "dp-packet.h"
> -#include "encaps.h"
> -#include "flow.h"
> -#include "ha-chassis.h"
> -#include "lport.h"
> -#include "nx-match.h"
> -#include "ovn-controller.h"
> -#include "latch.h"
> -#include "lib/packets.h"
> -#include "lib/sset.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofp-msgs.h"
> -#include "openvswitch/ofp-packet.h"
> -#include "openvswitch/ofp-print.h"
> -#include "openvswitch/ofp-switch.h"
> -#include "openvswitch/ofp-util.h"
> -#include "openvswitch/vlog.h"
> -
> -#include "lib/dhcp.h"
> -#include "ovn-controller.h"
> -#include "ovn/actions.h"
> -#include "ovn/lex.h"
> -#include "ovn/lib/acl-log.h"
> -#include "ovn/lib/ip-mcast-index.h"
> -#include "ovn/lib/mcast-group-index.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "ovn/logical-fields.h"
> -#include "openvswitch/poll-loop.h"
> -#include "openvswitch/rconn.h"
> -#include "socket-util.h"
> -#include "seq.h"
> -#include "timeval.h"
> -#include "vswitch-idl.h"
> -#include "lflow.h"
> -#include "ip-mcast.h"
> -
> -VLOG_DEFINE_THIS_MODULE(pinctrl);
> -
> -/* pinctrl module creates a thread - pinctrl_handler to handle
> - * the packet-ins from ovs-vswitchd. Some of the OVN actions
> - * are translated to OF 'controller' actions. See include/ovn/actions.h
> - * for more details.
> - *
> - * pinctrl_handler thread doesn't access the Southbound IDL object. But
> - * some of the OVN actions which gets translated to 'controller'
> - * OF action, require data from Southbound DB.  Below are the details
> - * on how these actions are implemented.
> - *
> - * pinctrl_run() function is called by ovn-controller main thread.
> - * A Mutex - 'pinctrl_mutex' is used between the pinctrl_handler() thread
> - * and pinctrl_run().
> - *
> - *   - dns_lookup -     In order to do a DNS lookup, this action needs
> - *                      to access the 'DNS' table. pinctrl_run() builds a
> - *                      local DNS cache - 'dns_cache'. See
> sync_dns_cache()
> - *                      for more details.
> - *                      The function 'pinctrl_handle_dns_lookup()' (which
> is
> - *                      called with in the pinctrl_handler thread) looks
> into
> - *                      the local DNS cache to resolve the DNS requests.
> - *
> - *   - put_arp/put_nd - These actions stores the IPv4/IPv6 and MAC
> addresses
> - *                      in the 'MAC_Binding' table.
> - *                      The function 'pinctrl_handle_put_mac_binding()'
> (which
> - *                      is called with in the pinctrl_handler thread),
> stores
> - *                      the IPv4/IPv6 and MAC addresses in the
> - *                      hmap - put_mac_bindings.
> - *
> - *                      pinctrl_run(), reads these mac bindings from the
> hmap
> - *                      'put_mac_bindings' and writes to the 'MAC_Binding'
> - *                      table in the Southbound DB.
> - *
> - *   - arp/nd_ns      - These actions generate an ARP/IPv6 Neighbor
> solicit
> - *                      requests. The original packets are buffered and
> - *                      injected back when put_arp/put_nd resolves
> - *                      corresponding ARP/IPv6 Neighbor solicit requests.
> - *                      When pinctrl_run(), writes the mac bindings from
> the
> - *                      'put_mac_bindings' hmap to the MAC_Binding table
> in
> - *                      SB DB, run_buffered_binding will add the buffered
> - *                      packets to buffered_mac_bindings and notify
> - *                      pinctrl_handler.
> - *
> - *                      The pinctrl_handler thread calls the function -
> - *                      send_mac_binding_buffered_pkts(), which uses
> - *                      the hmap - 'buffered_mac_bindings' and reinjects
> the
> - *                      buffered packets.
> - *
> - *    - igmp          - This action punts an IGMP packet to the controller
> - *                      which maintains multicast group information. The
> - *                      multicast groups (mcast_snoop_map) are synced to
> - *                      the 'IGMP_Group' table by ip_mcast_sync().
> - *                      ip_mcast_sync() also reads the 'IP_Multicast'
> - *                      (snooping and querier) configuration and builds a
> - *                      local configuration mcast_cfg_map.
> - *                      ip_mcast_snoop_run() which runs in the
> - *                      pinctrl_handler() thread configures the per
> datapath
> - *                      mcast_snoop_map entries according to
> mcast_cfg_map.
> - *
> - * pinctrl module also periodically sends IPv6 Router Solicitation
> requests
> - * and gARPs (for the router gateway IPs and configured NAT addresses).
> - *
> - * IPv6 RA handling - pinctrl_run() prepares the IPv6 RA information
> - *                    (see prepare_ipv6_ras()) in the shash 'ipv6_ras' by
> - *                    looking into the Southbound DB table - Port_Binding.
> - *
> - *                    pinctrl_handler thread sends the periodic IPv6 RAs
> using
> - *                    the shash - 'ipv6_ras'
> - *
> - * gARP handling    - pinctrl_run() prepares the gARP information
> - *                    (see send_garp_prepare()) in the shash
> 'send_garp_data'
> - *                    by looking into the Southbound DB table
> Port_Binding.
> - *
> - *                    pinctrl_handler() thread sends these gARPs using the
> - *                    shash 'send_garp_data'.
> - *
> - * IGMP Queries     - pinctrl_run() prepares the IGMP queries (at most one
> - *                    per local datapath) based on the mcast_snoop_map
> - *                    contents and stores them in mcast_query_list.
> - *
> - *                    pinctrl_handler thread sends the periodic IGMP
> queries
> - *                    by walking the mcast_query_list.
> - *
> - * Notification between pinctrl_handler() and pinctrl_run()
> - * -------------------------------------------------------
> - * 'struct seq' is used for notification between pinctrl_handler() thread
> - *  and pinctrl_run().
> - *  'pinctrl_handler_seq' is used by pinctrl_run() to
> - *  wake up pinctrl_handler thread from poll_block() if any changes
> happened
> - *  in 'send_garp_data', 'ipv6_ras' and 'buffered_mac_bindings'
> structures.
> - *
> - *  'pinctrl_main_seq' is used by pinctrl_handler() thread to wake up
> - *  the main thread from poll_block() when mac bindings/igmp groups need
> to
> - *  be updated in the Southboubd DB.
> - * */
> -
> -static struct ovs_mutex pinctrl_mutex = OVS_MUTEX_INITIALIZER;
> -static struct seq *pinctrl_handler_seq;
> -static struct seq *pinctrl_main_seq;
> -
> -static void *pinctrl_handler(void *arg);
> -
> -struct pinctrl {
> -    char *br_int_name;
> -    pthread_t pinctrl_thread;
> -    /* Latch to destroy the 'pinctrl_thread' */
> -    struct latch pinctrl_thread_exit;
> -};
> -
> -static struct pinctrl pinctrl;
> -
> -static void init_buffered_packets_map(void);
> -static void destroy_buffered_packets_map(void);
> -static void
> -run_buffered_binding(struct ovsdb_idl_index
> *sbrec_port_binding_by_datapath,
> -                     struct ovsdb_idl_index
> *sbrec_mac_binding_by_lport_ip,
> -                     const struct hmap *local_datapaths)
> -    OVS_REQUIRES(pinctrl_mutex);
> -
> -static void pinctrl_handle_put_mac_binding(const struct flow *md,
> -                                           const struct flow *headers,
> -                                           bool is_arp)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void init_put_mac_bindings(void);
> -static void destroy_put_mac_bindings(void);
> -static void run_put_mac_bindings(
> -    struct ovsdb_idl_txn *ovnsb_idl_txn,
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -    struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void wait_put_mac_bindings(struct ovsdb_idl_txn *ovnsb_idl_txn);
> -static void flush_put_mac_bindings(void);
> -static void send_mac_binding_buffered_pkts(struct rconn *swconn)
> -    OVS_REQUIRES(pinctrl_mutex);
> -
> -static void init_send_garps(void);
> -static void destroy_send_garps(void);
> -static void send_garp_wait(long long int send_garp_time);
> -static void send_garp_prepare(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct ovsrec_bridge *,
> -    const struct sbrec_chassis *,
> -    const struct hmap *local_datapaths,
> -    const struct sset *active_tunnels)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void send_garp_run(struct rconn *swconn, long long int
> *send_garp_time)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void pinctrl_handle_nd_na(struct rconn *swconn,
> -                                 const struct flow *ip_flow,
> -                                 const struct match *md,
> -                                 struct ofpbuf *userdata,
> -                                 bool is_router);
> -static void reload_metadata(struct ofpbuf *ofpacts,
> -                            const struct match *md);
> -static void pinctrl_handle_put_nd_ra_opts(
> -    struct rconn *swconn,
> -    const struct flow *ip_flow, struct dp_packet *pkt_in,
> -    struct ofputil_packet_in *pin, struct ofpbuf *userdata,
> -    struct ofpbuf *continuation);
> -static void pinctrl_handle_nd_ns(struct rconn *swconn,
> -                                 const struct flow *ip_flow,
> -                                 struct dp_packet *pkt_in,
> -                                 const struct match *md,
> -                                 struct ofpbuf *userdata);
> -static void pinctrl_handle_put_icmp4_frag_mtu(struct rconn *swconn,
> -                                              const struct flow *in_flow,
> -                                              struct dp_packet *pkt_in,
> -                                              struct ofputil_packet_in
> *pin,
> -                                              struct ofpbuf *userdata,
> -                                              struct ofpbuf
> *continuation);
> -static void
> -pinctrl_handle_event(struct ofpbuf *userdata)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void wait_controller_event(struct ovsdb_idl_txn *ovnsb_idl_txn);
> -static void init_ipv6_ras(void);
> -static void destroy_ipv6_ras(void);
> -static void ipv6_ra_wait(long long int send_ipv6_ra_time);
> -static void prepare_ipv6_ras(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct hmap *local_datapaths)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void send_ipv6_ras(struct rconn *swconn,
> -                          long long int *send_ipv6_ra_time)
> -    OVS_REQUIRES(pinctrl_mutex);
> -
> -static void ip_mcast_snoop_init(void);
> -static void ip_mcast_snoop_destroy(void);
> -static void ip_mcast_snoop_run(void)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void ip_mcast_querier_run(struct rconn *swconn,
> -                                 long long int *query_time);
> -static void ip_mcast_querier_wait(long long int query_time);
> -static void ip_mcast_sync(
> -    struct ovsdb_idl_txn *ovnsb_idl_txn,
> -    const struct sbrec_chassis *chassis,
> -    const struct hmap *local_datapaths,
> -    struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -    struct ovsdb_idl_index *sbrec_igmp_groups,
> -    struct ovsdb_idl_index *sbrec_ip_multicast)
> -    OVS_REQUIRES(pinctrl_mutex);
> -static void pinctrl_ip_mcast_handle_igmp(
> -    struct rconn *swconn,
> -    const struct flow *ip_flow,
> -    struct dp_packet *pkt_in,
> -    const struct match *md,
> -    struct ofpbuf *userdata);
> -
> -static bool may_inject_pkts(void);
> -
> -COVERAGE_DEFINE(pinctrl_drop_put_mac_binding);
> -COVERAGE_DEFINE(pinctrl_drop_buffered_packets_map);
> -COVERAGE_DEFINE(pinctrl_drop_controller_event);
> -
> -struct empty_lb_backends_event {
> -    struct hmap_node hmap_node;
> -    long long int timestamp;
> -
> -    char *vip;
> -    char *protocol;
> -    char *load_balancer;
> -};
> -
> -static struct hmap event_table[OVN_EVENT_MAX];
> -static int64_t event_seq_num;
> -
> -static void
> -init_event_table(void)
> -{
> -    for (size_t i = 0; i < OVN_EVENT_MAX; i++) {
> -        hmap_init(&event_table[i]);
> -    }
> -}
> -
> -#define EVENT_TIMEOUT   10000
> -static void
> -empty_lb_backends_event_gc(bool flush)
> -{
> -    struct empty_lb_backends_event *cur_ce, *next_ce;
> -    long long int now = time_msec();
> -
> -    HMAP_FOR_EACH_SAFE (cur_ce, next_ce, hmap_node,
> -                        &event_table[OVN_EVENT_EMPTY_LB_BACKENDS]) {
> -        if ((now < cur_ce->timestamp + EVENT_TIMEOUT) && !flush) {
> -            continue;
> -        }
> -
> -        free(cur_ce->vip);
> -        free(cur_ce->protocol);
> -        free(cur_ce->load_balancer);
> -        hmap_remove(&event_table[OVN_EVENT_EMPTY_LB_BACKENDS],
> -                    &cur_ce->hmap_node);
> -        free(cur_ce);
> -    }
> -}
> -
> -static void
> -event_table_gc(bool flush)
> -{
> -    empty_lb_backends_event_gc(flush);
> -}
> -
> -static void
> -event_table_destroy(void)
> -{
> -    event_table_gc(true);
> -    for (size_t i = 0; i < OVN_EVENT_MAX; i++) {
> -        hmap_destroy(&event_table[i]);
> -    }
> -}
> -
> -static struct empty_lb_backends_event *
> -pinctrl_find_empty_lb_backends_event(char *vip, char *protocol,
> -                                     char *load_balancer, uint32_t hash)
> -{
> -    struct empty_lb_backends_event *ce;
> -    HMAP_FOR_EACH_WITH_HASH (ce, hmap_node, hash,
> -                             &event_table[OVN_EVENT_EMPTY_LB_BACKENDS]) {
> -        if (!strcmp(ce->vip, vip) &&
> -            !strcmp(ce->protocol, protocol) &&
> -            !strcmp(ce->load_balancer, load_balancer)) {
> -            return ce;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static const struct sbrec_controller_event *
> -empty_lb_backends_lookup(struct empty_lb_backends_event *event,
> -                         const struct sbrec_controller_event_table
> *ce_table,
> -                         const struct sbrec_chassis *chassis)
> -{
> -    const struct sbrec_controller_event *sbrec_event;
> -    const char *event_type = event_to_string(OVN_EVENT_EMPTY_LB_BACKENDS);
> -    char ref_uuid[UUID_LEN + 1];
> -    sprintf(ref_uuid, UUID_FMT, UUID_ARGS(&chassis->header_.uuid));
> -
> -    SBREC_CONTROLLER_EVENT_TABLE_FOR_EACH (sbrec_event, ce_table) {
> -        if (strcmp(sbrec_event->event_type, event_type)) {
> -            continue;
> -        }
> -
> -        char chassis_uuid[UUID_LEN + 1];
> -        sprintf(chassis_uuid, UUID_FMT,
> -                UUID_ARGS(&sbrec_event->chassis->header_.uuid));
> -        if (strcmp(ref_uuid, chassis_uuid)) {
> -            continue;
> -        }
> -
> -        const char *vip = smap_get(&sbrec_event->event_info, "vip");
> -        const char *protocol = smap_get(&sbrec_event->event_info,
> "protocol");
> -        const char *load_balancer = smap_get(&sbrec_event->event_info,
> -                                             "load_balancer");
> -
> -        if (!strcmp(event->vip, vip) &&
> -            !strcmp(event->protocol, protocol) &&
> -            !strcmp(event->load_balancer, load_balancer)) {
> -            return sbrec_event;
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static void
> -controller_event_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                     const struct sbrec_controller_event_table *ce_table,
> -                     const struct sbrec_chassis *chassis)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    if (!ovnsb_idl_txn) {
> -        goto out;
> -    }
> -
> -    struct empty_lb_backends_event *empty_lbs;
> -    HMAP_FOR_EACH (empty_lbs, hmap_node,
> -                   &event_table[OVN_EVENT_EMPTY_LB_BACKENDS]) {
> -        const struct sbrec_controller_event *event;
> -
> -        event = empty_lb_backends_lookup(empty_lbs, ce_table, chassis);
> -        if (!event) {
> -            struct smap event_info = SMAP_INITIALIZER(&event_info);
> -
> -            smap_add(&event_info, "vip", empty_lbs->vip);
> -            smap_add(&event_info, "protocol", empty_lbs->protocol);
> -            smap_add(&event_info, "load_balancer",
> empty_lbs->load_balancer);
> -
> -            event = sbrec_controller_event_insert(ovnsb_idl_txn);
> -            sbrec_controller_event_set_event_type(event,
> -                    event_to_string(OVN_EVENT_EMPTY_LB_BACKENDS));
> -            sbrec_controller_event_set_seq_num(event, ++event_seq_num);
> -            sbrec_controller_event_set_event_info(event, &event_info);
> -            sbrec_controller_event_set_chassis(event, chassis);
> -        }
> -    }
> -
> -out:
> -    event_table_gc(!!ovnsb_idl_txn);
> -}
> -
> -void
> -pinctrl_init(void)
> -{
> -    init_put_mac_bindings();
> -    init_send_garps();
> -    init_ipv6_ras();
> -    init_buffered_packets_map();
> -    init_event_table();
> -    ip_mcast_snoop_init();
> -    pinctrl.br_int_name = NULL;
> -    pinctrl_handler_seq = seq_create();
> -    pinctrl_main_seq = seq_create();
> -
> -    latch_init(&pinctrl.pinctrl_thread_exit);
> -    pinctrl.pinctrl_thread = ovs_thread_create("ovn_pinctrl",
> pinctrl_handler,
> -                                                &pinctrl);
> -}
> -
> -static ovs_be32
> -queue_msg(struct rconn *swconn, struct ofpbuf *msg)
> -{
> -    const struct ofp_header *oh = msg->data;
> -    ovs_be32 xid = oh->xid;
> -
> -    rconn_send(swconn, msg, NULL);
> -    return xid;
> -}
> -
> -/* Sets up 'swconn', a newly (re)connected connection to a switch. */
> -static void
> -pinctrl_setup(struct rconn *swconn)
> -{
> -    /* Fetch the switch configuration.  The response later will allow us
> to
> -     * change the miss_send_len to UINT16_MAX, so that we can enable
> -     * asynchronous messages. */
> -    queue_msg(swconn, ofpraw_alloc(OFPRAW_OFPT_GET_CONFIG_REQUEST,
> -                                   rconn_get_version(swconn), 0));
> -
> -    /* Set a packet-in format that supports userdata.  */
> -    queue_msg(swconn,
> -
> ofputil_encode_set_packet_in_format(rconn_get_version(swconn),
> -
> OFPUTIL_PACKET_IN_NXT2));
> -}
> -
> -static void
> -set_switch_config(struct rconn *swconn,
> -                  const struct ofputil_switch_config *config)
> -{
> -    enum ofp_version version = rconn_get_version(swconn);
> -    struct ofpbuf *request = ofputil_encode_set_config(config, version);
> -    queue_msg(swconn, request);
> -}
> -
> -static void
> -set_actions_and_enqueue_msg(struct rconn *swconn,
> -                            const struct dp_packet *packet,
> -                            const struct match *md,
> -                            struct ofpbuf *userdata)
> -{
> -    /* Copy metadata from 'md' into the packet-out via "set_field"
> -     * actions, then add actions from 'userdata'.
> -     */
> -    uint64_t ofpacts_stub[4096 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -    enum ofp_version version = rconn_get_version(swconn);
> -
> -    reload_metadata(&ofpacts, md);
> -    enum ofperr error = ofpacts_pull_openflow_actions(userdata,
> userdata->size,
> -                                                      version, NULL, NULL,
> -                                                      &ofpacts);
> -    if (error) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "failed to parse actions from userdata (%s)",
> -                     ofperr_to_string(error));
> -        ofpbuf_uninit(&ofpacts);
> -        return;
> -    }
> -
> -    struct ofputil_packet_out po = {
> -        .packet = dp_packet_data(packet),
> -        .packet_len = dp_packet_size(packet),
> -        .buffer_id = UINT32_MAX,
> -        .ofpacts = ofpacts.data,
> -        .ofpacts_len = ofpacts.size,
> -    };
> -    match_set_in_port(&po.flow_metadata, OFPP_CONTROLLER);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    queue_msg(swconn, ofputil_encode_packet_out(&po, proto));
> -    ofpbuf_uninit(&ofpacts);
> -}
> -
> -struct buffer_info {
> -    struct ofpbuf ofpacts;
> -    struct dp_packet *p;
> -};
> -
> -#define BUFFER_QUEUE_DEPTH     4
> -struct buffered_packets {
> -    struct hmap_node hmap_node;
> -    struct ovs_list list;
> -
> -    /* key */
> -    struct in6_addr ip;
> -    struct eth_addr ea;
> -
> -    long long int timestamp;
> -
> -    struct buffer_info data[BUFFER_QUEUE_DEPTH];
> -    uint32_t head, tail;
> -};
> -
> -static struct hmap buffered_packets_map;
> -static struct ovs_list buffered_mac_bindings;
> -
> -static void
> -init_buffered_packets_map(void)
> -{
> -    hmap_init(&buffered_packets_map);
> -    ovs_list_init(&buffered_mac_bindings);
> -}
> -
> -static void
> -destroy_buffered_packets(struct buffered_packets *bp)
> -{
> -    struct buffer_info *bi;
> -
> -    while (bp->head != bp->tail) {
> -        bi = &bp->data[bp->head];
> -        dp_packet_delete(bi->p);
> -        ofpbuf_uninit(&bi->ofpacts);
> -
> -        bp->head = (bp->head + 1) % BUFFER_QUEUE_DEPTH;
> -    }
> -}
> -
> -static void
> -destroy_buffered_packets_map(void)
> -{
> -    struct buffered_packets *bp, *next;
> -    HMAP_FOR_EACH_SAFE (bp, next, hmap_node, &buffered_packets_map) {
> -        destroy_buffered_packets(bp);
> -        hmap_remove(&buffered_packets_map, &bp->hmap_node);
> -        free(bp);
> -    }
> -    hmap_destroy(&buffered_packets_map);
> -
> -    LIST_FOR_EACH_POP (bp, list, &buffered_mac_bindings) {
> -        destroy_buffered_packets(bp);
> -        free(bp);
> -    }
> -}
> -
> -static void
> -buffered_push_packet(struct buffered_packets *bp,
> -                     struct dp_packet *packet,
> -                     const struct match *md)
> -{
> -    uint32_t next = (bp->tail + 1) % BUFFER_QUEUE_DEPTH;
> -    struct buffer_info *bi = &bp->data[bp->tail];
> -
> -    ofpbuf_init(&bi->ofpacts, 4096);
> -
> -    reload_metadata(&bi->ofpacts, md);
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(&bi->ofpacts);
> -    resubmit->in_port = OFPP_CONTROLLER;
> -    resubmit->table_id = OFTABLE_REMOTE_OUTPUT;
> -
> -    bi->p = packet;
> -
> -    if (next == bp->head) {
> -        bi = &bp->data[bp->head];
> -        dp_packet_delete(bi->p);
> -        ofpbuf_uninit(&bi->ofpacts);
> -        bp->head = (bp->head + 1) % BUFFER_QUEUE_DEPTH;
> -    }
> -    bp->tail = next;
> -}
> -
> -static void
> -buffered_send_packets(struct rconn *swconn, struct buffered_packets *bp,
> -                      struct eth_addr *addr)
> -{
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -
> -    while (bp->head != bp->tail) {
> -        struct buffer_info *bi = &bp->data[bp->head];
> -        struct eth_header *eth = dp_packet_data(bi->p);
> -
> -        eth->eth_dst = *addr;
> -        struct ofputil_packet_out po = {
> -            .packet = dp_packet_data(bi->p),
> -            .packet_len = dp_packet_size(bi->p),
> -            .buffer_id = UINT32_MAX,
> -            .ofpacts = bi->ofpacts.data,
> -            .ofpacts_len = bi->ofpacts.size,
> -        };
> -        match_set_in_port(&po.flow_metadata, OFPP_CONTROLLER);
> -        queue_msg(swconn, ofputil_encode_packet_out(&po, proto));
> -
> -        ofpbuf_uninit(&bi->ofpacts);
> -        dp_packet_delete(bi->p);
> -
> -        bp->head = (bp->head + 1) % BUFFER_QUEUE_DEPTH;
> -    }
> -}
> -
> -#define BUFFER_MAP_TIMEOUT   10000
> -static void
> -buffered_packets_map_gc(void)
> -{
> -    struct buffered_packets *cur_qp, *next_qp;
> -    long long int now = time_msec();
> -
> -    HMAP_FOR_EACH_SAFE (cur_qp, next_qp, hmap_node,
> &buffered_packets_map) {
> -        if (now > cur_qp->timestamp + BUFFER_MAP_TIMEOUT) {
> -            destroy_buffered_packets(cur_qp);
> -            hmap_remove(&buffered_packets_map, &cur_qp->hmap_node);
> -            free(cur_qp);
> -        }
> -    }
> -}
> -
> -static struct buffered_packets *
> -pinctrl_find_buffered_packets(const struct in6_addr *ip, uint32_t hash)
> -{
> -    struct buffered_packets *qp;
> -
> -    HMAP_FOR_EACH_WITH_HASH (qp, hmap_node, hash,
> -                             &buffered_packets_map) {
> -        if (IN6_ARE_ADDR_EQUAL(&qp->ip, ip)) {
> -            return qp;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static int
> -pinctrl_handle_buffered_packets(const struct flow *ip_flow,
> -                                struct dp_packet *pkt_in,
> -                                const struct match *md, bool is_arp)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct buffered_packets *bp;
> -    struct dp_packet *clone;
> -    struct in6_addr addr;
> -
> -    if (is_arp) {
> -        addr = in6_addr_mapped_ipv4(ip_flow->nw_dst);
> -    } else {
> -        addr = ip_flow->ipv6_dst;
> -    }
> -
> -    uint32_t hash = hash_bytes(&addr, sizeof addr, 0);
> -    bp = pinctrl_find_buffered_packets(&addr, hash);
> -    if (!bp) {
> -        if (hmap_count(&buffered_packets_map) >= 1000) {
> -            COVERAGE_INC(pinctrl_drop_buffered_packets_map);
> -            return -ENOMEM;
> -        }
> -
> -        bp = xmalloc(sizeof *bp);
> -        hmap_insert(&buffered_packets_map, &bp->hmap_node, hash);
> -        bp->head = bp->tail = 0;
> -        bp->ip = addr;
> -    }
> -    bp->timestamp = time_msec();
> -    /* clone the packet to send it later with correct L2 address */
> -    clone = dp_packet_clone_data(dp_packet_data(pkt_in),
> -                                 dp_packet_size(pkt_in));
> -    buffered_push_packet(bp, clone, md);
> -
> -    return 0;
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_arp(struct rconn *swconn, const struct flow *ip_flow,
> -                   struct dp_packet *pkt_in,
> -                   const struct match *md, struct ofpbuf *userdata)
> -{
> -    /* This action only works for IP packets, and the switch should only
> send
> -     * us IP packets this way, but check here just to be sure. */
> -    if (ip_flow->dl_type != htons(ETH_TYPE_IP)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "ARP action on non-IP packet (Ethertype
> %"PRIx16")",
> -                     ntohs(ip_flow->dl_type));
> -        return;
> -    }
> -
> -    ovs_mutex_lock(&pinctrl_mutex);
> -    pinctrl_handle_buffered_packets(ip_flow, pkt_in, md, true);
> -    ovs_mutex_unlock(&pinctrl_mutex);
> -
> -    /* Compose an ARP packet. */
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    compose_arp__(&packet);
> -
> -    struct eth_header *eth = dp_packet_eth(&packet);
> -    eth->eth_dst = ip_flow->dl_dst;
> -    eth->eth_src = ip_flow->dl_src;
> -
> -    struct arp_eth_header *arp = dp_packet_l3(&packet);
> -    arp->ar_op = htons(ARP_OP_REQUEST);
> -    arp->ar_sha = ip_flow->dl_src;
> -    put_16aligned_be32(&arp->ar_spa, ip_flow->nw_src);
> -    arp->ar_tha = eth_addr_zero;
> -    put_16aligned_be32(&arp->ar_tpa, ip_flow->nw_dst);
> -
> -    if (ip_flow->vlans[0].tci & htons(VLAN_CFI)) {
> -        eth_push_vlan(&packet, htons(ETH_TYPE_VLAN_8021Q),
> -                      ip_flow->vlans[0].tci);
> -    }
> -
> -    set_actions_and_enqueue_msg(swconn, &packet, md, userdata);
> -    dp_packet_uninit(&packet);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_icmp(struct rconn *swconn, const struct flow *ip_flow,
> -                    struct dp_packet *pkt_in,
> -                    const struct match *md, struct ofpbuf *userdata,
> -                    bool include_orig_ip_datagram)
> -{
> -    /* This action only works for IP packets, and the switch should only
> send
> -     * us IP packets this way, but check here just to be sure. */
> -    if (ip_flow->dl_type != htons(ETH_TYPE_IP) &&
> -        ip_flow->dl_type != htons(ETH_TYPE_IPV6)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl,
> -                     "ICMP action on non-IP packet (eth_type
> 0x%"PRIx16")",
> -                     ntohs(ip_flow->dl_type));
> -        return;
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    dp_packet_clear(&packet);
> -    packet.packet_type = htonl(PT_ETH);
> -
> -    struct eth_header *eh = dp_packet_put_zeros(&packet, sizeof *eh);
> -    eh->eth_dst = ip_flow->dl_dst;
> -    eh->eth_src = ip_flow->dl_src;
> -
> -    if (get_dl_type(ip_flow) == htons(ETH_TYPE_IP)) {
> -        struct ip_header *in_ip = dp_packet_l3(pkt_in);
> -        uint16_t in_ip_len = ntohs(in_ip->ip_tot_len);
> -        if (in_ip_len < IP_HEADER_LEN) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -            VLOG_WARN_RL(&rl,
> -                        "ICMP action on IP packet with invalid length
> (%u)",
> -                        in_ip_len);
> -            return;
> -        }
> -
> -        struct ip_header *nh = dp_packet_put_zeros(&packet, sizeof *nh);
> -
> -        eh->eth_type = htons(ETH_TYPE_IP);
> -        dp_packet_set_l3(&packet, nh);
> -        nh->ip_ihl_ver = IP_IHL_VER(5, 4);
> -        nh->ip_tot_len = htons(sizeof(struct ip_header) +
> -                               sizeof(struct icmp_header));
> -        nh->ip_proto = IPPROTO_ICMP;
> -        nh->ip_frag_off = htons(IP_DF);
> -        packet_set_ipv4(&packet, ip_flow->nw_src, ip_flow->nw_dst,
> -                        ip_flow->nw_tos, 255);
> -
> -        struct icmp_header *ih = dp_packet_put_zeros(&packet, sizeof *ih);
> -        dp_packet_set_l4(&packet, ih);
> -        packet_set_icmp(&packet, ICMP4_DST_UNREACH, 1);
> -
> -        if (include_orig_ip_datagram) {
> -            /* RFC 1122: 3.2.2 MUST send at least the IP header and 8
> bytes
> -             * of header. MAY send more.
> -             * RFC says return as much as we can without exceeding 576
> -             * bytes.
> -             * So, lets return as much as we can. */
> -
> -            /* Calculate available room to include the original IP +
> data. */
> -            nh = dp_packet_l3(&packet);
> -            uint16_t room = 576 - (sizeof *eh + ntohs(nh->ip_tot_len));
> -            if (in_ip_len > room) {
> -                in_ip_len = room;
> -            }
> -            dp_packet_put(&packet, in_ip, in_ip_len);
> -
> -            /* dp_packet_put may reallocate the buffer. Get the l3 and l4
> -             * header pointers again. */
> -            nh = dp_packet_l3(&packet);
> -            ih = dp_packet_l4(&packet);
> -            uint16_t ip_total_len = ntohs(nh->ip_tot_len) + in_ip_len;
> -            nh->ip_tot_len = htons(ip_total_len);
> -            ih->icmp_csum = 0;
> -            ih->icmp_csum = csum(ih, sizeof *ih + in_ip_len);
> -            nh->ip_csum = 0;
> -            nh->ip_csum = csum(nh, sizeof *nh);
> -        }
> -    } else {
> -        struct ip6_hdr *nh = dp_packet_put_zeros(&packet, sizeof *nh);
> -        struct icmp6_error_header *ih;
> -        uint32_t icmpv6_csum;
> -
> -        eh->eth_type = htons(ETH_TYPE_IPV6);
> -        dp_packet_set_l3(&packet, nh);
> -        nh->ip6_vfc = 0x60;
> -        nh->ip6_nxt = IPPROTO_ICMPV6;
> -        nh->ip6_plen = htons(sizeof(*nh) + ICMP6_ERROR_HEADER_LEN);
> -        packet_set_ipv6(&packet, &ip_flow->ipv6_src, &ip_flow->ipv6_dst,
> -                        ip_flow->nw_tos, ip_flow->ipv6_label, 255);
> -
> -        ih = dp_packet_put_zeros(&packet, sizeof *ih);
> -        dp_packet_set_l4(&packet, ih);
> -        ih->icmp6_base.icmp6_type = ICMP6_DST_UNREACH;
> -        ih->icmp6_base.icmp6_code = 1;
> -        ih->icmp6_base.icmp6_cksum = 0;
> -
> -        uint8_t *data = dp_packet_put_zeros(&packet, sizeof *nh);
> -        memcpy(data, dp_packet_l3(pkt_in), sizeof(*nh));
> -
> -        icmpv6_csum = packet_csum_pseudoheader6(dp_packet_l3(&packet));
> -        ih->icmp6_base.icmp6_cksum = csum_finish(
> -            csum_continue(icmpv6_csum, ih,
> -                          sizeof(*nh) + ICMP6_ERROR_HEADER_LEN));
> -    }
> -
> -    if (ip_flow->vlans[0].tci & htons(VLAN_CFI)) {
> -        eth_push_vlan(&packet, htons(ETH_TYPE_VLAN_8021Q),
> -                      ip_flow->vlans[0].tci);
> -    }
> -
> -    set_actions_and_enqueue_msg(swconn, &packet, md, userdata);
> -    dp_packet_uninit(&packet);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_tcp_reset(struct rconn *swconn, const struct flow *ip_flow,
> -                         struct dp_packet *pkt_in,
> -                         const struct match *md, struct ofpbuf *userdata)
> -{
> -    /* This action only works for TCP segments, and the switch should
> only send
> -     * us TCP segments this way, but check here just to be sure. */
> -    if (ip_flow->nw_proto != IPPROTO_TCP) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "TCP_RESET action on non-TCP packet");
> -        return;
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    dp_packet_clear(&packet);
> -    packet.packet_type = htonl(PT_ETH);
> -
> -    struct eth_header *eh = dp_packet_put_zeros(&packet, sizeof *eh);
> -    eh->eth_dst = ip_flow->dl_dst;
> -    eh->eth_src = ip_flow->dl_src;
> -
> -    if (get_dl_type(ip_flow) == htons(ETH_TYPE_IPV6)) {
> -        struct ip6_hdr *nh = dp_packet_put_zeros(&packet, sizeof *nh);
> -
> -        eh->eth_type = htons(ETH_TYPE_IPV6);
> -        dp_packet_set_l3(&packet, nh);
> -        nh->ip6_vfc = 0x60;
> -        nh->ip6_nxt = IPPROTO_TCP;
> -        nh->ip6_plen = htons(TCP_HEADER_LEN);
> -        packet_set_ipv6(&packet, &ip_flow->ipv6_src, &ip_flow->ipv6_dst,
> -                        ip_flow->nw_tos, ip_flow->ipv6_label, 255);
> -    } else {
> -        struct ip_header *nh = dp_packet_put_zeros(&packet, sizeof *nh);
> -
> -        eh->eth_type = htons(ETH_TYPE_IP);
> -        dp_packet_set_l3(&packet, nh);
> -        nh->ip_ihl_ver = IP_IHL_VER(5, 4);
> -        nh->ip_tot_len = htons(IP_HEADER_LEN + TCP_HEADER_LEN);
> -        nh->ip_proto = IPPROTO_TCP;
> -        nh->ip_frag_off = htons(IP_DF);
> -        packet_set_ipv4(&packet, ip_flow->nw_src, ip_flow->nw_dst,
> -                        ip_flow->nw_tos, 255);
> -    }
> -
> -    struct tcp_header *th = dp_packet_put_zeros(&packet, sizeof *th);
> -    struct tcp_header *tcp_in = dp_packet_l4(pkt_in);
> -    dp_packet_set_l4(&packet, th);
> -    th->tcp_ctl = TCP_CTL(TCP_RST, 5);
> -    if (ip_flow->tcp_flags & htons(TCP_ACK)) {
> -        th->tcp_seq = tcp_in->tcp_ack;
> -    } else {
> -        uint32_t tcp_seq, ack_seq, tcp_len;
> -
> -        tcp_seq = ntohl(get_16aligned_be32(&tcp_in->tcp_seq));
> -        tcp_len = TCP_OFFSET(tcp_in->tcp_ctl) * 4;
> -        ack_seq = tcp_seq + dp_packet_l4_size(pkt_in) - tcp_len;
> -        put_16aligned_be32(&th->tcp_ack, htonl(ack_seq));
> -        put_16aligned_be32(&th->tcp_seq, 0);
> -    }
> -    packet_set_tcp_port(&packet, ip_flow->tp_dst, ip_flow->tp_src);
> -
> -    if (ip_flow->vlans[0].tci & htons(VLAN_CFI)) {
> -        eth_push_vlan(&packet, htons(ETH_TYPE_VLAN_8021Q),
> -                      ip_flow->vlans[0].tci);
> -    }
> -
> -    set_actions_and_enqueue_msg(swconn, &packet, md, userdata);
> -    dp_packet_uninit(&packet);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_put_dhcp_opts(
> -    struct rconn *swconn,
> -    struct dp_packet *pkt_in, struct ofputil_packet_in *pin,
> -    struct ofpbuf *userdata, struct ofpbuf *continuation)
> -{
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    struct dp_packet *pkt_out_ptr = NULL;
> -    uint32_t success = 0;
> -
> -    /* Parse result field. */
> -    const struct mf_field *f;
> -    enum ofperr ofperr = nx_pull_header(userdata, NULL, &f, NULL);
> -    if (ofperr) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "bad result OXM (%s)",
> ofperr_to_string(ofperr));
> -        goto exit;
> -    }
> -
> -    /* Parse result offset and offer IP. */
> -    ovs_be32 *ofsp = ofpbuf_try_pull(userdata, sizeof *ofsp);
> -    ovs_be32 *offer_ip = ofpbuf_try_pull(userdata, sizeof *offer_ip);
> -    if (!ofsp || !offer_ip) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "offset or offer_ip not present in the
> userdata");
> -        goto exit;
> -    }
> -
> -    /* Check that the result is valid and writable. */
> -    struct mf_subfield dst = { .field = f, .ofs = ntohl(*ofsp), .n_bits =
> 1 };
> -    ofperr = mf_check_dst(&dst, NULL);
> -    if (ofperr) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "bad result bit (%s)",
> ofperr_to_string(ofperr));
> -        goto exit;
> -    }
> -
> -    if (!userdata->size) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "DHCP options not present in the userdata");
> -        goto exit;
> -    }
> -
> -    /* Validate the DHCP request packet.
> -     * Format of the DHCP packet is
> -     *
> ------------------------------------------------------------------------
> -     *| UDP HEADER  | DHCP HEADER  | 4 Byte DHCP Cookie | DHCP
> OPTIONS(var len)|
> -     *
> ------------------------------------------------------------------------
> -     */
> -
> -    const char *end = (char *)dp_packet_l4(pkt_in) +
> dp_packet_l4_size(pkt_in);
> -    const char *in_dhcp_ptr = dp_packet_get_udp_payload(pkt_in);
> -    if (!in_dhcp_ptr) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Invalid or incomplete DHCP packet received");
> -        goto exit;
> -    }
> -
> -    const struct dhcp_header *in_dhcp_data
> -        = (const struct dhcp_header *) in_dhcp_ptr;
> -    in_dhcp_ptr += sizeof *in_dhcp_data;
> -    if (in_dhcp_ptr > end) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Invalid or incomplete DHCP packet received, "
> -                     "bad data length");
> -        goto exit;
> -    }
> -    if (in_dhcp_data->op != DHCP_OP_REQUEST) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Invalid opcode in the DHCP packet: %d",
> -                     in_dhcp_data->op);
> -        goto exit;
> -    }
> -
> -    /* DHCP options follow the DHCP header. The first 4 bytes of the DHCP
> -     * options is the DHCP magic cookie followed by the actual DHCP
> options.
> -     */
> -    ovs_be32 magic_cookie = htonl(DHCP_MAGIC_COOKIE);
> -    if (in_dhcp_ptr + sizeof magic_cookie > end ||
> -        get_unaligned_be32((const void *) in_dhcp_ptr) != magic_cookie) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "DHCP magic cookie not present in the DHCP
> packet");
> -        goto exit;
> -    }
> -    in_dhcp_ptr += sizeof magic_cookie;
> -
> -    const uint8_t *in_dhcp_msg_type = NULL;
> -    ovs_be32 request_ip = in_dhcp_data->ciaddr;
> -    while (in_dhcp_ptr < end) {
> -        const struct dhcp_opt_header *in_dhcp_opt =
> -            (const struct dhcp_opt_header *)in_dhcp_ptr;
> -        if (in_dhcp_opt->code == DHCP_OPT_END) {
> -            break;
> -        }
> -        if (in_dhcp_opt->code == DHCP_OPT_PAD) {
> -            in_dhcp_ptr += 1;
> -            continue;
> -        }
> -        in_dhcp_ptr += sizeof *in_dhcp_opt;
> -        if (in_dhcp_ptr > end) {
> -            break;
> -        }
> -        in_dhcp_ptr += in_dhcp_opt->len;
> -        if (in_dhcp_ptr > end) {
> -            break;
> -        }
> -
> -        switch (in_dhcp_opt->code) {
> -        case DHCP_OPT_MSG_TYPE:
> -            if (in_dhcp_opt->len == 1) {
> -                in_dhcp_msg_type = DHCP_OPT_PAYLOAD(in_dhcp_opt);
> -            }
> -            break;
> -        case DHCP_OPT_REQ_IP:
> -            if (in_dhcp_opt->len == 4) {
> -                request_ip =
> get_unaligned_be32(DHCP_OPT_PAYLOAD(in_dhcp_opt));
> -            }
> -            break;
> -        default:
> -            break;
> -        }
> -    }
> -
> -    /* Check that the DHCP Message Type (opt 53) is present or not with
> -     * valid values - DHCP_MSG_DISCOVER or DHCP_MSG_REQUEST.
> -     */
> -    if (!in_dhcp_msg_type) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Missing DHCP message type");
> -        goto exit;
> -    }
> -    if (*in_dhcp_msg_type != DHCP_MSG_DISCOVER &&
> -        *in_dhcp_msg_type != DHCP_MSG_REQUEST) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Invalid DHCP message type: %d",
> *in_dhcp_msg_type);
> -        goto exit;
> -    }
> -
> -    uint8_t msg_type;
> -    if (*in_dhcp_msg_type == DHCP_MSG_DISCOVER) {
> -        msg_type = DHCP_MSG_OFFER;
> -    } else {
> -        /* This is a DHCPREQUEST. If the client has requested an IP that
> -         * does not match the offered IP address, reply with a NAK. The
> -         * requested IP address may be supplied either via Requested IP
> Address
> -         * (opt 50) or via ciaddr, depending on the client's state.
> -         */
> -        msg_type = DHCP_MSG_ACK;
> -        if (request_ip != *offer_ip) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -            VLOG_WARN_RL(&rl, "DHCPREQUEST requested IP "IP_FMT" does not
> "
> -                         "match offer "IP_FMT, IP_ARGS(request_ip),
> -                         IP_ARGS(*offer_ip));
> -            msg_type = DHCP_MSG_NAK;
> -        }
> -    }
> -
> -    /* Frame the DHCP reply packet
> -     * Total DHCP options length will be options stored in the userdata +
> -     * 16 bytes. Note that the DHCP options stored in userdata are not
> included
> -     * in DHCPNAK messages.
> -     *
> -     * --------------------------------------------------------------
> -     *| 4 Bytes (dhcp cookie) | 3 Bytes (option type) | DHCP options |
> -     * --------------------------------------------------------------
> -     *| 4 Bytes padding | 1 Byte (option end 0xFF ) | 4 Bytes padding|
> -     * --------------------------------------------------------------
> -     */
> -    uint16_t new_l4_size = UDP_HEADER_LEN + DHCP_HEADER_LEN + 16;
> -    if (msg_type != DHCP_MSG_NAK) {
> -        new_l4_size += userdata->size;
> -    }
> -    size_t new_packet_size = pkt_in->l4_ofs + new_l4_size;
> -
> -    struct dp_packet pkt_out;
> -    dp_packet_init(&pkt_out, new_packet_size);
> -    dp_packet_clear(&pkt_out);
> -    dp_packet_prealloc_tailroom(&pkt_out, new_packet_size);
> -    pkt_out_ptr = &pkt_out;
> -
> -    /* Copy the L2 and L3 headers from the pkt_in as they would remain
> same*/
> -    dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, pkt_in->l4_ofs), pkt_in->l4_ofs);
> -
> -    pkt_out.l2_5_ofs = pkt_in->l2_5_ofs;
> -    pkt_out.l2_pad_size = pkt_in->l2_pad_size;
> -    pkt_out.l3_ofs = pkt_in->l3_ofs;
> -    pkt_out.l4_ofs = pkt_in->l4_ofs;
> -
> -    struct udp_header *udp = dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, UDP_HEADER_LEN), UDP_HEADER_LEN);
> -
> -    struct dhcp_header *dhcp_data = dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, DHCP_HEADER_LEN),
> DHCP_HEADER_LEN);
> -    dhcp_data->op = DHCP_OP_REPLY;
> -    dhcp_data->yiaddr = (msg_type == DHCP_MSG_NAK) ? 0 : *offer_ip;
> -    dp_packet_put(&pkt_out, &magic_cookie, sizeof(ovs_be32));
> -
> -    uint16_t out_dhcp_opts_size = 12;
> -    if (msg_type != DHCP_MSG_NAK) {
> -      out_dhcp_opts_size += userdata->size;
> -    }
> -    uint8_t *out_dhcp_opts = dp_packet_put_zeros(&pkt_out,
> -                                                 out_dhcp_opts_size);
> -    /* DHCP option - type */
> -    out_dhcp_opts[0] = DHCP_OPT_MSG_TYPE;
> -    out_dhcp_opts[1] = 1;
> -    out_dhcp_opts[2] = msg_type;
> -    out_dhcp_opts += 3;
> -
> -    if (msg_type != DHCP_MSG_NAK) {
> -      memcpy(out_dhcp_opts, userdata->data, userdata->size);
> -      out_dhcp_opts += userdata->size;
> -    }
> -
> -    /* Padding */
> -    out_dhcp_opts += 4;
> -    /* End */
> -    out_dhcp_opts[0] = DHCP_OPT_END;
> -
> -    udp->udp_len = htons(new_l4_size);
> -
> -    struct ip_header *out_ip = dp_packet_l3(&pkt_out);
> -    out_ip->ip_tot_len = htons(pkt_out.l4_ofs - pkt_out.l3_ofs +
> new_l4_size);
> -    udp->udp_csum = 0;
> -    /* Checksum needs to be initialized to zero. */
> -    out_ip->ip_csum = 0;
> -    out_ip->ip_csum = csum(out_ip, sizeof *out_ip);
> -
> -    pin->packet = dp_packet_data(&pkt_out);
> -    pin->packet_len = dp_packet_size(&pkt_out);
> -
> -    /* Log the response. */
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(20, 40);
> -    const struct eth_header *l2 = dp_packet_eth(&pkt_out);
> -    VLOG_INFO_RL(&rl, "DHCP%s "ETH_ADDR_FMT" "IP_FMT"",
> -                 msg_type == DHCP_MSG_OFFER ? "OFFER" :
> -                   (msg_type == DHCP_MSG_ACK ? "ACK": "NAK"),
> -                 ETH_ADDR_ARGS(l2->eth_src), IP_ARGS(*offer_ip));
> -
> -    success = 1;
> -exit:
> -    if (!ofperr) {
> -        union mf_subvalue sv;
> -        sv.u8_val = success;
> -        mf_write_subfield(&dst, &sv, &pin->flow_metadata);
> -    }
> -    queue_msg(swconn, ofputil_encode_resume(pin, continuation, proto));
> -    if (pkt_out_ptr) {
> -        dp_packet_uninit(pkt_out_ptr);
> -    }
> -}
> -
> -static bool
> -compose_out_dhcpv6_opts(struct ofpbuf *userdata,
> -                        struct ofpbuf *out_dhcpv6_opts, ovs_be32 iaid)
> -{
> -    while (userdata->size) {
> -        struct dhcp_opt6_header *userdata_opt = ofpbuf_try_pull(
> -            userdata, sizeof *userdata_opt);
> -        if (!userdata_opt) {
> -            return false;
> -        }
> -
> -        size_t size = ntohs(userdata_opt->size);
> -        uint8_t *userdata_opt_data = ofpbuf_try_pull(userdata, size);
> -        if (!userdata_opt_data) {
> -            return false;
> -        }
> -
> -        switch (ntohs(userdata_opt->opt_code)) {
> -        case DHCPV6_OPT_SERVER_ID_CODE:
> -        {
> -            /* The Server Identifier option carries a DUID
> -             * identifying a server between a client and a server.
> -             * See RFC 3315 Sec 9 and Sec 22.3.
> -             *
> -             * We use DUID Based on Link-layer Address [DUID-LL].
> -             */
> -
> -            struct dhcpv6_opt_server_id *opt_server_id = ofpbuf_put_zeros(
> -                out_dhcpv6_opts, sizeof *opt_server_id);
> -
> -            opt_server_id->opt.code = htons(DHCPV6_OPT_SERVER_ID_CODE);
> -            opt_server_id->opt.len = htons(size + 4);
> -            opt_server_id->duid_type = htons(DHCPV6_DUID_LL);
> -            opt_server_id->hw_type = htons(DHCPV6_HW_TYPE_ETH);
> -            memcpy(&opt_server_id->mac, userdata_opt_data,
> -                    sizeof(struct eth_addr));
> -            break;
> -        }
> -
> -        case DHCPV6_OPT_IA_ADDR_CODE:
> -        {
> -            if (size != sizeof(struct in6_addr)) {
> -                return false;
> -            }
> -
> -            if (!iaid) {
> -                /* If iaid is None, it means its an DHCPv6 information
> request.
> -                 * Don't put IA_NA option in the response. */
> -                 break;
> -            }
> -            /* IA Address option is used to specify IPv6 addresses
> associated
> -             * with an IA_NA or IA_TA. The IA Address option must be
> -             * encapsulated in the Options field of an IA_NA or IA_TA
> option.
> -             *
> -             * We will encapsulate the IA Address within the IA_NA option.
> -             * Please see RFC 3315 section 22.5 and 22.6
> -             */
> -            struct dhcpv6_opt_ia_na *opt_ia_na = ofpbuf_put_zeros(
> -                out_dhcpv6_opts, sizeof *opt_ia_na);
> -            opt_ia_na->opt.code = htons(DHCPV6_OPT_IA_NA_CODE);
> -            /* IA_NA length (in bytes)-
> -             *  IAID - 4
> -             *  T1   - 4
> -             *  T2   - 4
> -             *  IA Address - sizeof(struct dhcpv6_opt_ia_addr)
> -             */
> -            opt_ia_na->opt.len = htons(12 + sizeof(struct
> dhcpv6_opt_ia_addr));
> -            opt_ia_na->iaid = iaid;
> -            /* Set the lifetime of the address(es) to infinity */
> -            opt_ia_na->t1 = OVS_BE32_MAX;
> -            opt_ia_na->t2 = OVS_BE32_MAX;
> -
> -            struct dhcpv6_opt_ia_addr *opt_ia_addr = ofpbuf_put_zeros(
> -                out_dhcpv6_opts, sizeof *opt_ia_addr);
> -            opt_ia_addr->opt.code = htons(DHCPV6_OPT_IA_ADDR_CODE);
> -            opt_ia_addr->opt.len = htons(size + 8);
> -            memcpy(opt_ia_addr->ipv6.s6_addr, userdata_opt_data, size);
> -            opt_ia_addr->t1 = OVS_BE32_MAX;
> -            opt_ia_addr->t2 = OVS_BE32_MAX;
> -            break;
> -        }
> -
> -        case DHCPV6_OPT_DNS_SERVER_CODE:
> -        {
> -            struct dhcpv6_opt_header *opt_dns = ofpbuf_put_zeros(
> -                out_dhcpv6_opts, sizeof *opt_dns);
> -            opt_dns->code = htons(DHCPV6_OPT_DNS_SERVER_CODE);
> -            opt_dns->len = htons(size);
> -            ofpbuf_put(out_dhcpv6_opts, userdata_opt_data, size);
> -            break;
> -        }
> -
> -        case DHCPV6_OPT_DOMAIN_SEARCH_CODE:
> -        {
> -            struct dhcpv6_opt_header *opt_dsl = ofpbuf_put_zeros(
> -                out_dhcpv6_opts, sizeof *opt_dsl);
> -            opt_dsl->code = htons(DHCPV6_OPT_DOMAIN_SEARCH_CODE);
> -            opt_dsl->len = htons(size + 2);
> -            uint8_t *data = ofpbuf_put_zeros(out_dhcpv6_opts, size + 2);
> -            *data = size;
> -            memcpy(data + 1, userdata_opt_data, size);
> -            break;
> -        }
> -
> -        default:
> -            return false;
> -        }
> -    }
> -    return true;
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_put_dhcpv6_opts(
> -    struct rconn *swconn,
> -    struct dp_packet *pkt_in, struct ofputil_packet_in *pin,
> -    struct ofpbuf *userdata, struct ofpbuf *continuation OVS_UNUSED)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    struct dp_packet *pkt_out_ptr = NULL;
> -    uint32_t success = 0;
> -
> -    /* Parse result field. */
> -    const struct mf_field *f;
> -    enum ofperr ofperr = nx_pull_header(userdata, NULL, &f, NULL);
> -    if (ofperr) {
> -       VLOG_WARN_RL(&rl, "bad result OXM (%s)", ofperr_to_string(ofperr));
> -       goto exit;
> -    }
> -
> -    /* Parse result offset. */
> -    ovs_be32 *ofsp = ofpbuf_try_pull(userdata, sizeof *ofsp);
> -    if (!ofsp) {
> -        VLOG_WARN_RL(&rl, "offset not present in the userdata");
> -        goto exit;
> -    }
> -
> -    /* Check that the result is valid and writable. */
> -    struct mf_subfield dst = { .field = f, .ofs = ntohl(*ofsp), .n_bits =
> 1 };
> -    ofperr = mf_check_dst(&dst, NULL);
> -    if (ofperr) {
> -        VLOG_WARN_RL(&rl, "bad result bit (%s)",
> ofperr_to_string(ofperr));
> -        goto exit;
> -    }
> -
> -    if (!userdata->size) {
> -        VLOG_WARN_RL(&rl, "DHCPv6 options not present in the userdata");
> -        goto exit;
> -    }
> -
> -    struct udp_header *in_udp = dp_packet_l4(pkt_in);
> -    const uint8_t *in_dhcpv6_data = dp_packet_get_udp_payload(pkt_in);
> -    if (!in_udp || !in_dhcpv6_data) {
> -        VLOG_WARN_RL(&rl, "truncated dhcpv6 packet");
> -        goto exit;
> -    }
> -
> -    uint8_t out_dhcpv6_msg_type;
> -    uint8_t in_dhcpv6_msg_type = *in_dhcpv6_data;
> -    switch (in_dhcpv6_msg_type) {
> -    case DHCPV6_MSG_TYPE_SOLICIT:
> -        out_dhcpv6_msg_type = DHCPV6_MSG_TYPE_ADVT;
> -        break;
> -
> -    case DHCPV6_MSG_TYPE_REQUEST:
> -    case DHCPV6_MSG_TYPE_CONFIRM:
> -    case DHCPV6_MSG_TYPE_DECLINE:
> -    case DHCPV6_MSG_TYPE_INFO_REQ:
> -        out_dhcpv6_msg_type = DHCPV6_MSG_TYPE_REPLY;
> -        break;
> -
> -    default:
> -        /* Invalid or unsupported DHCPv6 message type */
> -        goto exit;
> -    }
> -
> -    /* Skip 4 bytes (message type (1 byte) + transaction ID (3 bytes). */
> -    in_dhcpv6_data += 4;
> -    /* We need to extract IAID from the IA-NA option of the client's
> DHCPv6
> -     * solicit/request/confirm packet and copy the same IAID in the
> Server's
> -     * response.
> -     * DHCPv6 information packet (for stateless request will not have
> IA-NA
> -     * option. So we don't need to copy that in the Server's response.
> -     * */
> -    ovs_be32 iaid = 0;
> -    struct dhcpv6_opt_header const *in_opt_client_id = NULL;
> -    size_t udp_len = ntohs(in_udp->udp_len);
> -    size_t l4_len = dp_packet_l4_size(pkt_in);
> -    uint8_t *end = (uint8_t *)in_udp + MIN(udp_len, l4_len);
> -    while (in_dhcpv6_data < end) {
> -        struct dhcpv6_opt_header const *in_opt =
> -             (struct dhcpv6_opt_header *)in_dhcpv6_data;
> -        switch(ntohs(in_opt->code)) {
> -        case DHCPV6_OPT_IA_NA_CODE:
> -        {
> -            struct dhcpv6_opt_ia_na *opt_ia_na = (
> -                struct dhcpv6_opt_ia_na *)in_opt;
> -            iaid = opt_ia_na->iaid;
> -            break;
> -        }
> -
> -        case DHCPV6_OPT_CLIENT_ID_CODE:
> -            in_opt_client_id = in_opt;
> -            break;
> -
> -        default:
> -            break;
> -        }
> -        in_dhcpv6_data += sizeof *in_opt + ntohs(in_opt->len);
> -    }
> -
> -    if (!in_opt_client_id) {
> -        VLOG_WARN_RL(&rl, "DHCPv6 option - Client id not present in the "
> -                     "DHCPv6 packet");
> -        goto exit;
> -    }
> -
> -    if (!iaid && in_dhcpv6_msg_type != DHCPV6_MSG_TYPE_INFO_REQ) {
> -        VLOG_WARN_RL(&rl, "DHCPv6 option - IA NA not present in the "
> -                     "DHCPv6 packet");
> -        goto exit;
> -    }
> -
> -    uint64_t out_ofpacts_dhcpv6_opts_stub[256 / 8];
> -    struct ofpbuf out_dhcpv6_opts =
> -        OFPBUF_STUB_INITIALIZER(out_ofpacts_dhcpv6_opts_stub);
> -
> -    if (!compose_out_dhcpv6_opts(userdata, &out_dhcpv6_opts, iaid)) {
> -        VLOG_WARN_RL(&rl, "Invalid userdata");
> -        goto exit;
> -    }
> -
> -    uint16_t new_l4_size
> -        = (UDP_HEADER_LEN + 4 + sizeof *in_opt_client_id +
> -           ntohs(in_opt_client_id->len) + out_dhcpv6_opts.size);
> -    size_t new_packet_size = pkt_in->l4_ofs + new_l4_size;
> -
> -    struct dp_packet pkt_out;
> -    dp_packet_init(&pkt_out, new_packet_size);
> -    dp_packet_clear(&pkt_out);
> -    dp_packet_prealloc_tailroom(&pkt_out, new_packet_size);
> -    pkt_out_ptr = &pkt_out;
> -
> -    /* Copy L2 and L3 headers from pkt_in. */
> -    dp_packet_put(&pkt_out, dp_packet_pull(pkt_in, pkt_in->l4_ofs),
> -                  pkt_in->l4_ofs);
> -
> -    pkt_out.l2_5_ofs = pkt_in->l2_5_ofs;
> -    pkt_out.l2_pad_size = pkt_in->l2_pad_size;
> -    pkt_out.l3_ofs = pkt_in->l3_ofs;
> -    pkt_out.l4_ofs = pkt_in->l4_ofs;
> -
> -    /* Pull the DHCPv6 message type and transaction id from the pkt_in.
> -     * Need to preserve the transaction id in the DHCPv6 reply packet. */
> -    struct udp_header *out_udp = dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, UDP_HEADER_LEN), UDP_HEADER_LEN);
> -    uint8_t *out_dhcpv6 = dp_packet_put(&pkt_out, dp_packet_pull(pkt_in,
> 4), 4);
> -
> -    /* Set the proper DHCPv6 message type. */
> -    *out_dhcpv6 = out_dhcpv6_msg_type;
> -
> -    /* Copy the Client Identifier. */
> -    dp_packet_put(&pkt_out, in_opt_client_id,
> -                  sizeof *in_opt_client_id +
> ntohs(in_opt_client_id->len));
> -
> -    /* Copy the DHCPv6 Options. */
> -    dp_packet_put(&pkt_out, out_dhcpv6_opts.data, out_dhcpv6_opts.size);
> -    out_udp->udp_len = htons(new_l4_size);
> -    out_udp->udp_csum = 0;
> -
> -    struct ovs_16aligned_ip6_hdr *out_ip6 = dp_packet_l3(&pkt_out);
> -    out_ip6->ip6_ctlun.ip6_un1.ip6_un1_plen = out_udp->udp_len;
> -
> -    uint32_t csum;
> -    csum = packet_csum_pseudoheader6(dp_packet_l3(&pkt_out));
> -    csum = csum_continue(csum, out_udp, dp_packet_size(&pkt_out) -
> -                         ((const unsigned char *)out_udp -
> -                         (const unsigned char *)dp_packet_eth(&pkt_out)));
> -    out_udp->udp_csum = csum_finish(csum);
> -    if (!out_udp->udp_csum) {
> -        out_udp->udp_csum = htons(0xffff);
> -    }
> -
> -    pin->packet = dp_packet_data(&pkt_out);
> -    pin->packet_len = dp_packet_size(&pkt_out);
> -    ofpbuf_uninit(&out_dhcpv6_opts);
> -    success = 1;
> -exit:
> -    if (!ofperr) {
> -        union mf_subvalue sv;
> -        sv.u8_val = success;
> -        mf_write_subfield(&dst, &sv, &pin->flow_metadata);
> -    }
> -    queue_msg(swconn, ofputil_encode_resume(pin, continuation, proto));
> -    dp_packet_uninit(pkt_out_ptr);
> -}
> -
> -static void
> -put_be16(struct ofpbuf *buf, ovs_be16 x)
> -{
> -    ofpbuf_put(buf, &x, sizeof x);
> -}
> -
> -static void
> -put_be32(struct ofpbuf *buf, ovs_be32 x)
> -{
> -    ofpbuf_put(buf, &x, sizeof x);
> -}
> -
> -struct dns_data {
> -    uint64_t *dps;
> -    size_t n_dps;
> -    struct smap records;
> -    bool delete;
> -};
> -
> -static struct shash dns_cache = SHASH_INITIALIZER(&dns_cache);
> -
> -/* Called by pinctrl_run(). Runs within the main ovn-controller
> - * thread context. */
> -static void
> -sync_dns_cache(const struct sbrec_dns_table *dns_table)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct shash_node *iter;
> -    SHASH_FOR_EACH (iter, &dns_cache) {
> -        struct dns_data *d = iter->data;
> -        d->delete = true;
> -    }
> -
> -    const struct sbrec_dns *sbrec_dns;
> -    SBREC_DNS_TABLE_FOR_EACH (sbrec_dns, dns_table) {
> -        const char *dns_id = smap_get(&sbrec_dns->external_ids, "dns_id");
> -        if (!dns_id) {
> -            continue;
> -        }
> -
> -        struct dns_data *dns_data = shash_find_data(&dns_cache, dns_id);
> -        if (!dns_data) {
> -            dns_data = xmalloc(sizeof *dns_data);
> -            smap_init(&dns_data->records);
> -            shash_add(&dns_cache, dns_id, dns_data);
> -            dns_data->n_dps = 0;
> -            dns_data->dps = NULL;
> -        } else {
> -            free(dns_data->dps);
> -        }
> -
> -        dns_data->delete = false;
> -
> -        if (!smap_equal(&dns_data->records, &sbrec_dns->records)) {
> -            smap_clear(&dns_data->records);
> -            smap_clone(&dns_data->records, &sbrec_dns->records);
> -        }
> -
> -        dns_data->n_dps = sbrec_dns->n_datapaths;
> -        dns_data->dps = xcalloc(dns_data->n_dps, sizeof(uint64_t));
> -        for (size_t i = 0; i < sbrec_dns->n_datapaths; i++) {
> -            dns_data->dps[i] = sbrec_dns->datapaths[i]->tunnel_key;
> -        }
> -    }
> -
> -    struct shash_node *next;
> -    SHASH_FOR_EACH_SAFE (iter, next, &dns_cache) {
> -        struct dns_data *d = iter->data;
> -        if (d->delete) {
> -            shash_delete(&dns_cache, iter);
> -            free(d);
> -        }
> -    }
> -}
> -
> -static void
> -destroy_dns_cache(void)
> -{
> -    struct shash_node *iter, *next;
> -    SHASH_FOR_EACH_SAFE (iter, next, &dns_cache) {
> -        struct dns_data *d = iter->data;
> -        shash_delete(&dns_cache, iter);
> -        free(d);
> -    }
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_dns_lookup(
> -    struct rconn *swconn,
> -    struct dp_packet *pkt_in, struct ofputil_packet_in *pin,
> -    struct ofpbuf *userdata, struct ofpbuf *continuation)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    struct dp_packet *pkt_out_ptr = NULL;
> -    uint32_t success = 0;
> -
> -    /* Parse result field. */
> -    const struct mf_field *f;
> -    enum ofperr ofperr = nx_pull_header(userdata, NULL, &f, NULL);
> -    if (ofperr) {
> -       VLOG_WARN_RL(&rl, "bad result OXM (%s)", ofperr_to_string(ofperr));
> -       goto exit;
> -    }
> -
> -    /* Parse result offset. */
> -    ovs_be32 *ofsp = ofpbuf_try_pull(userdata, sizeof *ofsp);
> -    if (!ofsp) {
> -        VLOG_WARN_RL(&rl, "offset not present in the userdata");
> -        goto exit;
> -    }
> -
> -    /* Check that the result is valid and writable. */
> -    struct mf_subfield dst = { .field = f, .ofs = ntohl(*ofsp), .n_bits =
> 1 };
> -    ofperr = mf_check_dst(&dst, NULL);
> -    if (ofperr) {
> -        VLOG_WARN_RL(&rl, "bad result bit (%s)",
> ofperr_to_string(ofperr));
> -        goto exit;
> -    }
> -
> -    /* Extract the DNS header */
> -    struct dns_header const *in_dns_header =
> dp_packet_get_udp_payload(pkt_in);
> -    if (!in_dns_header) {
> -        VLOG_WARN_RL(&rl, "truncated dns packet");
> -        goto exit;
> -    }
> -
> -    /* Check if it is DNS request or not */
> -    if (in_dns_header->lo_flag & 0x80) {
> -        /* It's a DNS response packet which we are not interested in */
> -        goto exit;
> -    }
> -
> -    /* Check if at least one query request is present */
> -    if (!in_dns_header->qdcount) {
> -        goto exit;
> -    }
> -
> -    struct udp_header *in_udp = dp_packet_l4(pkt_in);
> -    size_t udp_len = ntohs(in_udp->udp_len);
> -    size_t l4_len = dp_packet_l4_size(pkt_in);
> -    uint8_t *end = (uint8_t *)in_udp + MIN(udp_len, l4_len);
> -    uint8_t *in_dns_data = (uint8_t *)(in_dns_header + 1);
> -    uint8_t *in_queryname = in_dns_data;
> -    uint8_t idx = 0;
> -    struct ds query_name;
> -    ds_init(&query_name);
> -    /* Extract the query_name. If the query name is - 'www.ovn.org' it
> would be
> -     * encoded as (in hex) - 03 77 77 77 03 6f 76 63 03 6f 72 67 00.
> -     */
> -    while ((in_dns_data + idx) < end && in_dns_data[idx]) {
> -        uint8_t label_len = in_dns_data[idx++];
> -        if (in_dns_data + idx + label_len > end) {
> -            ds_destroy(&query_name);
> -            goto exit;
> -        }
> -        ds_put_buffer(&query_name, (const char *) in_dns_data + idx,
> label_len);
> -        idx += label_len;
> -        ds_put_char(&query_name, '.');
> -    }
> -
> -    idx++;
> -    ds_chomp(&query_name, '.');
> -    in_dns_data += idx;
> -
> -    /* Query should have TYPE and CLASS fields */
> -    if (in_dns_data + (2 * sizeof(ovs_be16)) > end) {
> -        ds_destroy(&query_name);
> -        goto exit;
> -    }
> -
> -    uint16_t query_type = ntohs(*ALIGNED_CAST(const ovs_be16 *,
> in_dns_data));
> -    /* Supported query types - A, AAAA and ANY */
> -    if (!(query_type == DNS_QUERY_TYPE_A || query_type ==
> DNS_QUERY_TYPE_AAAA
> -          || query_type == DNS_QUERY_TYPE_ANY)) {
> -        ds_destroy(&query_name);
> -        goto exit;
> -    }
> -
> -    uint64_t dp_key = ntohll(pin->flow_metadata.flow.metadata);
> -    const char *answer_ips = NULL;
> -    struct shash_node *iter;
> -    SHASH_FOR_EACH (iter, &dns_cache) {
> -        struct dns_data *d = iter->data;
> -        for (size_t i = 0; i < d->n_dps; i++) {
> -            if (d->dps[i] == dp_key) {
> -                answer_ips = smap_get(&d->records, ds_cstr(&query_name));
> -                if (answer_ips) {
> -                    break;
> -                }
> -            }
> -        }
> -
> -        if (answer_ips) {
> -            break;
> -        }
> -    }
> -
> -    ds_destroy(&query_name);
> -    if (!answer_ips) {
> -        goto exit;
> -    }
> -
> -    struct lport_addresses ip_addrs;
> -    if (!extract_ip_addresses(answer_ips, &ip_addrs)) {
> -        goto exit;
> -    }
> -
> -    uint16_t ancount = 0;
> -    uint64_t dns_ans_stub[128 / 8];
> -    struct ofpbuf dns_answer = OFPBUF_STUB_INITIALIZER(dns_ans_stub);
> -
> -    if (query_type == DNS_QUERY_TYPE_A || query_type ==
> DNS_QUERY_TYPE_ANY) {
> -        for (size_t i = 0; i < ip_addrs.n_ipv4_addrs; i++) {
> -            /* Copy the answer section */
> -            /* Format of the answer section is
> -             *  - NAME     -> The domain name
> -             *  - TYPE     -> 2 octets containing one of the RR type codes
> -             *  - CLASS    -> 2 octets which specify the class of the data
> -             *                in the RDATA field.
> -             *  - TTL      -> 32 bit unsigned int specifying the time
> -             *                interval (in secs) that the resource record
> -             *                 may be cached before it should be
> discarded.
> -             *  - RDLENGTH -> 16 bit integer specifying the length of the
> -             *                RDATA field.
> -             *  - RDATA    -> a variable length string of octets that
> -             *                describes the resource. In our case it will
> -             *                be IP address of the domain name.
> -             */
> -            ofpbuf_put(&dns_answer, in_queryname, idx);
> -            put_be16(&dns_answer, htons(DNS_QUERY_TYPE_A));
> -            put_be16(&dns_answer, htons(DNS_CLASS_IN));
> -            put_be32(&dns_answer, htonl(DNS_DEFAULT_RR_TTL));
> -            put_be16(&dns_answer, htons(sizeof(ovs_be32)));
> -            put_be32(&dns_answer, ip_addrs.ipv4_addrs[i].addr);
> -            ancount++;
> -        }
> -    }
> -
> -    if (query_type == DNS_QUERY_TYPE_AAAA ||
> -        query_type == DNS_QUERY_TYPE_ANY) {
> -        for (size_t i = 0; i < ip_addrs.n_ipv6_addrs; i++) {
> -            ofpbuf_put(&dns_answer, in_queryname, idx);
> -            put_be16(&dns_answer, htons(DNS_QUERY_TYPE_AAAA));
> -            put_be16(&dns_answer, htons(DNS_CLASS_IN));
> -            put_be32(&dns_answer, htonl(DNS_DEFAULT_RR_TTL));
> -            const struct in6_addr *ip6 = &ip_addrs.ipv6_addrs[i].addr;
> -            put_be16(&dns_answer, htons(sizeof *ip6));
> -            ofpbuf_put(&dns_answer, ip6, sizeof *ip6);
> -            ancount++;
> -        }
> -    }
> -
> -    destroy_lport_addresses(&ip_addrs);
> -
> -    if (!ancount) {
> -        ofpbuf_uninit(&dns_answer);
> -        goto exit;
> -    }
> -
> -    uint16_t new_l4_size = ntohs(in_udp->udp_len) +  dns_answer.size;
> -    size_t new_packet_size = pkt_in->l4_ofs + new_l4_size;
> -    struct dp_packet pkt_out;
> -    dp_packet_init(&pkt_out, new_packet_size);
> -    dp_packet_clear(&pkt_out);
> -    dp_packet_prealloc_tailroom(&pkt_out, new_packet_size);
> -    pkt_out_ptr = &pkt_out;
> -
> -    /* Copy the L2 and L3 headers from the pkt_in as they would remain
> same.*/
> -    dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, pkt_in->l4_ofs), pkt_in->l4_ofs);
> -
> -    pkt_out.l2_5_ofs = pkt_in->l2_5_ofs;
> -    pkt_out.l2_pad_size = pkt_in->l2_pad_size;
> -    pkt_out.l3_ofs = pkt_in->l3_ofs;
> -    pkt_out.l4_ofs = pkt_in->l4_ofs;
> -
> -    struct udp_header *out_udp = dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, UDP_HEADER_LEN), UDP_HEADER_LEN);
> -
> -    /* Copy the DNS header. */
> -    struct dns_header *out_dns_header = dp_packet_put(
> -        &pkt_out, dp_packet_pull(pkt_in, sizeof *out_dns_header),
> -        sizeof *out_dns_header);
> -
> -    /* Set the response bit to 1 in the flags. */
> -    out_dns_header->lo_flag |= 0x80;
> -
> -    /* Set the answer RR. */
> -    out_dns_header->ancount = htons(ancount);
> -
> -    /* Copy the Query section. */
> -    dp_packet_put(&pkt_out, dp_packet_data(pkt_in),
> dp_packet_size(pkt_in));
> -
> -    /* Copy the answer sections. */
> -    dp_packet_put(&pkt_out, dns_answer.data, dns_answer.size);
> -    ofpbuf_uninit(&dns_answer);
> -
> -    out_udp->udp_len = htons(new_l4_size);
> -    out_udp->udp_csum = 0;
> -
> -    struct eth_header *eth = dp_packet_data(&pkt_out);
> -    if (eth->eth_type == htons(ETH_TYPE_IP)) {
> -        struct ip_header *out_ip = dp_packet_l3(&pkt_out);
> -        out_ip->ip_tot_len = htons(pkt_out.l4_ofs - pkt_out.l3_ofs
> -                                   + new_l4_size);
> -        /* Checksum needs to be initialized to zero. */
> -        out_ip->ip_csum = 0;
> -        out_ip->ip_csum = csum(out_ip, sizeof *out_ip);
> -    } else {
> -        struct ovs_16aligned_ip6_hdr *nh = dp_packet_l3(&pkt_out);
> -        nh->ip6_plen = htons(new_l4_size);
> -
> -        /* IPv6 needs UDP checksum calculated */
> -        uint32_t csum;
> -        csum = packet_csum_pseudoheader6(nh);
> -        csum = csum_continue(csum, out_udp, dp_packet_size(&pkt_out) -
> -                             ((const unsigned char *)out_udp -
> -                             (const unsigned char *)eth));
> -        out_udp->udp_csum = csum_finish(csum);
> -        if (!out_udp->udp_csum) {
> -            out_udp->udp_csum = htons(0xffff);
> -        }
> -    }
> -
> -    pin->packet = dp_packet_data(&pkt_out);
> -    pin->packet_len = dp_packet_size(&pkt_out);
> -
> -    success = 1;
> -exit:
> -    if (!ofperr) {
> -        union mf_subvalue sv;
> -        sv.u8_val = success;
> -        mf_write_subfield(&dst, &sv, &pin->flow_metadata);
> -    }
> -    queue_msg(swconn, ofputil_encode_resume(pin, continuation, proto));
> -    dp_packet_uninit(pkt_out_ptr);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -process_packet_in(struct rconn *swconn, const struct ofp_header *msg)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -
> -    struct ofputil_packet_in pin;
> -    struct ofpbuf continuation;
> -    enum ofperr error = ofputil_decode_packet_in(msg, true, NULL, NULL,
> &pin,
> -                                                 NULL, NULL,
> &continuation);
> -
> -    if (error) {
> -        VLOG_WARN_RL(&rl, "error decoding packet-in: %s",
> -                     ofperr_to_string(error));
> -        return;
> -    }
> -    if (pin.reason != OFPR_ACTION) {
> -        return;
> -    }
> -
> -    struct ofpbuf userdata = ofpbuf_const_initializer(pin.userdata,
> -                                                      pin.userdata_len);
> -    const struct action_header *ah = ofpbuf_pull(&userdata, sizeof *ah);
> -    if (!ah) {
> -        VLOG_WARN_RL(&rl, "packet-in userdata lacks action header");
> -        return;
> -    }
> -
> -    struct dp_packet packet;
> -    dp_packet_use_const(&packet, pin.packet, pin.packet_len);
> -    struct flow headers;
> -    flow_extract(&packet, &headers);
> -
> -    switch (ntohl(ah->opcode)) {
> -    case ACTION_OPCODE_ARP:
> -        pinctrl_handle_arp(swconn, &headers, &packet, &pin.flow_metadata,
> -                           &userdata);
> -        break;
> -    case ACTION_OPCODE_IGMP:
> -        pinctrl_ip_mcast_handle_igmp(swconn, &headers, &packet,
> -                                     &pin.flow_metadata, &userdata);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_ARP:
> -        ovs_mutex_lock(&pinctrl_mutex);
> -        pinctrl_handle_put_mac_binding(&pin.flow_metadata.flow, &headers,
> -                                       true);
> -        ovs_mutex_unlock(&pinctrl_mutex);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_DHCP_OPTS:
> -        pinctrl_handle_put_dhcp_opts(swconn, &packet, &pin, &userdata,
> -                                     &continuation);
> -        break;
> -
> -    case ACTION_OPCODE_ND_NA:
> -        pinctrl_handle_nd_na(swconn, &headers, &pin.flow_metadata,
> &userdata,
> -                             false);
> -        break;
> -
> -    case ACTION_OPCODE_ND_NA_ROUTER:
> -        pinctrl_handle_nd_na(swconn, &headers, &pin.flow_metadata,
> &userdata,
> -                             true);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_ND:
> -        ovs_mutex_lock(&pinctrl_mutex);
> -        pinctrl_handle_put_mac_binding(&pin.flow_metadata.flow, &headers,
> -                                       false);
> -        ovs_mutex_unlock(&pinctrl_mutex);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_DHCPV6_OPTS:
> -        pinctrl_handle_put_dhcpv6_opts(swconn, &packet, &pin, &userdata,
> -                                       &continuation);
> -        break;
> -
> -    case ACTION_OPCODE_DNS_LOOKUP:
> -        ovs_mutex_lock(&pinctrl_mutex);
> -        pinctrl_handle_dns_lookup(swconn, &packet, &pin, &userdata,
> -                                  &continuation);
> -        ovs_mutex_unlock(&pinctrl_mutex);
> -        break;
> -
> -    case ACTION_OPCODE_LOG:
> -        handle_acl_log(&headers, &userdata);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_ND_RA_OPTS:
> -        pinctrl_handle_put_nd_ra_opts(swconn, &headers, &packet, &pin,
> -                                      &userdata, &continuation);
> -        break;
> -
> -    case ACTION_OPCODE_ND_NS:
> -        pinctrl_handle_nd_ns(swconn, &headers, &packet,
> &pin.flow_metadata,
> -                             &userdata);
> -        break;
> -
> -    case ACTION_OPCODE_ICMP:
> -        pinctrl_handle_icmp(swconn, &headers, &packet, &pin.flow_metadata,
> -                            &userdata, false);
> -        break;
> -
> -    case ACTION_OPCODE_ICMP4_ERROR:
> -        pinctrl_handle_icmp(swconn, &headers, &packet, &pin.flow_metadata,
> -                            &userdata, true);
> -        break;
> -
> -    case ACTION_OPCODE_TCP_RESET:
> -        pinctrl_handle_tcp_reset(swconn, &headers, &packet,
> &pin.flow_metadata,
> -                                 &userdata);
> -        break;
> -
> -    case ACTION_OPCODE_PUT_ICMP4_FRAG_MTU:
> -        pinctrl_handle_put_icmp4_frag_mtu(swconn, &headers, &packet,
> -                                          &pin, &userdata, &continuation);
> -        break;
> -
> -    case ACTION_OPCODE_EVENT:
> -        ovs_mutex_lock(&pinctrl_mutex);
> -        pinctrl_handle_event(&userdata);
> -        ovs_mutex_unlock(&pinctrl_mutex);
> -        break;
> -
> -    default:
> -        VLOG_WARN_RL(&rl, "unrecognized packet-in opcode %"PRIu32,
> -                     ntohl(ah->opcode));
> -        break;
> -    }
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_recv(struct rconn *swconn, const struct ofp_header *oh,
> -             enum ofptype type)
> -{
> -    if (type == OFPTYPE_ECHO_REQUEST) {
> -        queue_msg(swconn, ofputil_encode_echo_reply(oh));
> -    } else if (type == OFPTYPE_GET_CONFIG_REPLY) {
> -        /* Enable asynchronous messages */
> -        struct ofputil_switch_config config;
> -
> -        ofputil_decode_get_config_reply(oh, &config);
> -        config.miss_send_len = UINT16_MAX;
> -        set_switch_config(swconn, &config);
> -    } else if (type == OFPTYPE_PACKET_IN) {
> -        process_packet_in(swconn, oh);
> -    } else {
> -        if (VLOG_IS_DBG_ENABLED()) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(30,
> 300);
> -
> -            char *s = ofp_to_string(oh, ntohs(oh->length), NULL, NULL, 2);
> -
> -            VLOG_DBG_RL(&rl, "OpenFlow packet ignored: %s", s);
> -            free(s);
> -        }
> -    }
> -}
> -
> -/* Called with in the main ovn-controller thread context. */
> -static void
> -notify_pinctrl_handler(void)
> -{
> -    seq_change(pinctrl_handler_seq);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -notify_pinctrl_main(void)
> -{
> -    seq_change(pinctrl_main_seq);
> -}
> -
> -/* pinctrl_handler pthread function. */
> -static void *
> -pinctrl_handler(void *arg_)
> -{
> -    struct pinctrl *pctrl = arg_;
> -    /* OpenFlow connection to the switch. */
> -    struct rconn *swconn;
> -    /* Last seen sequence number for 'swconn'.  When this differs from
> -     * rconn_get_connection_seqno(rconn), 'swconn' has reconnected. */
> -    unsigned int conn_seq_no = 0;
> -
> -    char *br_int_name = NULL;
> -    uint64_t new_seq;
> -
> -    /* Next IPV6 RA in seconds. */
> -    static long long int send_ipv6_ra_time = LLONG_MAX;
> -    /* Next GARP announcement in ms. */
> -    static long long int send_garp_time = LLONG_MAX;
> -    /* Next multicast query (IGMP) in ms. */
> -    static long long int send_mcast_query_time = LLONG_MAX;
> -
> -    swconn = rconn_create(5, 0, DSCP_DEFAULT, 1 << OFP13_VERSION);
> -
> -    while (!latch_is_set(&pctrl->pinctrl_thread_exit)) {
> -        if (pctrl->br_int_name) {
> -            if (!br_int_name || strcmp(br_int_name, pctrl->br_int_name)) {
> -                free(br_int_name);
> -                br_int_name = xstrdup(pctrl->br_int_name);
> -            }
> -        }
> -
> -        if (br_int_name) {
> -            char *target;
> -
> -            target = xasprintf("unix:%s/%s.mgmt", ovs_rundir(),
> br_int_name);
> -            if (strcmp(target, rconn_get_target(swconn))) {
> -                VLOG_INFO("%s: connecting to switch", target);
> -                rconn_connect(swconn, target, target);
> -            }
> -            free(target);
> -        } else {
> -            rconn_disconnect(swconn);
> -        }
> -
> -        ovs_mutex_lock(&pinctrl_mutex);
> -        ip_mcast_snoop_run();
> -        ovs_mutex_unlock(&pinctrl_mutex);
> -
> -        rconn_run(swconn);
> -        if (rconn_is_connected(swconn)) {
> -            if (conn_seq_no != rconn_get_connection_seqno(swconn)) {
> -                pinctrl_setup(swconn);
> -                conn_seq_no = rconn_get_connection_seqno(swconn);
> -            }
> -
> -            for (int i = 0; i < 50; i++) {
> -                struct ofpbuf *msg = rconn_recv(swconn);
> -                if (!msg) {
> -                    break;
> -                }
> -
> -                const struct ofp_header *oh = msg->data;
> -                enum ofptype type;
> -
> -                ofptype_decode(&type, oh);
> -                pinctrl_recv(swconn, oh, type);
> -                ofpbuf_delete(msg);
> -            }
> -
> -            if (may_inject_pkts()) {
> -                ovs_mutex_lock(&pinctrl_mutex);
> -                send_garp_run(swconn, &send_garp_time);
> -                send_ipv6_ras(swconn, &send_ipv6_ra_time);
> -                send_mac_binding_buffered_pkts(swconn);
> -                ovs_mutex_unlock(&pinctrl_mutex);
> -
> -                ip_mcast_querier_run(swconn, &send_mcast_query_time);
> -            }
> -        }
> -
> -        rconn_run_wait(swconn);
> -        rconn_recv_wait(swconn);
> -        send_garp_wait(send_garp_time);
> -        ipv6_ra_wait(send_ipv6_ra_time);
> -        ip_mcast_querier_wait(send_mcast_query_time);
> -
> -        new_seq = seq_read(pinctrl_handler_seq);
> -        seq_wait(pinctrl_handler_seq, new_seq);
> -
> -        latch_wait(&pctrl->pinctrl_thread_exit);
> -        poll_block();
> -    }
> -
> -    free(br_int_name);
> -    rconn_destroy(swconn);
> -    return NULL;
> -}
> -
> -/* Called by ovn-controller. */
> -void
> -pinctrl_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -            struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -            struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -            struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -            struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -            struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip,
> -            struct ovsdb_idl_index *sbrec_igmp_groups,
> -            struct ovsdb_idl_index *sbrec_ip_multicast_opts,
> -            const struct sbrec_dns_table *dns_table,
> -            const struct sbrec_controller_event_table *ce_table,
> -            const struct ovsrec_bridge *br_int,
> -            const struct sbrec_chassis *chassis,
> -            const struct hmap *local_datapaths,
> -            const struct sset *active_tunnels)
> -{
> -    ovs_mutex_lock(&pinctrl_mutex);
> -    if (br_int && (!pinctrl.br_int_name || strcmp(pinctrl.br_int_name,
> -                                                  br_int->name))) {
> -        if (pinctrl.br_int_name) {
> -            free(pinctrl.br_int_name);
> -        }
> -        pinctrl.br_int_name = xstrdup(br_int->name);
> -        /* Notify pinctrl_handler that integration bridge is
> -         * set/changed. */
> -        notify_pinctrl_handler();
> -    }
> -    run_put_mac_bindings(ovnsb_idl_txn, sbrec_datapath_binding_by_key,
> -                         sbrec_port_binding_by_key,
> -                         sbrec_mac_binding_by_lport_ip);
> -    send_garp_prepare(sbrec_port_binding_by_datapath,
> -                      sbrec_port_binding_by_name, br_int, chassis,
> -                      local_datapaths, active_tunnels);
> -    prepare_ipv6_ras(sbrec_port_binding_by_datapath,
> -                     sbrec_port_binding_by_name, local_datapaths);
> -    sync_dns_cache(dns_table);
> -    controller_event_run(ovnsb_idl_txn, ce_table, chassis);
> -    ip_mcast_sync(ovnsb_idl_txn, chassis, local_datapaths,
> -                  sbrec_datapath_binding_by_key,
> -                  sbrec_port_binding_by_key,
> -                  sbrec_igmp_groups,
> -                  sbrec_ip_multicast_opts);
> -    run_buffered_binding(sbrec_port_binding_by_datapath,
> -                         sbrec_mac_binding_by_lport_ip,
> -                         local_datapaths);
> -    ovs_mutex_unlock(&pinctrl_mutex);
> -}
> -
> -/* Table of ipv6_ra_state structures, keyed on logical port name.
> - * Protected by pinctrl_mutex. */
> -static struct shash ipv6_ras;
> -
> -struct ipv6_ra_config {
> -    time_t min_interval;
> -    time_t max_interval;
> -    struct eth_addr eth_src;
> -    struct eth_addr eth_dst;
> -    struct in6_addr ipv6_src;
> -    struct in6_addr ipv6_dst;
> -    int32_t mtu;
> -    uint8_t mo_flags; /* Managed/Other flags for RAs */
> -    uint8_t la_flags; /* On-link/autonomous flags for address prefixes */
> -    struct lport_addresses prefixes;
> -};
> -
> -struct ipv6_ra_state {
> -    long long int next_announce;
> -    struct ipv6_ra_config *config;
> -    int64_t port_key;
> -    int64_t metadata;
> -    bool delete_me;
> -};
> -
> -static void
> -init_ipv6_ras(void)
> -{
> -    shash_init(&ipv6_ras);
> -}
> -
> -static void
> -ipv6_ra_config_delete(struct ipv6_ra_config *config)
> -{
> -    if (config) {
> -        destroy_lport_addresses(&config->prefixes);
> -        free(config);
> -    }
> -}
> -
> -static void
> -ipv6_ra_delete(struct ipv6_ra_state *ra)
> -{
> -    if (ra) {
> -        ipv6_ra_config_delete(ra->config);
> -        free(ra);
> -    }
> -}
> -
> -static void
> -destroy_ipv6_ras(void)
> -{
> -    struct shash_node *iter, *next;
> -    SHASH_FOR_EACH_SAFE (iter, next, &ipv6_ras) {
> -        struct ipv6_ra_state *ra = iter->data;
> -        ipv6_ra_delete(ra);
> -        shash_delete(&ipv6_ras, iter);
> -    }
> -    shash_destroy(&ipv6_ras);
> -}
> -
> -static struct ipv6_ra_config *
> -ipv6_ra_update_config(const struct sbrec_port_binding *pb)
> -{
> -    struct ipv6_ra_config *config;
> -
> -    config = xzalloc(sizeof *config);
> -
> -    config->max_interval = smap_get_int(&pb->options,
> "ipv6_ra_max_interval",
> -            ND_RA_MAX_INTERVAL_DEFAULT);
> -    config->min_interval = smap_get_int(&pb->options,
> "ipv6_ra_min_interval",
> -            nd_ra_min_interval_default(config->max_interval));
> -    config->mtu = smap_get_int(&pb->options, "ipv6_ra_mtu",
> ND_MTU_DEFAULT);
> -    config->la_flags = IPV6_ND_RA_OPT_PREFIX_ON_LINK;
> -
> -    const char *address_mode = smap_get(&pb->options,
> "ipv6_ra_address_mode");
> -    if (!address_mode) {
> -        VLOG_WARN("No address mode specified");
> -        goto fail;
> -    }
> -    if (!strcmp(address_mode, "dhcpv6_stateless")) {
> -        config->mo_flags = IPV6_ND_RA_FLAG_OTHER_ADDR_CONFIG;
> -        config->la_flags |= IPV6_ND_RA_OPT_PREFIX_AUTONOMOUS;
> -    } else if (!strcmp(address_mode, "dhcpv6_stateful")) {
> -        config->mo_flags = IPV6_ND_RA_FLAG_MANAGED_ADDR_CONFIG;
> -    } else if (!strcmp(address_mode, "slaac")) {
> -        config->la_flags |= IPV6_ND_RA_OPT_PREFIX_AUTONOMOUS;
> -    } else {
> -        VLOG_WARN("Invalid address mode %s", address_mode);
> -        goto fail;
> -    }
> -
> -    const char *prefixes = smap_get(&pb->options, "ipv6_ra_prefixes");
> -    if (prefixes && !extract_ip_addresses(prefixes, &config->prefixes)) {
> -        VLOG_WARN("Invalid IPv6 prefixes: %s", prefixes);
> -        goto fail;
> -    }
> -
> -    /* All nodes multicast addresses */
> -    config->eth_dst = (struct eth_addr) ETH_ADDR_C(33,33,00,00,00,01);
> -    ipv6_parse("ff02::1", &config->ipv6_dst);
> -
> -    const char *eth_addr = smap_get(&pb->options, "ipv6_ra_src_eth");
> -    if (!eth_addr || !eth_addr_from_string(eth_addr, &config->eth_src)) {
> -        VLOG_WARN("Invalid ethernet source %s", eth_addr);
> -        goto fail;
> -    }
> -    const char *ip_addr = smap_get(&pb->options, "ipv6_ra_src_addr");
> -    if (!ip_addr || !ipv6_parse(ip_addr, &config->ipv6_src)) {
> -        VLOG_WARN("Invalid IP source %s", ip_addr);
> -        goto fail;
> -    }
> -
> -    return config;
> -
> -fail:
> -    ipv6_ra_config_delete(config);
> -    return NULL;
> -}
> -
> -static long long int
> -ipv6_ra_calc_next_announce(time_t min_interval, time_t max_interval)
> -{
> -    long long int min_interval_ms = min_interval * 1000LL;
> -    long long int max_interval_ms = max_interval * 1000LL;
> -
> -    return time_msec() + min_interval_ms +
> -        random_range(max_interval_ms - min_interval_ms);
> -}
> -
> -static void
> -put_load(uint64_t value, enum mf_field_id dst, int ofs, int n_bits,
> -         struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts,
> -                                                       mf_from_id(dst),
> NULL,
> -                                                       NULL);
> -    ovs_be64 n_value = htonll(value);
> -    bitwise_copy(&n_value, 8, 0, sf->value, sf->field->n_bytes, ofs,
> n_bits);
> -    bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs,
> n_bits);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static long long int
> -ipv6_ra_send(struct rconn *swconn, struct ipv6_ra_state *ra)
> -{
> -    if (time_msec() < ra->next_announce) {
> -        return ra->next_announce;
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    compose_nd_ra(&packet, ra->config->eth_src, ra->config->eth_dst,
> -            &ra->config->ipv6_src, &ra->config->ipv6_dst,
> -            255, ra->config->mo_flags, htons(IPV6_ND_RA_LIFETIME), 0, 0,
> -            ra->config->mtu);
> -
> -    for (int i = 0; i < ra->config->prefixes.n_ipv6_addrs; i++) {
> -        ovs_be128 addr;
> -        memcpy(&addr, &ra->config->prefixes.ipv6_addrs[i].addr, sizeof
> addr);
> -        packet_put_ra_prefix_opt(&packet,
> -            ra->config->prefixes.ipv6_addrs[i].plen,
> -            ra->config->la_flags,
> htonl(IPV6_ND_RA_OPT_PREFIX_VALID_LIFETIME),
> -            htonl(IPV6_ND_RA_OPT_PREFIX_PREFERRED_LIFETIME), addr);
> -    }
> -
> -    uint64_t ofpacts_stub[4096 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -
> -    /* Set MFF_LOG_DATAPATH and MFF_LOG_INPORT. */
> -    uint32_t dp_key = ra->metadata;
> -    uint32_t port_key = ra->port_key;
> -    put_load(dp_key, MFF_LOG_DATAPATH, 0, 64, &ofpacts);
> -    put_load(port_key, MFF_LOG_INPORT, 0, 32, &ofpacts);
> -    put_load(1, MFF_LOG_FLAGS, MLF_LOCAL_ONLY_BIT, 1, &ofpacts);
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(&ofpacts);
> -    resubmit->in_port = OFPP_CONTROLLER;
> -    resubmit->table_id = OFTABLE_LOG_INGRESS_PIPELINE;
> -
> -    struct ofputil_packet_out po = {
> -        .packet = dp_packet_data(&packet),
> -        .packet_len = dp_packet_size(&packet),
> -        .buffer_id = UINT32_MAX,
> -        .ofpacts = ofpacts.data,
> -        .ofpacts_len = ofpacts.size,
> -    };
> -
> -    match_set_in_port(&po.flow_metadata, OFPP_CONTROLLER);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    queue_msg(swconn, ofputil_encode_packet_out(&po, proto));
> -    dp_packet_uninit(&packet);
> -    ofpbuf_uninit(&ofpacts);
> -
> -    ra->next_announce =
> ipv6_ra_calc_next_announce(ra->config->min_interval,
> -            ra->config->max_interval);
> -
> -    return ra->next_announce;
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -ipv6_ra_wait(long long int send_ipv6_ra_time)
> -{
> -    /* Set the poll timer for next IPv6 RA only if IPv6 RAs needs to
> -     * be sent. */
> -    if (!shash_is_empty(&ipv6_ras)) {
> -        poll_timer_wait_until(send_ipv6_ra_time);
> -    }
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -send_ipv6_ras(struct rconn *swconn, long long int *send_ipv6_ra_time)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    *send_ipv6_ra_time = LLONG_MAX;
> -    struct shash_node *iter;
> -    SHASH_FOR_EACH (iter, &ipv6_ras) {
> -        struct ipv6_ra_state *ra = iter->data;
> -        long long int next_ra = ipv6_ra_send(swconn, ra);
> -        if (*send_ipv6_ra_time > next_ra) {
> -            *send_ipv6_ra_time = next_ra;
> -        }
> -    }
> -}
> -
> -/* Called by pinctrl_run(). Runs with in the main ovn-controller
> - * thread context. */
> -static void
> -prepare_ipv6_ras(struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                 const struct hmap *local_datapaths)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct shash_node *iter, *iter_next;
> -
> -    SHASH_FOR_EACH (iter, &ipv6_ras) {
> -        struct ipv6_ra_state *ra = iter->data;
> -        ra->delete_me = true;
> -    }
> -
> -    bool changed = false;
> -    const struct local_datapath *ld;
> -    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> -        struct sbrec_port_binding *target =
> sbrec_port_binding_index_init_row(
> -            sbrec_port_binding_by_datapath);
> -        sbrec_port_binding_index_set_datapath(target, ld->datapath);
> -
> -        struct sbrec_port_binding *pb;
> -        SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target,
> -
>  sbrec_port_binding_by_datapath) {
> -            if (!smap_get_bool(&pb->options, "ipv6_ra_send_periodic",
> false)) {
> -                continue;
> -            }
> -
> -            const char *peer_s = smap_get(&pb->options, "peer");
> -            if (!peer_s) {
> -                continue;
> -            }
> -
> -            const struct sbrec_port_binding *peer
> -                = lport_lookup_by_name(sbrec_port_binding_by_name,
> peer_s);
> -            if (!peer) {
> -                continue;
> -            }
> -
> -            struct ipv6_ra_config *config = ipv6_ra_update_config(pb);
> -            if (!config) {
> -                continue;
> -            }
> -
> -            struct ipv6_ra_state *ra
> -                = shash_find_data(&ipv6_ras, pb->logical_port);
> -            if (!ra) {
> -                ra = xzalloc(sizeof *ra);
> -                ra->config = config;
> -                ra->next_announce = ipv6_ra_calc_next_announce(
> -                    ra->config->min_interval,
> -                    ra->config->max_interval);
> -                shash_add(&ipv6_ras, pb->logical_port, ra);
> -                changed = true;
> -            } else {
> -                if (config->min_interval != ra->config->min_interval ||
> -                    config->max_interval != ra->config->max_interval)
> -                    ra->next_announce = ipv6_ra_calc_next_announce(
> -                        config->min_interval,
> -                        config->max_interval);
> -                ipv6_ra_config_delete(ra->config);
> -                ra->config = config;
> -            }
> -
> -            /* Peer is the logical switch port that the logical
> -             * router port is connected to. The RA is injected
> -             * into that logical switch port.
> -             */
> -            ra->port_key = peer->tunnel_key;
> -            ra->metadata = peer->datapath->tunnel_key;
> -            ra->delete_me = false;
> -
> -            /* pinctrl_handler thread will send the IPv6 RAs. */
> -        }
> -        sbrec_port_binding_index_destroy_row(target);
> -    }
> -
> -    /* Remove those that are no longer in the SB database */
> -    SHASH_FOR_EACH_SAFE (iter, iter_next, &ipv6_ras) {
> -        struct ipv6_ra_state *ra = iter->data;
> -        if (ra->delete_me) {
> -            shash_delete(&ipv6_ras, iter);
> -            ipv6_ra_delete(ra);
> -        }
> -    }
> -
> -    if (changed) {
> -        notify_pinctrl_handler();
> -    }
> -
> -}
> -
> -/* Called by pinctrl_run(). Runs with in the main ovn-controller
> - * thread context. */
> -void
> -pinctrl_wait(struct ovsdb_idl_txn *ovnsb_idl_txn)
> -{
> -    wait_put_mac_bindings(ovnsb_idl_txn);
> -    wait_controller_event(ovnsb_idl_txn);
> -    int64_t new_seq = seq_read(pinctrl_main_seq);
> -    seq_wait(pinctrl_main_seq, new_seq);
> -}
> -
> -/* Called by ovn-controller. */
> -void
> -pinctrl_destroy(void)
> -{
> -    latch_set(&pinctrl.pinctrl_thread_exit);
> -    pthread_join(pinctrl.pinctrl_thread, NULL);
> -    latch_destroy(&pinctrl.pinctrl_thread_exit);
> -    free(pinctrl.br_int_name);
> -    destroy_send_garps();
> -    destroy_ipv6_ras();
> -    destroy_buffered_packets_map();
> -    event_table_destroy();
> -    destroy_put_mac_bindings();
> -    destroy_dns_cache();
> -    ip_mcast_snoop_destroy();
> -    seq_destroy(pinctrl_main_seq);
> -    seq_destroy(pinctrl_handler_seq);
> -}
> -
> -/* Implementation of the "put_arp" and "put_nd" OVN actions.  These
> - * actions send a packet to ovn-controller, using the flow as an API
> - * (see actions.h for details).  This code implements the actions by
> - * updating the MAC_Binding table in the southbound database.
> - *
> - * This code could be a lot simpler if the database could always be
> updated,
> - * but in fact we can only update it when 'ovnsb_idl_txn' is nonnull.
> Thus,
> - * we buffer up a few put_mac_bindings (but we don't keep them longer
> - * than 1 second) and apply them whenever a database transaction is
> - * available. */
> -
> -/* Buffered "put_mac_binding" operation. */
> -struct put_mac_binding {
> -    struct hmap_node hmap_node; /* In 'put_mac_bindings'. */
> -
> -    long long int timestamp;    /* In milliseconds. */
> -
> -    /* Key. */
> -    uint32_t dp_key;
> -    uint32_t port_key;
> -    struct in6_addr ip_key;
> -
> -    /* Value. */
> -    struct eth_addr mac;
> -};
> -
> -/* Contains "struct put_mac_binding"s. */
> -static struct hmap put_mac_bindings;
> -
> -static void
> -init_put_mac_bindings(void)
> -{
> -    hmap_init(&put_mac_bindings);
> -}
> -
> -static void
> -destroy_put_mac_bindings(void)
> -{
> -    flush_put_mac_bindings();
> -    hmap_destroy(&put_mac_bindings);
> -}
> -
> -static struct put_mac_binding *
> -pinctrl_find_put_mac_binding(uint32_t dp_key, uint32_t port_key,
> -                             const struct in6_addr *ip_key, uint32_t hash)
> -{
> -    struct put_mac_binding *pa;
> -    HMAP_FOR_EACH_WITH_HASH (pa, hmap_node, hash, &put_mac_bindings) {
> -        if (pa->dp_key == dp_key
> -            && pa->port_key == port_key
> -            && IN6_ARE_ADDR_EQUAL(&pa->ip_key, ip_key)) {
> -            return pa;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_put_mac_binding(const struct flow *md,
> -                               const struct flow *headers,
> -                               bool is_arp)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    uint32_t dp_key = ntohll(md->metadata);
> -    uint32_t port_key = md->regs[MFF_LOG_INPORT - MFF_REG0];
> -    struct in6_addr ip_key;
> -
> -    if (is_arp) {
> -        ip_key = in6_addr_mapped_ipv4(htonl(md->regs[0]));
> -    } else {
> -        ovs_be128 ip6 = hton128(flow_get_xxreg(md, 0));
> -        memcpy(&ip_key, &ip6, sizeof ip_key);
> -    }
> -    uint32_t hash = hash_bytes(&ip_key, sizeof ip_key,
> -                               hash_2words(dp_key, port_key));
> -    struct put_mac_binding *pmb
> -        = pinctrl_find_put_mac_binding(dp_key, port_key, &ip_key, hash);
> -    if (!pmb) {
> -        if (hmap_count(&put_mac_bindings) >= 1000) {
> -            COVERAGE_INC(pinctrl_drop_put_mac_binding);
> -            return;
> -        }
> -
> -        pmb = xmalloc(sizeof *pmb);
> -        hmap_insert(&put_mac_bindings, &pmb->hmap_node, hash);
> -        pmb->dp_key = dp_key;
> -        pmb->port_key = port_key;
> -        pmb->ip_key = ip_key;
> -    }
> -    pmb->timestamp = time_msec();
> -    pmb->mac = headers->dl_src;
> -
> -    /* We can send the buffered packet once the main ovn-controller
> -     * thread calls pinctrl_run() and it writes the mac_bindings stored
> -     * in 'put_mac_bindings' hmap into the Southbound MAC_Binding table.
> */
> -    notify_pinctrl_main();
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -send_mac_binding_buffered_pkts(struct rconn *swconn)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct buffered_packets *bp;
> -    LIST_FOR_EACH_POP (bp, list, &buffered_mac_bindings) {
> -        buffered_send_packets(swconn, bp, &bp->ea);
> -        free(bp);
> -    }
> -    ovs_list_init(&buffered_mac_bindings);
> -}
> -
> -static const struct sbrec_mac_binding *
> -mac_binding_lookup(struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip,
> -                   const char *logical_port,
> -                   const char *ip)
> -{
> -    struct sbrec_mac_binding *mb = sbrec_mac_binding_index_init_row(
> -        sbrec_mac_binding_by_lport_ip);
> -    sbrec_mac_binding_index_set_logical_port(mb, logical_port);
> -    sbrec_mac_binding_index_set_ip(mb, ip);
> -
> -    const struct sbrec_mac_binding *retval
> -        = sbrec_mac_binding_index_find(sbrec_mac_binding_by_lport_ip,
> -                                       mb);
> -
> -    sbrec_mac_binding_index_destroy_row(mb);
> -
> -    return retval;
> -}
> -
> -static void
> -run_put_mac_binding(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                    struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -                    struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -                    struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip,
> -                    const struct put_mac_binding *pmb)
> -{
> -    if (time_msec() > pmb->timestamp + 1000) {
> -        return;
> -    }
> -
> -    /* Convert logical datapath and logical port key into lport. */
> -    const struct sbrec_port_binding *pb = lport_lookup_by_key(
> -        sbrec_datapath_binding_by_key, sbrec_port_binding_by_key,
> -        pmb->dp_key, pmb->port_key);
> -    if (!pb) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -
> -        VLOG_WARN_RL(&rl, "unknown logical port with datapath %"PRIu32" "
> -                     "and port %"PRIu32, pmb->dp_key, pmb->port_key);
> -        return;
> -    }
> -
> -    /* Convert ethernet argument to string form for database. */
> -    char mac_string[ETH_ADDR_STRLEN + 1];
> -    snprintf(mac_string, sizeof mac_string,
> -             ETH_ADDR_FMT, ETH_ADDR_ARGS(pmb->mac));
> -
> -    struct ds ip_s = DS_EMPTY_INITIALIZER;
> -    ipv6_format_mapped(&pmb->ip_key, &ip_s);
> -
> -    /* Update or add an IP-MAC binding for this logical port. */
> -    const struct sbrec_mac_binding *b =
> -        mac_binding_lookup(sbrec_mac_binding_by_lport_ip,
> pb->logical_port,
> -                           ds_cstr(&ip_s));
> -    if (!b) {
> -        b = sbrec_mac_binding_insert(ovnsb_idl_txn);
> -        sbrec_mac_binding_set_logical_port(b, pb->logical_port);
> -        sbrec_mac_binding_set_ip(b, ds_cstr(&ip_s));
> -        sbrec_mac_binding_set_mac(b, mac_string);
> -        sbrec_mac_binding_set_datapath(b, pb->datapath);
> -    } else if (strcmp(b->mac, mac_string)) {
> -        sbrec_mac_binding_set_mac(b, mac_string);
> -    }
> -    ds_destroy(&ip_s);
> -}
> -
> -/* Called by pinctrl_run(). Runs with in the main ovn-controller
> - * thread context. */
> -static void
> -run_put_mac_bindings(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                     struct ovsdb_idl_index
> *sbrec_datapath_binding_by_key,
> -                     struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -                     struct ovsdb_idl_index
> *sbrec_mac_binding_by_lport_ip)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    if (!ovnsb_idl_txn) {
> -        return;
> -    }
> -
> -    const struct put_mac_binding *pmb;
> -    HMAP_FOR_EACH (pmb, hmap_node, &put_mac_bindings) {
> -        run_put_mac_binding(ovnsb_idl_txn, sbrec_datapath_binding_by_key,
> -                            sbrec_port_binding_by_key,
> -                            sbrec_mac_binding_by_lport_ip,
> -                            pmb);
> -    }
> -    flush_put_mac_bindings();
> -}
> -
> -static void
> -run_buffered_binding(struct ovsdb_idl_index
> *sbrec_port_binding_by_datapath,
> -                     struct ovsdb_idl_index
> *sbrec_mac_binding_by_lport_ip,
> -                     const struct hmap *local_datapaths)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    const struct local_datapath *ld;
> -    bool notify = false;
> -
> -    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> -        struct sbrec_port_binding *target =
> sbrec_port_binding_index_init_row(
> -                sbrec_port_binding_by_datapath);
> -        sbrec_port_binding_index_set_datapath(target, ld->datapath);
> -
> -        const struct sbrec_port_binding *pb;
> -        SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target,
> -
>  sbrec_port_binding_by_datapath) {
> -            struct buffered_packets *cur_qp, *next_qp;
> -            HMAP_FOR_EACH_SAFE (cur_qp, next_qp, hmap_node,
> -                                &buffered_packets_map) {
> -                struct ds ip_s = DS_EMPTY_INITIALIZER;
> -                ipv6_format_mapped(&cur_qp->ip, &ip_s);
> -                const struct sbrec_mac_binding *b = mac_binding_lookup(
> -                        sbrec_mac_binding_by_lport_ip, pb->logical_port,
> -                        ds_cstr(&ip_s));
> -                if (b && ovs_scan(b->mac, ETH_ADDR_SCAN_FMT,
> -                                  ETH_ADDR_SCAN_ARGS(cur_qp->ea))) {
> -                    hmap_remove(&buffered_packets_map,
> &cur_qp->hmap_node);
> -                    ovs_list_push_back(&buffered_mac_bindings,
> &cur_qp->list);
> -                    notify = true;
> -                }
> -                ds_destroy(&ip_s);
> -            }
> -        }
> -        sbrec_port_binding_index_destroy_row(target);
> -    }
> -    buffered_packets_map_gc();
> -
> -    if (notify) {
> -        notify_pinctrl_handler();
> -    }
> -}
> -
> -static void
> -wait_put_mac_bindings(struct ovsdb_idl_txn *ovnsb_idl_txn)
> -{
> -    if (ovnsb_idl_txn && !hmap_is_empty(&put_mac_bindings)) {
> -        poll_immediate_wake();
> -    }
> -}
> -
> -static void
> -flush_put_mac_bindings(void)
> -{
> -    struct put_mac_binding *pmb;
> -    HMAP_FOR_EACH_POP (pmb, hmap_node, &put_mac_bindings) {
> -        free(pmb);
> -    }
> -}
> -
> -/*
> - * Send gratuitous ARP for vif on localnet.
> - *
> - * When a new vif on localnet is added, gratuitous ARPs are sent
> announcing
> - * the port's mac,ip mapping.  On localnet, such announcements are needed
> for
> - * switches and routers on the broadcast segment to update their port-mac
> - * and ARP tables.
> - */
> -struct garp_data {
> -    struct eth_addr ea;          /* Ethernet address of port. */
> -    ovs_be32 ipv4;               /* Ipv4 address of port. */
> -    long long int announce_time; /* Next announcement in ms. */
> -    int backoff;                 /* Backoff for the next announcement. */
> -    uint32_t dp_key;             /* Datapath used to output this GARP. */
> -    uint32_t port_key;           /* Port to inject the GARP into. */
> -};
> -
> -/* Contains GARPs to be sent. Protected by pinctrl_mutex*/
> -static struct shash send_garp_data;
> -
> -static void
> -init_send_garps(void)
> -{
> -    shash_init(&send_garp_data);
> -}
> -
> -static void
> -destroy_send_garps(void)
> -{
> -    shash_destroy_free_data(&send_garp_data);
> -}
> -
> -/* Runs with in the main ovn-controller thread context. */
> -static void
> -add_garp(const char *name, const struct eth_addr ea, ovs_be32 ip,
> -         uint32_t dp_key, uint32_t port_key)
> -{
> -    struct garp_data *garp = xmalloc(sizeof *garp);
> -    garp->ea = ea;
> -    garp->ipv4 = ip;
> -    garp->announce_time = time_msec() + 1000;
> -    garp->backoff = 1;
> -    garp->dp_key = dp_key;
> -    garp->port_key = port_key;
> -    shash_add(&send_garp_data, name, garp);
> -
> -    /* Notify pinctrl_handler so that it can wakeup and process
> -     * these GARP requests. */
> -    notify_pinctrl_handler();
> -}
> -
> -/* Add or update a vif for which GARPs need to be announced. */
> -static void
> -send_garp_update(const struct sbrec_port_binding *binding_rec,
> -                 struct shash *nat_addresses)
> -{
> -    volatile struct garp_data *garp = NULL;
> -    /* Update GARP for NAT IP if it exists.  Consider port bindings with
> type
> -     * "l3gateway" for logical switch ports attached to gateway routers,
> and
> -     * port bindings with type "patch" for logical switch ports attached
> to
> -     * distributed gateway ports. */
> -    if (!strcmp(binding_rec->type, "l3gateway")
> -        || !strcmp(binding_rec->type, "patch")) {
> -        struct lport_addresses *laddrs = NULL;
> -        while ((laddrs = shash_find_and_delete(nat_addresses,
> -
>  binding_rec->logical_port))) {
> -            int i;
> -            for (i = 0; i < laddrs->n_ipv4_addrs; i++) {
> -                char *name = xasprintf("%s-%s", binding_rec->logical_port,
> -
> laddrs->ipv4_addrs[i].addr_s);
> -                garp = shash_find_data(&send_garp_data, name);
> -                if (garp) {
> -                    garp->dp_key = binding_rec->datapath->tunnel_key;
> -                    garp->port_key = binding_rec->tunnel_key;
> -                } else {
> -                    add_garp(name, laddrs->ea,
> -                             laddrs->ipv4_addrs[i].addr,
> -                             binding_rec->datapath->tunnel_key,
> -                             binding_rec->tunnel_key);
> -                }
> -                free(name);
> -            }
> -            destroy_lport_addresses(laddrs);
> -            free(laddrs);
> -        }
> -        return;
> -    }
> -
> -    /* Update GARP for vif if it exists. */
> -    garp = shash_find_data(&send_garp_data, binding_rec->logical_port);
> -    if (garp) {
> -        garp->dp_key = binding_rec->datapath->tunnel_key;
> -        garp->port_key = binding_rec->tunnel_key;
> -        return;
> -    }
> -
> -    /* Add GARP for new vif. */
> -    int i;
> -    for (i = 0; i < binding_rec->n_mac; i++) {
> -        struct lport_addresses laddrs;
> -        if (!extract_lsp_addresses(binding_rec->mac[i], &laddrs)
> -            || !laddrs.n_ipv4_addrs) {
> -            continue;
> -        }
> -
> -        add_garp(binding_rec->logical_port,
> -                 laddrs.ea, laddrs.ipv4_addrs[0].addr,
> -                 binding_rec->datapath->tunnel_key,
> binding_rec->tunnel_key);
> -
> -        destroy_lport_addresses(&laddrs);
> -        break;
> -    }
> -}
> -
> -/* Remove a vif from GARP announcements. */
> -static void
> -send_garp_delete(const char *lport)
> -{
> -    struct garp_data *garp = shash_find_and_delete(&send_garp_data,
> lport);
> -    free(garp);
> -    notify_pinctrl_handler();
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static long long int
> -send_garp(struct rconn *swconn, struct garp_data *garp,
> -          long long int current_time)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    if (current_time < garp->announce_time) {
> -        return garp->announce_time;
> -    }
> -
> -    /* Compose a GARP request packet. */
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    compose_arp(&packet, ARP_OP_REQUEST, garp->ea, eth_addr_zero,
> -                true, garp->ipv4, garp->ipv4);
> -
> -    /* Inject GARP request. */
> -    uint64_t ofpacts_stub[4096 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    put_load(garp->dp_key, MFF_LOG_DATAPATH, 0, 64, &ofpacts);
> -    put_load(garp->port_key, MFF_LOG_INPORT, 0, 32, &ofpacts);
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(&ofpacts);
> -    resubmit->in_port = OFPP_CONTROLLER;
> -    resubmit->table_id = OFTABLE_LOG_INGRESS_PIPELINE;
> -
> -    struct ofputil_packet_out po = {
> -        .packet = dp_packet_data(&packet),
> -        .packet_len = dp_packet_size(&packet),
> -        .buffer_id = UINT32_MAX,
> -        .ofpacts = ofpacts.data,
> -        .ofpacts_len = ofpacts.size,
> -    };
> -    match_set_in_port(&po.flow_metadata, OFPP_CONTROLLER);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    queue_msg(swconn, ofputil_encode_packet_out(&po, proto));
> -    dp_packet_uninit(&packet);
> -    ofpbuf_uninit(&ofpacts);
> -
> -    /* Set the next announcement.  At most 5 announcements are sent for a
> -     * vif. */
> -    if (garp->backoff < 16) {
> -        garp->backoff *= 2;
> -        garp->announce_time = current_time + garp->backoff * 1000;
> -    } else {
> -        garp->announce_time = LLONG_MAX;
> -    }
> -    return garp->announce_time;
> -}
> -
> -/*
> - * Multicast snooping configuration.
> - */
> -struct ip_mcast_snoop_cfg {
> -    bool enabled;
> -    bool querier_enabled;
> -
> -    uint32_t table_size;       /* Max number of allowed multicast groups.
> */
> -    uint32_t idle_time_s;      /* Idle timeout for multicast groups. */
> -    uint32_t query_interval_s; /* Multicast query interval. */
> -    uint32_t query_max_resp_s; /* Multicast query max-response field. */
> -    uint32_t seq_no;           /* Used for flushing learnt groups. */
> -
> -    struct eth_addr query_eth_src; /* Src ETH address used for queries. */
> -    struct eth_addr query_eth_dst; /* Dst ETH address used for queries. */
> -    ovs_be32 query_ipv4_src;       /* Src IPv4 address used for queries.
> */
> -    ovs_be32 query_ipv4_dst;       /* Dsc IPv4 address used for queries.
> */
> -};
> -
> -/*
> - * Holds per-datapath information about multicast snooping. Maintained by
> - * pinctrl_handler().
> - */
> -struct ip_mcast_snoop {
> -    struct hmap_node hmap_node;    /* Linkage in the hash map. */
> -    struct ovs_list query_node;    /* Linkage in the query list. */
> -    struct ip_mcast_snoop_cfg cfg; /* Multicast configuration. */
> -    struct mcast_snooping *ms;     /* Multicast group state. */
> -    int64_t dp_key;                /* Datapath running the snooping. */
> -
> -    long long int query_time_ms;   /* Next query time in ms. */
> -};
> -
> -/*
> - * Holds the per-datapath multicast configuration state. Maintained by
> - * pinctrl_run().
> - */
> -struct ip_mcast_snoop_state {
> -    struct hmap_node hmap_node;
> -    int64_t dp_key;
> -    struct ip_mcast_snoop_cfg cfg;
> -};
> -
> -/* Only default vlan supported for now. */
> -#define IP_MCAST_VLAN 1
> -
> -/* Multicast snooping information stored independently by datapath key.
> - * Protected by pinctrl_mutex. pinctrl_handler has RW access and
> pinctrl_main
> - * has RO access.
> - */
> -static struct hmap mcast_snoop_map OVS_GUARDED_BY(pinctrl_mutex);
> -
> -/* Contains multicast queries to be sent. Only used by pinctrl_handler so
> no
> - * locking needed.
> - */
> -static struct ovs_list mcast_query_list;
> -
> -/* Multicast config information stored independently by datapath key.
> - * Protected by pinctrl_mutex. pinctrl_handler has RO access and
> pinctrl_main
> - * has RW access. Read accesses from pinctrl_ip_mcast_handle_igmp() can be
> - * performed without taking the lock as they are executed in the
> pinctrl_main
> - * thread.
> - */
> -static struct hmap mcast_cfg_map OVS_GUARDED_BY(pinctrl_mutex);
> -
> -static void
> -ip_mcast_snoop_cfg_load(struct ip_mcast_snoop_cfg *cfg,
> -                        const struct sbrec_ip_multicast *ip_mcast)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -
> -    memset(cfg, 0, sizeof *cfg);
> -    cfg->enabled =
> -        (ip_mcast->enabled && ip_mcast->enabled[0]);
> -    cfg->querier_enabled =
> -        (cfg->enabled && ip_mcast->querier && ip_mcast->querier[0]);
> -
> -    if (ip_mcast->table_size) {
> -        cfg->table_size = ip_mcast->table_size[0];
> -    } else {
> -        cfg->table_size = OVN_MCAST_DEFAULT_MAX_ENTRIES;
> -    }
> -
> -    if (ip_mcast->idle_timeout) {
> -        cfg->idle_time_s = ip_mcast->idle_timeout[0];
> -    } else {
> -        cfg->idle_time_s = OVN_MCAST_DEFAULT_IDLE_TIMEOUT_S;
> -    }
> -
> -    if (ip_mcast->query_interval) {
> -        cfg->query_interval_s = ip_mcast->query_interval[0];
> -    } else {
> -        cfg->query_interval_s = cfg->idle_time_s / 2;
> -        if (cfg->query_interval_s < OVN_MCAST_MIN_QUERY_INTERVAL_S) {
> -            cfg->query_interval_s = OVN_MCAST_MIN_QUERY_INTERVAL_S;
> -        }
> -    }
> -
> -    if (ip_mcast->query_max_resp) {
> -        cfg->query_max_resp_s = ip_mcast->query_max_resp[0];
> -    } else {
> -        cfg->query_max_resp_s = OVN_MCAST_DEFAULT_QUERY_MAX_RESPONSE_S;
> -    }
> -
> -    cfg->seq_no = ip_mcast->seq_no;
> -
> -    if (cfg->querier_enabled) {
> -        /* Try to parse the source ETH address. */
> -        if (!ip_mcast->eth_src ||
> -                !eth_addr_from_string(ip_mcast->eth_src,
> -                                      &cfg->query_eth_src)) {
> -            VLOG_WARN_RL(&rl,
> -                         "IGMP Querier enabled with invalid ETH src
> address");
> -            /* Failed to parse the IPv4 source address. Disable the
> querier. */
> -            cfg->querier_enabled = false;
> -        }
> -
> -        /* Try to parse the source IP address. */
> -        if (!ip_mcast->ip4_src ||
> -                !ip_parse(ip_mcast->ip4_src, &cfg->query_ipv4_src)) {
> -            VLOG_WARN_RL(&rl,
> -                         "IGMP Querier enabled with invalid IPv4 src
> address");
> -            /* Failed to parse the IPv4 source address. Disable the
> querier. */
> -            cfg->querier_enabled = false;
> -        }
> -
> -        /* IGMP queries must be sent to 224.0.0.1. */
> -        cfg->query_eth_dst =
> -            (struct eth_addr)ETH_ADDR_C(01, 00, 5E, 00, 00, 01);
> -        cfg->query_ipv4_dst = htonl(0xe0000001);
> -    }
> -}
> -
> -static uint32_t
> -ip_mcast_snoop_hash(int64_t dp_key)
> -{
> -    return hash_uint64(dp_key);
> -}
> -
> -static struct ip_mcast_snoop_state *
> -ip_mcast_snoop_state_add(int64_t dp_key)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct ip_mcast_snoop_state *ms_state = xmalloc(sizeof *ms_state);
> -
> -    ms_state->dp_key = dp_key;
> -    hmap_insert(&mcast_cfg_map, &ms_state->hmap_node,
> -                ip_mcast_snoop_hash(dp_key));
> -    return ms_state;
> -}
> -
> -static struct ip_mcast_snoop_state *
> -ip_mcast_snoop_state_find(int64_t dp_key)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct ip_mcast_snoop_state *ms_state;
> -    uint32_t hash = ip_mcast_snoop_hash(dp_key);
> -
> -    HMAP_FOR_EACH_WITH_HASH (ms_state, hmap_node, hash, &mcast_cfg_map) {
> -        if (ms_state->dp_key == dp_key) {
> -            return ms_state;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static bool
> -ip_mcast_snoop_state_update(int64_t dp_key,
> -                            const struct ip_mcast_snoop_cfg *cfg)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    bool notify = false;
> -    struct ip_mcast_snoop_state *ms_state =
> ip_mcast_snoop_state_find(dp_key);
> -
> -    if (!ms_state) {
> -        ms_state = ip_mcast_snoop_state_add(dp_key);
> -        notify = true;
> -    } else if (memcmp(cfg, &ms_state->cfg, sizeof *cfg)) {
> -        notify = true;
> -    }
> -
> -    ms_state->cfg = *cfg;
> -    return notify;
> -}
> -
> -static void
> -ip_mcast_snoop_state_remove(struct ip_mcast_snoop_state *ms_state)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    hmap_remove(&mcast_cfg_map, &ms_state->hmap_node);
> -    free(ms_state);
> -}
> -
> -static bool
> -ip_mcast_snoop_enable(struct ip_mcast_snoop *ip_ms)
> -{
> -    if (ip_ms->cfg.enabled) {
> -        return true;
> -    }
> -
> -    ip_ms->ms = mcast_snooping_create();
> -    return ip_ms->ms != NULL;
> -}
> -
> -static void
> -ip_mcast_snoop_flush(struct ip_mcast_snoop *ip_ms)
> -{
> -    if (!ip_ms->cfg.enabled) {
> -        return;
> -    }
> -
> -    mcast_snooping_flush(ip_ms->ms);
> -}
> -
> -static void
> -ip_mcast_snoop_disable(struct ip_mcast_snoop *ip_ms)
> -{
> -    if (!ip_ms->cfg.enabled) {
> -        return;
> -    }
> -
> -    mcast_snooping_unref(ip_ms->ms);
> -    ip_ms->ms = NULL;
> -}
> -
> -static bool
> -ip_mcast_snoop_configure(struct ip_mcast_snoop *ip_ms,
> -                         const struct ip_mcast_snoop_cfg *cfg)
> -{
> -    if (cfg->enabled) {
> -        if (!ip_mcast_snoop_enable(ip_ms)) {
> -            return false;
> -        }
> -        if (ip_ms->cfg.seq_no != cfg->seq_no) {
> -            ip_mcast_snoop_flush(ip_ms);
> -        }
> -
> -        if (ip_ms->cfg.querier_enabled && !cfg->querier_enabled) {
> -            ovs_list_remove(&ip_ms->query_node);
> -        } else if (!ip_ms->cfg.querier_enabled && cfg->querier_enabled) {
> -            ovs_list_push_back(&mcast_query_list, &ip_ms->query_node);
> -        }
> -    } else {
> -        ip_mcast_snoop_disable(ip_ms);
> -        goto set_fields;
> -    }
> -
> -    ovs_rwlock_wrlock(&ip_ms->ms->rwlock);
> -    if (cfg->table_size != ip_ms->cfg.table_size) {
> -        mcast_snooping_set_max_entries(ip_ms->ms, cfg->table_size);
> -    }
> -
> -    if (cfg->idle_time_s != ip_ms->cfg.idle_time_s) {
> -        mcast_snooping_set_idle_time(ip_ms->ms, cfg->idle_time_s);
> -    }
> -    ovs_rwlock_unlock(&ip_ms->ms->rwlock);
> -
> -    if (cfg->query_interval_s != ip_ms->cfg.query_interval_s) {
> -        long long int now = time_msec();
> -
> -        if (ip_ms->query_time_ms > now + cfg->query_interval_s * 1000) {
> -            ip_ms->query_time_ms = now;
> -        }
> -    }
> -
> -set_fields:
> -    memcpy(&ip_ms->cfg, cfg, sizeof ip_ms->cfg);
> -    return true;
> -}
> -
> -static struct ip_mcast_snoop *
> -ip_mcast_snoop_add(int64_t dp_key, const struct ip_mcast_snoop_cfg *cfg)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct ip_mcast_snoop *ip_ms = xzalloc(sizeof *ip_ms);
> -
> -    ip_ms->dp_key = dp_key;
> -    if (!ip_mcast_snoop_configure(ip_ms, cfg)) {
> -        free(ip_ms);
> -        return NULL;
> -    }
> -
> -    hmap_insert(&mcast_snoop_map, &ip_ms->hmap_node,
> -                ip_mcast_snoop_hash(dp_key));
> -    return ip_ms;
> -}
> -
> -static struct ip_mcast_snoop *
> -ip_mcast_snoop_find(int64_t dp_key)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct ip_mcast_snoop *ip_ms;
> -
> -    HMAP_FOR_EACH_WITH_HASH (ip_ms, hmap_node,
> ip_mcast_snoop_hash(dp_key),
> -                             &mcast_snoop_map) {
> -        if (ip_ms->dp_key == dp_key) {
> -            return ip_ms;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -ip_mcast_snoop_remove(struct ip_mcast_snoop *ip_ms)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    hmap_remove(&mcast_snoop_map, &ip_ms->hmap_node);
> -
> -    if (ip_ms->cfg.querier_enabled) {
> -        ovs_list_remove(&ip_ms->query_node);
> -    }
> -
> -    ip_mcast_snoop_disable(ip_ms);
> -    free(ip_ms);
> -}
> -
> -static void
> -ip_mcast_snoop_init(void)
> -    OVS_NO_THREAD_SAFETY_ANALYSIS
> -{
> -    hmap_init(&mcast_snoop_map);
> -    ovs_list_init(&mcast_query_list);
> -    hmap_init(&mcast_cfg_map);
> -}
> -
> -static void
> -ip_mcast_snoop_destroy(void)
> -    OVS_NO_THREAD_SAFETY_ANALYSIS
> -{
> -    struct ip_mcast_snoop *ip_ms, *ip_ms_next;
> -
> -    HMAP_FOR_EACH_SAFE (ip_ms, ip_ms_next, hmap_node, &mcast_snoop_map) {
> -        ip_mcast_snoop_remove(ip_ms);
> -    }
> -    hmap_destroy(&mcast_snoop_map);
> -
> -    struct ip_mcast_snoop_state *ip_ms_state;
> -
> -    HMAP_FOR_EACH_POP (ip_ms_state, hmap_node, &mcast_cfg_map) {
> -        free(ip_ms_state);
> -    }
> -}
> -
> -static void
> -ip_mcast_snoop_run(void)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct ip_mcast_snoop *ip_ms, *ip_ms_next;
> -
> -    /* First read the config updated by pinctrl_main. If there's any new
> or
> -     * updated config then apply it.
> -     */
> -    struct ip_mcast_snoop_state *ip_ms_state;
> -
> -    HMAP_FOR_EACH (ip_ms_state, hmap_node, &mcast_cfg_map) {
> -        ip_ms = ip_mcast_snoop_find(ip_ms_state->dp_key);
> -
> -        if (!ip_ms) {
> -            ip_mcast_snoop_add(ip_ms_state->dp_key, &ip_ms_state->cfg);
> -        } else if (memcmp(&ip_ms_state->cfg, &ip_ms->cfg,
> -                          sizeof ip_ms_state->cfg)) {
> -            ip_mcast_snoop_configure(ip_ms, &ip_ms_state->cfg);
> -        }
> -    }
> -
> -    bool notify = false;
> -
> -    /* Then walk the multicast snoop instances. */
> -    HMAP_FOR_EACH_SAFE (ip_ms, ip_ms_next, hmap_node, &mcast_snoop_map) {
> -
> -        /* Delete the stale ones. */
> -        if (!ip_mcast_snoop_state_find(ip_ms->dp_key)) {
> -            ip_mcast_snoop_remove(ip_ms);
> -            continue;
> -        }
> -
> -        /* If enabled run the snooping instance to timeout old groups. */
> -        if (ip_ms->cfg.enabled) {
> -            if (mcast_snooping_run(ip_ms->ms)) {
> -                notify = true;
> -            }
> -
> -            mcast_snooping_wait(ip_ms->ms);
> -        }
> -    }
> -
> -    if (notify) {
> -        notify_pinctrl_main();
> -    }
> -}
> -
> -/*
> - * This runs in the pinctrl main thread, so it has access to the
> southbound
> - * database. It reads the IP_Multicast table and updates the local
> multicast
> - * configuration. Then writes to the southbound database the updated
> - * IGMP_Groups.
> - */
> -static void
> -ip_mcast_sync(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -              const struct sbrec_chassis *chassis,
> -              const struct hmap *local_datapaths,
> -              struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -              struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -              struct ovsdb_idl_index *sbrec_igmp_groups,
> -              struct ovsdb_idl_index *sbrec_ip_multicast)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    bool notify = false;
> -
> -    if (!ovnsb_idl_txn || !chassis) {
> -        return;
> -    }
> -
> -    struct sbrec_ip_multicast *ip_mcast;
> -    struct ip_mcast_snoop_state *ip_ms_state, *ip_ms_state_next;
> -
> -    /* First read and update our own local multicast configuration for the
> -     * local datapaths.
> -     */
> -    SBREC_IP_MULTICAST_FOR_EACH_BYINDEX (ip_mcast, sbrec_ip_multicast) {
> -
> -        int64_t dp_key = ip_mcast->datapath->tunnel_key;
> -        struct ip_mcast_snoop_cfg cfg;
> -
> -        ip_mcast_snoop_cfg_load(&cfg, ip_mcast);
> -        if (ip_mcast_snoop_state_update(dp_key, &cfg)) {
> -            notify = true;
> -        }
> -    }
> -
> -    /* Then delete the old entries. */
> -    HMAP_FOR_EACH_SAFE (ip_ms_state, ip_ms_state_next, hmap_node,
> -                        &mcast_cfg_map) {
> -        if (!get_local_datapath(local_datapaths, ip_ms_state->dp_key)) {
> -            ip_mcast_snoop_state_remove(ip_ms_state);
> -            notify = true;
> -        }
> -    }
> -
> -    const struct sbrec_igmp_group *sbrec_igmp;
> -
> -    /* Then flush any IGMP_Group entries that are not needed anymore:
> -     * - either multicast snooping was disabled on the datapath
> -     * - or the group has expired.
> -     */
> -    SBREC_IGMP_GROUP_FOR_EACH_BYINDEX (sbrec_igmp, sbrec_igmp_groups) {
> -        ovs_be32 group_addr;
> -
> -        if (!sbrec_igmp->datapath) {
> -            continue;
> -        }
> -
> -        int64_t dp_key = sbrec_igmp->datapath->tunnel_key;
> -        struct ip_mcast_snoop *ip_ms = ip_mcast_snoop_find(dp_key);
> -
> -        /* If the datapath doesn't exist anymore or IGMP snooping was
> disabled
> -         * on it then delete the IGMP_Group entry.
> -         */
> -        if (!ip_ms || !ip_ms->cfg.enabled) {
> -            igmp_group_delete(sbrec_igmp);
> -            continue;
> -        }
> -
> -        if (!ip_parse(sbrec_igmp->address, &group_addr)) {
> -            continue;
> -        }
> -
> -        ovs_rwlock_rdlock(&ip_ms->ms->rwlock);
> -        struct mcast_group *mc_group =
> -            mcast_snooping_lookup4(ip_ms->ms, group_addr,
> -                                   IP_MCAST_VLAN);
> -
> -        if (!mc_group || ovs_list_is_empty(&mc_group->bundle_lru)) {
> -            igmp_group_delete(sbrec_igmp);
> -        }
> -        ovs_rwlock_unlock(&ip_ms->ms->rwlock);
> -    }
> -
> -    struct ip_mcast_snoop *ip_ms, *ip_ms_next;
> -
> -    /* Last: write new IGMP_Groups to the southbound DB and update
> existing
> -     * ones (if needed). We also flush any old per-datapath multicast
> snoop
> -     * structures.
> -     */
> -    HMAP_FOR_EACH_SAFE (ip_ms, ip_ms_next, hmap_node, &mcast_snoop_map) {
> -        /* Flush any non-local snooping datapaths (e.g., stale). */
> -        struct local_datapath *local_dp =
> -            get_local_datapath(local_datapaths, ip_ms->dp_key);
> -
> -        if (!local_dp) {
> -            continue;
> -        }
> -
> -        /* Skip datapaths on which snooping is disabled. */
> -        if (!ip_ms->cfg.enabled) {
> -            continue;
> -        }
> -
> -        struct mcast_group *mc_group;
> -
> -        ovs_rwlock_rdlock(&ip_ms->ms->rwlock);
> -        LIST_FOR_EACH (mc_group, group_node, &ip_ms->ms->group_lru) {
> -            if (ovs_list_is_empty(&mc_group->bundle_lru)) {
> -                continue;
> -            }
> -            sbrec_igmp = igmp_group_lookup(sbrec_igmp_groups,
> &mc_group->addr,
> -                                           local_dp->datapath, chassis);
> -            if (!sbrec_igmp) {
> -                sbrec_igmp = igmp_group_create(ovnsb_idl_txn,
> &mc_group->addr,
> -                                               local_dp->datapath,
> chassis);
> -            }
> -
> -            igmp_group_update_ports(sbrec_igmp,
> sbrec_datapath_binding_by_key,
> -                                    sbrec_port_binding_by_key, ip_ms->ms,
> -                                    mc_group);
> -        }
> -        ovs_rwlock_unlock(&ip_ms->ms->rwlock);
> -    }
> -
> -    if (notify) {
> -        notify_pinctrl_handler();
> -    }
> -}
> -
> -static void
> -pinctrl_ip_mcast_handle_igmp(struct rconn *swconn OVS_UNUSED,
> -                             const struct flow *ip_flow,
> -                             struct dp_packet *pkt_in,
> -                             const struct match *md,
> -                             struct ofpbuf *userdata OVS_UNUSED)
> -    OVS_NO_THREAD_SAFETY_ANALYSIS
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -
> -    /* This action only works for IP packets, and the switch should only
> send
> -     * us IP packets this way, but check here just to be sure.
> -     */
> -    if (ip_flow->dl_type != htons(ETH_TYPE_IP)) {
> -        VLOG_WARN_RL(&rl,
> -                     "IGMP action on non-IP packet (eth_type
> 0x%"PRIx16")",
> -                     ntohs(ip_flow->dl_type));
> -        return;
> -    }
> -
> -    int64_t dp_key = ntohll(md->flow.metadata);
> -    uint32_t port_key = md->flow.regs[MFF_LOG_INPORT - MFF_REG0];
> -
> -    const struct igmp_header *igmp;
> -    size_t offset;
> -
> -    offset = (char *) dp_packet_l4(pkt_in) - (char *)
> dp_packet_data(pkt_in);
> -    igmp = dp_packet_at(pkt_in, offset, IGMP_HEADER_LEN);
> -    if (!igmp || csum(igmp, dp_packet_l4_size(pkt_in)) != 0) {
> -        VLOG_WARN_RL(&rl, "multicast snooping received bad IGMP
> checksum");
> -        return;
> -    }
> -
> -    ovs_be32 ip4 = ip_flow->igmp_group_ip4;
> -
> -    struct ip_mcast_snoop *ip_ms = ip_mcast_snoop_find(dp_key);
> -    if (!ip_ms || !ip_ms->cfg.enabled) {
> -        /* IGMP snooping is not configured or is disabled. */
> -        return;
> -    }
> -
> -    void *port_key_data = (void *)(uintptr_t)port_key;
> -
> -    bool group_change = false;
> -
> -    ovs_rwlock_wrlock(&ip_ms->ms->rwlock);
> -    switch (ntohs(ip_flow->tp_src)) {
> -     /* Only default VLAN is supported for now. */
> -    case IGMP_HOST_MEMBERSHIP_REPORT:
> -    case IGMPV2_HOST_MEMBERSHIP_REPORT:
> -        group_change =
> -            mcast_snooping_add_group4(ip_ms->ms, ip4, IP_MCAST_VLAN,
> -                                      port_key_data);
> -        break;
> -    case IGMP_HOST_LEAVE_MESSAGE:
> -        group_change =
> -            mcast_snooping_leave_group4(ip_ms->ms, ip4, IP_MCAST_VLAN,
> -                                        port_key_data);
> -        break;
> -    case IGMP_HOST_MEMBERSHIP_QUERY:
> -        /* Shouldn't be receiving any of these since we are the multicast
> -         * router. Store them for now.
> -         */
> -        group_change =
> -            mcast_snooping_add_mrouter(ip_ms->ms, IP_MCAST_VLAN,
> -                                       port_key_data);
> -        break;
> -    case IGMPV3_HOST_MEMBERSHIP_REPORT:
> -        group_change =
> -            mcast_snooping_add_report(ip_ms->ms, pkt_in, IP_MCAST_VLAN,
> -                                      port_key_data);
> -        break;
> -    }
> -    ovs_rwlock_unlock(&ip_ms->ms->rwlock);
> -
> -    if (group_change) {
> -        notify_pinctrl_main();
> -    }
> -}
> -
> -static long long int
> -ip_mcast_querier_send(struct rconn *swconn, struct ip_mcast_snoop *ip_ms,
> -                      long long int current_time)
> -{
> -    if (current_time < ip_ms->query_time_ms) {
> -        return ip_ms->query_time_ms;
> -    }
> -
> -    /* Compose a multicast query. */
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -
> -    uint8_t ip_tos = 0;
> -    uint8_t igmp_ttl = 1;
> -
> -    dp_packet_clear(&packet);
> -    packet.packet_type = htonl(PT_ETH);
> -
> -    struct eth_header *eh = dp_packet_put_zeros(&packet, sizeof *eh);
> -    eh->eth_dst = ip_ms->cfg.query_eth_dst;
> -    eh->eth_src = ip_ms->cfg.query_eth_src;
> -
> -    struct ip_header *nh = dp_packet_put_zeros(&packet, sizeof *nh);
> -
> -    eh->eth_type = htons(ETH_TYPE_IP);
> -    dp_packet_set_l3(&packet, nh);
> -    nh->ip_ihl_ver = IP_IHL_VER(5, 4);
> -    nh->ip_tot_len = htons(sizeof(struct ip_header) +
> -                            sizeof(struct igmpv3_query_header));
> -    nh->ip_tos = IP_DSCP_CS6;
> -    nh->ip_proto = IPPROTO_IGMP;
> -    nh->ip_frag_off = htons(IP_DF);
> -    packet_set_ipv4(&packet, ip_ms->cfg.query_ipv4_src,
> -                    ip_ms->cfg.query_ipv4_dst, ip_tos, igmp_ttl);
> -
> -    nh->ip_csum = 0;
> -    nh->ip_csum = csum(nh, sizeof *nh);
> -
> -    struct igmpv3_query_header *igh =
> -        dp_packet_put_zeros(&packet, sizeof *igh);
> -    dp_packet_set_l4(&packet, igh);
> -
> -    /* IGMP query max-response in tenths of seconds. */
> -    uint8_t max_response = ip_ms->cfg.query_max_resp_s * 10;
> -    uint8_t qqic = max_response;
> -    packet_set_igmp3_query(&packet, max_response, 0, false, 0, qqic);
> -
> -    /* Inject multicast query. */
> -    uint64_t ofpacts_stub[4096 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    put_load(ip_ms->dp_key, MFF_LOG_DATAPATH, 0, 64, &ofpacts);
> -    put_load(OVN_MCAST_FLOOD_TUNNEL_KEY, MFF_LOG_OUTPORT, 0, 32,
> &ofpacts);
> -    put_load(1, MFF_LOG_FLAGS, MLF_LOCAL_ONLY, 1, &ofpacts);
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(&ofpacts);
> -    resubmit->in_port = OFPP_CONTROLLER;
> -    resubmit->table_id = OFTABLE_LOCAL_OUTPUT;
> -
> -    struct ofputil_packet_out po = {
> -        .packet = dp_packet_data(&packet),
> -        .packet_len = dp_packet_size(&packet),
> -        .buffer_id = UINT32_MAX,
> -        .ofpacts = ofpacts.data,
> -        .ofpacts_len = ofpacts.size,
> -    };
> -    match_set_in_port(&po.flow_metadata, OFPP_CONTROLLER);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    queue_msg(swconn, ofputil_encode_packet_out(&po, proto));
> -    dp_packet_uninit(&packet);
> -    ofpbuf_uninit(&ofpacts);
> -
> -    /* Set the next query time. */
> -    ip_ms->query_time_ms = current_time + ip_ms->cfg.query_interval_s *
> 1000;
> -    return ip_ms->query_time_ms;
> -}
> -
> -static void
> -ip_mcast_querier_run(struct rconn *swconn, long long int *query_time)
> -{
> -    if (ovs_list_is_empty(&mcast_query_list)) {
> -        return;
> -    }
> -
> -    /* Send multicast queries and update the next query time. */
> -    long long int current_time = time_msec();
> -    *query_time = LLONG_MAX;
> -
> -    struct ip_mcast_snoop *ip_ms;
> -
> -    LIST_FOR_EACH (ip_ms, query_node, &mcast_query_list) {
> -        long long int next_query_time =
> -            ip_mcast_querier_send(swconn, ip_ms, current_time);
> -        if (*query_time > next_query_time) {
> -            *query_time = next_query_time;
> -        }
> -    }
> -}
> -
> -static void
> -ip_mcast_querier_wait(long long int query_time)
> -{
> -    if (!ovs_list_is_empty(&mcast_query_list)) {
> -        poll_timer_wait_until(query_time);
> -    }
> -}
> -
> -/* Get localnet vifs, local l3gw ports and ofport for localnet patch
> ports. */
> -static void
> -get_localnet_vifs_l3gwports(
> -    struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -    const struct ovsrec_bridge *br_int,
> -    const struct sbrec_chassis *chassis,
> -    const struct hmap *local_datapaths,
> -    struct sset *localnet_vifs,
> -    struct sset *local_l3gw_ports)
> -{
> -    for (int i = 0; i < br_int->n_ports; i++) {
> -        const struct ovsrec_port *port_rec = br_int->ports[i];
> -        if (!strcmp(port_rec->name, br_int->name)) {
> -            continue;
> -        }
> -        const char *tunnel_id = smap_get(&port_rec->external_ids,
> -                                          "ovn-chassis-id");
> -        if (tunnel_id &&
> -                encaps_tunnel_id_match(tunnel_id, chassis->name, NULL)) {
> -            continue;
> -        }
> -        const char *localnet = smap_get(&port_rec->external_ids,
> -                                        "ovn-localnet-port");
> -        if (localnet) {
> -            continue;
> -        }
> -        for (int j = 0; j < port_rec->n_interfaces; j++) {
> -            const struct ovsrec_interface *iface_rec =
> port_rec->interfaces[j];
> -            if (!iface_rec->n_ofport) {
> -                continue;
> -            }
> -            /* Get localnet vif. */
> -            const char *iface_id = smap_get(&iface_rec->external_ids,
> -                                            "iface-id");
> -            if (!iface_id) {
> -                continue;
> -            }
> -            const struct sbrec_port_binding *pb
> -                = lport_lookup_by_name(sbrec_port_binding_by_name,
> iface_id);
> -            if (!pb) {
> -                continue;
> -            }
> -            struct local_datapath *ld
> -                = get_local_datapath(local_datapaths,
> -                                     pb->datapath->tunnel_key);
> -            if (ld && ld->localnet_port) {
> -                sset_add(localnet_vifs, iface_id);
> -            }
> -        }
> -    }
> -
> -    struct sbrec_port_binding *target = sbrec_port_binding_index_init_row(
> -        sbrec_port_binding_by_datapath);
> -
> -    const struct local_datapath *ld;
> -    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> -        const struct sbrec_port_binding *pb;
> -
> -        if (!ld->localnet_port) {
> -            continue;
> -        }
> -
> -        /* Get l3gw ports.  Consider port bindings with type "l3gateway"
> -         * that connect to gateway routers (if local), and consider port
> -         * bindings of type "patch" since they might connect to
> -         * distributed gateway ports with NAT addresses. */
> -
> -        sbrec_port_binding_index_set_datapath(target, ld->datapath);
> -        SBREC_PORT_BINDING_FOR_EACH_EQUAL (pb, target,
> -
>  sbrec_port_binding_by_datapath) {
> -            if ((ld->has_local_l3gateway && !strcmp(pb->type,
> "l3gateway"))
> -                || !strcmp(pb->type, "patch")) {
> -                sset_add(local_l3gw_ports, pb->logical_port);
> -            }
> -        }
> -    }
> -    sbrec_port_binding_index_destroy_row(target);
> -}
> -
> -static bool
> -pinctrl_is_chassis_resident(struct ovsdb_idl_index
> *sbrec_port_binding_by_name,
> -                            const struct sbrec_chassis *chassis,
> -                            const struct sset *active_tunnels,
> -                            const char *port_name)
> -{
> -    const struct sbrec_port_binding *pb
> -        = lport_lookup_by_name(sbrec_port_binding_by_name, port_name);
> -    if (!pb || !pb->chassis) {
> -        return false;
> -    }
> -    if (strcmp(pb->type, "chassisredirect")) {
> -        return pb->chassis == chassis;
> -    } else {
> -        return ha_chassis_group_is_active(pb->ha_chassis_group,
> -                                          active_tunnels, chassis);
> -    }
> -}
> -
> -/* Extracts the mac, IPv4 and IPv6 addresses, and logical port from
> - * 'addresses' which should be of the format 'MAC [IP1 IP2 ..]
> - * [is_chassis_resident("LPORT_NAME")]', where IPn should be a valid IPv4
> - * or IPv6 address, and stores them in the 'ipv4_addrs' and 'ipv6_addrs'
> - * fields of 'laddrs'.  The logical port name is stored in 'lport'.
> - *
> - * Returns true if at least 'MAC' is found in 'address', false otherwise.
> - *
> - * The caller must call destroy_lport_addresses() and free(*lport). */
> -static bool
> -extract_addresses_with_port(const char *addresses,
> -                            struct lport_addresses *laddrs,
> -                            char **lport)
> -{
> -    int ofs;
> -    if (!extract_addresses(addresses, laddrs, &ofs)) {
> -        return false;
> -    } else if (ofs >= strlen(addresses)) {
> -        return true;
> -    }
> -
> -    struct lexer lexer;
> -    lexer_init(&lexer, addresses + ofs);
> -    lexer_get(&lexer);
> -
> -    if (lexer.error || lexer.token.type != LEX_T_ID
> -        || !lexer_match_id(&lexer, "is_chassis_resident")) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_INFO_RL(&rl, "invalid syntax '%s' in address", addresses);
> -        lexer_destroy(&lexer);
> -        return true;
> -    }
> -
> -    if (!lexer_match(&lexer, LEX_T_LPAREN)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_INFO_RL(&rl, "Syntax error: expecting '(' after "
> -                          "'is_chassis_resident' in address '%s'",
> addresses);
> -        lexer_destroy(&lexer);
> -        return false;
> -    }
> -
> -    if (lexer.token.type != LEX_T_STRING) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_INFO_RL(&rl,
> -                    "Syntax error: expecting quoted string after "
> -                    "'is_chassis_resident' in address '%s'", addresses);
> -        lexer_destroy(&lexer);
> -        return false;
> -    }
> -
> -    *lport = xstrdup(lexer.token.s);
> -
> -    lexer_get(&lexer);
> -    if (!lexer_match(&lexer, LEX_T_RPAREN)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_INFO_RL(&rl, "Syntax error: expecting ')' after quoted
> string in "
> -                          "'is_chassis_resident()' in address '%s'",
> -                          addresses);
> -        lexer_destroy(&lexer);
> -        return false;
> -    }
> -
> -    lexer_destroy(&lexer);
> -    return true;
> -}
> -
> -static void
> -consider_nat_address(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                     const char *nat_address,
> -                     const struct sbrec_port_binding *pb,
> -                     struct sset *nat_address_keys,
> -                     const struct sbrec_chassis *chassis,
> -                     const struct sset *active_tunnels,
> -                     struct shash *nat_addresses)
> -{
> -    struct lport_addresses *laddrs = xmalloc(sizeof *laddrs);
> -    char *lport = NULL;
> -    if (!extract_addresses_with_port(nat_address, laddrs, &lport)
> -        || (!lport && !strcmp(pb->type, "patch"))
> -        || (lport && !pinctrl_is_chassis_resident(
> -                sbrec_port_binding_by_name, chassis,
> -                active_tunnels, lport))) {
> -        destroy_lport_addresses(laddrs);
> -        free(laddrs);
> -        free(lport);
> -        return;
> -    }
> -    free(lport);
> -
> -    int i;
> -    for (i = 0; i < laddrs->n_ipv4_addrs; i++) {
> -        char *name = xasprintf("%s-%s", pb->logical_port,
> -                                        laddrs->ipv4_addrs[i].addr_s);
> -        sset_add(nat_address_keys, name);
> -        free(name);
> -    }
> -    shash_add(nat_addresses, pb->logical_port, laddrs);
> -}
> -
> -static void
> -get_nat_addresses_and_keys(struct ovsdb_idl_index
> *sbrec_port_binding_by_name,
> -                           struct sset *nat_address_keys,
> -                           struct sset *local_l3gw_ports,
> -                           const struct sbrec_chassis *chassis,
> -                           const struct sset *active_tunnels,
> -                           struct shash *nat_addresses)
> -{
> -    const char *gw_port;
> -    SSET_FOR_EACH(gw_port, local_l3gw_ports) {
> -        const struct sbrec_port_binding *pb;
> -
> -        pb = lport_lookup_by_name(sbrec_port_binding_by_name, gw_port);
> -        if (!pb) {
> -            continue;
> -        }
> -
> -        if (pb->n_nat_addresses) {
> -            for (int i = 0; i < pb->n_nat_addresses; i++) {
> -                consider_nat_address(sbrec_port_binding_by_name,
> -                                     pb->nat_addresses[i], pb,
> -                                     nat_address_keys, chassis,
> -                                     active_tunnels,
> -                                     nat_addresses);
> -            }
> -        } else {
> -            /* Continue to support options:nat-addresses for version
> -             * upgrade. */
> -            const char *nat_addresses_options = smap_get(&pb->options,
> -                                                         "nat-addresses");
> -            if (nat_addresses_options) {
> -                consider_nat_address(sbrec_port_binding_by_name,
> -                                     nat_addresses_options, pb,
> -                                     nat_address_keys, chassis,
> -                                     active_tunnels,
> -                                     nat_addresses);
> -            }
> -        }
> -    }
> -}
> -
> -static void
> -send_garp_wait(long long int send_garp_time)
> -{
> -    /* Set the poll timer for next garp only if there is garp data to
> -     * be sent. */
> -    if (!shash_is_empty(&send_garp_data)) {
> -        poll_timer_wait_until(send_garp_time);
> -    }
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -send_garp_run(struct rconn *swconn, long long int *send_garp_time)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    if (shash_is_empty(&send_garp_data)) {
> -        return;
> -    }
> -
> -    /* Send GARPs, and update the next announcement. */
> -    struct shash_node *iter;
> -    long long int current_time = time_msec();
> -    *send_garp_time = LLONG_MAX;
> -    SHASH_FOR_EACH (iter, &send_garp_data) {
> -        long long int next_announce = send_garp(swconn, iter->data,
> -                                                current_time);
> -        if (*send_garp_time > next_announce) {
> -            *send_garp_time = next_announce;
> -        }
> -    }
> -}
> -
> -/* Called by pinctrl_run(). Runs with in the main ovn-controller
> - * thread context. */
> -static void
> -send_garp_prepare(struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -                  struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                  const struct ovsrec_bridge *br_int,
> -                  const struct sbrec_chassis *chassis,
> -                  const struct hmap *local_datapaths,
> -                  const struct sset *active_tunnels)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    struct sset localnet_vifs = SSET_INITIALIZER(&localnet_vifs);
> -    struct sset local_l3gw_ports = SSET_INITIALIZER(&local_l3gw_ports);
> -    struct sset nat_ip_keys = SSET_INITIALIZER(&nat_ip_keys);
> -    struct shash nat_addresses;
> -
> -    shash_init(&nat_addresses);
> -
> -    get_localnet_vifs_l3gwports(sbrec_port_binding_by_datapath,
> -                                sbrec_port_binding_by_name,
> -                                br_int, chassis, local_datapaths,
> -                                &localnet_vifs, &local_l3gw_ports);
> -
> -    get_nat_addresses_and_keys(sbrec_port_binding_by_name,
> -                               &nat_ip_keys, &local_l3gw_ports,
> -                               chassis, active_tunnels,
> -                               &nat_addresses);
> -    /* For deleted ports and deleted nat ips, remove from send_garp_data.
> */
> -    struct shash_node *iter, *next;
> -    SHASH_FOR_EACH_SAFE (iter, next, &send_garp_data) {
> -        if (!sset_contains(&localnet_vifs, iter->name) &&
> -            !sset_contains(&nat_ip_keys, iter->name)) {
> -            send_garp_delete(iter->name);
> -        }
> -    }
> -
> -    /* Update send_garp_data. */
> -    const char *iface_id;
> -    SSET_FOR_EACH (iface_id, &localnet_vifs) {
> -        const struct sbrec_port_binding *pb = lport_lookup_by_name(
> -            sbrec_port_binding_by_name, iface_id);
> -        if (pb) {
> -            send_garp_update(pb, &nat_addresses);
> -        }
> -    }
> -
> -    /* Update send_garp_data for nat-addresses. */
> -    const char *gw_port;
> -    SSET_FOR_EACH (gw_port, &local_l3gw_ports) {
> -        const struct sbrec_port_binding *pb
> -            = lport_lookup_by_name(sbrec_port_binding_by_name, gw_port);
> -        if (pb) {
> -            send_garp_update(pb, &nat_addresses);
> -        }
> -    }
> -
> -    /* pinctrl_handler thread will send the GARPs. */
> -
> -    sset_destroy(&localnet_vifs);
> -    sset_destroy(&local_l3gw_ports);
> -
> -    SHASH_FOR_EACH_SAFE (iter, next, &nat_addresses) {
> -        struct lport_addresses *laddrs = iter->data;
> -        destroy_lport_addresses(laddrs);
> -        shash_delete(&nat_addresses, iter);
> -        free(laddrs);
> -    }
> -    shash_destroy(&nat_addresses);
> -
> -    sset_destroy(&nat_ip_keys);
> -}
> -
> -static bool
> -may_inject_pkts(void)
> -{
> -    return (!shash_is_empty(&ipv6_ras) ||
> -            !shash_is_empty(&send_garp_data) ||
> -            !ovs_list_is_empty(&mcast_query_list) ||
> -            !ovs_list_is_empty(&buffered_mac_bindings));
> -}
> -
> -static void
> -reload_metadata(struct ofpbuf *ofpacts, const struct match *md)
> -{
> -    enum mf_field_id md_fields[] = {
> -#if FLOW_N_REGS == 16
> -        MFF_REG0,
> -        MFF_REG1,
> -        MFF_REG2,
> -        MFF_REG3,
> -        MFF_REG4,
> -        MFF_REG5,
> -        MFF_REG6,
> -        MFF_REG7,
> -        MFF_REG8,
> -        MFF_REG9,
> -        MFF_REG10,
> -        MFF_REG11,
> -        MFF_REG12,
> -        MFF_REG13,
> -        MFF_REG14,
> -        MFF_REG15,
> -#else
> -#error
> -#endif
> -        MFF_METADATA,
> -    };
> -    for (size_t i = 0; i < ARRAY_SIZE(md_fields); i++) {
> -        const struct mf_field *field = mf_from_id(md_fields[i]);
> -        if (!mf_is_all_wild(field, &md->wc)) {
> -            union mf_value value;
> -            mf_get_value(field, &md->flow, &value);
> -            ofpact_put_set_field(ofpacts, field, &value, NULL);
> -        }
> -    }
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_nd_na(struct rconn *swconn, const struct flow *ip_flow,
> -                     const struct match *md,
> -                     struct ofpbuf *userdata, bool is_router)
> -{
> -    /* This action only works for IPv6 ND packets, and the switch should
> only
> -     * send us ND packets this way, but check here just to be sure. */
> -    if (!is_nd(ip_flow, NULL)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "NA action on non-ND packet");
> -        return;
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -
> -    /* These flags are not exactly correct.  Look at section 7.2.4
> -     * of RFC 4861. */
> -    uint32_t rso_flags = ND_RSO_SOLICITED | ND_RSO_OVERRIDE;
> -    if (is_router) {
> -        rso_flags |= ND_RSO_ROUTER;
> -    }
> -    compose_nd_na(&packet, ip_flow->dl_dst, ip_flow->dl_src,
> -                  &ip_flow->nd_target, &ip_flow->ipv6_src,
> -                  htonl(rso_flags));
> -
> -    /* Reload previous packet metadata and set actions from userdata. */
> -    set_actions_and_enqueue_msg(swconn, &packet, md, userdata);
> -    dp_packet_uninit(&packet);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_nd_ns(struct rconn *swconn, const struct flow *ip_flow,
> -                     struct dp_packet *pkt_in,
> -                     const struct match *md, struct ofpbuf *userdata)
> -{
> -    /* This action only works for IPv6 packets. */
> -    if (get_dl_type(ip_flow) != htons(ETH_TYPE_IPV6)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "NS action on non-IPv6 packet");
> -        return;
> -    }
> -
> -    ovs_mutex_lock(&pinctrl_mutex);
> -    pinctrl_handle_buffered_packets(ip_flow, pkt_in, md, false);
> -    ovs_mutex_unlock(&pinctrl_mutex);
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -
> -    compose_nd_ns(&packet, ip_flow->dl_src, &ip_flow->ipv6_src,
> -                  &ip_flow->ipv6_dst);
> -
> -    /* Reload previous packet metadata and set actions from userdata. */
> -    set_actions_and_enqueue_msg(swconn, &packet, md, userdata);
> -    dp_packet_uninit(&packet);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_put_nd_ra_opts(
> -    struct rconn *swconn,
> -    const struct flow *in_flow, struct dp_packet *pkt_in,
> -    struct ofputil_packet_in *pin, struct ofpbuf *userdata,
> -    struct ofpbuf *continuation)
> -{
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    struct dp_packet *pkt_out_ptr = NULL;
> -    uint32_t success = 0;
> -
> -    /* Parse result field. */
> -    const struct mf_field *f;
> -    enum ofperr ofperr = nx_pull_header(userdata, NULL, &f, NULL);
> -    if (ofperr) {
> -       VLOG_WARN_RL(&rl, "bad result OXM (%s)", ofperr_to_string(ofperr));
> -       goto exit;
> -    }
> -
> -    /* Parse result offset. */
> -    ovs_be32 *ofsp = ofpbuf_try_pull(userdata, sizeof *ofsp);
> -    if (!ofsp) {
> -        VLOG_WARN_RL(&rl, "offset not present in the userdata");
> -        goto exit;
> -    }
> -
> -    /* Check that the result is valid and writable. */
> -    struct mf_subfield dst = { .field = f, .ofs = ntohl(*ofsp), .n_bits =
> 1 };
> -    ofperr = mf_check_dst(&dst, NULL);
> -    if (ofperr) {
> -        VLOG_WARN_RL(&rl, "bad result bit (%s)",
> ofperr_to_string(ofperr));
> -        goto exit;
> -    }
> -
> -    if (!userdata->size) {
> -        VLOG_WARN_RL(&rl, "IPv6 ND RA options not present in the
> userdata");
> -        goto exit;
> -    }
> -
> -    if (!is_icmpv6(in_flow, NULL) || in_flow->tp_dst != htons(0) ||
> -        in_flow->tp_src != htons(ND_ROUTER_SOLICIT)) {
> -        VLOG_WARN_RL(&rl, "put_nd_ra action on invalid or unsupported
> packet");
> -        goto exit;
> -    }
> -
> -    size_t new_packet_size = pkt_in->l4_ofs + userdata->size;
> -    struct dp_packet pkt_out;
> -    dp_packet_init(&pkt_out, new_packet_size);
> -    dp_packet_clear(&pkt_out);
> -    dp_packet_prealloc_tailroom(&pkt_out, new_packet_size);
> -    pkt_out_ptr = &pkt_out;
> -
> -    /* Copy L2 and L3 headers from pkt_in. */
> -    dp_packet_put(&pkt_out, dp_packet_pull(pkt_in, pkt_in->l4_ofs),
> -                  pkt_in->l4_ofs);
> -
> -    pkt_out.l2_5_ofs = pkt_in->l2_5_ofs;
> -    pkt_out.l2_pad_size = pkt_in->l2_pad_size;
> -    pkt_out.l3_ofs = pkt_in->l3_ofs;
> -    pkt_out.l4_ofs = pkt_in->l4_ofs;
> -
> -    /* Copy the ICMPv6 Router Advertisement data from 'userdata' field. */
> -    dp_packet_put(&pkt_out, userdata->data, userdata->size);
> -
> -    /* Set the IPv6 payload length and calculate the ICMPv6 checksum. */
> -    struct ovs_16aligned_ip6_hdr *nh = dp_packet_l3(&pkt_out);
> -    nh->ip6_plen = htons(userdata->size);
> -    struct ovs_ra_msg *ra = dp_packet_l4(&pkt_out);
> -    ra->icmph.icmp6_cksum = 0;
> -    uint32_t icmp_csum = packet_csum_pseudoheader6(nh);
> -    ra->icmph.icmp6_cksum = csum_finish(csum_continue(
> -        icmp_csum, ra, userdata->size));
> -    pin->packet = dp_packet_data(&pkt_out);
> -    pin->packet_len = dp_packet_size(&pkt_out);
> -    success = 1;
> -
> -exit:
> -    if (!ofperr) {
> -        union mf_subvalue sv;
> -        sv.u8_val = success;
> -        mf_write_subfield(&dst, &sv, &pin->flow_metadata);
> -    }
> -    queue_msg(swconn, ofputil_encode_resume(pin, continuation, proto));
> -    dp_packet_uninit(pkt_out_ptr);
> -}
> -
> -/* Called with in the pinctrl_handler thread context. */
> -static void
> -pinctrl_handle_put_icmp4_frag_mtu(struct rconn *swconn,
> -                                  const struct flow *in_flow,
> -                                  struct dp_packet *pkt_in,
> -                                  struct ofputil_packet_in *pin,
> -                                  struct ofpbuf *userdata,
> -                                  struct ofpbuf *continuation)
> -{
> -    enum ofp_version version = rconn_get_version(swconn);
> -    enum ofputil_protocol proto =
> ofputil_protocol_from_ofp_version(version);
> -    struct dp_packet *pkt_out = NULL;
> -
> -    /* This action only works for ICMPv4 packets. */
> -    if (!is_icmpv4(in_flow, NULL)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "put_icmp4_frag_mtu action on non-ICMPv4
> packet");
> -        goto exit;
> -    }
> -
> -    ovs_be16 *mtu = ofpbuf_try_pull(userdata, sizeof *mtu);
> -    if (!mtu) {
> -        goto exit;
> -    }
> -
> -    pkt_out = dp_packet_clone(pkt_in);
> -    pkt_out->l2_5_ofs = pkt_in->l2_5_ofs;
> -    pkt_out->l2_pad_size = pkt_in->l2_pad_size;
> -    pkt_out->l3_ofs = pkt_in->l3_ofs;
> -    pkt_out->l4_ofs = pkt_in->l4_ofs;
> -
> -    struct ip_header *nh = dp_packet_l3(pkt_out);
> -    struct icmp_header *ih = dp_packet_l4(pkt_out);
> -    ovs_be16 old_frag_mtu = ih->icmp_fields.frag.mtu;
> -    ih->icmp_fields.frag.mtu = *mtu;
> -    ih->icmp_csum = recalc_csum16(ih->icmp_csum, old_frag_mtu, *mtu);
> -    nh->ip_csum = 0;
> -    nh->ip_csum = csum(nh, sizeof *nh);
> -
> -    pin->packet = dp_packet_data(pkt_out);
> -    pin->packet_len = dp_packet_size(pkt_out);
> -
> -exit:
> -    queue_msg(swconn, ofputil_encode_resume(pin, continuation, proto));
> -    if (pkt_out) {
> -        dp_packet_delete(pkt_out);
> -    }
> -}
> -
> -static void
> -wait_controller_event(struct ovsdb_idl_txn *ovnsb_idl_txn)
> -{
> -    if (!ovnsb_idl_txn) {
> -        return;
> -    }
> -
> -    for (size_t i = 0; i < OVN_EVENT_MAX; i++) {
> -        if (!hmap_is_empty(&event_table[i])) {
> -            poll_immediate_wake();
> -            break;
> -        }
> -    }
> -}
> -
> -static bool
> -pinctrl_handle_empty_lb_backends_opts(struct ofpbuf *userdata)
> -{
> -    struct controller_event_opt_header *userdata_opt;
> -    uint32_t hash = 0;
> -    char *vip = NULL;
> -    char *protocol = NULL;
> -    char *load_balancer = NULL;
> -
> -    while (userdata->size) {
> -        userdata_opt = ofpbuf_try_pull(userdata, sizeof *userdata_opt);
> -        if (!userdata_opt) {
> -            return false;
> -        }
> -        size_t size = ntohs(userdata_opt->size);
> -        char *userdata_opt_data = ofpbuf_try_pull(userdata, size);
> -        if (!userdata_opt_data) {
> -            return false;
> -        }
> -        switch (ntohs(userdata_opt->opt_code)) {
> -        case EMPTY_LB_VIP:
> -            vip = xmemdup0(userdata_opt_data, size);
> -            break;
> -        case EMPTY_LB_PROTOCOL:
> -            protocol = xmemdup0(userdata_opt_data, size);
> -            break;
> -        case EMPTY_LB_LOAD_BALANCER:
> -            load_balancer = xmemdup0(userdata_opt_data, size);
> -            break;
> -        default:
> -            OVS_NOT_REACHED();
> -        }
> -        hash = hash_bytes(userdata_opt_data, size, hash);
> -    }
> -    if (!vip || !protocol || !load_balancer) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "missing lb parameters in userdata");
> -        return false;
> -    }
> -
> -    struct empty_lb_backends_event *event;
> -
> -    event = pinctrl_find_empty_lb_backends_event(vip, protocol,
> -                                                 load_balancer, hash);
> -    if (!event) {
> -        if (hmap_count(&event_table[OVN_EVENT_EMPTY_LB_BACKENDS]) >=
> 1000) {
> -            COVERAGE_INC(pinctrl_drop_controller_event);
> -            return false;
> -        }
> -
> -        event = xzalloc(sizeof *event);
> -        hmap_insert(&event_table[OVN_EVENT_EMPTY_LB_BACKENDS],
> -                    &event->hmap_node, hash);
> -        event->vip = vip;
> -        event->protocol = protocol;
> -        event->load_balancer = load_balancer;
> -        event->timestamp = time_msec();
> -        notify_pinctrl_main();
> -    } else {
> -        free(vip);
> -        free(protocol);
> -        free(load_balancer);
> -    }
> -    return true;
> -}
> -
> -static void
> -pinctrl_handle_event(struct ofpbuf *userdata)
> -    OVS_REQUIRES(pinctrl_mutex)
> -{
> -    ovs_be32 *pevent;
> -
> -    pevent = ofpbuf_try_pull(userdata, sizeof *pevent);
> -    if (!pevent) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "event not present in the userdata");
> -        return;
> -    }
> -
> -    switch (ntohl(*pevent)) {
> -    case OVN_EVENT_EMPTY_LB_BACKENDS:
> -        pinctrl_handle_empty_lb_backends_opts(userdata);
> -        break;
> -    default:
> -        return;
> -    }
> -}
> diff --git a/ovn/controller/pinctrl.h b/ovn/controller/pinctrl.h
> deleted file mode 100644
> index fcfce6bcf..000000000
> --- a/ovn/controller/pinctrl.h
> +++ /dev/null
> @@ -1,51 +0,0 @@
> -
> -/* Copyright (c) 2015, 2016 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef PINCTRL_H
> -#define PINCTRL_H 1
> -
> -#include <stdint.h>
> -
> -#include "lib/sset.h"
> -#include "openvswitch/meta-flow.h"
> -
> -struct hmap;
> -struct lport_index;
> -struct ovsdb_idl_index;
> -struct ovsdb_idl_txn;
> -struct ovsrec_bridge;
> -struct sbrec_chassis;
> -struct sbrec_dns_table;
> -struct sbrec_controller_event_table;
> -
> -void pinctrl_init(void);
> -void pinctrl_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
> -                 struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_datapath,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_key,
> -                 struct ovsdb_idl_index *sbrec_port_binding_by_name,
> -                 struct ovsdb_idl_index *sbrec_mac_binding_by_lport_ip,
> -                 struct ovsdb_idl_index *sbrec_igmp_groups,
> -                 struct ovsdb_idl_index *sbrec_ip_multicast_opts,
> -                 const struct sbrec_dns_table *,
> -                 const struct sbrec_controller_event_table *,
> -                 const struct ovsrec_bridge *, const struct sbrec_chassis
> *,
> -                 const struct hmap *local_datapaths,
> -                 const struct sset *active_tunnels);
> -void pinctrl_wait(struct ovsdb_idl_txn *ovnsb_idl_txn);
> -void pinctrl_destroy(void);
> -
> -#endif /* ovn/pinctrl.h */
> diff --git a/ovn/lib/.gitignore b/ovn/lib/.gitignore
> deleted file mode 100644
> index a80a1bce1..000000000
> --- a/ovn/lib/.gitignore
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -/libovn.sym
> -/ovn-nb-idl.c
> -/ovn-nb-idl.h
> -/ovn-nb-idl.ovsidl
> -/ovn-sb-idl.c
> -/ovn-sb-idl.h
> -/ovn-sb-idl.ovsidl
> diff --git a/ovn/lib/acl-log.c b/ovn/lib/acl-log.c
> deleted file mode 100644
> index f47b0af43..000000000
> --- a/ovn/lib/acl-log.c
> +++ /dev/null
> @@ -1,105 +0,0 @@
> -/*
> - * Copyright (c) 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "ovn/lib/acl-log.h"
> -#include <string.h>
> -#include "flow.h"
> -#include "openvswitch/json.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -
> -
> -VLOG_DEFINE_THIS_MODULE(acl_log);
> -
> -const char *
> -log_verdict_to_string(uint8_t verdict)
> -{
> -    if (verdict == LOG_VERDICT_ALLOW) {
> -        return "allow";
> -    } else if (verdict == LOG_VERDICT_DROP) {
> -        return "drop";
> -    } else if (verdict == LOG_VERDICT_REJECT) {
> -        return "reject";
> -    } else {
> -        return "<unknown>";
> -    }
> -}
> -
> -const char *
> -log_severity_to_string(uint8_t severity)
> -{
> -    if (severity == LOG_SEVERITY_ALERT) {
> -        return "alert";
> -    } else if (severity == LOG_SEVERITY_WARNING) {
> -        return "warning";
> -    } else if (severity == LOG_SEVERITY_NOTICE) {
> -        return "notice";
> -    } else if (severity == LOG_SEVERITY_INFO) {
> -        return "info";
> -    } else if (severity == LOG_SEVERITY_DEBUG) {
> -        return "debug";
> -    } else {
> -        return "<unknown>";
> -    }
> -}
> -
> -uint8_t
> -log_severity_from_string(const char *name)
> -{
> -    if (!strcmp(name, "alert")) {
> -        return LOG_SEVERITY_ALERT;
> -    } else if (!strcmp(name, "warning")) {
> -        return LOG_SEVERITY_WARNING;
> -    } else if (!strcmp(name, "notice")) {
> -        return LOG_SEVERITY_NOTICE;
> -    } else if (!strcmp(name, "info")) {
> -        return LOG_SEVERITY_INFO;
> -    } else if (!strcmp(name, "debug")) {
> -        return LOG_SEVERITY_DEBUG;
> -    } else {
> -        return UINT8_MAX;
> -    }
> -}
> -
> -void
> -handle_acl_log(const struct flow *headers, struct ofpbuf *userdata)
> -{
> -    if (!VLOG_IS_INFO_ENABLED()) {
> -        return;
> -    }
> -
> -    struct log_pin_header *lph = ofpbuf_try_pull(userdata, sizeof *lph);
> -    if (!lph) {
> -        VLOG_WARN("log data missing");
> -        return;
> -    }
> -
> -    size_t name_len = userdata->size;
> -    char *name = name_len ? xmemdup0(userdata->data, name_len) : NULL;
> -
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -    ds_put_cstr(&ds, "name=");
> -    json_string_escape(name_len ? name : "<unnamed>", &ds);
> -    ds_put_format(&ds, ", verdict=%s, severity=%s: ",
> -                  log_verdict_to_string(lph->verdict),
> -                  log_severity_to_string(lph->severity));
> -    flow_format(&ds, headers, NULL);
> -
> -    VLOG_INFO("%s", ds_cstr(&ds));
> -    ds_destroy(&ds);
> -    free(name);
> -}
> diff --git a/ovn/lib/acl-log.h b/ovn/lib/acl-log.h
> deleted file mode 100644
> index 55dc75b7f..000000000
> --- a/ovn/lib/acl-log.h
> +++ /dev/null
> @@ -1,54 +0,0 @@
> -/*
> - * Copyright (c) 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef ACL_LOG_H
> -#define ACL_LOG_H 1
> -
> -#include <stdint.h>
> -#include "openvswitch/types.h"
> -
> -struct ofpbuf;
> -struct flow;
> -
> -struct log_pin_header {
> -    uint8_t verdict;            /* One of LOG_VERDICT_*. */
> -    uint8_t severity;           /* One of LOG_SEVERITY*. */
> -    /* Followed by an optional string containing the rule's name. */
> -};
> -
> -enum log_verdict {
> -    LOG_VERDICT_ALLOW,
> -    LOG_VERDICT_DROP,
> -    LOG_VERDICT_REJECT,
> -    LOG_VERDICT_UNKNOWN = UINT8_MAX
> -};
> -
> -const char *log_verdict_to_string(uint8_t verdict);
> -
> -
> -/* Severity levels.  Based on RFC5424 levels. */
> -#define LOG_SEVERITY_ALERT    1
> -#define LOG_SEVERITY_WARNING  4
> -#define LOG_SEVERITY_NOTICE   5
> -#define LOG_SEVERITY_INFO     6
> -#define LOG_SEVERITY_DEBUG    7
> -
> -const char *log_severity_to_string(uint8_t severity);
> -uint8_t log_severity_from_string(const char *name);
> -
> -void handle_acl_log(const struct flow *headers, struct ofpbuf *userdata);
> -
> -#endif /* ovn/lib/acl-log.h */
> diff --git a/ovn/lib/actions.c b/ovn/lib/actions.c
> deleted file mode 100644
> index 4eacc44ed..000000000
> --- a/ovn/lib/actions.c
> +++ /dev/null
> @@ -1,2902 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include <stdarg.h>
> -#include <stdbool.h>
> -#include "bitmap.h"
> -#include "byte-order.h"
> -#include "compiler.h"
> -#include "ovn-l7.h"
> -#include "hash.h"
> -#include "lib/packets.h"
> -#include "nx-match.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/json.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/actions.h"
> -#include "ovn/expr.h"
> -#include "ovn/lex.h"
> -#include "ovn/lib/acl-log.h"
> -#include "ovn/lib/extend-table.h"
> -#include "packets.h"
> -#include "openvswitch/shash.h"
> -#include "simap.h"
> -#include "uuid.h"
> -#include "socket-util.h"
> -
> -VLOG_DEFINE_THIS_MODULE(actions);
> -
> -/* Prototypes for functions to be defined by each action. */
> -#define OVNACT(ENUM, STRUCT)                                        \
> -    static void format_##ENUM(const struct STRUCT *, struct ds *);  \
> -    static void encode_##ENUM(const struct STRUCT *,                \
> -                              const struct ovnact_encode_params *,  \
> -                              struct ofpbuf *ofpacts);              \
> -    static void STRUCT##_free(struct STRUCT *a);
> -OVNACTS
> -#undef OVNACT
> -
> -/* Helpers. */
> -
> -/* Implementation of ovnact_put_<ENUM>(). */
> -void *
> -ovnact_put(struct ofpbuf *ovnacts, enum ovnact_type type, size_t len)
> -{
> -    ovs_assert(len == OVNACT_ALIGN(len));
> -
> -    ovnacts->header = ofpbuf_put_uninit(ovnacts, len);
> -    struct ovnact *ovnact = ovnacts->header;
> -    ovnact_init(ovnact, type, len);
> -    return ovnact;
> -}
> -
> -/* Implementation of ovnact_init_<ENUM>(). */
> -void
> -ovnact_init(struct ovnact *ovnact, enum ovnact_type type, size_t len)
> -{
> -    ovs_assert(len == OVNACT_ALIGN(len));
> -    memset(ovnact, 0, len);
> -    ovnact->type = type;
> -    ovnact->len = len;
> -}
> -
> -static size_t
> -encode_start_controller_op(enum action_opcode opcode, bool pause,
> -                           uint32_t meter_id, struct ofpbuf *ofpacts)
> -{
> -    size_t ofs = ofpacts->size;
> -
> -    struct ofpact_controller *oc = ofpact_put_CONTROLLER(ofpacts);
> -    oc->max_len = UINT16_MAX;
> -    oc->reason = OFPR_ACTION;
> -    oc->pause = pause;
> -    oc->meter_id = meter_id;
> -
> -    struct action_header ah = { .opcode = htonl(opcode) };
> -    ofpbuf_put(ofpacts, &ah, sizeof ah);
> -
> -    return ofs;
> -}
> -
> -static void
> -encode_finish_controller_op(size_t ofs, struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_controller *oc = ofpbuf_at_assert(ofpacts, ofs, sizeof
> *oc);
> -    ofpacts->header = oc;
> -    oc->userdata_len = ofpacts->size - (ofs + sizeof *oc);
> -    ofpact_finish_CONTROLLER(ofpacts, &oc);
> -}
> -
> -static void
> -encode_controller_op(enum action_opcode opcode, struct ofpbuf *ofpacts)
> -{
> -    size_t ofs = encode_start_controller_op(opcode, false,
> NX_CTLR_NO_METER,
> -                                            ofpacts);
> -    encode_finish_controller_op(ofs, ofpacts);
> -}
> -
> -static void
> -init_stack(struct ofpact_stack *stack, enum mf_field_id field)
> -{
> -    stack->subfield.field = mf_from_id(field);
> -    stack->subfield.ofs = 0;
> -    stack->subfield.n_bits = stack->subfield.field->n_bits;
> -}
> -
> -struct arg {
> -    const struct mf_subfield src;
> -    enum mf_field_id dst;
> -};
> -
> -static void
> -encode_setup_args(const struct arg args[], size_t n_args,
> -                  struct ofpbuf *ofpacts)
> -{
> -    /* 1. Save all of the destinations that will be modified. */
> -    for (const struct arg *a = args; a < &args[n_args]; a++) {
> -        ovs_assert(a->src.n_bits == mf_from_id(a->dst)->n_bits);
> -        if (a->src.field->id != a->dst) {
> -            init_stack(ofpact_put_STACK_PUSH(ofpacts), a->dst);
> -        }
> -    }
> -
> -    /* 2. Push the sources, in reverse order. */
> -    for (size_t i = n_args - 1; i < n_args; i--) {
> -        const struct arg *a = &args[i];
> -        if (a->src.field->id != a->dst) {
> -            ofpact_put_STACK_PUSH(ofpacts)->subfield = a->src;
> -        }
> -    }
> -
> -    /* 3. Pop the sources into the destinations. */
> -    for (const struct arg *a = args; a < &args[n_args]; a++) {
> -        if (a->src.field->id != a->dst) {
> -            init_stack(ofpact_put_STACK_POP(ofpacts), a->dst);
> -        }
> -    }
> -}
> -
> -static void
> -encode_restore_args(const struct arg args[], size_t n_args,
> -                    struct ofpbuf *ofpacts)
> -{
> -    for (size_t i = n_args - 1; i < n_args; i--) {
> -        const struct arg *a = &args[i];
> -        if (a->src.field->id != a->dst) {
> -            init_stack(ofpact_put_STACK_POP(ofpacts), a->dst);
> -        }
> -    }
> -}
> -
> -static void
> -put_load(uint64_t value, enum mf_field_id dst, int ofs, int n_bits,
> -         struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts,
> -                                                       mf_from_id(dst),
> NULL,
> -                                                       NULL);
> -    ovs_be64 n_value = htonll(value);
> -    bitwise_copy(&n_value, 8, 0, sf->value, sf->field->n_bytes, ofs,
> n_bits);
> -    bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs,
> n_bits);
> -}
> -
> -static uint8_t
> -first_ptable(const struct ovnact_encode_params *ep,
> -             enum ovnact_pipeline pipeline)
> -{
> -    return (pipeline == OVNACT_P_INGRESS
> -            ? ep->ingress_ptable
> -            : ep->egress_ptable);
> -}
> -
> -#define MAX_NESTED_ACTION_DEPTH 32
> -
> -/* Context maintained during ovnacts_parse(). */
> -struct action_context {
> -    const struct ovnact_parse_params *pp; /* Parameters. */
> -    struct lexer *lexer;        /* Lexer for pulling more tokens. */
> -    struct ofpbuf *ovnacts;     /* Actions. */
> -    struct expr *prereqs;       /* Prerequisites to apply to match. */
> -    int depth;                  /* Current nested action depth. */
> -};
> -
> -static void parse_actions(struct action_context *, enum lex_type
> sentinel);
> -
> -static bool
> -action_parse_field(struct action_context *ctx,
> -                   int n_bits, bool rw, struct expr_field *f)
> -{
> -    if (!expr_field_parse(ctx->lexer, ctx->pp->symtab, f, &ctx->prereqs))
> {
> -        return false;
> -    }
> -
> -    char *error = expr_type_check(f, n_bits, rw);
> -    if (error) {
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return false;
> -    }
> -
> -    return true;
> -}
> -
> -static bool
> -action_parse_port(struct action_context *ctx, uint16_t *port)
> -{
> -    if (lexer_is_int(ctx->lexer)) {
> -        int value = ntohll(ctx->lexer->token.value.integer);
> -        if (value <= UINT16_MAX) {
> -            *port = value;
> -            lexer_get(ctx->lexer);
> -            return true;
> -        }
> -    }
> -    lexer_syntax_error(ctx->lexer, "expecting port number");
> -    return false;
> -}
> -
> -/* Parses 'prerequisite' as an expression in the context of 'ctx', then
> adds it
> - * as a conjunction with the existing 'ctx->prereqs'. */
> -static void
> -add_prerequisite(struct action_context *ctx, const char *prerequisite)
> -{
> -    struct expr *expr;
> -    char *error;
> -
> -    expr = expr_parse_string(prerequisite, ctx->pp->symtab, NULL, NULL,
> NULL,
> -                             &error);
> -    ovs_assert(!error);
> -    ctx->prereqs = expr_combine(EXPR_T_AND, ctx->prereqs, expr);
> -}
> -
> -static void
> -ovnact_null_free(struct ovnact_null *a OVS_UNUSED)
> -{
> -}
> -
> -static void
> -format_OUTPUT(const struct ovnact_null *a OVS_UNUSED, struct ds *s)
> -{
> -    ds_put_cstr(s, "output;");
> -}
> -
> -static void
> -emit_resubmit(struct ofpbuf *ofpacts, uint8_t ptable)
> -{
> -    struct ofpact_resubmit *resubmit = ofpact_put_RESUBMIT(ofpacts);
> -    resubmit->in_port = OFPP_IN_PORT;
> -    resubmit->table_id = ptable;
> -}
> -
> -static void
> -encode_OUTPUT(const struct ovnact_null *a OVS_UNUSED,
> -              const struct ovnact_encode_params *ep,
> -              struct ofpbuf *ofpacts)
> -{
> -    emit_resubmit(ofpacts, ep->output_ptable);
> -}
> -
> -static void
> -parse_NEXT(struct action_context *ctx)
> -{
> -    if (!ctx->pp->n_tables) {
> -        lexer_error(ctx->lexer, "\"next\" action not allowed here.");
> -        return;
> -    }
> -
> -    int pipeline = ctx->pp->pipeline;
> -    int table = ctx->pp->cur_ltable + 1;
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        if (lexer_is_int(ctx->lexer)) {
> -            lexer_get_int(ctx->lexer, &table);
> -        } else {
> -            do {
> -                if (lexer_match_id(ctx->lexer, "pipeline")) {
> -                    if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -                        return;
> -                    }
> -                    if (lexer_match_id(ctx->lexer, "ingress")) {
> -                        pipeline = OVNACT_P_INGRESS;
> -                    } else if (lexer_match_id(ctx->lexer, "egress")) {
> -                        pipeline = OVNACT_P_EGRESS;
> -                    } else {
> -                        lexer_syntax_error(
> -                            ctx->lexer, "expecting \"ingress\" or
> \"egress\"");
> -                        return;
> -                    }
> -                } else if (lexer_match_id(ctx->lexer, "table")) {
> -                    if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS) ||
> -                        !lexer_force_int(ctx->lexer, &table)) {
> -                        return;
> -                    }
> -                } else {
> -                    lexer_syntax_error(ctx->lexer,
> -                                       "expecting \"pipeline\" or
> \"table\"");
> -                    return;
> -                }
> -            } while (lexer_match(ctx->lexer, LEX_T_COMMA));
> -        }
> -        if (!lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -            return;
> -        }
> -    }
> -
> -    if (pipeline == OVNACT_P_EGRESS && ctx->pp->pipeline ==
> OVNACT_P_INGRESS) {
> -        lexer_error(ctx->lexer,
> -                    "\"next\" action cannot advance from ingress to
> egress "
> -                    "pipeline (use \"output\" action instead)");
> -    } else if (table >= ctx->pp->n_tables) {
> -        lexer_error(ctx->lexer,
> -                    "\"next\" action cannot advance beyond table %d.",
> -                    ctx->pp->n_tables - 1);
> -        return;
> -    }
> -
> -    struct ovnact_next *next = ovnact_put_NEXT(ctx->ovnacts);
> -    next->pipeline = pipeline;
> -    next->ltable = table;
> -    next->src_pipeline = ctx->pp->pipeline;
> -    next->src_ltable = ctx->pp->cur_ltable;
> -}
> -
> -static void
> -format_NEXT(const struct ovnact_next *next, struct ds *s)
> -{
> -    if (next->pipeline != next->src_pipeline) {
> -        ds_put_format(s, "next(pipeline=%s, table=%d);",
> -                      (next->pipeline == OVNACT_P_INGRESS
> -                       ? "ingress" : "egress"),
> -                      next->ltable);
> -    } else if (next->ltable != next->src_ltable + 1) {
> -        ds_put_format(s, "next(%d);", next->ltable);
> -    } else {
> -        ds_put_cstr(s, "next;");
> -    }
> -}
> -
> -static void
> -encode_NEXT(const struct ovnact_next *next,
> -            const struct ovnact_encode_params *ep,
> -            struct ofpbuf *ofpacts)
> -{
> -    emit_resubmit(ofpacts, first_ptable(ep, next->pipeline) +
> next->ltable);
> -}
> -
> -static void
> -ovnact_next_free(struct ovnact_next *a OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_LOAD(struct action_context *ctx, const struct expr_field *lhs)
> -{
> -    size_t ofs = ctx->ovnacts->size;
> -    struct ovnact_load *load;
> -    if (lhs->symbol->ovn_field) {
> -        load = ovnact_put_OVNFIELD_LOAD(ctx->ovnacts);
> -    } else {
> -        load = ovnact_put_LOAD(ctx->ovnacts);
> -    }
> -
> -    load->dst = *lhs;
> -
> -    char *error = expr_type_check(lhs, lhs->n_bits, true);
> -    if (error) {
> -        ctx->ovnacts->size = ofs;
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return;
> -    }
> -    if (!expr_constant_parse(ctx->lexer, lhs, &load->imm)) {
> -        ctx->ovnacts->size = ofs;
> -        return;
> -    }
> -}
> -
> -static enum expr_constant_type
> -load_type(const struct ovnact_load *load)
> -{
> -    return load->dst.symbol->width > 0 ? EXPR_C_INTEGER : EXPR_C_STRING;
> -}
> -
> -static void
> -format_LOAD(const struct ovnact_load *load, struct ds *s)
> -{
> -    expr_field_format(&load->dst, s);
> -    ds_put_cstr(s, " = ");
> -    expr_constant_format(&load->imm, load_type(load), s);
> -    ds_put_char(s, ';');
> -}
> -
> -static void
> -encode_LOAD(const struct ovnact_load *load,
> -            const struct ovnact_encode_params *ep,
> -            struct ofpbuf *ofpacts)
> -{
> -    const union expr_constant *c = &load->imm;
> -    struct mf_subfield dst = expr_resolve_field(&load->dst);
> -    struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts, dst.field,
> -                                                       NULL, NULL);
> -
> -    if (load->dst.symbol->width) {
> -        bitwise_copy(&c->value, sizeof c->value, 0,
> -                     sf->value, dst.field->n_bytes, dst.ofs,
> -                     dst.n_bits);
> -        if (c->masked) {
> -            bitwise_copy(&c->mask, sizeof c->mask, 0,
> -                         ofpact_set_field_mask(sf), dst.field->n_bytes,
> -                         dst.ofs, dst.n_bits);
> -        } else {
> -            bitwise_one(ofpact_set_field_mask(sf), dst.field->n_bytes,
> -                        dst.ofs, dst.n_bits);
> -        }
> -    } else {
> -        uint32_t port;
> -        if (!ep->lookup_port(ep->aux, load->imm.string, &port)) {
> -            port = 0;
> -        }
> -        bitwise_put(port, sf->value,
> -                    sf->field->n_bytes, 0, sf->field->n_bits);
> -        bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, 0,
> -                    sf->field->n_bits);
> -    }
> -}
> -
> -static void
> -ovnact_load_free(struct ovnact_load *load)
> -{
> -    expr_constant_destroy(&load->imm, load_type(load));
> -}
> -
> -static void
> -format_assignment(const struct ovnact_move *move, const char *operator,
> -                  struct ds *s)
> -{
> -    expr_field_format(&move->lhs, s);
> -    ds_put_format(s, " %s ", operator);
> -    expr_field_format(&move->rhs, s);
> -    ds_put_char(s, ';');
> -}
> -
> -static void
> -format_MOVE(const struct ovnact_move *move, struct ds *s)
> -{
> -    format_assignment(move, "=", s);
> -}
> -
> -static void
> -format_EXCHANGE(const struct ovnact_move *move, struct ds *s)
> -{
> -    format_assignment(move, "<->", s);
> -}
> -
> -static void
> -parse_assignment_action(struct action_context *ctx, bool exchange,
> -                        const struct expr_field *lhs)
> -{
> -    struct expr_field rhs;
> -    if (!expr_field_parse(ctx->lexer, ctx->pp->symtab, &rhs,
> &ctx->prereqs)) {
> -        return;
> -    }
> -
> -    const struct expr_symbol *ls = lhs->symbol;
> -    const struct expr_symbol *rs = rhs.symbol;
> -    if ((ls->width != 0) != (rs->width != 0)) {
> -        if (exchange) {
> -            lexer_error(ctx->lexer,
> -                        "Can't exchange %s field (%s) with %s field
> (%s).",
> -                        ls->width ? "integer" : "string",
> -                        ls->name,
> -                        rs->width ? "integer" : "string",
> -                        rs->name);
> -        } else {
> -            lexer_error(ctx->lexer,
> -                        "Can't assign %s field (%s) to %s field (%s).",
> -                        rs->width ? "integer" : "string",
> -                        rs->name,
> -                        ls->width ? "integer" : "string",
> -                        ls->name);
> -        }
> -        return;
> -    }
> -
> -    if (lhs->n_bits != rhs.n_bits) {
> -        if (exchange) {
> -            lexer_error(ctx->lexer,
> -                        "Can't exchange %d-bit field with %d-bit field.",
> -                        lhs->n_bits, rhs.n_bits);
> -        } else {
> -            lexer_error(ctx->lexer,
> -                        "Can't assign %d-bit value to %d-bit
> destination.",
> -                        rhs.n_bits, lhs->n_bits);
> -        }
> -        return;
> -    } else if (!lhs->n_bits &&
> -               ls->field->n_bits != rs->field->n_bits) {
> -        lexer_error(ctx->lexer, "String fields %s and %s are incompatible
> for "
> -                    "%s.", ls->name, rs->name,
> -                    exchange ? "exchange" : "assignment");
> -        return;
> -    }
> -
> -    char *error = expr_type_check(lhs, lhs->n_bits, true);
> -    if (!error) {
> -        error = expr_type_check(&rhs, rhs.n_bits, true);
> -    }
> -    if (error) {
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return;
> -    }
> -
> -    struct ovnact_move *move;
> -    move = (exchange
> -            ? ovnact_put_EXCHANGE(ctx->ovnacts)
> -            : ovnact_put_MOVE(ctx->ovnacts));
> -    move->lhs = *lhs;
> -    move->rhs = rhs;
> -}
> -
> -static void
> -encode_MOVE(const struct ovnact_move *move,
> -            const struct ovnact_encode_params *ep OVS_UNUSED,
> -            struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_reg_move *orm = ofpact_put_REG_MOVE(ofpacts);
> -    orm->src = expr_resolve_field(&move->rhs);
> -    orm->dst = expr_resolve_field(&move->lhs);
> -}
> -
> -static void
> -encode_EXCHANGE(const struct ovnact_move *xchg,
> -                const struct ovnact_encode_params *ep OVS_UNUSED,
> -                struct ofpbuf *ofpacts)
> -{
> -    ofpact_put_STACK_PUSH(ofpacts)->subfield =
> expr_resolve_field(&xchg->rhs);
> -    ofpact_put_STACK_PUSH(ofpacts)->subfield =
> expr_resolve_field(&xchg->lhs);
> -    ofpact_put_STACK_POP(ofpacts)->subfield =
> expr_resolve_field(&xchg->rhs);
> -    ofpact_put_STACK_POP(ofpacts)->subfield =
> expr_resolve_field(&xchg->lhs);
> -}
> -
> -static void
> -ovnact_move_free(struct ovnact_move *move OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_DEC_TTL(struct action_context *ctx)
> -{
> -    lexer_force_match(ctx->lexer, LEX_T_DECREMENT);
> -    ovnact_put_DEC_TTL(ctx->ovnacts);
> -    add_prerequisite(ctx, "ip");
> -}
> -
> -static void
> -format_DEC_TTL(const struct ovnact_null *null OVS_UNUSED, struct ds *s)
> -{
> -    ds_put_cstr(s, "ip.ttl--;");
> -}
> -
> -static void
> -encode_DEC_TTL(const struct ovnact_null *null OVS_UNUSED,
> -               const struct ovnact_encode_params *ep OVS_UNUSED,
> -               struct ofpbuf *ofpacts)
> -{
> -    ofpact_put_DEC_TTL(ofpacts);
> -}
> -
> -static void
> -parse_CT_NEXT(struct action_context *ctx)
> -{
> -    if (ctx->pp->cur_ltable >= ctx->pp->n_tables) {
> -        lexer_error(ctx->lexer,
> -                    "\"ct_next\" action not allowed in last table.");
> -        return;
> -    }
> -
> -    add_prerequisite(ctx, "ip");
> -    ovnact_put_CT_NEXT(ctx->ovnacts)->ltable = ctx->pp->cur_ltable + 1;
> -}
> -
> -static void
> -format_CT_NEXT(const struct ovnact_ct_next *ct_next OVS_UNUSED, struct ds
> *s)
> -{
> -    ds_put_cstr(s, "ct_next;");
> -}
> -
> -static void
> -encode_CT_NEXT(const struct ovnact_ct_next *ct_next,
> -                const struct ovnact_encode_params *ep,
> -                struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_conntrack *ct = ofpact_put_CT(ofpacts);
> -    ct->recirc_table = first_ptable(ep, ep->pipeline) + ct_next->ltable;
> -    ct->zone_src.field = ep->is_switch ? mf_from_id(MFF_LOG_CT_ZONE)
> -                            : mf_from_id(MFF_LOG_DNAT_ZONE);
> -    ct->zone_src.ofs = 0;
> -    ct->zone_src.n_bits = 16;
> -    ofpact_finish(ofpacts, &ct->ofpact);
> -}
> -
> -static void
> -ovnact_ct_next_free(struct ovnact_ct_next *a OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_ct_commit_arg(struct action_context *ctx,
> -                    struct ovnact_ct_commit *cc)
> -{
> -    if (lexer_match_id(ctx->lexer, "ct_mark")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        if (ctx->lexer->token.type == LEX_T_INTEGER) {
> -            cc->ct_mark = ntohll(ctx->lexer->token.value.integer);
> -            cc->ct_mark_mask = UINT32_MAX;
> -        } else if (ctx->lexer->token.type == LEX_T_MASKED_INTEGER) {
> -            cc->ct_mark = ntohll(ctx->lexer->token.value.integer);
> -            cc->ct_mark_mask = ntohll(ctx->lexer->token.mask.integer);
> -        } else {
> -            lexer_syntax_error(ctx->lexer, "expecting integer");
> -            return;
> -        }
> -        lexer_get(ctx->lexer);
> -    } else if (lexer_match_id(ctx->lexer, "ct_label")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        if (ctx->lexer->token.type == LEX_T_INTEGER) {
> -            cc->ct_label = ctx->lexer->token.value.be128_int;
> -            cc->ct_label_mask = OVS_BE128_MAX;
> -        } else if (ctx->lexer->token.type == LEX_T_MASKED_INTEGER) {
> -            cc->ct_label = ctx->lexer->token.value.be128_int;
> -            cc->ct_label_mask = ctx->lexer->token.mask.be128_int;
> -        } else {
> -            lexer_syntax_error(ctx->lexer, "expecting integer");
> -            return;
> -        }
> -        lexer_get(ctx->lexer);
> -    } else {
> -        lexer_syntax_error(ctx->lexer, NULL);
> -    }
> -}
> -
> -static void
> -parse_CT_COMMIT(struct action_context *ctx)
> -{
> -    add_prerequisite(ctx, "ip");
> -
> -    struct ovnact_ct_commit *ct_commit =
> ovnact_put_CT_COMMIT(ctx->ovnacts);
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        while (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -            parse_ct_commit_arg(ctx, ct_commit);
> -            if (ctx->lexer->error) {
> -                return;
> -            }
> -            lexer_match(ctx->lexer, LEX_T_COMMA);
> -        }
> -    }
> -}
> -
> -static void
> -format_CT_COMMIT(const struct ovnact_ct_commit *cc, struct ds *s)
> -{
> -    ds_put_cstr(s, "ct_commit(");
> -    if (cc->ct_mark_mask) {
> -        ds_put_format(s, "ct_mark=%#"PRIx32, cc->ct_mark);
> -        if (cc->ct_mark_mask != UINT32_MAX) {
> -            ds_put_format(s, "/%#"PRIx32, cc->ct_mark_mask);
> -        }
> -    }
> -    if (!ovs_be128_is_zero(cc->ct_label_mask)) {
> -        if (ds_last(s) != '(') {
> -            ds_put_cstr(s, ", ");
> -        }
> -
> -        ds_put_format(s, "ct_label=");
> -        ds_put_hex(s, &cc->ct_label, sizeof cc->ct_label);
> -        if (!ovs_be128_equals(cc->ct_label_mask, OVS_BE128_MAX)) {
> -            ds_put_char(s, '/');
> -            ds_put_hex(s, &cc->ct_label_mask, sizeof cc->ct_label_mask);
> -        }
> -    }
> -    if (!ds_chomp(s, '(')) {
> -        ds_put_char(s, ')');
> -    }
> -    ds_put_char(s, ';');
> -}
> -
> -static void
> -encode_CT_COMMIT(const struct ovnact_ct_commit *cc,
> -                 const struct ovnact_encode_params *ep OVS_UNUSED,
> -                 struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_conntrack *ct = ofpact_put_CT(ofpacts);
> -    ct->flags = NX_CT_F_COMMIT;
> -    ct->recirc_table = NX_CT_RECIRC_NONE;
> -    ct->zone_src.field = mf_from_id(MFF_LOG_CT_ZONE);
> -    ct->zone_src.ofs = 0;
> -    ct->zone_src.n_bits = 16;
> -
> -    size_t set_field_offset = ofpacts->size;
> -    ofpbuf_pull(ofpacts, set_field_offset);
> -
> -    if (cc->ct_mark_mask) {
> -        const ovs_be32 value = htonl(cc->ct_mark);
> -        const ovs_be32 mask = htonl(cc->ct_mark_mask);
> -        ofpact_put_set_field(ofpacts, mf_from_id(MFF_CT_MARK), &value,
> &mask);
> -    }
> -
> -    if (!ovs_be128_is_zero(cc->ct_label_mask)) {
> -        ofpact_put_set_field(ofpacts, mf_from_id(MFF_CT_LABEL),
> &cc->ct_label,
> -                             &cc->ct_label_mask);
> -    }
> -
> -    ofpacts->header = ofpbuf_push_uninit(ofpacts, set_field_offset);
> -    ct = ofpacts->header;
> -    ofpact_finish(ofpacts, &ct->ofpact);
> -}
> -
> -static void
> -ovnact_ct_commit_free(struct ovnact_ct_commit *cc OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_ct_nat(struct action_context *ctx, const char *name,
> -             struct ovnact_ct_nat *cn)
> -{
> -    add_prerequisite(ctx, "ip");
> -
> -    if (ctx->pp->cur_ltable >= ctx->pp->n_tables) {
> -        lexer_error(ctx->lexer,
> -                    "\"%s\" action not allowed in last table.", name);
> -        return;
> -    }
> -    cn->ltable = ctx->pp->cur_ltable + 1;
> -
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        if (ctx->lexer->token.type != LEX_T_INTEGER
> -            || ctx->lexer->token.format != LEX_F_IPV4) {
> -            lexer_syntax_error(ctx->lexer, "expecting IPv4 address");
> -            return;
> -        }
> -        cn->ip = ctx->lexer->token.value.ipv4;
> -        lexer_get(ctx->lexer);
> -
> -        if (!lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -            return;
> -        }
> -    }
> -}
> -
> -static void
> -parse_CT_DNAT(struct action_context *ctx)
> -{
> -    parse_ct_nat(ctx, "ct_dnat", ovnact_put_CT_DNAT(ctx->ovnacts));
> -}
> -
> -static void
> -parse_CT_SNAT(struct action_context *ctx)
> -{
> -    parse_ct_nat(ctx, "ct_snat", ovnact_put_CT_SNAT(ctx->ovnacts));
> -}
> -
> -static void
> -format_ct_nat(const struct ovnact_ct_nat *cn, const char *name, struct ds
> *s)
> -{
> -    ds_put_cstr(s, name);
> -    if (cn->ip) {
> -        ds_put_format(s, "("IP_FMT")", IP_ARGS(cn->ip));
> -    }
> -    ds_put_char(s, ';');
> -}
> -
> -static void
> -format_CT_DNAT(const struct ovnact_ct_nat *cn, struct ds *s)
> -{
> -    format_ct_nat(cn, "ct_dnat", s);
> -}
> -
> -static void
> -format_CT_SNAT(const struct ovnact_ct_nat *cn, struct ds *s)
> -{
> -    format_ct_nat(cn, "ct_snat", s);
> -}
> -
> -static void
> -encode_ct_nat(const struct ovnact_ct_nat *cn,
> -              const struct ovnact_encode_params *ep,
> -              bool snat, struct ofpbuf *ofpacts)
> -{
> -    const size_t ct_offset = ofpacts->size;
> -    ofpbuf_pull(ofpacts, ct_offset);
> -
> -    struct ofpact_conntrack *ct = ofpact_put_CT(ofpacts);
> -    ct->recirc_table = cn->ltable + first_ptable(ep, ep->pipeline);
> -    if (snat) {
> -        ct->zone_src.field = mf_from_id(MFF_LOG_SNAT_ZONE);
> -    } else {
> -        ct->zone_src.field = mf_from_id(MFF_LOG_DNAT_ZONE);
> -    }
> -    ct->zone_src.ofs = 0;
> -    ct->zone_src.n_bits = 16;
> -    ct->flags = 0;
> -    ct->alg = 0;
> -
> -    struct ofpact_nat *nat;
> -    size_t nat_offset;
> -    nat_offset = ofpacts->size;
> -    ofpbuf_pull(ofpacts, nat_offset);
> -
> -    nat = ofpact_put_NAT(ofpacts);
> -    nat->flags = 0;
> -    nat->range_af = AF_UNSPEC;
> -
> -    if (cn->ip) {
> -        nat->range_af = AF_INET;
> -        nat->range.addr.ipv4.min = cn->ip;
> -        if (snat) {
> -            nat->flags |= NX_NAT_F_SRC;
> -        } else {
> -            nat->flags |= NX_NAT_F_DST;
> -        }
> -    }
> -
> -    ofpacts->header = ofpbuf_push_uninit(ofpacts, nat_offset);
> -    ct = ofpacts->header;
> -    if (cn->ip) {
> -        ct->flags |= NX_CT_F_COMMIT;
> -    }
> -    ofpact_finish(ofpacts, &ct->ofpact);
> -    ofpbuf_push_uninit(ofpacts, ct_offset);
> -}
> -
> -static void
> -encode_CT_DNAT(const struct ovnact_ct_nat *cn,
> -               const struct ovnact_encode_params *ep,
> -               struct ofpbuf *ofpacts)
> -{
> -    encode_ct_nat(cn, ep, false, ofpacts);
> -}
> -
> -static void
> -encode_CT_SNAT(const struct ovnact_ct_nat *cn,
> -               const struct ovnact_encode_params *ep,
> -               struct ofpbuf *ofpacts)
> -{
> -    encode_ct_nat(cn, ep, true, ofpacts);
> -}
> -
> -static void
> -ovnact_ct_nat_free(struct ovnact_ct_nat *ct_nat OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_ct_lb_action(struct action_context *ctx)
> -{
> -    if (ctx->pp->cur_ltable >= ctx->pp->n_tables) {
> -        lexer_error(ctx->lexer, "\"ct_lb\" action not allowed in last
> table.");
> -        return;
> -    }
> -
> -    add_prerequisite(ctx, "ip");
> -
> -    struct ovnact_ct_lb_dst *dsts = NULL;
> -    size_t allocated_dsts = 0;
> -    size_t n_dsts = 0;
> -
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        while (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -            struct ovnact_ct_lb_dst dst;
> -            if (lexer_match(ctx->lexer, LEX_T_LSQUARE)) {
> -                /* IPv6 address and port */
> -                if (ctx->lexer->token.type != LEX_T_INTEGER
> -                    || ctx->lexer->token.format != LEX_F_IPV6) {
> -                    free(dsts);
> -                    lexer_syntax_error(ctx->lexer, "expecting IPv6
> address");
> -                    return;
> -                }
> -                dst.family = AF_INET6;
> -                dst.ipv6 = ctx->lexer->token.value.ipv6;
> -
> -                lexer_get(ctx->lexer);
> -                if (!lexer_match(ctx->lexer, LEX_T_RSQUARE)) {
> -                    free(dsts);
> -                    lexer_syntax_error(ctx->lexer, "no closing square "
> -                                                   "bracket");
> -                    return;
> -                }
> -                dst.port = 0;
> -                if (lexer_match(ctx->lexer, LEX_T_COLON)
> -                    && !action_parse_port(ctx, &dst.port)) {
> -                    free(dsts);
> -                    return;
> -                }
> -            } else {
> -                if (ctx->lexer->token.type != LEX_T_INTEGER
> -                    || (ctx->lexer->token.format != LEX_F_IPV4
> -                    && ctx->lexer->token.format != LEX_F_IPV6)) {
> -                    free(dsts);
> -                    lexer_syntax_error(ctx->lexer, "expecting IP
> address");
> -                    return;
> -                }
> -
> -                /* Parse IP. */
> -                if (ctx->lexer->token.format == LEX_F_IPV4) {
> -                    dst.family = AF_INET;
> -                    dst.ipv4 = ctx->lexer->token.value.ipv4;
> -                } else {
> -                    dst.family = AF_INET6;
> -                    dst.ipv6 = ctx->lexer->token.value.ipv6;
> -                }
> -
> -                lexer_get(ctx->lexer);
> -                dst.port = 0;
> -                if (lexer_match(ctx->lexer, LEX_T_COLON)) {
> -                    if (dst.family == AF_INET6) {
> -                        free(dsts);
> -                        lexer_syntax_error(ctx->lexer, "IPv6 address
> needs "
> -                                "square brackets if port is included");
> -                        return;
> -                    } else if (!action_parse_port(ctx, &dst.port)) {
> -                        free(dsts);
> -                        return;
> -                    }
> -                }
> -            }
> -            lexer_match(ctx->lexer, LEX_T_COMMA);
> -
> -            /* Append to dsts. */
> -            if (n_dsts >= allocated_dsts) {
> -                dsts = x2nrealloc(dsts, &allocated_dsts, sizeof *dsts);
> -            }
> -            dsts[n_dsts++] = dst;
> -        }
> -    }
> -
> -    struct ovnact_ct_lb *cl = ovnact_put_CT_LB(ctx->ovnacts);
> -    cl->ltable = ctx->pp->cur_ltable + 1;
> -    cl->dsts = dsts;
> -    cl->n_dsts = n_dsts;
> -}
> -
> -static void
> -format_CT_LB(const struct ovnact_ct_lb *cl, struct ds *s)
> -{
> -    ds_put_cstr(s, "ct_lb");
> -    if (cl->n_dsts) {
> -        ds_put_char(s, '(');
> -        for (size_t i = 0; i < cl->n_dsts; i++) {
> -            if (i) {
> -                ds_put_cstr(s, ", ");
> -            }
> -
> -            const struct ovnact_ct_lb_dst *dst = &cl->dsts[i];
> -            if (dst->family == AF_INET) {
> -                ds_put_format(s, IP_FMT, IP_ARGS(dst->ipv4));
> -                if (dst->port) {
> -                    ds_put_format(s, ":%"PRIu16, dst->port);
> -                }
> -            } else {
> -                if (dst->port) {
> -                    ds_put_char(s, '[');
> -                }
> -                ipv6_format_addr(&dst->ipv6, s);
> -                if (dst->port) {
> -                    ds_put_format(s, "]:%"PRIu16, dst->port);
> -                }
> -            }
> -        }
> -        ds_put_char(s, ')');
> -    }
> -    ds_put_char(s, ';');
> -}
> -
> -static void
> -encode_CT_LB(const struct ovnact_ct_lb *cl,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    uint8_t recirc_table = cl->ltable + first_ptable(ep, ep->pipeline);
> -    if (!cl->n_dsts) {
> -        /* ct_lb without any destinations means that this is an
> established
> -         * connection and we just need to do a NAT. */
> -        const size_t ct_offset = ofpacts->size;
> -        ofpbuf_pull(ofpacts, ct_offset);
> -
> -        struct ofpact_conntrack *ct = ofpact_put_CT(ofpacts);
> -        struct ofpact_nat *nat;
> -        size_t nat_offset;
> -        ct->zone_src.field = ep->is_switch ? mf_from_id(MFF_LOG_CT_ZONE)
> -                                : mf_from_id(MFF_LOG_DNAT_ZONE);
> -        ct->zone_src.ofs = 0;
> -        ct->zone_src.n_bits = 16;
> -        ct->flags = 0;
> -        ct->recirc_table = recirc_table;
> -        ct->alg = 0;
> -
> -        nat_offset = ofpacts->size;
> -        ofpbuf_pull(ofpacts, nat_offset);
> -
> -        nat = ofpact_put_NAT(ofpacts);
> -        nat->flags = 0;
> -        nat->range_af = AF_UNSPEC;
> -
> -        ofpacts->header = ofpbuf_push_uninit(ofpacts, nat_offset);
> -        ct = ofpacts->header;
> -        ofpact_finish(ofpacts, &ct->ofpact);
> -        ofpbuf_push_uninit(ofpacts, ct_offset);
> -        return;
> -    }
> -
> -    uint32_t table_id = 0;
> -    struct ofpact_group *og;
> -    uint32_t zone_reg = ep->is_switch ? MFF_LOG_CT_ZONE - MFF_REG0
> -                            : MFF_LOG_DNAT_ZONE - MFF_REG0;
> -
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&ds, "type=select,selection_method=dp_hash");
> -
> -    BUILD_ASSERT(MFF_LOG_CT_ZONE >= MFF_REG0);
> -    BUILD_ASSERT(MFF_LOG_CT_ZONE < MFF_REG0 + FLOW_N_REGS);
> -    BUILD_ASSERT(MFF_LOG_DNAT_ZONE >= MFF_REG0);
> -    BUILD_ASSERT(MFF_LOG_DNAT_ZONE < MFF_REG0 + FLOW_N_REGS);
> -    for (size_t bucket_id = 0; bucket_id < cl->n_dsts; bucket_id++) {
> -        const struct ovnact_ct_lb_dst *dst = &cl->dsts[bucket_id];
> -        char ip_addr[INET6_ADDRSTRLEN];
> -        if (dst->family == AF_INET) {
> -            inet_ntop(AF_INET, &dst->ipv4, ip_addr, sizeof ip_addr);
> -        } else {
> -            inet_ntop(AF_INET6, &dst->ipv6, ip_addr, sizeof ip_addr);
> -        }
> -        ds_put_format(&ds,
> ",bucket=bucket_id=%"PRIuSIZE",weight:100,actions="
> -                      "ct(nat(dst=%s%s%s", bucket_id,
> -                      dst->family == AF_INET6 && dst->port ? "[" : "",
> -                      ip_addr,
> -                      dst->family == AF_INET6 && dst->port ? "]" : "");
> -        if (dst->port) {
> -            ds_put_format(&ds, ":%"PRIu16, dst->port);
> -        }
> -        ds_put_format(&ds, "),commit,table=%d,zone=NXM_NX_REG%d[0..15])",
> -                      recirc_table, zone_reg);
> -    }
> -
> -    table_id = ovn_extend_table_assign_id(ep->group_table, ds_cstr(&ds),
> -                                          ep->lflow_uuid);
> -    ds_destroy(&ds);
> -    if (table_id == EXT_TABLE_ID_INVALID) {
> -        return;
> -    }
> -
> -    /* Create an action to set the group. */
> -    og = ofpact_put_GROUP(ofpacts);
> -    og->group_id = table_id;
> -}
> -
> -static void
> -ovnact_ct_lb_free(struct ovnact_ct_lb *ct_lb)
> -{
> -    free(ct_lb->dsts);
> -}
> -
> -static void
> -format_CT_CLEAR(const struct ovnact_null *null OVS_UNUSED, struct ds *s)
> -{
> -    ds_put_cstr(s, "ct_clear;");
> -}
> -
> -static void
> -encode_CT_CLEAR(const struct ovnact_null *null OVS_UNUSED,
> -                const struct ovnact_encode_params *ep OVS_UNUSED,
> -                struct ofpbuf *ofpacts)
> -{
> -    ofpact_put_CT_CLEAR(ofpacts);
> -}
> -
> -/* Implements the "arp", "nd_na", and "clone" actions, which execute
> nested
> - * actions on a packet derived from the one being processed. */
> -static void
> -parse_nested_action(struct action_context *ctx, enum ovnact_type type,
> -                    const char *prereq)
> -{
> -    if (!lexer_force_match(ctx->lexer, LEX_T_LCURLY)) {
> -        return;
> -    }
> -
> -    if (ctx->depth + 1 == MAX_NESTED_ACTION_DEPTH) {
> -        lexer_error(ctx->lexer, "maximum depth of nested actions
> reached");
> -        return;
> -    }
> -
> -    uint64_t stub[1024 / 8];
> -    struct ofpbuf nested = OFPBUF_STUB_INITIALIZER(stub);
> -
> -    struct action_context inner_ctx = {
> -        .pp = ctx->pp,
> -        .lexer = ctx->lexer,
> -        .ovnacts = &nested,
> -        .prereqs = NULL,
> -        .depth = ctx->depth + 1,
> -    };
> -    parse_actions(&inner_ctx, LEX_T_RCURLY);
> -
> -    if (prereq) {
> -        /* XXX Not really sure what we should do with prerequisites for
> "arp"
> -         * and "nd_na" actions. */
> -        expr_destroy(inner_ctx.prereqs);
> -        add_prerequisite(ctx, prereq);
> -    } else {
> -        /* For "clone", the inner prerequisites should just add to the
> outer
> -         * ones. */
> -        ctx->prereqs = expr_combine(EXPR_T_AND,
> -                                    inner_ctx.prereqs, ctx->prereqs);
> -    }
> -
> -    if (inner_ctx.lexer->error) {
> -        ovnacts_free(nested.data, nested.size);
> -        ofpbuf_uninit(&nested);
> -        return;
> -    }
> -
> -    struct ovnact_nest *on = ovnact_put(ctx->ovnacts, type,
> -                                        OVNACT_ALIGN(sizeof *on));
> -    on->nested_len = nested.size;
> -    on->nested = ofpbuf_steal_data(&nested);
> -}
> -
> -static void
> -parse_ARP(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ARP, "ip4");
> -}
> -
> -static void
> -parse_ICMP4(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ICMP4, "ip4");
> -}
> -
> -static void
> -parse_ICMP4_ERROR(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ICMP4_ERROR, "ip4");
> -}
> -
> -static void
> -parse_ICMP6(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ICMP6, "ip6");
> -}
> -
> -static void
> -parse_TCP_RESET(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_TCP_RESET, "tcp");
> -}
> -
> -static void
> -parse_ND_NA(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ND_NA, "nd_ns");
> -}
> -
> -static void
> -parse_ND_NA_ROUTER(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ND_NA_ROUTER, "nd_ns");
> -}
> -
> -static void
> -parse_ND_NS(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_ND_NS, "ip6");
> -}
> -
> -static void
> -parse_CLONE(struct action_context *ctx)
> -{
> -    parse_nested_action(ctx, OVNACT_CLONE, NULL);
> -}
> -
> -static void
> -format_nested_action(const struct ovnact_nest *on, const char *name,
> -                     struct ds *s)
> -{
> -    ds_put_format(s, "%s { ", name);
> -    ovnacts_format(on->nested, on->nested_len, s);
> -    ds_put_format(s, " };");
> -}
> -
> -static void
> -format_ARP(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "arp", s);
> -}
> -
> -static void
> -format_ICMP4(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "icmp4", s);
> -}
> -
> -static void
> -format_ICMP4_ERROR(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "icmp4_error", s);
> -}
> -
> -static void
> -format_ICMP6(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "icmp6", s);
> -}
> -
> -static void
> -format_IGMP(const struct ovnact_null *a OVS_UNUSED, struct ds *s)
> -{
> -    ds_put_cstr(s, "igmp;");
> -}
> -
> -static void
> -format_TCP_RESET(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "tcp_reset", s);
> -}
> -
> -static void
> -format_ND_NA(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "nd_na", s);
> -}
> -
> -static void
> -format_ND_NA_ROUTER(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "nd_na_router", s);
> -}
> -
> -static void
> -format_ND_NS(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "nd_ns", s);
> -}
> -
> -static void
> -format_CLONE(const struct ovnact_nest *nest, struct ds *s)
> -{
> -    format_nested_action(nest, "clone", s);
> -}
> -
> -static void
> -format_TRIGGER_EVENT(const struct ovnact_controller_event *event,
> -                     struct ds *s)
> -{
> -    ds_put_format(s, "trigger_event(event = \"%s\"",
> -                  event_to_string(event->event_type));
> -    for (const struct ovnact_gen_option *o = event->options;
> -         o < &event->options[event->n_options]; o++) {
> -        ds_put_cstr(s, ", ");
> -        ds_put_format(s, "%s = ", o->option->name);
> -        expr_constant_set_format(&o->value, s);
> -    }
> -    ds_put_cstr(s, ");");
> -}
> -
> -static void
> -encode_nested_actions(const struct ovnact_nest *on,
> -                      const struct ovnact_encode_params *ep,
> -                      enum action_opcode opcode,
> -                      struct ofpbuf *ofpacts)
> -{
> -    /* Convert nested actions into ofpacts. */
> -    uint64_t inner_ofpacts_stub[1024 / 8];
> -    struct ofpbuf inner_ofpacts =
> OFPBUF_STUB_INITIALIZER(inner_ofpacts_stub);
> -    ovnacts_encode(on->nested, on->nested_len, ep, &inner_ofpacts);
> -
> -    /* Add a "controller" action with the actions nested inside "{...}",
> -     * converted to OpenFlow, as its userdata.  ovn-controller will
> convert the
> -     * packet to ARP or NA and then send the packet and actions back to
> the
> -     * switch inside an OFPT_PACKET_OUT message. */
> -    size_t oc_offset = encode_start_controller_op(opcode, false,
> -                                                  NX_CTLR_NO_METER,
> ofpacts);
> -    ofpacts_put_openflow_actions(inner_ofpacts.data, inner_ofpacts.size,
> -                                 ofpacts, OFP13_VERSION);
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -
> -    /* Free memory. */
> -    ofpbuf_uninit(&inner_ofpacts);
> -}
> -
> -static void
> -encode_ARP(const struct ovnact_nest *on,
> -           const struct ovnact_encode_params *ep,
> -           struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ARP, ofpacts);
> -}
> -
> -static void
> -encode_ICMP4(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ICMP, ofpacts);
> -}
> -
> -static void
> -encode_ICMP4_ERROR(const struct ovnact_nest *on,
> -                   const struct ovnact_encode_params *ep,
> -                   struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ICMP4_ERROR, ofpacts);
> -}
> -
> -static void
> -encode_ICMP6(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ICMP, ofpacts);
> -}
> -
> -static void
> -encode_IGMP(const struct ovnact_null *a OVS_UNUSED,
> -            const struct ovnact_encode_params *ep OVS_UNUSED,
> -            struct ofpbuf *ofpacts)
> -{
> -    encode_controller_op(ACTION_OPCODE_IGMP, ofpacts);
> -}
> -
> -static void
> -encode_TCP_RESET(const struct ovnact_nest *on,
> -                 const struct ovnact_encode_params *ep,
> -                 struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_TCP_RESET, ofpacts);
> -}
> -
> -static void
> -encode_ND_NA(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ND_NA, ofpacts);
> -}
> -
> -static void
> -encode_ND_NA_ROUTER(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ND_NA_ROUTER, ofpacts);
> -}
> -
> -static void
> -encode_ND_NS(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    encode_nested_actions(on, ep, ACTION_OPCODE_ND_NS, ofpacts);
> -}
> -
> -static void
> -encode_CLONE(const struct ovnact_nest *on,
> -             const struct ovnact_encode_params *ep,
> -             struct ofpbuf *ofpacts)
> -{
> -    size_t ofs = ofpacts->size;
> -    ofpact_put_CLONE(ofpacts);
> -    ovnacts_encode(on->nested, on->nested_len, ep, ofpacts);
> -
> -    struct ofpact_nest *clone = ofpbuf_at_assert(ofpacts, ofs, sizeof
> *clone);
> -    ofpacts->header = clone;
> -    ofpact_finish_CLONE(ofpacts, &clone);
> -}
> -
> -static void
> -encode_event_empty_lb_backends_opts(struct ofpbuf *ofpacts,
> -        const struct ovnact_controller_event *event)
> -{
> -    for (const struct ovnact_gen_option *o = event->options;
> -         o < &event->options[event->n_options]; o++) {
> -        struct controller_event_opt_header *hdr =
> -            ofpbuf_put_uninit(ofpacts, sizeof *hdr);
> -        const union expr_constant *c = o->value.values;
> -        size_t size;
> -        hdr->opt_code = htons(o->option->code);
> -        if (!strcmp(o->option->type, "str")) {
> -            size = strlen(c->string);
> -            hdr->size = htons(size);
> -            ofpbuf_put(ofpacts, c->string, size);
> -        } else {
> -            /* All empty_lb_backends fields are of type 'str' */
> -            OVS_NOT_REACHED();
> -        }
> -    }
> -}
> -
> -static void
> -encode_TRIGGER_EVENT(const struct ovnact_controller_event *event,
> -                     const struct ovnact_encode_params *ep OVS_UNUSED,
> -                     struct ofpbuf *ofpacts)
> -{
> -    size_t oc_offset;
> -
> -    oc_offset = encode_start_controller_op(ACTION_OPCODE_EVENT, false,
> -                                           NX_CTLR_NO_METER, ofpacts);
> -    ovs_be32 ofs = htonl(event->event_type);
> -    ofpbuf_put(ofpacts, &ofs, sizeof ofs);
> -
> -    switch (event->event_type) {
> -    case OVN_EVENT_EMPTY_LB_BACKENDS:
> -        encode_event_empty_lb_backends_opts(ofpacts, event);
> -        break;
> -    case OVN_EVENT_MAX:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -static void
> -ovnact_nest_free(struct ovnact_nest *on)
> -{
> -    ovnacts_free(on->nested, on->nested_len);
> -    free(on->nested);
> -}
> -
> -static void
> -parse_get_mac_bind(struct action_context *ctx, int width,
> -                   struct ovnact_get_mac_bind *get_mac)
> -{
> -    lexer_force_match(ctx->lexer, LEX_T_LPAREN);
> -    action_parse_field(ctx, 0, false, &get_mac->port);
> -    lexer_force_match(ctx->lexer, LEX_T_COMMA);
> -    action_parse_field(ctx, width, false, &get_mac->ip);
> -    lexer_force_match(ctx->lexer, LEX_T_RPAREN);
> -}
> -
> -static void
> -format_get_mac_bind(const struct ovnact_get_mac_bind *get_mac,
> -                    const char *name, struct ds *s)
> -{
> -    ds_put_format(s, "%s(", name);
> -    expr_field_format(&get_mac->port, s);
> -    ds_put_cstr(s, ", ");
> -    expr_field_format(&get_mac->ip, s);
> -    ds_put_cstr(s, ");");
> -}
> -
> -static void
> -format_GET_ARP(const struct ovnact_get_mac_bind *get_mac, struct ds *s)
> -{
> -    format_get_mac_bind(get_mac, "get_arp", s);
> -}
> -
> -static void
> -format_GET_ND(const struct ovnact_get_mac_bind *get_mac, struct ds *s)
> -{
> -    format_get_mac_bind(get_mac, "get_nd", s);
> -}
> -
> -static void
> -encode_get_mac(const struct ovnact_get_mac_bind *get_mac,
> -               enum mf_field_id ip_field,
> -               const struct ovnact_encode_params *ep,
> -               struct ofpbuf *ofpacts)
> -{
> -    const struct arg args[] = {
> -        { expr_resolve_field(&get_mac->port), MFF_LOG_OUTPORT },
> -        { expr_resolve_field(&get_mac->ip), ip_field },
> -    };
> -    encode_setup_args(args, ARRAY_SIZE(args), ofpacts);
> -
> -    put_load(0, MFF_ETH_DST, 0, 48, ofpacts);
> -    emit_resubmit(ofpacts, ep->mac_bind_ptable);
> -
> -    encode_restore_args(args, ARRAY_SIZE(args), ofpacts);
> -}
> -
> -static void
> -encode_GET_ARP(const struct ovnact_get_mac_bind *get_mac,
> -               const struct ovnact_encode_params *ep,
> -               struct ofpbuf *ofpacts)
> -{
> -    encode_get_mac(get_mac, MFF_REG0, ep, ofpacts);
> -}
> -
> -static void
> -encode_GET_ND(const struct ovnact_get_mac_bind *get_mac,
> -              const struct ovnact_encode_params *ep,
> -              struct ofpbuf *ofpacts)
> -{
> -    encode_get_mac(get_mac, MFF_XXREG0, ep, ofpacts);
> -}
> -
> -static void
> -ovnact_get_mac_bind_free(struct ovnact_get_mac_bind *get_mac OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_put_mac_bind(struct action_context *ctx, int width,
> -                   struct ovnact_put_mac_bind *put_mac)
> -{
> -    lexer_force_match(ctx->lexer, LEX_T_LPAREN);
> -    action_parse_field(ctx, 0, false, &put_mac->port);
> -    lexer_force_match(ctx->lexer, LEX_T_COMMA);
> -    action_parse_field(ctx, width, false, &put_mac->ip);
> -    lexer_force_match(ctx->lexer, LEX_T_COMMA);
> -    action_parse_field(ctx, 48, false, &put_mac->mac);
> -    lexer_force_match(ctx->lexer, LEX_T_RPAREN);
> -}
> -
> -static void
> -format_put_mac_bind(const struct ovnact_put_mac_bind *put_mac,
> -                    const char *name, struct ds *s)
> -{
> -    ds_put_format(s, "%s(", name);
> -    expr_field_format(&put_mac->port, s);
> -    ds_put_cstr(s, ", ");
> -    expr_field_format(&put_mac->ip, s);
> -    ds_put_cstr(s, ", ");
> -    expr_field_format(&put_mac->mac, s);
> -    ds_put_cstr(s, ");");
> -}
> -
> -static void
> -format_PUT_ARP(const struct ovnact_put_mac_bind *put_mac, struct ds *s)
> -{
> -    format_put_mac_bind(put_mac, "put_arp", s);
> -}
> -
> -static void
> -format_PUT_ND(const struct ovnact_put_mac_bind *put_mac, struct ds *s)
> -{
> -    format_put_mac_bind(put_mac, "put_nd", s);
> -}
> -
> -static void
> -encode_put_mac(const struct ovnact_put_mac_bind *put_mac,
> -               enum mf_field_id ip_field, enum action_opcode opcode,
> -               struct ofpbuf *ofpacts)
> -{
> -    const struct arg args[] = {
> -        { expr_resolve_field(&put_mac->port), MFF_LOG_INPORT },
> -        { expr_resolve_field(&put_mac->ip), ip_field },
> -        { expr_resolve_field(&put_mac->mac), MFF_ETH_SRC }
> -    };
> -    encode_setup_args(args, ARRAY_SIZE(args), ofpacts);
> -    encode_controller_op(opcode, ofpacts);
> -    encode_restore_args(args, ARRAY_SIZE(args), ofpacts);
> -}
> -
> -static void
> -encode_PUT_ARP(const struct ovnact_put_mac_bind *put_mac,
> -               const struct ovnact_encode_params *ep OVS_UNUSED,
> -               struct ofpbuf *ofpacts)
> -{
> -    encode_put_mac(put_mac, MFF_REG0, ACTION_OPCODE_PUT_ARP, ofpacts);
> -}
> -
> -static void
> -encode_PUT_ND(const struct ovnact_put_mac_bind *put_mac,
> -              const struct ovnact_encode_params *ep OVS_UNUSED,
> -              struct ofpbuf *ofpacts)
> -{
> -    encode_put_mac(put_mac, MFF_XXREG0, ACTION_OPCODE_PUT_ND, ofpacts);
> -}
> -
> -static void
> -ovnact_put_mac_bind_free(struct ovnact_put_mac_bind *put_mac OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_gen_opt(struct action_context *ctx, struct ovnact_gen_option *o,
> -              const struct hmap *gen_opts, const char *opts_type)
> -{
> -    if (ctx->lexer->token.type != LEX_T_ID) {
> -        lexer_syntax_error(ctx->lexer, NULL);
> -        return;
> -    }
> -
> -    o->option = gen_opts ? gen_opts_find(gen_opts, ctx->lexer->token.s) :
> NULL;
> -    if (!o->option) {
> -        lexer_syntax_error(ctx->lexer, "expecting %s option name",
> opts_type);
> -        return;
> -    }
> -    lexer_get(ctx->lexer);
> -
> -    if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -        return;
> -    }
> -
> -    if (!expr_constant_set_parse(ctx->lexer, &o->value)) {
> -        memset(&o->value, 0, sizeof o->value);
> -        return;
> -    }
> -
> -    if (!strcmp(o->option->type, "str")) {
> -        if (o->value.type != EXPR_C_STRING) {
> -            lexer_error(ctx->lexer, "%s option %s requires string value.",
> -                        opts_type, o->option->name);
> -            return;
> -        }
> -    } else {
> -        if (o->value.type != EXPR_C_INTEGER) {
> -            lexer_error(ctx->lexer, "%s option %s requires numeric
> value.",
> -                        opts_type, o->option->name);
> -            return;
> -        }
> -    }
> -}
> -
> -static const struct ovnact_gen_option *
> -find_offerip(const struct ovnact_gen_option *options, size_t n)
> -{
> -    for (const struct ovnact_gen_option *o = options; o < &options[n];
> o++) {
> -        if (o->option->code == 0) {
> -            return o;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -free_gen_options(struct ovnact_gen_option *options, size_t n)
> -{
> -    for (struct ovnact_gen_option *o = options; o < &options[n]; o++) {
> -        expr_constant_set_destroy(&o->value);
> -    }
> -    free(options);
> -}
> -
> -static void
> -validate_empty_lb_backends(struct action_context *ctx,
> -                           const struct ovnact_gen_option *options,
> -                           size_t n_options)
> -{
> -    for (const struct ovnact_gen_option *o = options;
> -         o < &options[n_options]; o++) {
> -        const union expr_constant *c = o->value.values;
> -        struct sockaddr_storage ss;
> -        struct uuid uuid;
> -
> -        if (o->value.n_values > 1 || !c->string) {
> -            lexer_error(ctx->lexer, "Invalid value for \"%s\" option",
> -                        o->option->name);
> -            return;
> -        }
> -
> -        switch (o->option->code) {
> -        case EMPTY_LB_VIP:
> -            if (!inet_parse_active(c->string, 0, &ss, false)) {
> -                lexer_error(ctx->lexer, "Invalid load balancer VIP '%s'",
> -                            c->string);
> -                return;
> -            }
> -            break;
> -        case EMPTY_LB_PROTOCOL:
> -            if (strcmp(c->string, "tcp") && strcmp(c->string, "udp")) {
> -                lexer_error(ctx->lexer,
> -                    "Load balancer protocol '%s' is not 'tcp' or 'udp'",
> -                    c->string);
> -                return;
> -            }
> -            break;
> -        case EMPTY_LB_LOAD_BALANCER:
> -            if (!uuid_from_string(&uuid, c->string)) {
> -                lexer_error(ctx->lexer, "Load balancer '%s' is not a
> UUID",
> -                            c->string);
> -                return;
> -            }
> -            break;
> -        }
> -    }
> -}
> -
> -static void
> -parse_trigger_event(struct action_context *ctx,
> -                    struct ovnact_controller_event *event)
> -{
> -    int event_type = 0;
> -
> -    lexer_force_match(ctx->lexer, LEX_T_LPAREN);
> -
> -    /* Event type must be listed first */
> -    if (!lexer_match_id(ctx->lexer, "event")) {
> -        lexer_syntax_error(ctx->lexer, "Expecting 'event' option");
> -        return;
> -    }
> -    if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -        return;
> -    }
> -
> -    if (ctx->lexer->token.type != LEX_T_STRING ||
> -        strlen(ctx->lexer->token.s) >= 64) {
> -        lexer_syntax_error(ctx->lexer, "Expecting string");
> -        return;
> -    }
> -
> -    event_type = string_to_event(ctx->lexer->token.s);
> -    if (event_type < 0 || event_type >= OVN_EVENT_MAX) {
> -        lexer_syntax_error(ctx->lexer, "Unknown event '%d'", event_type);
> -        return;
> -    }
> -
> -    event->event_type = event_type;
> -    lexer_get(ctx->lexer);
> -
> -    lexer_match(ctx->lexer, LEX_T_COMMA);
> -
> -    size_t allocated_options = 0;
> -    while (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -        if (event->n_options >= allocated_options) {
> -            event->options = x2nrealloc(event->options,
> &allocated_options,
> -                                     sizeof *event->options);
> -        }
> -
> -        struct ovnact_gen_option *o = &event->options[event->n_options++];
> -        memset(o, 0, sizeof *o);
> -        parse_gen_opt(ctx, o,
> -
> &ctx->pp->controller_event_opts->event_opts[event_type],
> -                      event_to_string(event_type));
> -        if (ctx->lexer->error) {
> -            return;
> -        }
> -
> -        lexer_match(ctx->lexer, LEX_T_COMMA);
> -    }
> -
> -    switch (event_type) {
> -    case OVN_EVENT_EMPTY_LB_BACKENDS:
> -        validate_empty_lb_backends(ctx, event->options, event->n_options);
> -        break;
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -ovnact_controller_event_free(struct ovnact_controller_event *event
> OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_put_opts(struct action_context *ctx, const struct expr_field *dst,
> -               struct ovnact_put_opts *po, const struct hmap *gen_opts,
> -               const char *opts_type)
> -{
> -    lexer_get(ctx->lexer); /* Skip put_dhcp[v6]_opts / put_nd_ra_opts. */
> -    lexer_get(ctx->lexer); /* Skip '('. */
> -
> -    /* Validate that the destination is a 1-bit, modifiable field. */
> -    char *error = expr_type_check(dst, 1, true);
> -    if (error) {
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return;
> -    }
> -    po->dst = *dst;
> -
> -    size_t allocated_options = 0;
> -    while (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -        if (po->n_options >= allocated_options) {
> -            po->options = x2nrealloc(po->options, &allocated_options,
> -                                     sizeof *po->options);
> -        }
> -
> -        struct ovnact_gen_option *o = &po->options[po->n_options++];
> -        memset(o, 0, sizeof *o);
> -        parse_gen_opt(ctx, o, gen_opts, opts_type);
> -        if (ctx->lexer->error) {
> -            return;
> -        }
> -
> -        lexer_match(ctx->lexer, LEX_T_COMMA);
> -    }
> -}
> -
> -/* Parses the "put_dhcp_opts" and "put_dhcpv6_opts" actions.
> - *
> - * The caller has already consumed "<dst> =", so this just parses the
> rest. */
> -static void
> -parse_put_dhcp_opts(struct action_context *ctx,
> -                    const struct expr_field *dst,
> -                    struct ovnact_put_opts *po)
> -{
> -    const struct hmap *dhcp_opts =
> -        (po->ovnact.type == OVNACT_PUT_DHCPV6_OPTS) ?
> -            ctx->pp->dhcpv6_opts : ctx->pp->dhcp_opts;
> -    const char *opts_type =
> -        (po->ovnact.type == OVNACT_PUT_DHCPV6_OPTS) ? "DHCPv6" : "DHCPv4";
> -
> -    parse_put_opts(ctx, dst, po, dhcp_opts, opts_type);
> -
> -    if (!ctx->lexer->error && po->ovnact.type == OVNACT_PUT_DHCPV4_OPTS
> -        && !find_offerip(po->options, po->n_options)) {
> -        lexer_error(ctx->lexer,
> -                    "put_dhcp_opts requires offerip to be specified.");
> -        return;
> -    }
> -}
> -
> -static void
> -format_put_opts(const char *name, const struct ovnact_put_opts *pdo,
> -                struct ds *s)
> -{
> -    expr_field_format(&pdo->dst, s);
> -    ds_put_format(s, " = %s(", name);
> -    for (const struct ovnact_gen_option *o = pdo->options;
> -         o < &pdo->options[pdo->n_options]; o++) {
> -        if (o != pdo->options) {
> -            ds_put_cstr(s, ", ");
> -        }
> -        ds_put_format(s, "%s = ", o->option->name);
> -        expr_constant_set_format(&o->value, s);
> -    }
> -    ds_put_cstr(s, ");");
> -}
> -
> -static void
> -format_PUT_DHCPV4_OPTS(const struct ovnact_put_opts *pdo, struct ds *s)
> -{
> -    format_put_opts("put_dhcp_opts", pdo, s);
> -}
> -
> -static void
> -format_PUT_DHCPV6_OPTS(const struct ovnact_put_opts *pdo, struct ds *s)
> -{
> -    format_put_opts("put_dhcpv6_opts", pdo, s);
> -}
> -
> -static void
> -encode_put_dhcpv4_option(const struct ovnact_gen_option *o,
> -                         struct ofpbuf *ofpacts)
> -{
> -    uint8_t *opt_header = ofpbuf_put_zeros(ofpacts, 2);
> -    opt_header[0] = o->option->code;
> -
> -    const union expr_constant *c = o->value.values;
> -    size_t n_values = o->value.n_values;
> -    if (!strcmp(o->option->type, "bool") ||
> -        !strcmp(o->option->type, "uint8")) {
> -        opt_header[1] = 1;
> -        ofpbuf_put(ofpacts, &c->value.u8_val, 1);
> -    } else if (!strcmp(o->option->type, "uint16")) {
> -        opt_header[1] = 2;
> -        ofpbuf_put(ofpacts, &c->value.be16_int, 2);
> -    } else if (!strcmp(o->option->type, "uint32")) {
> -        opt_header[1] = 4;
> -        ofpbuf_put(ofpacts, &c->value.be32_int, 4);
> -    } else if (!strcmp(o->option->type, "ipv4")) {
> -        opt_header[1] = n_values * sizeof(ovs_be32);
> -        for (size_t i = 0; i < n_values; i++) {
> -            ofpbuf_put(ofpacts, &c[i].value.ipv4, sizeof(ovs_be32));
> -        }
> -    } else if (!strcmp(o->option->type, "static_routes")) {
> -        size_t no_of_routes = n_values;
> -        if (no_of_routes % 2) {
> -            no_of_routes -= 1;
> -        }
> -        opt_header[1] = 0;
> -
> -        /* Calculating the length of this option first because when
> -         * we call ofpbuf_put, it might reallocate the buffer if the
> -         * tail room is short making "opt_header" pointer invalid.
> -         * So running the for loop twice.
> -         */
> -        for (size_t i = 0; i < no_of_routes; i += 2) {
> -            uint8_t plen = 32;
> -            if (c[i].masked) {
> -                plen = (uint8_t) ip_count_cidr_bits(c[i].mask.ipv4);
> -            }
> -            opt_header[1] += (1 + DIV_ROUND_UP(plen, 8) +
> sizeof(ovs_be32));
> -        }
> -
> -        /* Copied from RFC 3442. Please refer to this RFC for the format
> of
> -         * the classless static route option.
> -         *
> -         *  The following table contains some examples of how various
> subnet
> -         *  number/mask combinations can be encoded:
> -         *
> -         *  Subnet number   Subnet mask      Destination descriptor
> -         *  0               0                0
> -         *  10.0.0.0        255.0.0.0        8.10
> -         *  10.0.0.0        255.255.255.0    24.10.0.0
> -         *  10.17.0.0       255.255.0.0      16.10.17
> -         *  10.27.129.0     255.255.255.0    24.10.27.129
> -         *  10.229.0.128    255.255.255.128  25.10.229.0.128
> -         *  10.198.122.47   255.255.255.255  32.10.198.122.47
> -         */
> -
> -        for (size_t i = 0; i < no_of_routes; i += 2) {
> -            uint8_t plen = 32;
> -            if (c[i].masked) {
> -                plen = ip_count_cidr_bits(c[i].mask.ipv4);
> -            }
> -            ofpbuf_put(ofpacts, &plen, 1);
> -            ofpbuf_put(ofpacts, &c[i].value.ipv4, DIV_ROUND_UP(plen, 8));
> -            ofpbuf_put(ofpacts, &c[i + 1].value.ipv4,
> -                       sizeof(ovs_be32));
> -        }
> -    } else if (!strcmp(o->option->type, "str")) {
> -        opt_header[1] = strlen(c->string);
> -        ofpbuf_put(ofpacts, c->string, opt_header[1]);
> -    }
> -}
> -
> -static void
> -encode_put_dhcpv6_option(const struct ovnact_gen_option *o,
> -                         struct ofpbuf *ofpacts)
> -{
> -    struct dhcp_opt6_header *opt = ofpbuf_put_uninit(ofpacts, sizeof
> *opt);
> -    const union expr_constant *c = o->value.values;
> -    size_t n_values = o->value.n_values;
> -    size_t size;
> -
> -    opt->opt_code = htons(o->option->code);
> -
> -    if (!strcmp(o->option->type, "ipv6")) {
> -        size = n_values * sizeof(struct in6_addr);
> -        opt->size = htons(size);
> -        for (size_t i = 0; i < n_values; i++) {
> -            ofpbuf_put(ofpacts, &c[i].value.ipv6, sizeof(struct
> in6_addr));
> -        }
> -    } else if (!strcmp(o->option->type, "mac")) {
> -        size = sizeof(struct eth_addr);
> -        opt->size = htons(size);
> -        ofpbuf_put(ofpacts, &c->value.mac, size);
> -    } else if (!strcmp(o->option->type, "str")) {
> -        size = strlen(c->string);
> -        opt->size = htons(size);
> -        ofpbuf_put(ofpacts, c->string, size);
> -    }
> -}
> -
> -static void
> -encode_PUT_DHCPV4_OPTS(const struct ovnact_put_opts *pdo,
> -                       const struct ovnact_encode_params *ep OVS_UNUSED,
> -                       struct ofpbuf *ofpacts)
> -{
> -    struct mf_subfield dst = expr_resolve_field(&pdo->dst);
> -
> -    size_t oc_offset =
> encode_start_controller_op(ACTION_OPCODE_PUT_DHCP_OPTS,
> -                                                  true, NX_CTLR_NO_METER,
> -                                                  ofpacts);
> -    nx_put_header(ofpacts, dst.field->id, OFP13_VERSION, false);
> -    ovs_be32 ofs = htonl(dst.ofs);
> -    ofpbuf_put(ofpacts, &ofs, sizeof ofs);
> -
> -    /* Encode the offerip option first, because it's a special case and
> needs
> -     * to be first in the actual DHCP response, and then encode the rest
> -     * (skipping offerip the second time around). */
> -    const struct ovnact_gen_option *offerip_opt = find_offerip(
> -        pdo->options, pdo->n_options);
> -    ovs_be32 offerip = offerip_opt->value.values[0].value.ipv4;
> -    ofpbuf_put(ofpacts, &offerip, sizeof offerip);
> -
> -    for (const struct ovnact_gen_option *o = pdo->options;
> -         o < &pdo->options[pdo->n_options]; o++) {
> -        if (o != offerip_opt) {
> -            encode_put_dhcpv4_option(o, ofpacts);
> -        }
> -    }
> -
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -static void
> -encode_PUT_DHCPV6_OPTS(const struct ovnact_put_opts *pdo,
> -                       const struct ovnact_encode_params *ep OVS_UNUSED,
> -                       struct ofpbuf *ofpacts)
> -{
> -    struct mf_subfield dst = expr_resolve_field(&pdo->dst);
> -
> -    size_t oc_offset = encode_start_controller_op(
> -        ACTION_OPCODE_PUT_DHCPV6_OPTS, true, NX_CTLR_NO_METER, ofpacts);
> -    nx_put_header(ofpacts, dst.field->id, OFP13_VERSION, false);
> -    ovs_be32 ofs = htonl(dst.ofs);
> -    ofpbuf_put(ofpacts, &ofs, sizeof ofs);
> -
> -    for (const struct ovnact_gen_option *o = pdo->options;
> -         o < &pdo->options[pdo->n_options]; o++) {
> -        encode_put_dhcpv6_option(o, ofpacts);
> -    }
> -
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -static void
> -ovnact_put_opts_free(struct ovnact_put_opts *pdo)
> -{
> -    free_gen_options(pdo->options, pdo->n_options);
> -}
> -
> -static void
> -parse_SET_QUEUE(struct action_context *ctx)
> -{
> -    int queue_id;
> -
> -    if (!lexer_force_match(ctx->lexer, LEX_T_LPAREN)
> -        || !lexer_get_int(ctx->lexer, &queue_id)
> -        || !lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -        return;
> -    }
> -
> -    if (queue_id < QDISC_MIN_QUEUE_ID || queue_id > QDISC_MAX_QUEUE_ID) {
> -        lexer_error(ctx->lexer, "Queue ID %d for set_queue is "
> -                    "not in valid range %d to %d.",
> -                    queue_id, QDISC_MIN_QUEUE_ID, QDISC_MAX_QUEUE_ID);
> -        return;
> -    }
> -
> -    ovnact_put_SET_QUEUE(ctx->ovnacts)->queue_id = queue_id;
> -}
> -
> -static void
> -format_SET_QUEUE(const struct ovnact_set_queue *set_queue, struct ds *s)
> -{
> -    ds_put_format(s, "set_queue(%d);", set_queue->queue_id);
> -}
> -
> -static void
> -encode_SET_QUEUE(const struct ovnact_set_queue *set_queue,
> -                 const struct ovnact_encode_params *ep OVS_UNUSED,
> -                 struct ofpbuf *ofpacts)
> -{
> -    ofpact_put_SET_QUEUE(ofpacts)->queue_id = set_queue->queue_id;
> -}
> -
> -static void
> -ovnact_set_queue_free(struct ovnact_set_queue *a OVS_UNUSED)
> -{
> -}
> -
> -static void
> -parse_dns_lookup(struct action_context *ctx, const struct expr_field *dst,
> -                 struct ovnact_dns_lookup *dl)
> -{
> -    lexer_get(ctx->lexer); /* Skip dns_lookup. */
> -    lexer_get(ctx->lexer); /* Skip '('. */
> -    if (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -        lexer_error(ctx->lexer, "dns_lookup doesn't take any parameters");
> -        return;
> -    }
> -    /* Validate that the destination is a 1-bit, modifiable field. */
> -    char *error = expr_type_check(dst, 1, true);
> -    if (error) {
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return;
> -    }
> -    dl->dst = *dst;
> -    add_prerequisite(ctx, "udp");
> -}
> -
> -static void
> -format_DNS_LOOKUP(const struct ovnact_dns_lookup *dl, struct ds *s)
> -{
> -    expr_field_format(&dl->dst, s);
> -    ds_put_cstr(s, " = dns_lookup();");
> -}
> -
> -static void
> -encode_DNS_LOOKUP(const struct ovnact_dns_lookup *dl,
> -                  const struct ovnact_encode_params *ep OVS_UNUSED,
> -                  struct ofpbuf *ofpacts)
> -{
> -    struct mf_subfield dst = expr_resolve_field(&dl->dst);
> -
> -    size_t oc_offset =
> encode_start_controller_op(ACTION_OPCODE_DNS_LOOKUP,
> -                                                  true, NX_CTLR_NO_METER,
> -                                                  ofpacts);
> -    nx_put_header(ofpacts, dst.field->id, OFP13_VERSION, false);
> -    ovs_be32 ofs = htonl(dst.ofs);
> -    ofpbuf_put(ofpacts, &ofs, sizeof ofs);
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -
> -static void
> -ovnact_dns_lookup_free(struct ovnact_dns_lookup *dl OVS_UNUSED)
> -{
> -}
> -
> -/* Parses the "put_nd_ra_opts" action.
> - * The caller has already consumed "<dst> =", so this just parses the
> rest. */
> -static void
> -parse_put_nd_ra_opts(struct action_context *ctx, const struct expr_field
> *dst,
> -                     struct ovnact_put_opts *po)
> -{
> -    parse_put_opts(ctx, dst, po, ctx->pp->nd_ra_opts, "IPv6 ND RA");
> -
> -    if (ctx->lexer->error) {
> -        return;
> -    }
> -
> -    bool addr_mode_stateful = false;
> -    bool prefix_set = false;
> -    bool slla_present = false;
> -    /* Let's validate the options. */
> -    for (struct ovnact_gen_option *o = po->options;
> -            o < &po->options[po->n_options]; o++) {
> -        const union expr_constant *c = o->value.values;
> -        if (o->value.n_values > 1) {
> -            lexer_error(ctx->lexer, "Invalid value for \"%s\" option",
> -                        o->option->name);
> -            return;
> -        }
> -
> -        bool ok = true;
> -        switch (o->option->code) {
> -        case ND_RA_FLAG_ADDR_MODE:
> -            ok = (c->string && (!strcmp(c->string, "slaac") ||
> -                                !strcmp(c->string, "dhcpv6_stateful") ||
> -                                !strcmp(c->string, "dhcpv6_stateless")));
> -            if (ok && !strcmp(c->string, "dhcpv6_stateful")) {
> -                addr_mode_stateful = true;
> -            }
> -            break;
> -
> -        case ND_OPT_SOURCE_LINKADDR:
> -            ok = c->format == LEX_F_ETHERNET;
> -            slla_present = true;
> -            break;
> -
> -        case ND_OPT_PREFIX_INFORMATION:
> -            ok = c->format == LEX_F_IPV6 && c->masked;
> -            prefix_set = true;
> -            break;
> -
> -        case ND_OPT_MTU:
> -            ok = c->format == LEX_F_DECIMAL;
> -            break;
> -        }
> -
> -        if (!ok) {
> -            lexer_error(ctx->lexer, "Invalid value for \"%s\" option",
> -                        o->option->name);
> -            return;
> -        }
> -    }
> -
> -    if (!slla_present) {
> -        lexer_error(ctx->lexer, "slla option not present");
> -        return;
> -    }
> -
> -    if (!addr_mode_stateful && !prefix_set) {
> -        lexer_error(ctx->lexer, "prefix option needs "
> -                    "to be set when address mode is
> slaac/dhcpv6_stateless.");
> -        return;
> -    }
> -
> -    add_prerequisite(ctx, "ip6");
> -}
> -
> -static void
> -format_PUT_ND_RA_OPTS(const struct ovnact_put_opts *po,
> -                      struct ds *s)
> -{
> -    format_put_opts("put_nd_ra_opts", po, s);
> -}
> -
> -static void
> -encode_put_nd_ra_option(const struct ovnact_gen_option *o,
> -                        struct ofpbuf *ofpacts, ptrdiff_t ra_offset)
> -{
> -    const union expr_constant *c = o->value.values;
> -
> -    switch (o->option->code) {
> -    case ND_RA_FLAG_ADDR_MODE:
> -    {
> -        struct ovs_ra_msg *ra = ofpbuf_at(ofpacts, ra_offset, sizeof *ra);
> -        if (!strcmp(c->string, "dhcpv6_stateful")) {
> -            ra->mo_flags = IPV6_ND_RA_FLAG_MANAGED_ADDR_CONFIG;
> -        } else if (!strcmp(c->string, "dhcpv6_stateless")) {
> -            ra->mo_flags = IPV6_ND_RA_FLAG_OTHER_ADDR_CONFIG;
> -        }
> -        break;
> -    }
> -
> -    case ND_OPT_SOURCE_LINKADDR:
> -    {
> -        struct ovs_nd_lla_opt *lla_opt =
> -            ofpbuf_put_uninit(ofpacts, sizeof *lla_opt);
> -        lla_opt->type = ND_OPT_SOURCE_LINKADDR;
> -        lla_opt->len = 1;
> -        lla_opt->mac = c->value.mac;
> -        break;
> -    }
> -
> -    case ND_OPT_MTU:
> -    {
> -        struct ovs_nd_mtu_opt *mtu_opt =
> -            ofpbuf_put_uninit(ofpacts, sizeof *mtu_opt);
> -        mtu_opt->type = ND_OPT_MTU;
> -        mtu_opt->len = 1;
> -        mtu_opt->reserved = 0;
> -        put_16aligned_be32(&mtu_opt->mtu, c->value.be32_int);
> -        break;
> -    }
> -
> -    case ND_OPT_PREFIX_INFORMATION:
> -    {
> -        struct ovs_nd_prefix_opt *prefix_opt =
> -            ofpbuf_put_uninit(ofpacts, sizeof *prefix_opt);
> -        uint8_t prefix_len = ipv6_count_cidr_bits(&c->mask.ipv6);
> -        struct ovs_ra_msg *ra = ofpbuf_at(ofpacts, ra_offset, sizeof *ra);
> -        prefix_opt->type = ND_OPT_PREFIX_INFORMATION;
> -        prefix_opt->len = 4;
> -        prefix_opt->prefix_len = prefix_len;
> -        prefix_opt->la_flags = IPV6_ND_RA_OPT_PREFIX_ON_LINK;
> -        if (!(ra->mo_flags & IPV6_ND_RA_FLAG_MANAGED_ADDR_CONFIG)) {
> -            prefix_opt->la_flags |= IPV6_ND_RA_OPT_PREFIX_AUTONOMOUS;
> -        }
> -        put_16aligned_be32(&prefix_opt->valid_lifetime,
> -                           htonl(IPV6_ND_RA_OPT_PREFIX_VALID_LIFETIME));
> -        put_16aligned_be32(&prefix_opt->preferred_lifetime,
> -
>  htonl(IPV6_ND_RA_OPT_PREFIX_PREFERRED_LIFETIME));
> -        put_16aligned_be32(&prefix_opt->reserved, 0);
> -        memcpy(prefix_opt->prefix.be32, &c->value.be128[7].be32,
> -               sizeof(ovs_be32[4]));
> -        break;
> -    }
> -    }
> -}
> -
> -static void
> -encode_PUT_ND_RA_OPTS(const struct ovnact_put_opts *po,
> -                      const struct ovnact_encode_params *ep OVS_UNUSED,
> -                      struct ofpbuf *ofpacts)
> -{
> -    struct mf_subfield dst = expr_resolve_field(&po->dst);
> -
> -    size_t oc_offset = encode_start_controller_op(
> -        ACTION_OPCODE_PUT_ND_RA_OPTS, true, NX_CTLR_NO_METER, ofpacts);
> -    nx_put_header(ofpacts, dst.field->id, OFP13_VERSION, false);
> -    ovs_be32 ofs = htonl(dst.ofs);
> -    ofpbuf_put(ofpacts, &ofs, sizeof ofs);
> -
> -    /* Frame the complete ICMPv6 Router Advertisement data encoding
> -     * the ND RA options in it, in the userdata field, so that when
> -     * pinctrl module receives the ICMPv6 Router Solicitation packet
> -     * it can copy the userdata field AS IS and resume the packet.
> -     */
> -    size_t ra_offset = ofpacts->size;
> -    struct ovs_ra_msg *ra = ofpbuf_put_zeros(ofpacts, sizeof *ra);
> -    ra->icmph.icmp6_type = ND_ROUTER_ADVERT;
> -    ra->cur_hop_limit = IPV6_ND_RA_CUR_HOP_LIMIT;
> -    ra->mo_flags = 0;
> -    ra->router_lifetime = htons(IPV6_ND_RA_LIFETIME);
> -
> -    for (const struct ovnact_gen_option *o = po->options;
> -         o < &po->options[po->n_options]; o++) {
> -        encode_put_nd_ra_option(o, ofpacts, ra_offset);
> -    }
> -
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -
> -static void
> -parse_log_arg(struct action_context *ctx, struct ovnact_log *log)
> -{
> -    if (lexer_match_id(ctx->lexer, "verdict")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        if (lexer_match_id(ctx->lexer, "drop")) {
> -            log->verdict = LOG_VERDICT_DROP;
> -        } else if (lexer_match_id(ctx->lexer, "reject")) {
> -            log->verdict = LOG_VERDICT_REJECT;
> -        } else if (lexer_match_id(ctx->lexer, "allow")) {
> -            log->verdict = LOG_VERDICT_ALLOW;
> -        } else {
> -            lexer_syntax_error(ctx->lexer, "unknown verdict");
> -            return;
> -        }
> -    } else if (lexer_match_id(ctx->lexer, "name")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        /* If multiple names are given, use the most recent. */
> -        if (ctx->lexer->token.type == LEX_T_STRING) {
> -            /* Arbitrarily limit the name length to 64 bytes, since
> -             * these will be encoded in datapath actions. */
> -            if (strlen(ctx->lexer->token.s) >= 64) {
> -                lexer_syntax_error(ctx->lexer, "name must be shorter "
> -                                               "than 64 characters");
> -                return;
> -            }
> -            free(log->name);
> -            log->name = xstrdup(ctx->lexer->token.s);
> -        } else {
> -            lexer_syntax_error(ctx->lexer, "expecting string");
> -            return;
> -        }
> -        lexer_get(ctx->lexer);
> -    } else if (lexer_match_id(ctx->lexer, "severity")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        if (ctx->lexer->token.type == LEX_T_ID) {
> -            uint8_t severity =
> log_severity_from_string(ctx->lexer->token.s);
> -            if (severity != UINT8_MAX) {
> -                log->severity = severity;
> -                lexer_get(ctx->lexer);
> -                return;
> -            } else {
> -                lexer_syntax_error(ctx->lexer, "unknown severity");
> -                return;
> -            }
> -        }
> -        lexer_syntax_error(ctx->lexer, "expecting severity");
> -    } else if (lexer_match_id(ctx->lexer, "meter")) {
> -        if (!lexer_force_match(ctx->lexer, LEX_T_EQUALS)) {
> -            return;
> -        }
> -        /* If multiple meters are given, use the most recent. */
> -        if (ctx->lexer->token.type == LEX_T_STRING) {
> -            free(log->meter);
> -            log->meter = xstrdup(ctx->lexer->token.s);
> -        } else {
> -            lexer_syntax_error(ctx->lexer, "expecting string");
> -            return;
> -        }
> -        lexer_get(ctx->lexer);
> -    } else {
> -        lexer_syntax_error(ctx->lexer, NULL);
> -    }
> -}
> -
> -static void
> -parse_LOG(struct action_context *ctx)
> -{
> -    struct ovnact_log *log = ovnact_put_LOG(ctx->ovnacts);
> -
> -    /* Provide default values. */
> -    log->severity = LOG_SEVERITY_INFO;
> -    log->verdict = LOG_VERDICT_UNKNOWN;
> -
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        while (!lexer_match(ctx->lexer, LEX_T_RPAREN)) {
> -            parse_log_arg(ctx, log);
> -            if (ctx->lexer->error) {
> -                return;
> -            }
> -            lexer_match(ctx->lexer, LEX_T_COMMA);
> -        }
> -    }
> -    if (log->verdict == LOG_VERDICT_UNKNOWN) {
> -        lexer_syntax_error(ctx->lexer, "expecting verdict");
> -    }
> -}
> -
> -static void
> -format_LOG(const struct ovnact_log *log, struct ds *s)
> -{
> -    ds_put_cstr(s, "log(");
> -
> -    if (log->name) {
> -        ds_put_format(s, "name=\"%s\", ", log->name);
> -    }
> -
> -    ds_put_format(s, "verdict=%s, ", log_verdict_to_string(log->verdict));
> -    ds_put_format(s, "severity=%s",
> log_severity_to_string(log->severity));
> -
> -    if (log->meter) {
> -        ds_put_format(s, ", meter=\"%s\"", log->meter);
> -    }
> -
> -    ds_put_cstr(s, ");");
> -}
> -
> -static void
> -encode_LOG(const struct ovnact_log *log,
> -           const struct ovnact_encode_params *ep, struct ofpbuf *ofpacts)
> -{
> -    uint32_t meter_id = NX_CTLR_NO_METER;
> -
> -    if (log->meter) {
> -        meter_id = ovn_extend_table_assign_id(ep->meter_table, log->meter,
> -                                              ep->lflow_uuid);
> -        if (meter_id == EXT_TABLE_ID_INVALID) {
> -            VLOG_WARN("Unable to assign id for log meter: %s",
> log->meter);
> -            return;
> -        }
> -    }
> -
> -    size_t oc_offset = encode_start_controller_op(ACTION_OPCODE_LOG,
> false,
> -                                                  meter_id, ofpacts);
> -
> -    struct log_pin_header *lph = ofpbuf_put_uninit(ofpacts, sizeof *lph);
> -    lph->verdict = log->verdict;
> -    lph->severity = log->severity;
> -
> -    if (log->name) {
> -        int name_len = strlen(log->name);
> -        ofpbuf_put(ofpacts, log->name, name_len);
> -    }
> -
> -    encode_finish_controller_op(oc_offset, ofpacts);
> -}
> -
> -static void
> -ovnact_log_free(struct ovnact_log *log)
> -{
> -    free(log->name);
> -    free(log->meter);
> -}
> -
> -static void
> -parse_set_meter_action(struct action_context *ctx)
> -{
> -    uint64_t rate = 0;
> -    uint64_t burst = 0;
> -
> -    lexer_force_match(ctx->lexer, LEX_T_LPAREN); /* Skip '('. */
> -    if (ctx->lexer->token.type == LEX_T_INTEGER
> -        && ctx->lexer->token.format == LEX_F_DECIMAL) {
> -        rate = ntohll(ctx->lexer->token.value.integer);
> -    }
> -    lexer_get(ctx->lexer);
> -    if (lexer_match(ctx->lexer, LEX_T_COMMA)) {  /* Skip ','. */
> -        if (ctx->lexer->token.type == LEX_T_INTEGER
> -            && ctx->lexer->token.format == LEX_F_DECIMAL) {
> -            burst = ntohll(ctx->lexer->token.value.integer);
> -        }
> -        lexer_get(ctx->lexer);
> -    }
> -    lexer_force_match(ctx->lexer, LEX_T_RPAREN); /* Skip ')'. */
> -
> -    if (!rate) {
> -        lexer_error(ctx->lexer,
> -                    "Rate %"PRId64" for set_meter is not in valid.",
> -                    rate);
> -        return;
> -    }
> -
> -    struct ovnact_set_meter *cl = ovnact_put_SET_METER(ctx->ovnacts);
> -    cl->rate = rate;
> -    cl->burst = burst;
> -}
> -
> -static void
> -format_SET_METER(const struct ovnact_set_meter *cl, struct ds *s)
> -{
> -    if (cl->burst) {
> -        ds_put_format(s, "set_meter(%"PRId64", %"PRId64");",
> -                      cl->rate, cl->burst);
> -    } else {
> -        ds_put_format(s, "set_meter(%"PRId64");", cl->rate);
> -    }
> -}
> -
> -static void
> -encode_SET_METER(const struct ovnact_set_meter *cl,
> -                 const struct ovnact_encode_params *ep,
> -                 struct ofpbuf *ofpacts)
> -{
> -    uint32_t table_id;
> -    struct ofpact_meter *om;
> -
> -    /* Use the special "__string:" prefix to indicate that the name
> -     * describes the meter itself. */
> -    char *name;
> -    if (cl->burst) {
> -        name = xasprintf("__string: kbps burst stats bands=type=drop "
> -                         "rate=%"PRId64" burst_size=%"PRId64"", cl->rate,
> -                         cl->burst);
> -    } else {
> -        name = xasprintf("__string: kbps stats bands=type=drop "
> -                         "rate=%"PRId64"", cl->rate);
> -    }
> -
> -    table_id = ovn_extend_table_assign_id(ep->meter_table, name,
> -                                          ep->lflow_uuid);
> -    free(name);
> -    if (table_id == EXT_TABLE_ID_INVALID) {
> -        return;
> -    }
> -
> -    /* Create an action to set the meter. */
> -    om = ofpact_put_METER(ofpacts);
> -    om->meter_id = table_id;
> -}
> -
> -static void
> -ovnact_set_meter_free(struct ovnact_set_meter *ct OVS_UNUSED)
> -{
> -}
> -
> -static void
> -format_OVNFIELD_LOAD(const struct ovnact_load *load , struct ds *s)
> -{
> -    const struct ovn_field *f =
> ovn_field_from_name(load->dst.symbol->name);
> -    switch (f->id) {
> -    case OVN_ICMP4_FRAG_MTU:
> -        ds_put_format(s, "%s = %u;", f->name,
> -                      ntohs(load->imm.value.be16_int));
> -        break;
> -
> -    case OVN_FIELD_N_IDS:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -encode_OVNFIELD_LOAD(const struct ovnact_load *load,
> -            const struct ovnact_encode_params *ep OVS_UNUSED,
> -            struct ofpbuf *ofpacts)
> -{
> -    const struct ovn_field *f =
> ovn_field_from_name(load->dst.symbol->name);
> -    switch (f->id) {
> -    case OVN_ICMP4_FRAG_MTU: {
> -        size_t oc_offset = encode_start_controller_op(
> -            ACTION_OPCODE_PUT_ICMP4_FRAG_MTU, true, NX_CTLR_NO_METER,
> -            ofpacts);
> -        ofpbuf_put(ofpacts, &load->imm.value.be16_int, sizeof(ovs_be16));
> -        encode_finish_controller_op(oc_offset, ofpacts);
> -        break;
> -    }
> -    case OVN_FIELD_N_IDS:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -parse_check_pkt_larger(struct action_context *ctx,
> -                       const struct expr_field *dst,
> -                       struct ovnact_check_pkt_larger *cipl)
> -{
> -     /* Validate that the destination is a 1-bit, modifiable field. */
> -    char *error = expr_type_check(dst, 1, true);
> -    if (error) {
> -        lexer_error(ctx->lexer, "%s", error);
> -        free(error);
> -        return;
> -    }
> -
> -    int pkt_len;
> -    lexer_get(ctx->lexer); /* Skip check_pkt_len. */
> -    if (!lexer_force_match(ctx->lexer, LEX_T_LPAREN)
> -        || !lexer_get_int(ctx->lexer, &pkt_len)
> -        || !lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -        return;
> -    }
> -
> -    cipl->dst = *dst;
> -    cipl->pkt_len = pkt_len;
> -}
> -
> -static void
> -format_CHECK_PKT_LARGER(const struct ovnact_check_pkt_larger *cipl,
> -                        struct ds *s)
> -{
> -    expr_field_format(&cipl->dst, s);
> -    ds_put_format(s, " = check_pkt_larger(%d);", cipl->pkt_len);
> -}
> -
> -static void
> -encode_CHECK_PKT_LARGER(const struct ovnact_check_pkt_larger *cipl,
> -                        const struct ovnact_encode_params *ep OVS_UNUSED,
> -                        struct ofpbuf *ofpacts)
> -{
> -    struct ofpact_check_pkt_larger *check_pkt_larger =
> -        ofpact_put_CHECK_PKT_LARGER(ofpacts);
> -    check_pkt_larger->pkt_len = cipl->pkt_len;
> -    check_pkt_larger->dst = expr_resolve_field(&cipl->dst);
> -}
> -
> -static void
> -ovnact_check_pkt_larger_free(struct ovnact_check_pkt_larger *cipl
> OVS_UNUSED)
> -{
> -}
> -
> -/* Parses an assignment or exchange or put_dhcp_opts action. */
> -static void
> -parse_set_action(struct action_context *ctx)
> -{
> -    struct expr_field lhs;
> -    if (!expr_field_parse(ctx->lexer, ctx->pp->symtab, &lhs,
> &ctx->prereqs)) {
> -        return;
> -    }
> -
> -    if (lexer_match(ctx->lexer, LEX_T_EXCHANGE)) {
> -        parse_assignment_action(ctx, true, &lhs);
> -    } else if (lexer_match(ctx->lexer, LEX_T_EQUALS)) {
> -        if (ctx->lexer->token.type != LEX_T_ID) {
> -            parse_LOAD(ctx, &lhs);
> -        } else if (!strcmp(ctx->lexer->token.s, "put_dhcp_opts")
> -                   && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            parse_put_dhcp_opts(ctx, &lhs, ovnact_put_PUT_DHCPV4_OPTS(
> -                                    ctx->ovnacts));
> -        } else if (!strcmp(ctx->lexer->token.s, "put_dhcpv6_opts")
> -                   && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            parse_put_dhcp_opts(ctx, &lhs, ovnact_put_PUT_DHCPV6_OPTS(
> -                                    ctx->ovnacts));
> -        } else if (!strcmp(ctx->lexer->token.s, "dns_lookup")
> -                   && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            parse_dns_lookup(ctx, &lhs,
> ovnact_put_DNS_LOOKUP(ctx->ovnacts));
> -        } else if (!strcmp(ctx->lexer->token.s, "put_nd_ra_opts")
> -                && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            parse_put_nd_ra_opts(ctx, &lhs,
> -                                 ovnact_put_PUT_ND_RA_OPTS(ctx->ovnacts));
> -        } else if (!strcmp(ctx->lexer->token.s, "check_pkt_larger")
> -                && lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            parse_check_pkt_larger(ctx, &lhs,
> -
>  ovnact_put_CHECK_PKT_LARGER(ctx->ovnacts));
> -        } else {
> -            parse_assignment_action(ctx, false, &lhs);
> -        }
> -    } else {
> -        lexer_syntax_error(ctx->lexer, "expecting `=' or `<->'");
> -    }
> -}
> -
> -static bool
> -parse_action(struct action_context *ctx)
> -{
> -    if (ctx->lexer->token.type != LEX_T_ID) {
> -        lexer_syntax_error(ctx->lexer, NULL);
> -        return false;
> -    }
> -
> -    enum lex_type lookahead = lexer_lookahead(ctx->lexer);
> -    if (lookahead == LEX_T_EQUALS || lookahead == LEX_T_EXCHANGE
> -        || lookahead == LEX_T_LSQUARE) {
> -        parse_set_action(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "next")) {
> -        parse_NEXT(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "output")) {
> -        ovnact_put_OUTPUT(ctx->ovnacts);
> -    } else if (lexer_match_id(ctx->lexer, "ip.ttl")) {
> -        parse_DEC_TTL(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_next")) {
> -        parse_CT_NEXT(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_commit")) {
> -        parse_CT_COMMIT(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_dnat")) {
> -        parse_CT_DNAT(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_snat")) {
> -        parse_CT_SNAT(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_lb")) {
> -        parse_ct_lb_action(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "ct_clear")) {
> -        ovnact_put_CT_CLEAR(ctx->ovnacts);
> -    } else if (lexer_match_id(ctx->lexer, "clone")) {
> -        parse_CLONE(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "arp")) {
> -        parse_ARP(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "icmp4")) {
> -        parse_ICMP4(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "icmp4_error")) {
> -        parse_ICMP4_ERROR(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "icmp6")) {
> -        parse_ICMP6(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "igmp")) {
> -        ovnact_put_IGMP(ctx->ovnacts);
> -    } else if (lexer_match_id(ctx->lexer, "tcp_reset")) {
> -        parse_TCP_RESET(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "nd_na")) {
> -        parse_ND_NA(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "nd_na_router")) {
> -        parse_ND_NA_ROUTER(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "nd_ns")) {
> -        parse_ND_NS(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "get_arp")) {
> -        parse_get_mac_bind(ctx, 32, ovnact_put_GET_ARP(ctx->ovnacts));
> -    } else if (lexer_match_id(ctx->lexer, "put_arp")) {
> -        parse_put_mac_bind(ctx, 32, ovnact_put_PUT_ARP(ctx->ovnacts));
> -    } else if (lexer_match_id(ctx->lexer, "get_nd")) {
> -        parse_get_mac_bind(ctx, 128, ovnact_put_GET_ND(ctx->ovnacts));
> -    } else if (lexer_match_id(ctx->lexer, "put_nd")) {
> -        parse_put_mac_bind(ctx, 128, ovnact_put_PUT_ND(ctx->ovnacts));
> -    } else if (lexer_match_id(ctx->lexer, "set_queue")) {
> -        parse_SET_QUEUE(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "log")) {
> -        parse_LOG(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "set_meter")) {
> -        parse_set_meter_action(ctx);
> -    } else if (lexer_match_id(ctx->lexer, "trigger_event")) {
> -        parse_trigger_event(ctx, ovnact_put_TRIGGER_EVENT(ctx->ovnacts));
> -    } else {
> -        lexer_syntax_error(ctx->lexer, "expecting action");
> -    }
> -    lexer_force_match(ctx->lexer, LEX_T_SEMICOLON);
> -    return !ctx->lexer->error;
> -}
> -
> -static void
> -parse_actions(struct action_context *ctx, enum lex_type sentinel)
> -{
> -    /* "drop;" by itself is a valid (empty) set of actions, but it can't
> be
> -     * combined with other actions because that doesn't make sense. */
> -    if (ctx->lexer->token.type == LEX_T_ID
> -        && !strcmp(ctx->lexer->token.s, "drop")
> -        && lexer_lookahead(ctx->lexer) == LEX_T_SEMICOLON) {
> -        lexer_get(ctx->lexer);  /* Skip "drop". */
> -        lexer_get(ctx->lexer);  /* Skip ";". */
> -        lexer_force_match(ctx->lexer, sentinel);
> -        return;
> -    }
> -
> -    while (!lexer_match(ctx->lexer, sentinel)) {
> -        if (!parse_action(ctx)) {
> -            return;
> -        }
> -    }
> -}
> -
> -/* Parses OVN actions, in the format described for the "actions" column
> in the
> - * Logical_Flow table in ovn-sb(5), and appends the parsed versions of the
> - * actions to 'ovnacts' as "struct ovnact"s.  The caller must eventually
> free
> - * the parsed ovnacts with ovnacts_free().
> - *
> - * 'pp' provides most of the parameters for translation.
> - *
> - * Some actions add extra requirements (prerequisites) to the flow's
> match.  If
> - * so, this function sets '*prereqsp' to the actions' prerequisites;
> otherwise,
> - * it sets '*prereqsp' to NULL.  The caller owns '*prereqsp' and must
> - * eventually free it.
> - *
> - * Returns true if successful, false if an error occurred.  Upon return,
> - * returns true if and only if lexer->error is NULL.
> - */
> -bool
> -ovnacts_parse(struct lexer *lexer, const struct ovnact_parse_params *pp,
> -              struct ofpbuf *ovnacts, struct expr **prereqsp)
> -{
> -    size_t ovnacts_start = ovnacts->size;
> -
> -    struct action_context ctx = {
> -        .pp = pp,
> -        .lexer = lexer,
> -        .ovnacts = ovnacts,
> -        .prereqs = NULL,
> -    };
> -    if (!lexer->error) {
> -        parse_actions(&ctx, LEX_T_END);
> -    }
> -
> -    if (!lexer->error) {
> -        *prereqsp = ctx.prereqs;
> -        return true;
> -    } else {
> -        ofpbuf_pull(ovnacts, ovnacts_start);
> -        ovnacts_free(ovnacts->data, ovnacts->size);
> -        ofpbuf_push_uninit(ovnacts, ovnacts_start);
> -
> -        ovnacts->size = ovnacts_start;
> -        expr_destroy(ctx.prereqs);
> -        *prereqsp = NULL;
> -        return false;
> -    }
> -}
> -
> -/* Like ovnacts_parse(), but the actions are taken from 's'. */
> -char * OVS_WARN_UNUSED_RESULT
> -ovnacts_parse_string(const char *s, const struct ovnact_parse_params *pp,
> -                     struct ofpbuf *ofpacts, struct expr **prereqsp)
> -{
> -    struct lexer lexer;
> -
> -    lexer_init(&lexer, s);
> -    lexer_get(&lexer);
> -    ovnacts_parse(&lexer, pp, ofpacts, prereqsp);
> -    char *error = lexer_steal_error(&lexer);
> -    lexer_destroy(&lexer);
> -
> -    return error;
> -}
> -
> -/* Formatting ovnacts. */
> -
> -static void
> -ovnact_format(const struct ovnact *a, struct ds *s)
> -{
> -    switch (a->type) {
> -#define OVNACT(ENUM, STRUCT)                                            \
> -        case OVNACT_##ENUM:                                             \
> -            format_##ENUM(ALIGNED_CAST(const struct STRUCT *, a), s);   \
> -            break;
> -        OVNACTS
> -#undef OVNACT
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Appends a string representing the 'ovnacts_len' bytes of ovnacts in
> - * 'ovnacts' to 'string'. */
> -void
> -ovnacts_format(const struct ovnact *ovnacts, size_t ovnacts_len,
> -               struct ds *string)
> -{
> -    if (!ovnacts_len) {
> -        ds_put_cstr(string, "drop;");
> -    } else {
> -        const struct ovnact *a;
> -
> -        OVNACT_FOR_EACH (a, ovnacts, ovnacts_len) {
> -            if (a != ovnacts) {
> -                ds_put_char(string, ' ');
> -            }
> -            ovnact_format(a, string);
> -        }
> -    }
> -}
> -
> -/* Encoding ovnacts to OpenFlow. */
> -
> -static void
> -ovnact_encode(const struct ovnact *a, const struct ovnact_encode_params
> *ep,
> -              struct ofpbuf *ofpacts)
> -{
> -    switch (a->type) {
> -#define OVNACT(ENUM, STRUCT)                                            \
> -        case OVNACT_##ENUM:                                             \
> -            encode_##ENUM(ALIGNED_CAST(const struct STRUCT *, a),       \
> -                          ep, ofpacts);                                 \
> -            break;
> -        OVNACTS
> -#undef OVNACT
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Appends ofpacts to 'ofpacts' that represent the actions in the
> 'ovnacts_len'
> - * bytes of actions starting at 'ovnacts'. */
> -void
> -ovnacts_encode(const struct ovnact *ovnacts, size_t ovnacts_len,
> -               const struct ovnact_encode_params *ep,
> -               struct ofpbuf *ofpacts)
> -{
> -    if (ovnacts) {
> -        const struct ovnact *a;
> -
> -        OVNACT_FOR_EACH (a, ovnacts, ovnacts_len) {
> -            ovnact_encode(a, ep, ofpacts);
> -        }
> -    }
> -}
> -
> -/* Freeing ovnacts. */
> -
> -static void
> -ovnact_free(struct ovnact *a)
> -{
> -    switch (a->type) {
> -#define OVNACT(ENUM, STRUCT)                                            \
> -        case OVNACT_##ENUM:                                             \
> -            STRUCT##_free(ALIGNED_CAST(struct STRUCT *, a));            \
> -            break;
> -        OVNACTS
> -#undef OVNACT
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Frees each of the actions in the 'ovnacts_len' bytes of actions
> starting at
> - * 'ovnacts'.
> - *
> - * Does not call free(ovnacts); the caller must do so if desirable. */
> -void
> -ovnacts_free(struct ovnact *ovnacts, size_t ovnacts_len)
> -{
> -    if (ovnacts) {
> -        struct ovnact *a;
> -
> -        OVNACT_FOR_EACH (a, ovnacts, ovnacts_len) {
> -            ovnact_free(a);
> -        }
> -    }
> -}
> diff --git a/ovn/lib/automake.mk b/ovn/lib/automake.mk
> deleted file mode 100644
> index 7cac67fb6..000000000
> --- a/ovn/lib/automake.mk
> +++ /dev/null
> @@ -1,57 +0,0 @@
> -lib_LTLIBRARIES += ovn/lib/libovn.la
> -ovn_lib_libovn_la_LDFLAGS = \
> -        $(OVS_LTINFO) \
> -        -Wl,--version-script=$(top_builddir)/ovn/lib/libovn.sym \
> -        $(AM_LDFLAGS)
> -ovn_lib_libovn_la_SOURCES = \
> -       ovn/lib/acl-log.c \
> -       ovn/lib/acl-log.h \
> -       ovn/lib/actions.c \
> -       ovn/lib/chassis-index.c \
> -       ovn/lib/chassis-index.h \
> -       ovn/lib/expr.c \
> -       ovn/lib/extend-table.h \
> -       ovn/lib/extend-table.c \
> -       ovn/lib/ip-mcast-index.c \
> -       ovn/lib/ip-mcast-index.h \
> -       ovn/lib/mcast-group-index.c \
> -       ovn/lib/mcast-group-index.h \
> -       ovn/lib/lex.c \
> -       ovn/lib/ovn-l7.h \
> -       ovn/lib/ovn-util.c \
> -       ovn/lib/ovn-util.h \
> -       ovn/lib/logical-fields.c \
> -       ovn/lib/inc-proc-eng.c \
> -       ovn/lib/inc-proc-eng.h
> -nodist_ovn_lib_libovn_la_SOURCES = \
> -       ovn/lib/ovn-nb-idl.c \
> -       ovn/lib/ovn-nb-idl.h \
> -       ovn/lib/ovn-sb-idl.c \
> -       ovn/lib/ovn-sb-idl.h
> -
> -# ovn-sb IDL
> -OVSIDL_BUILT += \
> -       ovn/lib/ovn-sb-idl.c \
> -       ovn/lib/ovn-sb-idl.h \
> -       ovn/lib/ovn-sb-idl.ovsidl
> -EXTRA_DIST += ovn/lib/ovn-sb-idl.ann
> -OVN_SB_IDL_FILES = \
> -       $(srcdir)/ovn/ovn-sb.ovsschema \
> -       $(srcdir)/ovn/lib/ovn-sb-idl.ann
> -ovn/lib/ovn-sb-idl.ovsidl: $(OVN_SB_IDL_FILES)
> -       $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_SB_IDL_FILES) > $@.tmp && \
> -       mv $@.tmp $@
> -
> -# ovn-nb IDL
> -OVSIDL_BUILT += \
> -       ovn/lib/ovn-nb-idl.c \
> -       ovn/lib/ovn-nb-idl.h \
> -       ovn/lib/ovn-nb-idl.ovsidl
> -EXTRA_DIST += ovn/lib/ovn-nb-idl.ann
> -OVN_NB_IDL_FILES = \
> -       $(srcdir)/ovn/ovn-nb.ovsschema \
> -       $(srcdir)/ovn/lib/ovn-nb-idl.ann
> -ovn/lib/ovn-nb-idl.ovsidl: $(OVN_NB_IDL_FILES)
> -       $(AM_V_GEN)$(OVSDB_IDLC) annotate $(OVN_NB_IDL_FILES) > $@.tmp && \
> -       mv $@.tmp $@
> -
> diff --git a/ovn/lib/chassis-index.c b/ovn/lib/chassis-index.c
> deleted file mode 100644
> index 10f70fb4a..000000000
> --- a/ovn/lib/chassis-index.c
> +++ /dev/null
> @@ -1,67 +0,0 @@
> -/* Copyright (c) 2016, 2017 Red Hat, Inc.
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -struct ovsdb_idl_index *
> -chassis_index_create(struct ovsdb_idl *idl)
> -{
> -    return ovsdb_idl_index_create1(idl, &sbrec_chassis_col_name);
> -}
> -
> -/* Finds and returns the chassis with the given 'name', or NULL if no such
> - * chassis exists. */
> -const struct sbrec_chassis *
> -chassis_lookup_by_name(struct ovsdb_idl_index *sbrec_chassis_by_name,
> -                       const char *name)
> -{
> -    struct sbrec_chassis *target = sbrec_chassis_index_init_row(
> -        sbrec_chassis_by_name);
> -    sbrec_chassis_index_set_name(target, name);
> -
> -    struct sbrec_chassis *retval = sbrec_chassis_index_find(
> -        sbrec_chassis_by_name, target);
> -
> -    sbrec_chassis_index_destroy_row(target);
> -
> -    return retval;
> -}
> -
> -struct ovsdb_idl_index *
> -ha_chassis_group_index_create(struct ovsdb_idl *idl)
> -{
> -    return ovsdb_idl_index_create1(idl, &sbrec_ha_chassis_group_col_name);
> -}
> -
> -/* Finds and returns the HA chassis group with the given 'name', or NULL
> - * if no such HA chassis group exists. */
> -const struct sbrec_ha_chassis_group *
> -ha_chassis_group_lookup_by_name(
> -    struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name,
> -    const char *name)
> -{
> -    struct sbrec_ha_chassis_group *target =
> -
> sbrec_ha_chassis_group_index_init_row(sbrec_ha_chassis_grp_by_name);
> -    sbrec_ha_chassis_group_index_set_name(target, name);
> -
> -    struct sbrec_ha_chassis_group *retval =
> -        sbrec_ha_chassis_group_index_find(sbrec_ha_chassis_grp_by_name,
> -                                          target);
> -
> -    sbrec_ha_chassis_group_index_destroy_row(target);
> -
> -    return retval;
> -}
> diff --git a/ovn/lib/chassis-index.h b/ovn/lib/chassis-index.h
> deleted file mode 100644
> index 9bc610ad2..000000000
> --- a/ovn/lib/chassis-index.h
> +++ /dev/null
> @@ -1,30 +0,0 @@
> -/* Copyright (c) 2017, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_CHASSIS_INDEX_H
> -#define OVN_CHASSIS_INDEX_H 1
> -
> -struct ovsdb_idl;
> -
> -struct ovsdb_idl_index *chassis_index_create(struct ovsdb_idl *);
> -
> -const struct sbrec_chassis *chassis_lookup_by_name(
> -    struct ovsdb_idl_index *sbrec_chassis_by_name, const char *name);
> -
> -struct ovsdb_idl_index *ha_chassis_group_index_create(struct ovsdb_idl
> *idl);
> -const struct sbrec_ha_chassis_group *ha_chassis_group_lookup_by_name(
> -    struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name, const char
> *name);
> -
> -#endif /* ovn/lib/chassis-index.h */
> diff --git a/ovn/lib/expr.c b/ovn/lib/expr.c
> deleted file mode 100644
> index e4c650f7c..000000000
> --- a/ovn/lib/expr.c
> +++ /dev/null
> @@ -1,3450 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "byte-order.h"
> -#include "openvswitch/json.h"
> -#include "nx-match.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/vlog.h"
> -#include "openvswitch/shash.h"
> -#include "ovn/expr.h"
> -#include "ovn/lex.h"
> -#include "ovn/logical-fields.h"
> -#include "simap.h"
> -#include "sset.h"
> -#include "util.h"
> -
> -VLOG_DEFINE_THIS_MODULE(expr);
> -
> -static struct expr *parse_and_annotate(const char *s,
> -                                       const struct shash *symtab,
> -                                       struct ovs_list *nesting,
> -                                       char **errorp);
> -
> -/* Returns the name of measurement level 'level'. */
> -const char *
> -expr_level_to_string(enum expr_level level)
> -{
> -    switch (level) {
> -    case EXPR_L_NOMINAL: return "nominal";
> -    case EXPR_L_BOOLEAN: return "Boolean";
> -    case EXPR_L_ORDINAL: return "ordinal";
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Relational operators. */
> -
> -/* Returns a string form of relational operator 'relop'. */
> -const char *
> -expr_relop_to_string(enum expr_relop relop)
> -{
> -    switch (relop) {
> -    case EXPR_R_EQ: return "==";
> -    case EXPR_R_NE: return "!=";
> -    case EXPR_R_LT: return "<";
> -    case EXPR_R_LE: return "<=";
> -    case EXPR_R_GT: return ">";
> -    case EXPR_R_GE: return ">=";
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -bool
> -expr_relop_from_token(enum lex_type type, enum expr_relop *relop)
> -{
> -    enum expr_relop r;
> -
> -    switch ((int) type) {
> -    case LEX_T_EQ: r = EXPR_R_EQ; break;
> -    case LEX_T_NE: r = EXPR_R_NE; break;
> -    case LEX_T_LT: r = EXPR_R_LT; break;
> -    case LEX_T_LE: r = EXPR_R_LE; break;
> -    case LEX_T_GT: r = EXPR_R_GT; break;
> -    case LEX_T_GE: r = EXPR_R_GE; break;
> -    default: return false;
> -    }
> -
> -    if (relop) {
> -        *relop = r;
> -    }
> -    return true;
> -}
> -
> -/* Returns the relational operator that 'relop' becomes if you turn the
> - * relation's operands around, e.g. EXPR_R_EQ does not change because "a
> == b"
> - * and "b == a" are equivalent, but EXPR_R_LE becomes EXPR_R_GE because
> "a <=
> - * b" is equivalent to "b >= a". */
> -static enum expr_relop
> -expr_relop_turn(enum expr_relop relop)
> -{
> -    switch (relop) {
> -    case EXPR_R_EQ: return EXPR_R_EQ;
> -    case EXPR_R_NE: return EXPR_R_NE;
> -    case EXPR_R_LT: return EXPR_R_GT;
> -    case EXPR_R_LE: return EXPR_R_GE;
> -    case EXPR_R_GT: return EXPR_R_LT;
> -    case EXPR_R_GE: return EXPR_R_LE;
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Returns the relational operator that is the opposite of 'relop'. */
> -static enum expr_relop
> -expr_relop_invert(enum expr_relop relop)
> -{
> -    switch (relop) {
> -    case EXPR_R_EQ: return EXPR_R_NE;
> -    case EXPR_R_NE: return EXPR_R_EQ;
> -    case EXPR_R_LT: return EXPR_R_GE;
> -    case EXPR_R_LE: return EXPR_R_GT;
> -    case EXPR_R_GT: return EXPR_R_LE;
> -    case EXPR_R_GE: return EXPR_R_LT;
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Checks whether 'relop' is true for strcmp()-like 3-way comparison
> result
> - * 'cmp'. */
> -static bool
> -expr_relop_test(enum expr_relop relop, int cmp)
> -{
> -    switch (relop) {
> -    case EXPR_R_EQ: return cmp == 0;
> -    case EXPR_R_NE: return cmp != 0;
> -    case EXPR_R_LT: return cmp < 0;
> -    case EXPR_R_LE: return cmp <= 0;
> -    case EXPR_R_GT: return cmp > 0;
> -    case EXPR_R_GE: return cmp >= 0;
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Constructing and manipulating expressions. */
> -
> -/* Creates and returns a logical AND or OR expression (according to
> 'type',
> - * which must be EXPR_T_AND or EXPR_T_OR) that initially has no
> - * sub-expressions.  (To satisfy the invariants for expressions, the
> caller
> - * must add at least two sub-expressions whose types are different from
> - * 'type'.) */
> -struct expr *
> -expr_create_andor(enum expr_type type)
> -{
> -    struct expr *e = xmalloc(sizeof *e);
> -    e->type = type;
> -    ovs_list_init(&e->andor);
> -    return e;
> -}
> -
> -/* Returns a logical AND or OR expression (according to 'type', which
> must be
> - * EXPR_T_AND or EXPR_T_OR) whose sub-expressions are 'a' and 'b', with
> some
> - * flexibility:
> - *
> - *     - If 'a' or 'b' is NULL, just returns the other one (which means
> that if
> - *       that other one is not of the given 'type', then the returned
> - *       expression is not either).
> - *
> - *     - If 'a' or 'b', or both, have type 'type', then they are combined
> into
> - *       a single node that satisfies the invariants for expressions. */
> -struct expr *
> -expr_combine(enum expr_type type, struct expr *a, struct expr *b)
> -{
> -    if (!a) {
> -        return b;
> -    } else if (!b) {
> -        return a;
> -    } else if (a->type == type) {
> -        if (b->type == type) {
> -            ovs_list_splice(&a->andor, b->andor.next, &b->andor);
> -            free(b);
> -        } else {
> -            ovs_list_push_back(&a->andor, &b->node);
> -        }
> -        return a;
> -    } else if (b->type == type) {
> -        ovs_list_push_front(&b->andor, &a->node);
> -        return b;
> -    } else {
> -        struct expr *e = expr_create_andor(type);
> -        ovs_list_push_back(&e->andor, &a->node);
> -        ovs_list_push_back(&e->andor, &b->node);
> -        return e;
> -    }
> -}
> -
> -static void
> -expr_insert_andor(struct expr *andor, struct expr *before, struct expr
> *new)
> -{
> -    if (new->type == andor->type) {
> -        if (andor->type == EXPR_T_AND) {
> -            /* Conjunction junction, what's your function? */
> -        }
> -        ovs_list_splice(&before->node, new->andor.next, &new->andor);
> -        free(new);
> -    } else {
> -        ovs_list_insert(&before->node, &new->node);
> -    }
> -}
> -
> -/* Returns an EXPR_T_BOOLEAN expression with value 'b'. */
> -struct expr *
> -expr_create_boolean(bool b)
> -{
> -    struct expr *e = xmalloc(sizeof *e);
> -    e->type = EXPR_T_BOOLEAN;
> -    e->boolean = b;
> -    return e;
> -}
> -
> -static void
> -expr_not(struct expr *expr)
> -{
> -    struct expr *sub;
> -
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        expr->cmp.relop = expr_relop_invert(expr->cmp.relop);
> -        break;
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        LIST_FOR_EACH (sub, node, &expr->andor) {
> -            expr_not(sub);
> -        }
> -        expr->type = expr->type == EXPR_T_AND ? EXPR_T_OR : EXPR_T_AND;
> -        break;
> -
> -    case EXPR_T_BOOLEAN:
> -        expr->boolean = !expr->boolean;
> -        break;
> -
> -    case EXPR_T_CONDITION:
> -        expr->cond.not = !expr->cond.not;
> -        break;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static struct expr *
> -expr_fix_andor(struct expr *expr, bool short_circuit)
> -{
> -    struct expr *sub, *next;
> -
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        if (sub->type == EXPR_T_BOOLEAN) {
> -            if (sub->boolean == short_circuit) {
> -                expr_destroy(expr);
> -                return expr_create_boolean(short_circuit);
> -            } else {
> -                ovs_list_remove(&sub->node);
> -                expr_destroy(sub);
> -            }
> -        }
> -    }
> -
> -    if (ovs_list_is_short(&expr->andor)) {
> -        if (ovs_list_is_empty(&expr->andor)) {
> -            free(expr);
> -            return expr_create_boolean(!short_circuit);
> -        } else {
> -            sub = expr_from_node(ovs_list_front(&expr->andor));
> -            free(expr);
> -            return sub;
> -        }
> -    } else {
> -        return expr;
> -    }
> -}
> -
> -/* Returns 'expr' modified so that top-level oddities are fixed up:
> - *
> - *     - Eliminates any EXPR_T_BOOLEAN operands at the top level.
> - *
> - *     - Replaces one-operand EXPR_T_AND or EXPR_T_OR by its
> subexpression. */
> -static struct expr *
> -expr_fix(struct expr *expr)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return expr;
> -
> -    case EXPR_T_AND:
> -        return expr_fix_andor(expr, false);
> -
> -    case EXPR_T_OR:
> -        return expr_fix_andor(expr, true);
> -
> -    case EXPR_T_BOOLEAN:
> -        return expr;
> -
> -    case EXPR_T_CONDITION:
> -        return expr;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Formatting. */
> -
> -/* Searches bits [0,width) in 'sv' for a contiguous sequence of 1-bits.
> If one
> - * such sequence exists, stores the index of the first 1-bit into
> '*startp' and
> - * the number of 1-bits into '*n_bitsp'.  Stores 0 into both variables if
> no
> - * such sequence, or more than one, exists. */
> -static void
> -find_bitwise_range(const union mf_subvalue *sv, int width,
> -                   int *startp, int *n_bitsp)
> -{
> -    unsigned int start = bitwise_scan(sv, sizeof *sv, true, 0, width);
> -    if (start < width) {
> -        unsigned int end = bitwise_scan(sv, sizeof *sv, false, start,
> width);
> -        if (end >= width
> -            || bitwise_scan(sv, sizeof *sv, true, end, width) >= width) {
> -            *startp = start;
> -            *n_bitsp = end - start;
> -            return;
> -        }
> -    }
> -    *startp = *n_bitsp = 0;
> -}
> -
> -static void
> -expr_format_cmp(const struct expr *e, struct ds *s)
> -{
> -    /* The common case is numerical comparisons.
> -     * Handle string comparisons as a special case. */
> -    if (!e->cmp.symbol->width) {
> -        ds_put_format(s, "%s %s ", e->cmp.symbol->name,
> -                      expr_relop_to_string(e->cmp.relop));
> -        json_string_escape(e->cmp.string, s);
> -        return;
> -    }
> -
> -    int ofs, n;
> -    find_bitwise_range(&e->cmp.mask, e->cmp.symbol->width, &ofs, &n);
> -    if (n == 1 && (e->cmp.relop == EXPR_R_EQ || e->cmp.relop ==
> EXPR_R_NE)) {
> -        bool positive;
> -
> -        positive = bitwise_get_bit(&e->cmp.value, sizeof e->cmp.value,
> ofs);
> -        positive ^= e->cmp.relop == EXPR_R_NE;
> -        if (!positive) {
> -            ds_put_char(s, '!');
> -        }
> -        ds_put_cstr(s, e->cmp.symbol->name);
> -        if (e->cmp.symbol->width > 1) {
> -            ds_put_format(s, "[%d]", ofs);
> -        }
> -        return;
> -    }
> -
> -    ds_put_cstr(s, e->cmp.symbol->name);
> -    if (n > 0 && n < e->cmp.symbol->width) {
> -        if (n > 1) {
> -            ds_put_format(s, "[%d..%d]", ofs, ofs + n - 1);
> -        } else {
> -            ds_put_format(s, "[%d]", ofs);
> -        }
> -    }
> -
> -    ds_put_format(s, " %s ", expr_relop_to_string(e->cmp.relop));
> -
> -    if (n) {
> -        union mf_subvalue value;
> -
> -        memset(&value, 0, sizeof value);
> -        bitwise_copy(&e->cmp.value, sizeof e->cmp.value, ofs,
> -                     &value, sizeof value, 0,
> -                     n);
> -        mf_format_subvalue(&value, s);
> -    } else {
> -        mf_format_subvalue(&e->cmp.value, s);
> -        ds_put_char(s, '/');
> -        mf_format_subvalue(&e->cmp.mask, s);
> -    }
> -}
> -
> -static void
> -expr_format_andor(const struct expr *e, const char *op, struct ds *s)
> -{
> -    struct expr *sub;
> -    int i = 0;
> -
> -    LIST_FOR_EACH (sub, node, &e->andor) {
> -        if (i++) {
> -            ds_put_format(s, " %s ", op);
> -        }
> -
> -        if (sub->type == EXPR_T_AND || sub->type == EXPR_T_OR) {
> -            ds_put_char(s, '(');
> -            expr_format(sub, s);
> -            ds_put_char(s, ')');
> -        } else {
> -            expr_format(sub, s);
> -        }
> -    }
> -}
> -
> -static void
> -expr_format_condition(const struct expr *e, struct ds *s)
> -{
> -    if (e->cond.not) {
> -        ds_put_char(s, '!');
> -    }
> -    switch (e->cond.type) {
> -    case EXPR_COND_CHASSIS_RESIDENT:
> -        ds_put_format(s, "is_chassis_resident(");
> -        json_string_escape(e->cond.string, s);
> -        ds_put_char(s, ')');
> -        break;
> -    }
> -}
> -
> -/* Appends a string form of 'e' to 's'.  The string form is acceptable for
> - * parsing back into an equivalent expression. */
> -void
> -expr_format(const struct expr *e, struct ds *s)
> -{
> -    switch (e->type) {
> -    case EXPR_T_CMP:
> -        expr_format_cmp(e, s);
> -        break;
> -
> -    case EXPR_T_AND:
> -        expr_format_andor(e, "&&", s);
> -        break;
> -
> -    case EXPR_T_OR:
> -        expr_format_andor(e, "||", s);
> -        break;
> -
> -    case EXPR_T_BOOLEAN:
> -        ds_put_char(s, e->boolean ? '1' : '0');
> -        break;
> -
> -    case EXPR_T_CONDITION:
> -        expr_format_condition(e, s);
> -        break;
> -    }
> -}
> -
> -/* Prints a string form of 'e' on stdout, followed by a new-line. */
> -void
> -expr_print(const struct expr *e)
> -{
> -    struct ds output;
> -
> -    ds_init(&output);
> -    expr_format(e, &output);
> -    puts(ds_cstr(&output));
> -    ds_destroy(&output);
> -}
> -
> -/* Parsing. */
> -
> -#define MAX_PAREN_DEPTH 100
> -
> -/* Context maintained during expr_parse(). */
> -struct expr_context {
> -    struct lexer *lexer;           /* Lexer for pulling more tokens. */
> -    const struct shash *symtab;    /* Symbol table. */
> -    const struct shash *addr_sets; /* Address set table. */
> -    const struct shash *port_groups; /* Port group table. */
> -    struct sset *addr_sets_ref;       /* The set of address set
> referenced. */
> -    bool not;                    /* True inside odd number of NOT
> operators. */
> -    unsigned int paren_depth;    /* Depth of nested parentheses. */
> -};
> -
> -struct expr *expr_parse__(struct expr_context *);
> -static void expr_not(struct expr *);
> -static bool parse_field(struct expr_context *, struct expr_field *);
> -
> -static struct expr *
> -make_cmp__(const struct expr_field *f, enum expr_relop r,
> -             const union expr_constant *c)
> -{
> -    struct expr *e = xzalloc(sizeof *e);
> -    e->type = EXPR_T_CMP;
> -    e->cmp.symbol = f->symbol;
> -    e->cmp.relop = r;
> -    if (f->symbol->width) {
> -        bitwise_copy(&c->value, sizeof c->value, 0,
> -                     &e->cmp.value, sizeof e->cmp.value, f->ofs,
> -                     f->n_bits);
> -        if (c->masked) {
> -            bitwise_copy(&c->mask, sizeof c->mask, 0,
> -                         &e->cmp.mask, sizeof e->cmp.mask, f->ofs,
> -                         f->n_bits);
> -        } else {
> -            bitwise_one(&e->cmp.mask, sizeof e->cmp.mask, f->ofs,
> -                        f->n_bits);
> -        }
> -    } else {
> -        e->cmp.string = xstrdup(c->string);
> -    }
> -    return e;
> -}
> -
> -/* Returns the minimum reasonable width for integer constant 'c'. */
> -static int
> -expr_constant_width(const union expr_constant *c)
> -{
> -    if (c->masked) {
> -        return mf_subvalue_width(&c->mask);
> -    }
> -
> -    switch (c->format) {
> -    case LEX_F_DECIMAL:
> -    case LEX_F_HEXADECIMAL:
> -        return mf_subvalue_width(&c->value);
> -
> -    case LEX_F_IPV4:
> -        return 32;
> -
> -    case LEX_F_IPV6:
> -        return 128;
> -
> -    case LEX_F_ETHERNET:
> -        return 48;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static bool
> -type_check(struct expr_context *ctx, const struct expr_field *f,
> -           struct expr_constant_set *cs)
> -{
> -    if (cs->type != (f->symbol->width ? EXPR_C_INTEGER : EXPR_C_STRING)) {
> -        lexer_error(ctx->lexer,
> -                    "%s field %s is not compatible with %s constant.",
> -                    f->symbol->width ? "Integer" : "String",
> -                    f->symbol->name,
> -                    cs->type == EXPR_C_INTEGER ? "integer" : "string");
> -        return false;
> -    }
> -
> -    if (f->symbol->width) {
> -        for (size_t i = 0; i < cs->n_values; i++) {
> -            int w = expr_constant_width(&cs->values[i]);
> -            if (w > f->symbol->width) {
> -                lexer_error(ctx->lexer,
> -                            "%d-bit constant is not compatible with
> %d-bit "
> -                            "field %s.", w, f->symbol->width,
> f->symbol->name);
> -                return false;
> -            }
> -        }
> -    }
> -
> -    return true;
> -}
> -
> -static struct expr *
> -make_cmp(struct expr_context *ctx,
> -         const struct expr_field *f, enum expr_relop r,
> -         struct expr_constant_set *cs)
> -{
> -    struct expr *e = NULL;
> -
> -    if (!type_check(ctx, f, cs)) {
> -        goto exit;
> -    }
> -
> -    if (r != EXPR_R_EQ && r != EXPR_R_NE) {
> -        if (cs->in_curlies) {
> -            lexer_error(ctx->lexer, "Only == and != operators may be used
> "
> -                        "with value sets.");
> -            goto exit;
> -        }
> -        if (f->symbol->level == EXPR_L_NOMINAL ||
> -            f->symbol->level == EXPR_L_BOOLEAN) {
> -            lexer_error(ctx->lexer, "Only == and != operators may be used
> "
> -                        "with %s field %s.",
> -                        expr_level_to_string(f->symbol->level),
> -                        f->symbol->name);
> -            goto exit;
> -        }
> -        if (!cs->n_values) {
> -            lexer_error(ctx->lexer, "Only == and != operators may be used
> "
> -                        "to compare a field against an empty value set.");
> -            goto exit;
> -        }
> -        if (cs->values[0].masked) {
> -            lexer_error(ctx->lexer, "Only == and != operators may be used
> "
> -                        "with masked constants.  Consider using subfields
> "
> -                        "instead (e.g. eth.src[0..15] > 0x1111 in place
> of "
> -                        "eth.src >
> 00:00:00:00:11:11/00:00:00:00:ff:ff).");
> -            goto exit;
> -        }
> -    }
> -
> -    if (f->symbol->level == EXPR_L_NOMINAL) {
> -        if (f->symbol->predicate) {
> -            ovs_assert(f->symbol->width > 0);
> -            for (size_t i = 0; i < cs->n_values; i++) {
> -                const union mf_subvalue *value = &cs->values[i].value;
> -                bool positive = (value->integer & htonll(1)) != 0;
> -                positive ^= r == EXPR_R_NE;
> -                positive ^= ctx->not;
> -                if (!positive) {
> -                    const char *name = f->symbol->name;
> -                    lexer_error(ctx->lexer,
> -                                "Nominal predicate %s may only be tested "
> -                                "positively, e.g. `%s' or `%s == 1' but
> not "
> -                                "`!%s' or `%s == 0'.",
> -                                name, name, name, name, name);
> -                    goto exit;
> -                }
> -            }
> -        } else if (r != (ctx->not ? EXPR_R_NE : EXPR_R_EQ)) {
> -            lexer_error(ctx->lexer, "Nominal field %s may only be tested
> for "
> -                        "equality (taking enclosing `!' operators into "
> -                        "account).", f->symbol->name);
> -            goto exit;
> -        }
> -    }
> -
> -    if (!cs->n_values) {
> -        e = expr_create_boolean(r == EXPR_R_NE);
> -        goto exit;
> -    }
> -    e = make_cmp__(f, r, &cs->values[0]);
> -    for (size_t i = 1; i < cs->n_values; i++) {
> -        e = expr_combine(r == EXPR_R_EQ ? EXPR_T_OR : EXPR_T_AND,
> -                         e, make_cmp__(f, r, &cs->values[i]));
> -    }
> -exit:
> -    expr_constant_set_destroy(cs);
> -    return e;
> -}
> -
> -static bool
> -parse_field(struct expr_context *ctx, struct expr_field *f)
> -{
> -    const struct expr_symbol *symbol;
> -
> -    if (ctx->lexer->token.type != LEX_T_ID) {
> -        lexer_syntax_error(ctx->lexer, "expecting field name");
> -        return false;
> -    }
> -
> -    symbol = shash_find_data(ctx->symtab, ctx->lexer->token.s);
> -    if (!symbol) {
> -        lexer_syntax_error(ctx->lexer, "expecting field name");
> -        return false;
> -    }
> -    lexer_get(ctx->lexer);
> -
> -    f->symbol = symbol;
> -    if (lexer_match(ctx->lexer, LEX_T_LSQUARE)) {
> -        int low, high;
> -
> -        if (!symbol->width) {
> -            lexer_error(ctx->lexer,
> -                        "Cannot select subfield of string field %s.",
> -                        symbol->name);
> -            return false;
> -        }
> -
> -        if (!lexer_force_int(ctx->lexer, &low)) {
> -            return false;
> -        }
> -        if (lexer_match(ctx->lexer, LEX_T_ELLIPSIS)) {
> -            if (!lexer_force_int(ctx->lexer, &high)) {
> -                return false;
> -            }
> -        } else {
> -            high = low;
> -        }
> -
> -        if (!lexer_force_match(ctx->lexer, LEX_T_RSQUARE)) {
> -            return false;
> -        }
> -
> -        if (low > high) {
> -            lexer_error(ctx->lexer, "Invalid bit range %d to %d.", low,
> high);
> -            return false;
> -        } else if (high >= symbol->width) {
> -            lexer_error(ctx->lexer,
> -                        "Cannot select bits %d to %d of %d-bit field %s.",
> -                        low, high, symbol->width, symbol->name);
> -            return false;
> -        } else if (symbol->level == EXPR_L_NOMINAL
> -                   && (low != 0 || high != symbol->width - 1)) {
> -            lexer_error(ctx->lexer,
> -                        "Cannot select subfield of nominal field %s.",
> -                        symbol->name);
> -            return false;
> -        }
> -
> -        f->ofs = low;
> -        f->n_bits = high - low + 1;
> -    } else {
> -        f->ofs = 0;
> -        f->n_bits = symbol->width;
> -    }
> -
> -    return true;
> -}
> -
> -static bool
> -parse_relop(struct expr_context *ctx, enum expr_relop *relop)
> -{
> -    if (expr_relop_from_token(ctx->lexer->token.type, relop)) {
> -        lexer_get(ctx->lexer);
> -        return true;
> -    } else {
> -        lexer_syntax_error(ctx->lexer, "expecting relational operator");
> -        return false;
> -    }
> -}
> -
> -static bool
> -assign_constant_set_type(struct expr_context *ctx,
> -                         struct expr_constant_set *cs,
> -                         enum expr_constant_type type)
> -{
> -    if (!cs->n_values || cs->type == type) {
> -        cs->type = type;
> -        return true;
> -    } else {
> -        lexer_syntax_error(ctx->lexer, "expecting %s",
> -                           cs->type == EXPR_C_INTEGER ? "integer" :
> "string");
> -        return false;
> -    }
> -}
> -
> -static bool
> -parse_addr_sets(struct expr_context *ctx, struct expr_constant_set *cs,
> -                size_t *allocated_values)
> -{
> -    if (ctx->addr_sets_ref) {
> -        sset_add(ctx->addr_sets_ref, ctx->lexer->token.s);
> -    }
> -
> -    struct expr_constant_set *addr_sets
> -        = (ctx->addr_sets
> -           ? shash_find_data(ctx->addr_sets, ctx->lexer->token.s)
> -           : NULL);
> -    if (!addr_sets) {
> -        lexer_syntax_error(ctx->lexer, "expecting address set name");
> -        return false;
> -    }
> -
> -    if (!assign_constant_set_type(ctx, cs, EXPR_C_INTEGER)) {
> -        return false;
> -    }
> -
> -    size_t n_values = cs->n_values + addr_sets->n_values;
> -    if (n_values >= *allocated_values) {
> -        cs->values = xrealloc(cs->values, n_values * sizeof *cs->values);
> -        *allocated_values = n_values;
> -    }
> -    for (size_t i = 0; i < addr_sets->n_values; i++) {
> -        cs->values[cs->n_values++] = addr_sets->values[i];
> -    }
> -
> -    return true;
> -}
> -
> -static bool
> -parse_port_group(struct expr_context *ctx, struct expr_constant_set *cs,
> -                 size_t *allocated_values)
> -{
> -    struct expr_constant_set *port_group
> -        = (ctx->port_groups
> -           ? shash_find_data(ctx->port_groups, ctx->lexer->token.s)
> -           : NULL);
> -    if (!port_group) {
> -        lexer_syntax_error(ctx->lexer, "expecting port group name");
> -        return false;
> -    }
> -
> -    if (!assign_constant_set_type(ctx, cs, EXPR_C_STRING)) {
> -        return false;
> -    }
> -
> -    size_t n_values = cs->n_values + port_group->n_values;
> -    if (n_values >= *allocated_values) {
> -        cs->values = xrealloc(cs->values, n_values * sizeof *cs->values);
> -        *allocated_values = n_values;
> -    }
> -    for (size_t i = 0; i < port_group->n_values; i++) {
> -        cs->values[cs->n_values++].string =
> -            xstrdup(port_group->values[i].string);
> -    }
> -
> -    return true;
> -}
> -
> -static bool
> -parse_constant(struct expr_context *ctx, struct expr_constant_set *cs,
> -               size_t *allocated_values)
> -{
> -    if (cs->n_values >= *allocated_values) {
> -        cs->values = x2nrealloc(cs->values, allocated_values,
> -                                sizeof *cs->values);
> -    }
> -
> -    if (ctx->lexer->token.type == LEX_T_STRING) {
> -        if (!assign_constant_set_type(ctx, cs, EXPR_C_STRING)) {
> -            return false;
> -        }
> -        cs->values[cs->n_values++].string = xstrdup(ctx->lexer->token.s);
> -        lexer_get(ctx->lexer);
> -        return true;
> -    } else if (ctx->lexer->token.type == LEX_T_INTEGER ||
> -               ctx->lexer->token.type == LEX_T_MASKED_INTEGER) {
> -        if (!assign_constant_set_type(ctx, cs, EXPR_C_INTEGER)) {
> -            return false;
> -        }
> -
> -        union expr_constant *c = &cs->values[cs->n_values++];
> -        c->value = ctx->lexer->token.value;
> -        c->format = ctx->lexer->token.format;
> -        c->masked = ctx->lexer->token.type == LEX_T_MASKED_INTEGER;
> -        if (c->masked) {
> -            c->mask = ctx->lexer->token.mask;
> -        }
> -        lexer_get(ctx->lexer);
> -        return true;
> -    } else if (ctx->lexer->token.type == LEX_T_MACRO) {
> -        if (!parse_addr_sets(ctx, cs, allocated_values)) {
> -            return false;
> -        }
> -        lexer_get(ctx->lexer);
> -        return true;
> -    } else if (ctx->lexer->token.type == LEX_T_PORT_GROUP) {
> -        if (!parse_port_group(ctx, cs, allocated_values)) {
> -            return false;
> -        }
> -        lexer_get(ctx->lexer);
> -        return true;
> -    } else {
> -        lexer_syntax_error(ctx->lexer, "expecting constant");
> -        return false;
> -    }
> -}
> -
> -/* Parses a single or {}-enclosed set of integer or string constants into
> 'cs',
> - * which the caller need not have initialized.  Returns true on success,
> in
> - * which case the caller owns 'cs', false on failure, in which case 'cs'
> is
> - * indeterminate. */
> -static bool
> -parse_constant_set(struct expr_context *ctx, struct expr_constant_set *cs)
> -{
> -    size_t allocated_values = 0;
> -    bool ok;
> -
> -    memset(cs, 0, sizeof *cs);
> -    if (lexer_match(ctx->lexer, LEX_T_LCURLY)) {
> -        ok = true;
> -        cs->in_curlies = true;
> -        do {
> -            if (!parse_constant(ctx, cs, &allocated_values)) {
> -                ok = false;
> -                break;
> -            }
> -            lexer_match(ctx->lexer, LEX_T_COMMA);
> -        } while (!lexer_match(ctx->lexer, LEX_T_RCURLY));
> -    } else {
> -        ok = parse_constant(ctx, cs, &allocated_values);
> -    }
> -    if (!ok) {
> -        expr_constant_set_destroy(cs);
> -    }
> -    return ok;
> -}
> -
> -/* Parses from 'lexer' a single integer or string constant compatible
> with the
> - * type of 'f' into 'c'.
> - *
> - * Returns true if successful, false if an error occurred.  Upon return,
> - * returns true if and only if lexer->error is NULL.  On failure, 'c' is
> - * indeterminate. */
> -bool
> -expr_constant_parse(struct lexer *lexer, const struct expr_field *f,
> -                    union expr_constant *c)
> -{
> -    if (lexer->error) {
> -        return false;
> -    }
> -
> -    struct expr_context ctx = { .lexer = lexer };
> -
> -    struct expr_constant_set cs;
> -    memset(&cs, 0, sizeof cs);
> -    size_t allocated_values = 0;
> -    if (parse_constant(&ctx, &cs, &allocated_values)
> -        && type_check(&ctx, f, &cs)) {
> -        *c = cs.values[0];
> -        cs.n_values = 0;
> -    }
> -    expr_constant_set_destroy(&cs);
> -
> -    return !lexer->error;
> -}
> -
> -/* Appends to 's' a re-parseable representation of constant 'c' with the
> given
> - * 'type'. */
> -void
> -expr_constant_format(const union expr_constant *c,
> -                     enum expr_constant_type type, struct ds *s)
> -{
> -    if (type == EXPR_C_STRING) {
> -        json_string_escape(c->string, s);
> -    } else {
> -        struct lex_token token;
> -        token.type = c->masked ? LEX_T_MASKED_INTEGER : LEX_T_INTEGER;
> -        token.s = NULL;
> -        token.format = c->format;
> -        token.value = c->value;
> -        if (c->masked) {
> -            token.mask = c->mask;
> -        }
> -
> -        lex_token_format(&token, s);
> -    }
> -}
> -
> -/* Frees the contents of 'c', which has the specified 'type'.
> - *
> - * Does not free(c). */
> -void
> -expr_constant_destroy(const union expr_constant *c,
> -                      enum expr_constant_type type)
> -{
> -    if (c && type == EXPR_C_STRING) {
> -        free(c->string);
> -    }
> -}
> -
> -/* Parses from 'lexer' a single or {}-enclosed set of at least one
> integer or
> - * string constants into 'cs', which the caller need not have initialized.
> - *
> - * Returns true if successful, false if an error occurred.  Upon return,
> - * returns true if and only if lexer->error is NULL.  On failure, 'cs' is
> - * indeterminate. */
> -bool
> -expr_constant_set_parse(struct lexer *lexer, struct expr_constant_set *cs)
> -{
> -    if (!lexer->error) {
> -        struct expr_context ctx = { .lexer = lexer };
> -        parse_constant_set(&ctx, cs);
> -    }
> -    return !lexer->error;
> -}
> -
> -/* Appends to 's' a re-parseable representation of 'cs'. */
> -void
> -expr_constant_set_format(const struct expr_constant_set *cs, struct ds *s)
> -{
> -    bool curlies = cs->in_curlies || cs->n_values != 1;
> -    if (curlies) {
> -        ds_put_char(s, '{');
> -    }
> -
> -    for (const union expr_constant *c = cs->values;
> -         c < &cs->values[cs->n_values]; c++) {
> -        if (c != cs->values) {
> -            ds_put_cstr(s, ", ");
> -        }
> -
> -        expr_constant_format(c, cs->type, s);
> -    }
> -
> -    if (curlies) {
> -        ds_put_char(s, '}');
> -    }
> -}
> -
> -void
> -expr_constant_set_destroy(struct expr_constant_set *cs)
> -{
> -    if (cs) {
> -        if (cs->type == EXPR_C_STRING) {
> -            for (size_t i = 0; i < cs->n_values; i++) {
> -                free(cs->values[i].string);
> -            }
> -        }
> -        free(cs->values);
> -    }
> -}
> -
> -/* Adds an constant set named 'name' to 'const_sets', replacing any
> existing
> - * constant set entry with the given name. */
> -void
> -expr_const_sets_add(struct shash *const_sets, const char *name,
> -                    const char *const *values, size_t n_values,
> -                    bool convert_to_integer)
> -{
> -    /* Replace any existing entry for this name. */
> -    expr_const_sets_remove(const_sets, name);
> -
> -    struct expr_constant_set *cs = xzalloc(sizeof *cs);
> -    cs->in_curlies = true;
> -    cs->n_values = 0;
> -    cs->values = xmalloc(n_values * sizeof *cs->values);
> -    if (convert_to_integer) {
> -        cs->type = EXPR_C_INTEGER;
> -        for (size_t i = 0; i < n_values; i++) {
> -            /* Use the lexer to convert each constant set into the proper
> -             * integer format. */
> -            struct lexer lex;
> -            lexer_init(&lex, values[i]);
> -            lexer_get(&lex);
> -            if (lex.token.type != LEX_T_INTEGER
> -                && lex.token.type != LEX_T_MASKED_INTEGER) {
> -                VLOG_WARN("Invalid constant set entry: '%s', token type:
> %d",
> -                          values[i], lex.token.type);
> -            } else {
> -                union expr_constant *c = &cs->values[cs->n_values++];
> -                c->value = lex.token.value;
> -                c->format = lex.token.format;
> -                c->masked = lex.token.type == LEX_T_MASKED_INTEGER;
> -                if (c->masked) {
> -                    c->mask = lex.token.mask;
> -                }
> -            }
> -            lexer_destroy(&lex);
> -        }
> -    } else {
> -        cs->type = EXPR_C_STRING;
> -        for (size_t i = 0; i < n_values; i++) {
> -            union expr_constant *c = &cs->values[cs->n_values++];
> -            c->string = xstrdup(values[i]);
> -        }
> -    }
> -
> -    shash_add(const_sets, name, cs);
> -}
> -
> -void
> -expr_const_sets_remove(struct shash *const_sets, const char *name)
> -{
> -    struct expr_constant_set *cs = shash_find_and_delete(const_sets,
> name);
> -    if (cs) {
> -        expr_constant_set_destroy(cs);
> -        free(cs);
> -    }
> -}
> -
> -/* Destroy all contents of 'const_sets'. */
> -void
> -expr_const_sets_destroy(struct shash *const_sets)
> -{
> -    struct shash_node *node, *next;
> -
> -    SHASH_FOR_EACH_SAFE (node, next, const_sets) {
> -        struct expr_constant_set *cs = node->data;
> -
> -        shash_delete(const_sets, node);
> -        expr_constant_set_destroy(cs);
> -        free(cs);
> -    }
> -}
> -
> -static struct expr *
> -parse_chassis_resident(struct expr_context *ctx)
> -{
> -    if (ctx->lexer->token.type != LEX_T_STRING) {
> -        lexer_syntax_error(ctx->lexer, "expecting string");
> -        return NULL;
> -    }
> -
> -    struct expr *e = xzalloc(sizeof *e);
> -    e->type = EXPR_T_CONDITION;
> -    e->cond.type = EXPR_COND_CHASSIS_RESIDENT;
> -    e->cond.not = false;
> -    e->cond.string = xstrdup(ctx->lexer->token.s);
> -
> -    lexer_get(ctx->lexer);
> -    if (!lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -        expr_destroy(e);
> -        return NULL;
> -    }
> -
> -    return e;
> -}
> -
> -static struct expr *
> -expr_parse_primary(struct expr_context *ctx, bool *atomic)
> -{
> -    *atomic = false;
> -    if (lexer_match(ctx->lexer, LEX_T_LPAREN)) {
> -        if (ctx->paren_depth >= MAX_PAREN_DEPTH) {
> -            lexer_error(ctx->lexer, "Parentheses nested too deeply.");
> -            return NULL;
> -        }
> -
> -        ctx->paren_depth++;
> -        struct expr *e = expr_parse__(ctx);
> -        ctx->paren_depth--;
> -
> -        if (!lexer_force_match(ctx->lexer, LEX_T_RPAREN)) {
> -            expr_destroy(e);
> -            return NULL;
> -        }
> -        *atomic = true;
> -        return e;
> -    }
> -
> -    if (ctx->lexer->token.type == LEX_T_ID) {
> -        struct expr_field f;
> -        enum expr_relop r;
> -        struct expr_constant_set c;
> -
> -        if (lexer_lookahead(ctx->lexer) == LEX_T_LPAREN) {
> -            if (lexer_match_id(ctx->lexer, "is_chassis_resident")) {
> -                lexer_get(ctx->lexer); /* Skip "(". */
> -                *atomic = true;
> -                return parse_chassis_resident(ctx);
> -            }
> -            lexer_error(ctx->lexer, "parsing function name");
> -            return NULL;
> -        }
> -
> -        if (!parse_field(ctx, &f)) {
> -            return NULL;
> -        }
> -
> -        if (!expr_relop_from_token(ctx->lexer->token.type, &r)) {
> -            if (!f.n_bits || ctx->lexer->token.type == LEX_T_EQUALS) {
> -                lexer_syntax_error(ctx->lexer,
> -                                   "expecting relational operator");
> -                return NULL;
> -            } else if (f.n_bits > 1 && !ctx->not) {
> -                lexer_error(ctx->lexer,
> -                            "Explicit `!= 0' is required for inequality "
> -                            "test of multibit field against 0.");
> -                return NULL;
> -            }
> -
> -            *atomic = true;
> -
> -            union expr_constant *cst = xzalloc(sizeof *cst);
> -            cst->format = LEX_F_HEXADECIMAL;
> -            cst->masked = false;
> -
> -            c.type = EXPR_C_INTEGER;
> -            c.values = cst;
> -            c.n_values = 1;
> -            c.in_curlies = false;
> -            return make_cmp(ctx, &f, EXPR_R_NE, &c);
> -        } else if (parse_relop(ctx, &r) && parse_constant_set(ctx, &c)) {
> -            return make_cmp(ctx, &f, r, &c);
> -        } else {
> -            return NULL;
> -        }
> -    } else {
> -        struct expr_constant_set c1;
> -        if (!parse_constant_set(ctx, &c1)) {
> -            return NULL;
> -        }
> -
> -        if (!expr_relop_from_token(ctx->lexer->token.type, NULL)
> -            && c1.n_values == 1
> -            && c1.type == EXPR_C_INTEGER
> -            && c1.values[0].format == LEX_F_DECIMAL
> -            && !c1.values[0].masked
> -            && !c1.in_curlies) {
> -            uint64_t x = ntohll(c1.values[0].value.integer);
> -            if (x <= 1) {
> -                *atomic = true;
> -                expr_constant_set_destroy(&c1);
> -                return expr_create_boolean(x);
> -            }
> -        }
> -
> -        enum expr_relop r1;
> -        struct expr_field f;
> -        if (!parse_relop(ctx, &r1) || !parse_field(ctx, &f)) {
> -            expr_constant_set_destroy(&c1);
> -            return NULL;
> -        }
> -
> -        if (!expr_relop_from_token(ctx->lexer->token.type, NULL)) {
> -            return make_cmp(ctx, &f, expr_relop_turn(r1), &c1);
> -        }
> -
> -        enum expr_relop r2;
> -        struct expr_constant_set c2;
> -        if (!parse_relop(ctx, &r2) || !parse_constant_set(ctx, &c2)) {
> -            expr_constant_set_destroy(&c1);
> -            return NULL;
> -        } else {
> -            /* Reject "1 == field == 2", "1 < field > 2", and so on. */
> -            if (!(((r1 == EXPR_R_LT || r1 == EXPR_R_LE) &&
> -                   (r2 == EXPR_R_LT || r2 == EXPR_R_LE)) ||
> -                  ((r1 == EXPR_R_GT || r1 == EXPR_R_GE) &&
> -                   (r2 == EXPR_R_GT || r2 == EXPR_R_GE)))) {
> -                lexer_error(ctx->lexer, "Range expressions must have the "
> -                            "form `x < field < y' or `x > field > y',
> with "
> -                            "each `<' optionally replaced by `<=' or `>'
> by "
> -                            "`>=').");
> -                expr_constant_set_destroy(&c1);
> -                expr_constant_set_destroy(&c2);
> -                return NULL;
> -            }
> -
> -            struct expr *e1 = make_cmp(ctx, &f, expr_relop_turn(r1), &c1);
> -            struct expr *e2 = make_cmp(ctx, &f, r2, &c2);
> -            if (ctx->lexer->error) {
> -                expr_destroy(e1);
> -                expr_destroy(e2);
> -                return NULL;
> -            }
> -            return expr_combine(EXPR_T_AND, e1, e2);
> -        }
> -    }
> -}
> -
> -static struct expr *
> -expr_parse_not(struct expr_context *ctx)
> -{
> -    bool atomic;
> -
> -    if (lexer_match(ctx->lexer, LEX_T_LOG_NOT)) {
> -        ctx->not = !ctx->not;
> -        struct expr *expr = expr_parse_primary(ctx, &atomic);
> -        ctx->not = !ctx->not;
> -
> -        if (expr) {
> -            if (!atomic) {
> -                lexer_error(ctx->lexer,
> -                            "Missing parentheses around operand of !.");
> -                expr_destroy(expr);
> -                return NULL;
> -            }
> -            expr_not(expr);
> -        }
> -        return expr;
> -    } else {
> -        return expr_parse_primary(ctx, &atomic);
> -    }
> -}
> -
> -struct expr *
> -expr_parse__(struct expr_context *ctx)
> -{
> -    struct expr *e = expr_parse_not(ctx);
> -    if (!e) {
> -        return NULL;
> -    }
> -
> -    enum lex_type lex_type = ctx->lexer->token.type;
> -    if (lex_type == LEX_T_LOG_AND || lex_type == LEX_T_LOG_OR) {
> -        enum expr_type expr_type
> -            = lex_type == LEX_T_LOG_AND ? EXPR_T_AND : EXPR_T_OR;
> -
> -        lexer_get(ctx->lexer);
> -        do {
> -            struct expr *e2 = expr_parse_not(ctx);
> -            if (!e2) {
> -                expr_destroy(e);
> -                return NULL;
> -            }
> -            e = expr_combine(expr_type, e, e2);
> -        } while (lexer_match(ctx->lexer, lex_type));
> -        if (ctx->lexer->token.type == LEX_T_LOG_AND
> -            || ctx->lexer->token.type == LEX_T_LOG_OR) {
> -            expr_destroy(e);
> -            lexer_error(ctx->lexer,
> -                        "&& and || must be parenthesized when used
> together.");
> -            return NULL;
> -        }
> -    }
> -    return e;
> -}
> -
> -/* Parses an expression from 'lexer' using the symbols in 'symtab' and
> - * address set table in 'addr_sets'.  If successful, returns the new
> - * expression; on failure, returns NULL.  Returns nonnull if and only if
> - * lexer->error is NULL. */
> -struct expr *
> -expr_parse(struct lexer *lexer, const struct shash *symtab,
> -           const struct shash *addr_sets,
> -           const struct shash *port_groups,
> -           struct sset *addr_sets_ref)
> -{
> -    struct expr_context ctx = { .lexer = lexer,
> -                                .symtab = symtab,
> -                                .addr_sets = addr_sets,
> -                                .port_groups = port_groups,
> -                                .addr_sets_ref = addr_sets_ref };
> -    return lexer->error ? NULL : expr_parse__(&ctx);
> -}
> -
> -/* Parses the expression in 's' using the symbols in 'symtab' and
> - * address set table in 'addr_sets'.  If successful, returns the new
> - * expression and sets '*errorp' to NULL.  On failure, returns NULL and
> - * sets '*errorp' to an explanatory error message.  The caller must
> - * eventually free the returned expression (with expr_destroy()) or
> - * error (with free()). */
> -struct expr *
> -expr_parse_string(const char *s, const struct shash *symtab,
> -                  const struct shash *addr_sets,
> -                  const struct shash *port_groups,
> -                  struct sset *addr_sets_ref,
> -                  char **errorp)
> -{
> -    struct lexer lexer;
> -
> -    lexer_init(&lexer, s);
> -    lexer_get(&lexer);
> -    struct expr *expr = expr_parse(&lexer, symtab, addr_sets, port_groups,
> -                                   addr_sets_ref);
> -    lexer_force_end(&lexer);
> -    *errorp = lexer_steal_error(&lexer);
> -    if (*errorp) {
> -        expr_destroy(expr);
> -        expr = NULL;
> -    }
> -    lexer_destroy(&lexer);
> -
> -    return expr;
> -}
> -
> -/* Parses a field or subfield from 'lexer' into 'field', obtaining field
> names
> - * from 'symtab'.  Returns true if successful, false if an error occurred.
> - * Upon return, returns true if and only if lexer->error is NULL. */
> -bool
> -expr_field_parse(struct lexer *lexer, const struct shash *symtab,
> -                 struct expr_field *field, struct expr **prereqsp)
> -{
> -    struct expr_context ctx = { .lexer = lexer, .symtab = symtab };
> -    if (parse_field(&ctx, field) && field->symbol->predicate) {
> -        lexer_error(lexer, "Predicate symbol %s used where lvalue
> required.",
> -                    field->symbol->name);
> -    }
> -    if (!lexer->error) {
> -        const struct expr_symbol *symbol = field->symbol;
> -        while (symbol) {
> -            if (symbol->prereqs) {
> -                char *error;
> -                struct ovs_list nesting = OVS_LIST_INITIALIZER(&nesting);
> -                struct expr *e = parse_and_annotate(symbol->prereqs,
> symtab,
> -                                                    &nesting, &error);
> -                if (error) {
> -                    lexer_error(lexer, "%s", error);
> -                    free(error);
> -                    break;
> -                }
> -                *prereqsp = expr_combine(EXPR_T_AND, *prereqsp, e);
> -            }
> -
> -            if (!symbol->parent) {
> -                break;
> -            }
> -            symbol = symbol->parent;
> -        }
> -    }
> -    if (!lexer->error) {
> -        return true;
> -    }
> -    memset(field, 0, sizeof *field);
> -    return false;
> -}
> -
> -/* Appends to 's' a re-parseable representation of 'field'. */
> -void
> -expr_field_format(const struct expr_field *field, struct ds *s)
> -{
> -    ds_put_cstr(s, field->symbol->name);
> -    if (field->ofs || field->n_bits != field->symbol->width) {
> -        if (field->n_bits != 1) {
> -            ds_put_format(s, "[%d..%d]",
> -                          field->ofs, field->ofs + field->n_bits - 1);
> -        } else {
> -            ds_put_format(s, "[%d]", field->ofs);
> -        }
> -    }
> -}
> -
> -void
> -expr_symbol_format(const struct expr_symbol *symbol, struct ds *s)
> -{
> -    ds_put_format(s, "%s = ", symbol->name);
> -    if (symbol->parent) {
> -        struct expr_field f = { symbol->parent,
> -                                symbol->parent_ofs,
> -                                symbol->width };
> -        expr_field_format(&f, s);
> -    } else if (symbol->predicate) {
> -        ds_put_cstr(s, symbol->predicate);
> -    } else if (symbol->ovn_field) {
> -        ds_put_cstr(s, symbol->name);
> -    } else {
> -        nx_format_field_name(symbol->field->id, OFP13_VERSION, s);
> -    }
> -}
> -
> -static struct expr_symbol *
> -add_symbol(struct shash *symtab, const char *name, int width,
> -           const char *prereqs, enum expr_level level,
> -           bool must_crossproduct, bool rw)
> -{
> -    struct expr_symbol *symbol = xzalloc(sizeof *symbol);
> -    symbol->name = xstrdup(name);
> -    symbol->prereqs = prereqs && prereqs[0] ? xstrdup(prereqs) : NULL;
> -    symbol->width = width;
> -    symbol->level = level;
> -    symbol->must_crossproduct = must_crossproduct;
> -    symbol->rw = rw;
> -    shash_add_assert(symtab, symbol->name, symbol);
> -    return symbol;
> -}
> -
> -/* Adds field 'id' to symbol table 'symtab' under the given 'name'.
> Whenever
> - * 'name' is referenced, expression annotation (see expr_annotate()) will
> - * ensure that 'prereqs' are also true.  If 'must_crossproduct' is true,
> then
> - * conversion to flows will never attempt to use the field as a
> conjunctive
> - * match dimension (see "Crossproducting" in the large comment on struct
> - * expr_symbol in expr.h for an example).
> - *
> - * A given field 'id' must only be used for a single symbol in a symbol
> table.
> - * Use subfields to duplicate or subset a field (you can even make a
> subfield
> - * include all the bits of the "parent" field if you like). */
> -struct expr_symbol *
> -expr_symtab_add_field(struct shash *symtab, const char *name,
> -                      enum mf_field_id id, const char *prereqs,
> -                      bool must_crossproduct)
> -{
> -    const struct mf_field *field = mf_from_id(id);
> -    struct expr_symbol *symbol;
> -
> -    symbol = add_symbol(symtab, name, field->n_bits, prereqs,
> -                        (field->maskable == MFM_FULLY
> -                         ? EXPR_L_ORDINAL
> -                         : EXPR_L_NOMINAL),
> -                        must_crossproduct, field->writable);
> -    symbol->field = field;
> -    return symbol;
> -}
> -
> -static bool
> -parse_field_from_string(const char *s, const struct shash *symtab,
> -                        struct expr_field *field, char **errorp)
> -{
> -    struct lexer lexer;
> -    lexer_init(&lexer, s);
> -    lexer_get(&lexer);
> -
> -    struct expr_context ctx = { .lexer = &lexer, .symtab = symtab };
> -    parse_field(&ctx, field);
> -    lexer_force_end(&lexer);
> -    *errorp = lexer_steal_error(&lexer);
> -    lexer_destroy(&lexer);
> -
> -    return !*errorp;
> -}
> -
> -/* Adds 'name' as a subfield of a larger field in 'symtab'.  Whenever
> - * 'name' is referenced, expression annotation (see expr_annotate()) will
> - * ensure that 'prereqs' are also true.
> - *
> - * 'subfield' must describe the subfield as a string, e.g.
> "vlan.tci[0..11]"
> - * for the low 12 bits of a larger field named "vlan.tci". */
> -struct expr_symbol *
> -expr_symtab_add_subfield(struct shash *symtab, const char *name,
> -                         const char *prereqs, const char *subfield)
> -{
> -    struct expr_symbol *symbol;
> -    struct expr_field f;
> -    char *error;
> -
> -    if (!parse_field_from_string(subfield, symtab, &f, &error)) {
> -        VLOG_WARN("%s: error parsing %s subfield (%s)", subfield, name,
> error);
> -        free(error);
> -        return NULL;
> -    }
> -
> -    enum expr_level level = f.symbol->level;
> -    if (level != EXPR_L_ORDINAL) {
> -        VLOG_WARN("can't define %s as subfield of %s field %s",
> -                  name, expr_level_to_string(level), f.symbol->name);
> -    }
> -
> -    symbol = add_symbol(symtab, name, f.n_bits, prereqs, level, false,
> -                        f.symbol->rw);
> -    symbol->parent = f.symbol;
> -    symbol->parent_ofs = f.ofs;
> -    return symbol;
> -}
> -
> -/* Adds a string-valued symbol named 'name' to 'symtab' with the specified
> - * 'prereqs'. */
> -struct expr_symbol *
> -expr_symtab_add_string(struct shash *symtab, const char *name,
> -                       enum mf_field_id id, const char *prereqs)
> -{
> -    const struct mf_field *field = mf_from_id(id);
> -    struct expr_symbol *symbol;
> -
> -    symbol = add_symbol(symtab, name, 0, prereqs, EXPR_L_NOMINAL, false,
> -                        field->writable);
> -    symbol->field = field;
> -    return symbol;
> -}
> -
> -static enum expr_level
> -expr_get_level(const struct expr *expr)
> -{
> -    const struct expr *sub;
> -    enum expr_level level = EXPR_L_ORDINAL;
> -
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return (expr->cmp.symbol->level == EXPR_L_NOMINAL
> -                ? EXPR_L_NOMINAL
> -                : EXPR_L_BOOLEAN);
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        LIST_FOR_EACH (sub, node, &expr->andor) {
> -            enum expr_level sub_level = expr_get_level(sub);
> -            level = MIN(level, sub_level);
> -        }
> -        return level;
> -
> -    case EXPR_T_BOOLEAN:
> -    case EXPR_T_CONDITION:
> -        return EXPR_L_BOOLEAN;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static enum expr_level
> -expr_parse_level(const char *s, const struct shash *symtab, char **errorp)
> -{
> -    struct expr *expr = expr_parse_string(s, symtab, NULL, NULL, NULL,
> errorp);
> -    enum expr_level level = expr ? expr_get_level(expr) : EXPR_L_NOMINAL;
> -    expr_destroy(expr);
> -    return level;
> -}
> -
> -/* Adds a predicate symbol, whose value is the given Boolean 'expression',
> - * named 'name' to 'symtab'.  For example, "ip4 && ip4.proto == 6" might
> be an
> - * appropriate predicate named "tcp4". */
> -struct expr_symbol *
> -expr_symtab_add_predicate(struct shash *symtab, const char *name,
> -                          const char *expansion)
> -{
> -    struct expr_symbol *symbol;
> -    enum expr_level level;
> -    char *error;
> -
> -    level = expr_parse_level(expansion, symtab, &error);
> -    if (error) {
> -        VLOG_WARN("%s: error parsing %s expansion (%s)",
> -                  expansion, name, error);
> -        free(error);
> -        return NULL;
> -    }
> -
> -    symbol = add_symbol(symtab, name, 1, NULL, level, false, false);
> -    symbol->predicate = xstrdup(expansion);
> -    return symbol;
> -}
> -
> -struct expr_symbol *
> -expr_symtab_add_ovn_field(struct shash *symtab, const char *name,
> -                          enum ovn_field_id id)
> -{
> -    const struct ovn_field *ovn_field = ovn_field_from_id(id);
> -    struct expr_symbol *symbol;
> -
> -    symbol = add_symbol(symtab, name, ovn_field->n_bits, NULL,
> -                        EXPR_L_NOMINAL, false, true);
> -    symbol->ovn_field = ovn_field;
> -    return symbol;
> -}
> -
> -/* Destroys 'symtab' and all of its symbols. */
> -void
> -expr_symtab_destroy(struct shash *symtab)
> -{
> -    struct shash_node *node, *next;
> -
> -    SHASH_FOR_EACH_SAFE (node, next, symtab) {
> -        struct expr_symbol *symbol = node->data;
> -
> -        shash_delete(symtab, node);
> -        free(symbol->name);
> -        free(symbol->prereqs);
> -        free(symbol->predicate);
> -        free(symbol);
> -    }
> -}
> -
> -/* Cloning. */
> -
> -static struct expr *
> -expr_clone_cmp(struct expr *expr)
> -{
> -    struct expr *new = xmemdup(expr, sizeof *expr);
> -    if (!new->cmp.symbol->width) {
> -        new->cmp.string = xstrdup(new->cmp.string);
> -    }
> -    return new;
> -}
> -
> -static struct expr *
> -expr_clone_andor(struct expr *expr)
> -{
> -    struct expr *new = expr_create_andor(expr->type);
> -    struct expr *sub;
> -
> -    LIST_FOR_EACH (sub, node, &expr->andor) {
> -        struct expr *new_sub = expr_clone(sub);
> -        ovs_list_push_back(&new->andor, &new_sub->node);
> -    }
> -    return new;
> -}
> -
> -static struct expr *
> -expr_clone_condition(struct expr *expr)
> -{
> -    struct expr *new = xmemdup(expr, sizeof *expr);
> -    new->cond.string = xstrdup(new->cond.string);
> -    return new;
> -}
> -
> -/* Returns a clone of 'expr'.  This is a "deep copy": neither the returned
> - * expression nor any of its substructure will be shared with 'expr'. */
> -struct expr *
> -expr_clone(struct expr *expr)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return expr_clone_cmp(expr);
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        return expr_clone_andor(expr);
> -
> -    case EXPR_T_BOOLEAN:
> -        return expr_create_boolean(expr->boolean);
> -
> -    case EXPR_T_CONDITION:
> -        return expr_clone_condition(expr);
> -    }
> -    OVS_NOT_REACHED();
> -}
> -
> -/* Destroys 'expr' and all of the sub-expressions it references. */
> -void
> -expr_destroy(struct expr *expr)
> -{
> -    if (!expr) {
> -        return;
> -    }
> -
> -    struct expr *sub, *next;
> -
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        if (!expr->cmp.symbol->width) {
> -            free(expr->cmp.string);
> -        }
> -        break;
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -            ovs_list_remove(&sub->node);
> -            expr_destroy(sub);
> -        }
> -        break;
> -
> -    case EXPR_T_BOOLEAN:
> -        break;
> -
> -    case EXPR_T_CONDITION:
> -        free(expr->cond.string);
> -        break;
> -    }
> -    free(expr);
> -}
> -
> -/* Annotation. */
> -
> -/* An element in a linked list of symbols.
> - *
> - * Used to detect when a symbol is being expanded recursively, to allow
> - * flagging an error. */
> -struct annotation_nesting {
> -    struct ovs_list node;
> -    const struct expr_symbol *symbol;
> -};
> -
> -static struct expr *expr_annotate_(struct expr *, const struct shash
> *symtab,
> -                                   struct ovs_list *nesting, char
> **errorp);
> -
> -static struct expr *
> -parse_and_annotate(const char *s, const struct shash *symtab,
> -                   struct ovs_list *nesting, char **errorp)
> -{
> -    char *error;
> -    struct expr *expr;
> -
> -    expr = expr_parse_string(s, symtab, NULL, NULL, NULL, &error);
> -    if (expr) {
> -        expr = expr_annotate_(expr, symtab, nesting, &error);
> -    }
> -    if (expr) {
> -        *errorp = NULL;
> -    } else {
> -        *errorp = xasprintf("Error parsing expression `%s' encountered as
> "
> -                            "prerequisite or predicate of initial
> expression: "
> -                            "%s", s, error);
> -        free(error);
> -    }
> -    return expr;
> -}
> -
> -static struct expr *
> -expr_annotate_cmp(struct expr *expr, const struct shash *symtab,
> -                  bool append_prereqs, struct ovs_list *nesting, char
> **errorp)
> -{
> -    const struct expr_symbol *symbol = expr->cmp.symbol;
> -    const struct annotation_nesting *iter;
> -    LIST_FOR_EACH (iter, node, nesting) {
> -        if (iter->symbol == symbol) {
> -            *errorp = xasprintf("Recursive expansion of symbol `%s'.",
> -                                symbol->name);
> -            expr_destroy(expr);
> -            return NULL;
> -        }
> -    }
> -
> -    struct annotation_nesting an;
> -    an.symbol = symbol;
> -    ovs_list_push_back(nesting, &an.node);
> -
> -    struct expr *prereqs = NULL;
> -    if (append_prereqs && symbol->prereqs) {
> -        prereqs = parse_and_annotate(symbol->prereqs, symtab, nesting,
> errorp);
> -        if (!prereqs) {
> -            goto error;
> -        }
> -    }
> -
> -    if (symbol->parent) {
> -        expr->cmp.symbol = symbol->parent;
> -        mf_subvalue_shift(&expr->cmp.value, symbol->parent_ofs);
> -        mf_subvalue_shift(&expr->cmp.mask, symbol->parent_ofs);
> -    } else if (symbol->predicate) {
> -        struct expr *predicate;
> -
> -        predicate = parse_and_annotate(symbol->predicate, symtab,
> -                                       nesting, errorp);
> -        if (!predicate) {
> -            goto error;
> -        }
> -
> -        bool positive = (expr->cmp.value.integer & htonll(1)) != 0;
> -        positive ^= expr->cmp.relop == EXPR_R_NE;
> -        if (!positive) {
> -            expr_not(predicate);
> -        }
> -
> -        expr_destroy(expr);
> -        expr = predicate;
> -    }
> -
> -    *errorp = NULL;
> -    ovs_list_remove(&an.node);
> -    return prereqs ? expr_combine(EXPR_T_AND, expr, prereqs) : expr;
> -
> -error:
> -    expr_destroy(expr);
> -    expr_destroy(prereqs);
> -    ovs_list_remove(&an.node);
> -    return NULL;
> -}
> -
> -/* Append (logical AND) prerequisites for given symbol to the expression.
> */
> -static struct expr *
> -expr_append_prereqs(struct expr *expr, const struct expr_symbol *symbol,
> -                    const struct shash *symtab, struct ovs_list *nesting,
> -                    char **errorp)
> -{
> -    struct expr *prereqs = NULL;
> -
> -    if (symbol->prereqs) {
> -        prereqs = parse_and_annotate(symbol->prereqs, symtab, nesting,
> errorp);
> -        if (!prereqs) {
> -            expr_destroy(expr);
> -            return NULL;
> -        }
> -    }
> -
> -    return prereqs ? expr_combine(EXPR_T_AND, expr, prereqs) : expr;
> -}
> -
> -static const struct expr_symbol *expr_get_unique_symbol(
> -    const struct expr *expr);
> -
> -/* Ordinarily, annotation adds prerequisites to the expression, and
> that's what
> - * this function does if 'append_prereqs' is true.  If 'append_prereqs' is
> - * false, this function ignores prerequisites (in which case the caller
> must
> - * have arranged to deal with them). */
> -static struct expr *
> -expr_annotate__(struct expr *expr, const struct shash *symtab,
> -                bool append_prereqs, struct ovs_list *nesting, char
> **errorp)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return expr_annotate_cmp(expr, symtab, append_prereqs, nesting,
> -                                 errorp);
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR: {
> -        struct expr *sub, *next;
> -
> -        /* Detect whether every term in 'expr' mentions the same symbol.
> If
> -         * so, then suppress prerequisites for that symbol for those
> terms and
> -         * instead apply them once at our higher level.
> -         *
> -         * If 'append_prereqs' is false, though, we're not supposed to
> handle
> -         * prereqs at all (because our caller is already doing it). */
> -        if (append_prereqs) {
> -            const struct expr_symbol *sym = expr_get_unique_symbol(expr);
> -            if (sym) {
> -                append_prereqs = false;
> -                expr = expr_append_prereqs(expr, sym, symtab, nesting,
> errorp);
> -                if (!expr) {
> -                    return NULL;
> -                }
> -            }
> -        }
> -
> -        LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -            ovs_list_remove(&sub->node);
> -            struct expr *new_sub = expr_annotate__(sub, symtab,
> append_prereqs,
> -                                                   nesting, errorp);
> -            if (!new_sub) {
> -                expr_destroy(expr);
> -                return NULL;
> -            }
> -            expr_insert_andor(expr, next, new_sub);
> -        }
> -        *errorp = NULL;
> -        return expr;
> -    }
> -
> -    case EXPR_T_BOOLEAN:
> -    case EXPR_T_CONDITION:
> -        *errorp = NULL;
> -        return expr;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Same interface and purpose as expr_annotate(), with an additional
> parameter
> - * for internal bookkeeping.
> - *
> - * Uses 'nesting' to ensure that a given symbol is not recursively
> expanded. */
> -static struct expr *
> -expr_annotate_(struct expr *expr, const struct shash *symtab,
> -               struct ovs_list *nesting, char **errorp)
> -{
> -    return expr_annotate__(expr, symtab, true, nesting, errorp);
> -}
> -
> -/* "Annotates" 'expr', which does the following:
> - *
> - *     - Applies prerequisites, by locating each comparison operator whose
> - *       field has a prerequisite and adding a logical AND against those
> - *       prerequisites.
> - *
> - *     - Expands references to subfield symbols, by replacing them by
> - *       references to their underlying field symbols (suitably shifted).
> - *
> - *     - Expands references to predicate symbols, by replacing them by the
> - *       expressions that they expand to.
> - *
> - * In each case, annotation occurs recursively as necessary.
> - *
> - * If successful, returns the annotated expression and sets '*errorp' to
> NULL.
> - * On failure, returns NULL and sets '*errorp' to an explanatory error
> message,
> - * which the caller must free.  In either case, the caller transfers
> ownership
> - * of 'expr' and receives ownership of the returned expression, if any. */
> -struct expr *
> -expr_annotate(struct expr *expr, const struct shash *symtab, char
> **errorp)
> -{
> -    struct ovs_list nesting = OVS_LIST_INITIALIZER(&nesting);
> -    return expr_annotate_(expr, symtab, &nesting, errorp);
> -}
> -
> -static struct expr *
> -expr_simplify_eq(struct expr *expr)
> -{
> -    const union mf_subvalue *mask = &expr->cmp.mask;
> -    if (is_all_zeros(mask, sizeof *mask)) {
> -        /* Simplify "ip4.dst == 0/0" to just "1" (plus a prerequisite). */
> -        expr_destroy(expr);
> -        return expr_create_boolean(true);
> -    }
> -    return expr;
> -}
> -
> -static struct expr *
> -expr_simplify_ne(struct expr *expr)
> -{
> -    struct expr *new = NULL;
> -    const union mf_subvalue *value = &expr->cmp.value;
> -    const union mf_subvalue *mask = &expr->cmp.mask;
> -    int w = expr->cmp.symbol->width;
> -    int i;
> -
> -    for (i = 0; (i = bitwise_scan(mask, sizeof *mask, true, i, w)) < w;
> i++) {
> -        struct expr *e;
> -
> -        e = xzalloc(sizeof *e);
> -        e->type = EXPR_T_CMP;
> -        e->cmp.symbol = expr->cmp.symbol;
> -        e->cmp.relop = EXPR_R_EQ;
> -        bitwise_put_bit(&e->cmp.value, sizeof e->cmp.value, i,
> -                        !bitwise_get_bit(value, sizeof *value, i));
> -        bitwise_put1(&e->cmp.mask, sizeof e->cmp.mask, i);
> -
> -        new = expr_combine(EXPR_T_OR, new, e);
> -    }
> -    if (!new) {
> -        /* Handle a comparison like "ip4.dst != 0/0", where the mask has
> no
> -         * 1-bits.
> -         *
> -         * The correct result for this expression may not be obvious.
> It's
> -         * easier to understand that "ip4.dst == 0/0" should be true,
> since 0/0
> -         * matches every IPv4 address; then, "ip4.dst != 0/0" should have
> the
> -         * opposite result. */
> -        new = expr_create_boolean(false);
> -    }
> -
> -    expr_destroy(expr);
> -
> -    return new;
> -}
> -
> -static struct expr *
> -expr_simplify_relational(struct expr *expr)
> -{
> -    const union mf_subvalue *value = &expr->cmp.value;
> -    int start, n_bits, end;
> -
> -    find_bitwise_range(&expr->cmp.mask, expr->cmp.symbol->width,
> -                       &start, &n_bits);
> -    ovs_assert(n_bits > 0);
> -    end = start + n_bits;
> -
> -    /* Handle some special cases.
> -     *
> -     * These optimize to just "true":
> -     *
> -     *    tcp.dst >= 0
> -     *    tcp.dst <= 65535
> -     *
> -     * These are easier to understand, and equivalent, when treated as if
> -     * > or < were !=:
> -     *
> -     *    tcp.dst > 0
> -     *    tcp.dst < 65535
> -     */
> -    bool lt = expr->cmp.relop == EXPR_R_LT || expr->cmp.relop ==
> EXPR_R_LE;
> -    bool eq = expr->cmp.relop == EXPR_R_LE || expr->cmp.relop ==
> EXPR_R_GE;
> -    if (bitwise_scan(value, sizeof *value, !lt, start, end) == end) {
> -        if (eq) {
> -            expr_destroy(expr);
> -            return expr_create_boolean(true);
> -        } else {
> -            return expr_simplify_ne(expr);
> -        }
> -    }
> -
> -    /* Reduce "tcp.dst >= 1234" to "tcp.dst == 1234 || tcp.dst > 1234",
> -     * and similarly for "tcp.dst <= 1234". */
> -    struct expr *new = NULL;
> -    if (eq) {
> -        new = xmemdup(expr, sizeof *expr);
> -        new->cmp.relop = EXPR_R_EQ;
> -    }
> -
> -    for (int z = bitwise_scan(value, sizeof *value, lt, start, end);
> -         z < end;
> -         z = bitwise_scan(value, sizeof *value, lt, z + 1, end)) {
> -        struct expr *e;
> -
> -        e = xmemdup(expr, sizeof *expr);
> -        e->cmp.relop = EXPR_R_EQ;
> -        bitwise_toggle_bit(&e->cmp.value, sizeof e->cmp.value, z);
> -        bitwise_zero(&e->cmp.value, sizeof e->cmp.value, start, z -
> start);
> -        bitwise_zero(&e->cmp.mask, sizeof e->cmp.mask, start, z - start);
> -        new = expr_combine(EXPR_T_OR, new, e);
> -    }
> -    expr_destroy(expr);
> -    return new ? new : expr_create_boolean(false);
> -}
> -
> -/* Resolves condition and replaces the expression with a boolean. */
> -static struct expr *
> -expr_simplify_condition(struct expr *expr,
> -                        bool (*is_chassis_resident)(const void *c_aux,
> -                                                    const char
> *port_name),
> -                        const void *c_aux)
> -{
> -    bool result;
> -
> -    switch (expr->cond.type) {
> -    case EXPR_COND_CHASSIS_RESIDENT:
> -        result = is_chassis_resident(c_aux, expr->cond.string);
> -        break;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -    result ^= expr->cond.not;
> -    expr_destroy(expr);
> -    return expr_create_boolean(result);
> -}
> -
> -/* Takes ownership of 'expr' and returns an equivalent expression whose
> - * EXPR_T_CMP nodes use only tests for equality (EXPR_R_EQ). */
> -struct expr *
> -expr_simplify(struct expr *expr,
> -              bool (*is_chassis_resident)(const void *c_aux,
> -                                          const char *port_name),
> -              const void *c_aux)
> -{
> -    struct expr *sub, *next;
> -
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return (!expr->cmp.symbol->width ? expr
> -                : expr->cmp.relop == EXPR_R_EQ ? expr_simplify_eq(expr)
> -                : expr->cmp.relop == EXPR_R_NE ? expr_simplify_ne(expr)
> -                : expr_simplify_relational(expr));
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -            ovs_list_remove(&sub->node);
> -            expr_insert_andor(expr, next,
> -                              expr_simplify(sub, is_chassis_resident,
> c_aux));
> -        }
> -        return expr_fix(expr);
> -
> -    case EXPR_T_BOOLEAN:
> -        return expr;
> -
> -    case EXPR_T_CONDITION:
> -        return expr_simplify_condition(expr, is_chassis_resident, c_aux);
> -    }
> -    OVS_NOT_REACHED();
> -}
> -
> -/* Tests whether 'expr' is an expression over exactly one symbol: that is,
> - * whether it is either a EXPR_T_CMP node or a tree of ANDs and ORs all
> over
> - * the same symbol.  If it is, returns the symbol in question.  If it is
> not
> - * (that is, if there is more than one symbol or no symbols at all),
> returns
> - * NULL. */
> -static const struct expr_symbol *
> -expr_get_unique_symbol(const struct expr *expr)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return expr->cmp.symbol;
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR: {
> -        const struct expr_symbol *prev = NULL;
> -        struct expr *sub;
> -
> -        LIST_FOR_EACH (sub, node, &expr->andor) {
> -            const struct expr_symbol *symbol =
> expr_get_unique_symbol(sub);
> -            if (!symbol || (prev && symbol != prev)) {
> -                return NULL;
> -            }
> -            prev = symbol;
> -        }
> -        return prev;
> -    }
> -
> -    case EXPR_T_BOOLEAN:
> -    case EXPR_T_CONDITION:
> -        return NULL;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -struct expr_sort {
> -    struct expr *expr;
> -    const struct expr_symbol *symbol;
> -    enum expr_type type;
> -};
> -
> -static int
> -compare_expr_sort(const void *a_, const void *b_)
> -{
> -    const struct expr_sort *a = a_;
> -    const struct expr_sort *b = b_;
> -
> -    if (a->type != b->type) {
> -        return a->type < b->type ? -1 : 1;
> -    } else if (a->symbol) {
> -        int cmp = strcmp(a->symbol->name, b->symbol->name);
> -        if (cmp) {
> -            return cmp;
> -        }
> -
> -        enum expr_type a_type = a->expr->type;
> -        enum expr_type b_type = a->expr->type;
> -        return a_type < b_type ? -1 : a_type > b_type;
> -    } else if (a->type == EXPR_T_AND || a->type == EXPR_T_OR) {
> -        size_t a_len = ovs_list_size(&a->expr->andor);
> -        size_t b_len = ovs_list_size(&b->expr->andor);
> -        return a_len < b_len ? -1 : a_len > b_len;
> -    } else {
> -        return 0;
> -    }
> -}
> -
> -static struct expr *crush_cmps(struct expr *, const struct expr_symbol *);
> -
> -static bool
> -disjunction_matches_string(const struct expr *or, const char *s)
> -{
> -    const struct expr *sub;
> -
> -    LIST_FOR_EACH (sub, node, &or->andor) {
> -        if (!strcmp(sub->cmp.string, s)) {
> -            return true;
> -        }
> -    }
> -
> -    return false;
> -}
> -
> -/* Implementation of crush_cmps() for expr->type == EXPR_T_AND and a
> - * string-typed 'symbol'. */
> -static struct expr *
> -crush_and_string(struct expr *expr, const struct expr_symbol *symbol)
> -{
> -    ovs_assert(!ovs_list_is_short(&expr->andor));
> -
> -    struct expr *singleton = NULL;
> -
> -    /* First crush each subexpression into either a single EXPR_T_CMP or
> an
> -     * EXPR_T_OR with EXPR_T_CMP subexpressions. */
> -    struct expr *sub, *next = NULL;
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        ovs_list_remove(&sub->node);
> -        struct expr *new = crush_cmps(sub, symbol);
> -        switch (new->type) {
> -        case EXPR_T_CMP:
> -            if (!singleton) {
> -                ovs_list_insert(&next->node, &new->node);
> -                singleton = new;
> -            } else {
> -                bool match = !strcmp(new->cmp.string,
> singleton->cmp.string);
> -                expr_destroy(new);
> -                if (!match) {
> -                    expr_destroy(expr);
> -                    return expr_create_boolean(false);
> -                }
> -            }
> -            break;
> -        case EXPR_T_AND:
> -            OVS_NOT_REACHED();
> -        case EXPR_T_OR:
> -            ovs_list_insert(&next->node, &new->node);
> -            break;
> -        case EXPR_T_BOOLEAN:
> -            if (!new->boolean) {
> -                expr_destroy(expr);
> -                return new;
> -            }
> -            free(new);
> -            break;
> -        case EXPR_T_CONDITION:
> -            OVS_NOT_REACHED();
> -        }
> -    }
> -
> -    /* If we have a singleton, then the result is either the singleton
> itself
> -     * (if the ORs allow the singleton) or false. */
> -    if (singleton) {
> -        LIST_FOR_EACH (sub, node, &expr->andor) {
> -            if (sub->type == EXPR_T_OR
> -                && !disjunction_matches_string(sub,
> singleton->cmp.string)) {
> -                expr_destroy(expr);
> -                return expr_create_boolean(false);
> -            }
> -        }
> -        ovs_list_remove(&singleton->node);
> -        expr_destroy(expr);
> -        return singleton;
> -    }
> -
> -    /* Otherwise the result is the intersection of all of the ORs. */
> -    struct sset result = SSET_INITIALIZER(&result);
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        struct sset strings = SSET_INITIALIZER(&strings);
> -        const struct expr *s;
> -        LIST_FOR_EACH (s, node, &sub->andor) {
> -            sset_add(&strings, s->cmp.string);
> -        }
> -        if (sset_is_empty(&result)) {
> -            sset_swap(&result, &strings);
> -        } else {
> -            sset_intersect(&result, &strings);
> -        }
> -        sset_destroy(&strings);
> -
> -        if (sset_is_empty(&result)) {
> -            expr_destroy(expr);
> -            sset_destroy(&result);
> -            return expr_create_boolean(false);
> -        }
> -    }
> -
> -    expr_destroy(expr);
> -    expr = expr_create_andor(EXPR_T_OR);
> -
> -    const char *string;
> -    SSET_FOR_EACH (string, &result) {
> -        sub = xmalloc(sizeof *sub);
> -        sub->type = EXPR_T_CMP;
> -        sub->cmp.relop = EXPR_R_EQ;
> -        sub->cmp.symbol = symbol;
> -        sub->cmp.string = xstrdup(string);
> -        ovs_list_push_back(&expr->andor, &sub->node);
> -    }
> -    sset_destroy(&result);
> -    return expr_fix(expr);
> -}
> -
> -/* Implementation of crush_cmps() for expr->type == EXPR_T_AND and a
> - * numeric-typed 'symbol'. */
> -static struct expr *
> -crush_and_numeric(struct expr *expr, const struct expr_symbol *symbol)
> -{
> -    ovs_assert(!ovs_list_is_short(&expr->andor));
> -
> -    union mf_subvalue value, mask;
> -    memset(&value, 0, sizeof value);
> -    memset(&mask, 0, sizeof mask);
> -
> -    struct expr *sub, *next = NULL;
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        ovs_list_remove(&sub->node);
> -        struct expr *new = crush_cmps(sub, symbol);
> -        switch (new->type) {
> -        case EXPR_T_CMP:
> -            if (!mf_subvalue_intersect(&value, &mask,
> -                                       &new->cmp.value, &new->cmp.mask,
> -                                       &value, &mask)) {
> -                expr_destroy(new);
> -                expr_destroy(expr);
> -                return expr_create_boolean(false);
> -            }
> -            expr_destroy(new);
> -            break;
> -        case EXPR_T_AND:
> -            OVS_NOT_REACHED();
> -        case EXPR_T_OR:
> -            ovs_list_insert(&next->node, &new->node);
> -            break;
> -        case EXPR_T_BOOLEAN:
> -            if (!new->boolean) {
> -                expr_destroy(expr);
> -                return new;
> -            }
> -            expr_destroy(new);
> -            break;
> -        case EXPR_T_CONDITION:
> -            OVS_NOT_REACHED();
> -        }
> -    }
> -    if (ovs_list_is_empty(&expr->andor)) {
> -        if (is_all_zeros(&mask, sizeof mask)) {
> -            expr_destroy(expr);
> -            return expr_create_boolean(true);
> -        } else {
> -            struct expr *cmp;
> -            cmp = xmalloc(sizeof *cmp);
> -            cmp->type = EXPR_T_CMP;
> -            cmp->cmp.symbol = symbol;
> -            cmp->cmp.relop = EXPR_R_EQ;
> -            cmp->cmp.value = value;
> -            cmp->cmp.mask = mask;
> -            expr_destroy(expr);
> -            return cmp;
> -        }
> -    } else if (ovs_list_is_short(&expr->andor)) {
> -        /* Transform "a && (b || c || d)" into "ab || ac || ad" where
> "ab" is
> -         * computed as "a && b", etc. */
> -        struct expr *disjuncts =
> expr_from_node(ovs_list_pop_front(&expr->andor));
> -        struct expr *or;
> -
> -        or = xmalloc(sizeof *or);
> -        or->type = EXPR_T_OR;
> -        ovs_list_init(&or->andor);
> -
> -        ovs_assert(disjuncts->type == EXPR_T_OR);
> -        LIST_FOR_EACH_SAFE (sub, next, node, &disjuncts->andor) {
> -            ovs_assert(sub->type == EXPR_T_CMP);
> -            ovs_list_remove(&sub->node);
> -            if (mf_subvalue_intersect(&value, &mask,
> -                                      &sub->cmp.value, &sub->cmp.mask,
> -                                      &sub->cmp.value, &sub->cmp.mask)) {
> -                ovs_list_push_back(&or->andor, &sub->node);
> -            } else {
> -                expr_destroy(sub);
> -            }
> -        }
> -        free(disjuncts);
> -        free(expr);
> -        if (ovs_list_is_empty(&or->andor)) {
> -            free(or);
> -            return expr_create_boolean(false);
> -        } else if (ovs_list_is_short(&or->andor)) {
> -            struct expr *cmp =
> expr_from_node(ovs_list_pop_front(&or->andor));
> -            free(or);
> -            return cmp;
> -        } else {
> -            return or;
> -        }
> -    } else {
> -        /* Transform "x && (a0 || a1) && (b0 || b1) && ..." into
> -         *           "(xa0b0 || xa0b1 || xa1b0 || xa1b1) && ...". */
> -        struct expr *as =
> expr_from_node(ovs_list_pop_front(&expr->andor));
> -        struct expr *bs =
> expr_from_node(ovs_list_pop_front(&expr->andor));
> -        struct expr *new = NULL;
> -        struct expr *or;
> -
> -        or = xmalloc(sizeof *or);
> -        or->type = EXPR_T_OR;
> -        ovs_list_init(&or->andor);
> -
> -        struct expr *a;
> -        LIST_FOR_EACH (a, node, &as->andor) {
> -            union mf_subvalue a_value, a_mask;
> -
> -            ovs_assert(a->type == EXPR_T_CMP);
> -            if (!mf_subvalue_intersect(&value, &mask,
> -                                       &a->cmp.value, &a->cmp.mask,
> -                                       &a_value, &a_mask)) {
> -                continue;
> -            }
> -
> -            struct expr *b;
> -            LIST_FOR_EACH (b, node, &bs->andor) {
> -                ovs_assert(b->type == EXPR_T_CMP);
> -                if (!new) {
> -                    new = xmalloc(sizeof *new);
> -                    new->type = EXPR_T_CMP;
> -                    new->cmp.symbol = symbol;
> -                    new->cmp.relop = EXPR_R_EQ;
> -                }
> -                if (mf_subvalue_intersect(&a_value, &a_mask,
> -                                          &b->cmp.value, &b->cmp.mask,
> -                                          &new->cmp.value,
> &new->cmp.mask)) {
> -                    ovs_list_push_back(&or->andor, &new->node);
> -                    new = NULL;
> -                }
> -            }
> -        }
> -        expr_destroy(as);
> -        expr_destroy(bs);
> -        free(new);
> -
> -        if (ovs_list_is_empty(&or->andor)) {
> -            expr_destroy(expr);
> -            free(or);
> -            return expr_create_boolean(false);
> -        } else if (ovs_list_is_short(&or->andor)) {
> -            struct expr *cmp =
> expr_from_node(ovs_list_pop_front(&or->andor));
> -            free(or);
> -            if (ovs_list_is_empty(&expr->andor)) {
> -                expr_destroy(expr);
> -                return crush_cmps(cmp, symbol);
> -            } else {
> -                return crush_cmps(expr_combine(EXPR_T_AND, cmp, expr),
> symbol);
> -            }
> -        } else if (!ovs_list_is_empty(&expr->andor)) {
> -            struct expr *e = expr_combine(EXPR_T_AND, or, expr);
> -            ovs_assert(!ovs_list_is_short(&e->andor));
> -            return crush_cmps(e, symbol);
> -        } else {
> -            expr_destroy(expr);
> -            return crush_cmps(or, symbol);
> -        }
> -    }
> -}
> -
> -static int
> -compare_cmps_3way(const struct expr *a, const struct expr *b)
> -{
> -    ovs_assert(a->cmp.symbol == b->cmp.symbol);
> -    if (!a->cmp.symbol->width) {
> -        return strcmp(a->cmp.string, b->cmp.string);
> -    } else {
> -        int d = memcmp(&a->cmp.value, &b->cmp.value, sizeof a->cmp.value);
> -        if (!d) {
> -            d = memcmp(&a->cmp.mask, &b->cmp.mask, sizeof a->cmp.mask);
> -        }
> -        return d;
> -    }
> -}
> -
> -static int
> -compare_cmps_cb(const void *a_, const void *b_)
> -{
> -    const struct expr *const *ap = a_;
> -    const struct expr *const *bp = b_;
> -    const struct expr *a = *ap;
> -    const struct expr *b = *bp;
> -    return compare_cmps_3way(a, b);
> -}
> -
> -/* Implementation of crush_cmps() for expr->type == EXPR_T_OR. */
> -static struct expr *
> -crush_or(struct expr *expr, const struct expr_symbol *symbol)
> -{
> -    struct expr *sub, *next = NULL;
> -
> -    /* First, crush all the subexpressions.  That might eliminate the
> -     * OR-expression entirely; if so, return the result.  Otherwise,
> 'expr'
> -     * is now a disjunction of cmps over the same symbol. */
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        ovs_list_remove(&sub->node);
> -        expr_insert_andor(expr, next, crush_cmps(sub, symbol));
> -    }
> -    expr = expr_fix(expr);
> -    if (expr->type != EXPR_T_OR) {
> -        return expr;
> -    }
> -
> -    /* Sort subexpressions by value and mask, to bring together
> duplicates. */
> -    size_t n = ovs_list_size(&expr->andor);
> -    struct expr **subs = xmalloc(n * sizeof *subs);
> -
> -    size_t i = 0;
> -    LIST_FOR_EACH (sub, node, &expr->andor) {
> -        subs[i++] = sub;
> -    }
> -    ovs_assert(i == n);
> -
> -    qsort(subs, n, sizeof *subs, compare_cmps_cb);
> -
> -    /* Eliminate duplicates. */
> -    ovs_list_init(&expr->andor);
> -    ovs_list_push_back(&expr->andor, &subs[0]->node);
> -    for (i = 1; i < n; i++) {
> -        struct expr *a = expr_from_node(ovs_list_back(&expr->andor));
> -        struct expr *b = subs[i];
> -        if (compare_cmps_3way(a, b)) {
> -            ovs_list_push_back(&expr->andor, &b->node);
> -        } else {
> -            expr_destroy(b);
> -        }
> -    }
> -    free(subs);
> -    return expr_fix(expr);
> -}
> -
> -/* Takes ownership of 'expr', which must have a unique symbol in the
> sense of
> - * 'expr_get_unique_symbol(expr)', where 'symbol' is the symbol returned
> by
> - * that function.  Returns an equivalent expression owned by the caller
> that is
> - * a single EXPR_T_CMP or a disjunction of them or a EXPR_T_BOOLEAN. */
> -static struct expr *
> -crush_cmps(struct expr *expr, const struct expr_symbol *symbol)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_OR:
> -        return crush_or(expr, symbol);
> -
> -    case EXPR_T_AND:
> -        return (symbol->width
> -                ? crush_and_numeric(expr, symbol)
> -                : crush_and_string(expr, symbol));
> -
> -    case EXPR_T_CMP:
> -        return expr;
> -
> -    case EXPR_T_BOOLEAN:
> -        return expr;
> -
> -    /* Should not hit expression type condition, since crush_cmps is only
> -     * called during expr_normalize, after expr_simplify which resolves
> -     * all conditions. */
> -    case EXPR_T_CONDITION:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Applied to an EXPR_T_AND 'expr' whose subexpressions are in terms of
> only
> - * EXPR_T_CMP, EXPR_T_AND, and EXPR_T_OR, this takes ownership of 'expr'
> and
> - * returns a new expression in terms of EXPR_T_CMP, EXPR_T_AND,
> EXPR_T_OR, or
> - * EXPR_T_BOOLEAN.
> - *
> - * The function attempts to bring together and combine clauses of the
> original
> - * 'expr' that were in terms of a single variable.  For example, it
> combines
> - * (x[0] == 1 && x[1] == 1) into the single x[0..1] == 3. */
> -static struct expr *
> -expr_sort(struct expr *expr)
> -{
> -    ovs_assert(expr->type == EXPR_T_AND);
> -
> -    size_t n = ovs_list_size(&expr->andor);
> -    struct expr_sort *subs = xmalloc(n * sizeof *subs);
> -    struct expr *sub;
> -    size_t i;
> -
> -    i = 0;
> -    LIST_FOR_EACH (sub, node, &expr->andor) {
> -        subs[i].expr = sub;
> -        subs[i].symbol = expr_get_unique_symbol(sub);
> -        subs[i].type = subs[i].symbol ? EXPR_T_CMP : sub->type;
> -        i++;
> -    }
> -    ovs_assert(i == n);
> -
> -    qsort(subs, n, sizeof *subs, compare_expr_sort);
> -
> -    ovs_list_init(&expr->andor);
> -    free(expr);
> -    expr = NULL;
> -
> -    for (i = 0; i < n; ) {
> -        if (subs[i].symbol) {
> -            size_t j;
> -            for (j = i + 1; j < n; j++) {
> -                if (subs[i].symbol != subs[j].symbol) {
> -                    break;
> -                }
> -            }
> -
> -            struct expr *crushed;
> -            if (j == i + 1) {
> -                crushed = crush_cmps(subs[i].expr, subs[i].symbol);
> -            } else {
> -                struct expr *combined = subs[i].expr;
> -                for (size_t k = i + 1; k < j; k++) {
> -                    combined = expr_combine(EXPR_T_AND, combined,
> -                                            subs[k].expr);
> -                }
> -                ovs_assert(!ovs_list_is_short(&combined->andor));
> -                crushed = crush_cmps(combined, subs[i].symbol);
> -            }
> -            if (crushed->type == EXPR_T_BOOLEAN) {
> -                if (!crushed->boolean) {
> -                    for (size_t k = j; k < n; k++) {
> -                        expr_destroy(subs[k].expr);
> -                    }
> -                    expr_destroy(expr);
> -                    expr = crushed;
> -                    break;
> -                } else {
> -                    free(crushed);
> -                }
> -            } else {
> -                expr = expr_combine(EXPR_T_AND, expr, crushed);
> -            }
> -            i = j;
> -        } else {
> -            expr = expr_combine(EXPR_T_AND, expr, subs[i++].expr);
> -        }
> -    }
> -    free(subs);
> -
> -    return expr;
> -}
> -
> -static struct expr *expr_normalize_or(struct expr *expr);
> -
> -/* Returns 'expr', which is an AND, reduced to OR(AND(clause)) where
> - * a clause is a cmp or a disjunction of cmps on a single field. */
> -static struct expr *
> -expr_normalize_and(struct expr *expr)
> -{
> -    expr = expr_sort(expr);
> -    if (expr->type != EXPR_T_AND) {
> -        return expr;
> -    }
> -
> -    struct expr *a, *b;
> -    LIST_FOR_EACH_SAFE (a, b, node, &expr->andor) {
> -        if (&b->node == &expr->andor
> -            || a->type != EXPR_T_CMP || b->type != EXPR_T_CMP
> -            || a->cmp.symbol != b->cmp.symbol) {
> -            continue;
> -        } else if (a->cmp.symbol->width
> -                   ? mf_subvalue_intersect(&a->cmp.value, &a->cmp.mask,
> -                                           &b->cmp.value, &b->cmp.mask,
> -                                           &b->cmp.value, &b->cmp.mask)
> -                   : !strcmp(a->cmp.string, b->cmp.string)) {
> -            ovs_list_remove(&a->node);
> -            expr_destroy(a);
> -        } else {
> -            expr_destroy(expr);
> -            return expr_create_boolean(false);
> -        }
> -    }
> -    if (ovs_list_is_short(&expr->andor)) {
> -        struct expr *sub = expr_from_node(ovs_list_front(&expr->andor));
> -        free(expr);
> -        return sub;
> -    }
> -
> -    struct expr *sub;
> -    LIST_FOR_EACH (sub, node, &expr->andor) {
> -        if (sub->type == EXPR_T_CMP) {
> -            continue;
> -        }
> -
> -        ovs_assert(sub->type == EXPR_T_OR);
> -        const struct expr_symbol *symbol = expr_get_unique_symbol(sub);
> -        if (!symbol || symbol->must_crossproduct) {
> -            struct expr *or = expr_create_andor(EXPR_T_OR);
> -            struct expr *k;
> -
> -            LIST_FOR_EACH (k, node, &sub->andor) {
> -                struct expr *and = expr_create_andor(EXPR_T_AND);
> -                struct expr *m;
> -
> -                LIST_FOR_EACH (m, node, &expr->andor) {
> -                    struct expr *term = m == sub ? k : m;
> -                    if (term->type == EXPR_T_AND) {
> -                        struct expr *p;
> -
> -                        LIST_FOR_EACH (p, node, &term->andor) {
> -                            struct expr *new = expr_clone(p);
> -                            ovs_list_push_back(&and->andor, &new->node);
> -                        }
> -                    } else {
> -                        struct expr *new = expr_clone(term);
> -                        ovs_list_push_back(&and->andor, &new->node);
> -                    }
> -                }
> -                ovs_list_push_back(&or->andor, &and->node);
> -            }
> -            expr_destroy(expr);
> -            return expr_normalize_or(or);
> -        }
> -    }
> -    return expr;
> -}
> -
> -static struct expr *
> -expr_normalize_or(struct expr *expr)
> -{
> -    struct expr *sub, *next;
> -
> -    LIST_FOR_EACH_SAFE (sub, next, node, &expr->andor) {
> -        if (sub->type == EXPR_T_AND) {
> -            ovs_list_remove(&sub->node);
> -
> -            struct expr *new = expr_normalize_and(sub);
> -            if (new->type == EXPR_T_BOOLEAN) {
> -                if (new->boolean) {
> -                    expr_destroy(expr);
> -                    return new;
> -                }
> -                free(new);
> -            } else {
> -                expr_insert_andor(expr, next, new);
> -            }
> -        } else {
> -            ovs_assert(sub->type == EXPR_T_CMP);
> -        }
> -    }
> -    if (ovs_list_is_empty(&expr->andor)) {
> -        free(expr);
> -        return expr_create_boolean(false);
> -    }
> -    if (ovs_list_is_short(&expr->andor)) {
> -        struct expr *e = expr_from_node(ovs_list_pop_front(&expr->andor));
> -        free(expr);
> -        return e;
> -    }
> -
> -    return expr;
> -}
> -
> -/* Takes ownership of 'expr', which is either a constant "true" or
> "false" or
> - * an expression in terms of only relationals, AND, and OR.  Returns
> either a
> - * constant "true" or "false" or 'expr' reduced to OR(AND(clause)) where a
> - * clause is a cmp or a disjunction of cmps on a single field.  This form
> is
> - * significant because it is a form that can be directly converted to
> OpenFlow
> - * flows with the Open vSwitch "conjunctive match" extension.
> - *
> - * 'expr' must already have been simplified, with expr_simplify(). */
> -struct expr *
> -expr_normalize(struct expr *expr)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return expr;
> -
> -    case EXPR_T_AND:
> -        return expr_normalize_and(expr);
> -
> -    case EXPR_T_OR:
> -        return expr_normalize_or(expr);
> -
> -    case EXPR_T_BOOLEAN:
> -        return expr;
> -
> -    /* Should not hit expression type condition, since expr_normalize is
> -     * only called after expr_simplify, which resolves all conditions. */
> -    case EXPR_T_CONDITION:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Creates, initializes, and returns a new 'struct expr_match'.  If 'm' is
> - * nonnull then it is copied into the new expr_match, otherwise the new
> - * expr_match's 'match' member is initialized to a catch-all match for the
> - * caller to refine in-place.
> - *
> - * If 'conj_id' is nonzero, adds one conjunction based on 'conj_id',
> 'clause',
> - * and 'n_clauses' to the returned 'struct expr_match', otherwise the
> - * expr_match will not have any conjunctions.
> - *
> - * The caller should use expr_match_add() to add the expr_match to a hash
> table
> - * after it is finalized. */
> -static struct expr_match *
> -expr_match_new(const struct match *m, uint8_t clause, uint8_t n_clauses,
> -               uint32_t conj_id)
> -{
> -    struct expr_match *match = xmalloc(sizeof *match);
> -    if (m) {
> -        match->match = *m;
> -    } else {
> -        match_init_catchall(&match->match);
> -    }
> -    if (conj_id) {
> -        match->conjunctions = xmalloc(sizeof *match->conjunctions);
> -        match->conjunctions[0].id = conj_id;
> -        match->conjunctions[0].clause = clause;
> -        match->conjunctions[0].n_clauses = n_clauses;
> -        match->n = 1;
> -        match->allocated = 1;
> -    } else {
> -        match->conjunctions = NULL;
> -        match->n = 0;
> -        match->allocated = 0;
> -    }
> -    return match;
> -}
> -
> -/* Adds 'match' to hash table 'matches', which becomes the new owner of
> - * 'match'.
> - *
> - * This might actually destroy 'match' because it gets merged together
> with
> - * some existing conjunction.*/
> -static void
> -expr_match_add(struct hmap *matches, struct expr_match *match)
> -{
> -    uint32_t hash = match_hash(&match->match, 0);
> -    struct expr_match *m;
> -
> -    HMAP_FOR_EACH_WITH_HASH (m, hmap_node, hash, matches) {
> -        if (match_equal(&m->match, &match->match)) {
> -            if (!m->n || !match->n) {
> -                free(m->conjunctions);
> -                m->conjunctions = NULL;
> -                m->n = 0;
> -                m->allocated = 0;
> -            } else {
> -                ovs_assert(match->n == 1);
> -                if (m->n >= m->allocated) {
> -                    m->conjunctions = x2nrealloc(m->conjunctions,
> -                                                 &m->allocated,
> -                                                 sizeof *m->conjunctions);
> -                }
> -                m->conjunctions[m->n++] = match->conjunctions[0];
> -            }
> -            free(match->conjunctions);
> -            free(match);
> -            return;
> -        }
> -    }
> -
> -    hmap_insert(matches, &match->hmap_node, hash);
> -}
> -
> -/* Applies EXPR_T_CMP-typed 'expr' to 'm'.  This will only work properly
> if 'm'
> - * doesn't already match on 'expr->cmp.symbol', because it replaces any
> - * existing match on that symbol instead of intersecting with it.
> - *
> - * If 'expr' is a comparison on a string field, uses 'lookup_port' and
> 'aux' to
> - * convert the string to a port number.  In such a case, if the port
> can't be
> - * found, returns false.  In all other cases, returns true. */
> -static bool
> -constrain_match(const struct expr *expr,
> -                bool (*lookup_port)(const void *aux,
> -                                    const char *port_name,
> -                                    unsigned int *portp),
> -                const void *aux, struct match *m)
> -{
> -    ovs_assert(expr->type == EXPR_T_CMP);
> -    if (expr->cmp.symbol->width) {
> -        mf_mask_subfield(expr->cmp.symbol->field, &expr->cmp.value,
> -                         &expr->cmp.mask, m);
> -    } else {
> -        unsigned int port;
> -        if (!lookup_port(aux, expr->cmp.string, &port)) {
> -            return false;
> -        }
> -
> -        struct mf_subfield sf;
> -        sf.field = expr->cmp.symbol->field;
> -        sf.ofs = 0;
> -        sf.n_bits = expr->cmp.symbol->field->n_bits;
> -
> -        union mf_subvalue x;
> -        memset(&x, 0, sizeof x);
> -        x.integer = htonll(port);
> -
> -        mf_write_subfield(&sf, &x, m);
> -    }
> -    return true;
> -}
> -
> -static bool
> -add_disjunction(const struct expr *or,
> -                bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                    unsigned int *portp),
> -                const void *aux,
> -                struct match *m, uint8_t clause, uint8_t n_clauses,
> -                uint32_t conj_id, struct hmap *matches)
> -{
> -    struct expr *sub;
> -    int n = 0;
> -
> -    ovs_assert(or->type == EXPR_T_OR);
> -    LIST_FOR_EACH (sub, node, &or->andor) {
> -        struct expr_match *match = expr_match_new(m, clause, n_clauses,
> -                                                  conj_id);
> -        if (constrain_match(sub, lookup_port, aux, &match->match)) {
> -            expr_match_add(matches, match);
> -            n++;
> -        } else {
> -            free(match->conjunctions);
> -            free(match);
> -        }
> -    }
> -
> -    /* If n == 1, then this didn't really need to be a disjunction.  Oh
> well,
> -     * that shouldn't happen much. */
> -    return n > 0;
> -}
> -
> -static void
> -add_conjunction(const struct expr *and,
> -                bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                    unsigned int *portp),
> -                const void *aux, uint32_t *n_conjsp, struct hmap *matches)
> -{
> -    struct match match;
> -    int n_clauses = 0;
> -    struct expr *sub;
> -
> -    match_init_catchall(&match);
> -
> -    ovs_assert(and->type == EXPR_T_AND);
> -    LIST_FOR_EACH (sub, node, &and->andor) {
> -        switch (sub->type) {
> -        case EXPR_T_CMP:
> -            if (!constrain_match(sub, lookup_port, aux, &match)) {
> -                return;
> -            }
> -            break;
> -        case EXPR_T_OR:
> -            n_clauses++;
> -            break;
> -        case EXPR_T_AND:
> -        case EXPR_T_BOOLEAN:
> -        case EXPR_T_CONDITION:
> -        default:
> -            OVS_NOT_REACHED();
> -        }
> -    }
> -
> -    if (!n_clauses) {
> -        expr_match_add(matches, expr_match_new(&match, 0, 0, 0));
> -    } else if (n_clauses == 1) {
> -        LIST_FOR_EACH (sub, node, &and->andor) {
> -            if (sub->type == EXPR_T_OR) {
> -                add_disjunction(sub, lookup_port, aux, &match, 0, 0, 0,
> -                                matches);
> -            }
> -        }
> -    } else {
> -        int clause = 0;
> -        (*n_conjsp)++;
> -        LIST_FOR_EACH (sub, node, &and->andor) {
> -            if (sub->type == EXPR_T_OR) {
> -                if (!add_disjunction(sub, lookup_port, aux, &match,
> clause++,
> -                                     n_clauses, *n_conjsp, matches)) {
> -                    /* This clause can't ever match, so we might as well
> skip
> -                     * adding the other clauses--the overall disjunctive
> flow
> -                     * can't ever match.  Ideally we would also back out
> all of
> -                     * the clauses we already added, but that seems like
> a lot
> -                     * of trouble for a case that might never occur in
> -                     * practice. */
> -                    return;
> -                }
> -            }
> -        }
> -
> -        /* Add the flow that matches on conj_id. */
> -        match_set_conj_id(&match, *n_conjsp);
> -        expr_match_add(matches, expr_match_new(&match, 0, 0, 0));
> -    }
> -}
> -
> -static void
> -add_cmp_flow(const struct expr *cmp,
> -             bool (*lookup_port)(const void *aux, const char *port_name,
> -                                 unsigned int *portp),
> -             const void *aux, struct hmap *matches)
> -{
> -    struct expr_match *m = expr_match_new(NULL, 0, 0, 0);
> -    if (constrain_match(cmp, lookup_port, aux, &m->match)) {
> -        expr_match_add(matches, m);
> -    } else {
> -        free(m);
> -    }
> -}
> -
> -/* Converts 'expr', which must be in the form returned by
> expr_normalize(), to
> - * a collection of Open vSwitch flows in 'matches', which this function
> - * initializes to an hmap of "struct expr_match" structures.  Returns the
> - * number of conjunctive match IDs consumed by 'matches', which uses
> - * conjunctive match IDs beginning with 0; the caller must offset or
> remap them
> - * into the desired range as necessary.
> - *
> - * The matches inserted into 'matches' will be of three distinct kinds:
> - *
> - *     - Ordinary flows.  The caller should add these OpenFlow flows with
> - *       its desired actions.
> - *
> - *     - Conjunctive flows, distinguished by 'n > 0' in the expr_match
> - *       structure.  The caller should add these OpenFlow flows with the
> - *       conjunction(id, k/n) actions as specified in the 'conjunctions'
> array,
> - *       remapping the ids.
> - *
> - *     - conj_id flows, distinguished by matching on the "conj_id"
> field.  The
> - *       caller should remap the conj_id and add the OpenFlow flow with
> its
> - *       desired actions.
> - *
> - * 'lookup_port' must be a function to map from a port name to a port
> number.
> - * When successful, 'lookup_port' stores the port number into '*portp' and
> - * returns true; when there is no port by the given name, it returns
> false.
> - * 'aux' is passed to 'lookup_port' as auxiliary data.  Any comparisons
> against
> - * string fields in 'expr' are translated into integers through this
> function.
> - * A comparison against a string that is not in 'ports' acts like a
> Boolean
> - * "false"; that is, it will always fail to match.  For a simple
> expression,
> - * this means that the overall expression always fails to match, but an
> - * expression with a disjunction on the string field might still match on
> other
> - * port names.
> - *
> - * (This treatment of string fields might be too simplistic in general,
> but it
> - * seems reasonable for now when string fields are used only for ports.)
> */
> -uint32_t
> -expr_to_matches(const struct expr *expr,
> -                bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                    unsigned int *portp),
> -                const void *aux, struct hmap *matches)
> -{
> -    uint32_t n_conjs = 0;
> -
> -    hmap_init(matches);
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        add_cmp_flow(expr, lookup_port, aux, matches);
> -        break;
> -
> -    case EXPR_T_AND:
> -        add_conjunction(expr, lookup_port, aux, &n_conjs, matches);
> -        break;
> -
> -    case EXPR_T_OR:
> -        if (expr_get_unique_symbol(expr)) {
> -            struct expr *sub;
> -
> -            LIST_FOR_EACH (sub, node, &expr->andor) {
> -                add_cmp_flow(sub, lookup_port, aux, matches);
> -            }
> -        } else {
> -            struct expr *sub;
> -
> -            LIST_FOR_EACH (sub, node, &expr->andor) {
> -                if (sub->type == EXPR_T_AND) {
> -                    add_conjunction(sub, lookup_port, aux, &n_conjs,
> matches);
> -                } else {
> -                    add_cmp_flow(sub, lookup_port, aux, matches);
> -                }
> -            }
> -        }
> -        break;
> -
> -    case EXPR_T_BOOLEAN:
> -        if (expr->boolean) {
> -            struct expr_match *m = expr_match_new(NULL, 0, 0, 0);
> -            expr_match_add(matches, m);
> -        } else {
> -            /* No match. */
> -        }
> -        break;
> -
> -    /* Should not hit expression type condition, since expr_to_matches is
> -     * only called after expr_simplify, which resolves all conditions. */
> -    case EXPR_T_CONDITION:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -    return n_conjs;
> -}
> -
> -/* Destroys all of the 'struct expr_match'es in 'matches', as well as the
> - * 'matches' hmap itself. */
> -void
> -expr_matches_destroy(struct hmap *matches)
> -{
> -    struct expr_match *m;
> -
> -    HMAP_FOR_EACH_POP (m, hmap_node, matches) {
> -        free(m->conjunctions);
> -        free(m);
> -    }
> -    hmap_destroy(matches);
> -}
> -
> -/* Prints a representation of the 'struct expr_match'es in 'matches' to
> - * 'stream'. */
> -void
> -expr_matches_print(const struct hmap *matches, FILE *stream)
> -{
> -    if (hmap_is_empty(matches)) {
> -        fputs("(no flows)\n", stream);
> -        return;
> -    }
> -
> -    const struct expr_match *m;
> -    HMAP_FOR_EACH (m, hmap_node, matches) {
> -        char *s = match_to_string(&m->match, NULL, OFP_DEFAULT_PRIORITY);
> -        fputs(s, stream);
> -        free(s);
> -
> -        if (m->n) {
> -            for (int i = 0; i < m->n; i++) {
> -                const struct cls_conjunction *c = &m->conjunctions[i];
> -                fprintf(stream, "%c conjunction(%"PRIu32", %d/%d)",
> -                        i == 0 ? ':' : ',', c->id, c->clause,
> c->n_clauses);
> -            }
> -        }
> -        putc('\n', stream);
> -    }
> -}
> -
> -/* Returns true if 'expr' honors the invariants for expressions (see the
> large
> - * comment above "struct expr" in expr.h), false otherwise. */
> -bool
> -expr_honors_invariants(const struct expr *expr)
> -{
> -    const struct expr *sub;
> -
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        if (expr->cmp.symbol->width) {
> -            for (int i = 0; i < ARRAY_SIZE(expr->cmp.value.be64); i++) {
> -                if (expr->cmp.value.be64[i] & ~expr->cmp.mask.be64[i]) {
> -                    return false;
> -                }
> -            }
> -        }
> -        return true;
> -
> -    case EXPR_T_AND:
> -    case EXPR_T_OR:
> -        if (ovs_list_is_short(&expr->andor)) {
> -            return false;
> -        }
> -        LIST_FOR_EACH (sub, node, &expr->andor) {
> -            if (sub->type == expr->type || !expr_honors_invariants(sub)) {
> -                return false;
> -            }
> -        }
> -        return true;
> -
> -    case EXPR_T_BOOLEAN:
> -    case EXPR_T_CONDITION:
> -        return true;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static bool
> -expr_is_normalized_and(const struct expr *expr)
> -{
> -    /* XXX should also check that no symbol is repeated. */
> -    const struct expr *sub;
> -
> -    LIST_FOR_EACH (sub, node, &expr->andor) {
> -        if (!expr_get_unique_symbol(sub)) {
> -            return false;
> -        }
> -    }
> -    return true;
> -}
> -
> -/* Returns true if 'expr' is in the form returned by expr_normalize(),
> false
> - * otherwise. */
> -bool
> -expr_is_normalized(const struct expr *expr)
> -{
> -    switch (expr->type) {
> -    case EXPR_T_CMP:
> -        return true;
> -
> -    case EXPR_T_AND:
> -        return expr_is_normalized_and(expr);
> -
> -    case EXPR_T_OR:
> -        if (!expr_get_unique_symbol(expr)) {
> -            const struct expr *sub;
> -
> -            LIST_FOR_EACH (sub, node, &expr->andor) {
> -                if (!expr_get_unique_symbol(sub)
> -                    && !expr_is_normalized_and(sub)) {
> -                    return false;
> -                }
> -            }
> -        }
> -        return true;
> -
> -    case EXPR_T_BOOLEAN:
> -        return true;
> -
> -    case EXPR_T_CONDITION:
> -        return false;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static bool
> -expr_evaluate_andor(const struct expr *e, const struct flow *f,
> -                    bool short_circuit,
> -                    bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                        unsigned int *portp),
> -                    const void *aux)
> -{
> -    const struct expr *sub;
> -
> -    LIST_FOR_EACH (sub, node, &e->andor) {
> -        if (expr_evaluate(sub, f, lookup_port, aux) == short_circuit) {
> -            return short_circuit;
> -        }
> -    }
> -    return !short_circuit;
> -}
> -
> -static bool
> -expr_evaluate_cmp(const struct expr *e, const struct flow *f,
> -                  bool (*lookup_port)(const void *aux, const char
> *port_name,
> -                                      unsigned int *portp),
> -                  const void *aux)
> -{
> -    const struct expr_symbol *s = e->cmp.symbol;
> -    const struct mf_field *field = s->field;
> -
> -    int cmp;
> -    if (e->cmp.symbol->width) {
> -        int n_bytes = field->n_bytes;
> -        const uint8_t *cst = &e->cmp.value.u8[sizeof e->cmp.value -
> n_bytes];
> -        const uint8_t *mask = &e->cmp.mask.u8[sizeof e->cmp.mask -
> n_bytes];
> -
> -        /* Get field value and mask off undesired bits. */
> -        union mf_value value;
> -        mf_get_value(field, f, &value);
> -        for (int i = 0; i < field->n_bytes; i++) {
> -            value.b[i] &= mask[i];
> -        }
> -
> -        /* Compare against constant. */
> -        cmp = memcmp(&value, cst, n_bytes);
> -    } else {
> -        /* Get field value. */
> -        struct mf_subfield sf = { .field = field, .ofs = 0,
> -                                  .n_bits = field->n_bits };
> -        uint64_t value = mf_get_subfield(&sf, f);
> -
> -        /* Get constant. */
> -        unsigned int cst;
> -        if (!lookup_port(aux, e->cmp.string, &cst)) {
> -            return false;
> -        }
> -
> -        /* Compare. */
> -        cmp = value < cst ? -1 : value > cst;
> -    }
> -
> -    return expr_relop_test(e->cmp.relop, cmp);
> -}
> -
> -/* Evaluates 'e' against microflow 'uflow' and returns the result.
> - *
> - * 'lookup_port' must be a function to map from a port name to a port
> number
> - * and 'aux' auxiliary data to pass to it; see expr_to_matches() for more
> - * details.
> - *
> - * This isn't particularly fast.  For performance-sensitive tasks, use
> - * expr_to_matches() and the classifier. */
> -bool
> -expr_evaluate(const struct expr *e, const struct flow *uflow,
> -              bool (*lookup_port)(const void *aux, const char *port_name,
> -                                  unsigned int *portp),
> -              const void *aux)
> -{
> -    switch (e->type) {
> -    case EXPR_T_CMP:
> -        return expr_evaluate_cmp(e, uflow, lookup_port, aux);
> -
> -    case EXPR_T_AND:
> -        return expr_evaluate_andor(e, uflow, false, lookup_port, aux);
> -
> -    case EXPR_T_OR:
> -        return expr_evaluate_andor(e, uflow, true, lookup_port, aux);
> -
> -    case EXPR_T_BOOLEAN:
> -        return e->boolean;
> -
> -    case EXPR_T_CONDITION:
> -        /* Assume tests calling expr_evaluate are not chassis specific, so
> -         * is_chassis_resident evaluates as true. */
> -        return (e->cond.not ? false : true);
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Action parsing helper. */
> -
> -/* Checks that 'f' is 'n_bits' wide (where 'n_bits == 0' means that 'f'
> must be
> - * a string field) and, if 'rw' is true, that 'f' is modifiable.  Returns
> NULL
> - * if 'f' is acceptable, otherwise a malloc()'d error message that the
> caller
> - * must free(). */
> -char * OVS_WARN_UNUSED_RESULT
> -expr_type_check(const struct expr_field *f, int n_bits, bool rw)
> -{
> -    if (n_bits != f->n_bits) {
> -        if (n_bits && f->n_bits) {
> -            return xasprintf("Cannot use %d-bit field %s[%d..%d] "
> -                             "where %d-bit field is required.",
> -                             f->n_bits, f->symbol->name,
> -                             f->ofs, f->ofs + f->n_bits - 1,
> -                             n_bits);
> -        } else if (n_bits) {
> -            return xasprintf("Cannot use string field %s where numeric "
> -                             "field is required.", f->symbol->name);
> -        } else {
> -            return xasprintf("Cannot use numeric field %s where string "
> -                             "field is required.", f->symbol->name);
> -        }
> -    }
> -
> -    if (rw && !f->symbol->rw) {
> -        return xasprintf("Field %s is not modifiable.", f->symbol->name);
> -    }
> -
> -    return NULL;
> -}
> -
> -/* Returns the mf_subfield that corresponds to 'f'. */
> -struct mf_subfield
> -expr_resolve_field(const struct expr_field *f)
> -{
> -    const struct expr_symbol *symbol = f->symbol;
> -    int ofs = f->ofs;
> -
> -    while (symbol->parent) {
> -        ofs += symbol->parent_ofs;
> -        symbol = symbol->parent;
> -    }
> -
> -    int n_bits = symbol->width ? f->n_bits : symbol->field->n_bits;
> -    return (struct mf_subfield) { symbol->field, ofs, n_bits };
> -}
> -
> -static bool
> -microflow_is_chassis_resident_cb(const void *c_aux OVS_UNUSED,
> -                                 const char *port_name OVS_UNUSED)
> -{
> -    /* Assume tests calling expr_parse_microflow are not chassis
> specific, so
> -     * is_chassis_resident need not be supplied and should return true. */
> -    return true;
> -}
> -
> -static struct expr *
> -expr_parse_microflow__(struct lexer *lexer,
> -                       const struct shash *symtab,
> -                       bool (*lookup_port)(const void *aux,
> -                                           const char *port_name,
> -                                           unsigned int *portp),
> -                       const void *aux,
> -                       struct expr *e, struct flow *uflow)
> -{
> -    char *error;
> -    e = expr_annotate(e, symtab, &error);
> -    if (error) {
> -        lexer_error(lexer, "%s", error);
> -        free(error);
> -        return NULL;
> -    }
> -
> -    struct ds annotated = DS_EMPTY_INITIALIZER;
> -    expr_format(e, &annotated);
> -
> -    e = expr_simplify(e, microflow_is_chassis_resident_cb, NULL);
> -    e = expr_normalize(e);
> -
> -    struct match m = MATCH_CATCHALL_INITIALIZER;
> -
> -    switch (e->type) {
> -    case EXPR_T_BOOLEAN:
> -        if (!e->boolean) {
> -            lexer_error(lexer, "Constraints are contradictory.");
> -        }
> -        break;
> -
> -    case EXPR_T_OR:
> -        lexer_error(lexer, "Constraints are ambiguous: %s.",
> -                    ds_cstr(&annotated));
> -        break;
> -
> -    case EXPR_T_CMP:
> -        constrain_match(e, lookup_port, aux, &m);
> -        break;
> -
> -    case EXPR_T_AND: {
> -        struct expr *sub;
> -        LIST_FOR_EACH (sub, node, &e->andor) {
> -            if (sub->type == EXPR_T_CMP) {
> -                constrain_match(sub, lookup_port, aux, &m);
> -            } else {
> -                ovs_assert(sub->type == EXPR_T_OR);
> -                lexer_error(lexer, "Constraints are ambiguous: %s.",
> -                            ds_cstr(&annotated));
> -                break;
> -            }
> -        }
> -    }
> -        break;
> -
> -    /* Should not hit expression type condition, since
> -     * expr_simplify was called above. */
> -    case EXPR_T_CONDITION:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -    ds_destroy(&annotated);
> -
> -    *uflow = m.flow;
> -    return e;
> -}
> -
> -/* Parses 's' as a microflow, using symbols from 'symtab', address set
> - * table from 'addr_sets', and looking up port numbers using 'lookup_port'
> - * and 'aux'.  On success, stores the result in 'uflow' and returns
> - * NULL, otherwise zeros 'uflow' and returns an error message that the
> - * caller must free().
> - *
> - * A "microflow" is a description of a single stream of packets, such as
> half a
> - * TCP connection.  's' uses the syntax of an OVN logical expression to
> express
> - * constraints that describe the microflow.  For example, "ip4 && tcp.src
> ==
> - * 80" would set uflow->dl_type to ETH_TYPE_IP, uflow->nw_proto to
> IPPROTO_TCP,
> - * and uflow->tp_src to 80.
> - *
> - * Microflow expressions can be erroneous in two ways.  First, they can be
> - * ambiguous.  For example, "tcp.src == 80" is ambiguous because it does
> not
> - * state IPv4 or IPv6 as the Ethernet type.  "ip4 && tcp.src > 1024" is
> also
> - * ambiguous because it does not constrain bits of tcp.src to particular
> - * values.  Second, they can be contradictory, e.g. "ip4 && ip6".  This
> - * function will report both types of errors.
> - *
> - * This function isn't that smart, so it can yield errors for some
> "clever"
> - * formulations of particular microflows that area accepted other ways.
> For
> - * example, all of the following expressions are equivalent:
> - *     ip4 && tcp.src[1..15] == 0x28
> - *     ip4 && tcp.src > 79 && tcp.src < 82
> - *     ip4 && 80 <= tcp.src <= 81
> - *     ip4 && tcp.src == {80, 81}
> - * but as of this writing this function only accepts the first two,
> rejecting
> - * the last two as ambiguous.  Just don't be too clever. */
> -char * OVS_WARN_UNUSED_RESULT
> -expr_parse_microflow(const char *s, const struct shash *symtab,
> -                     const struct shash *addr_sets,
> -                     const struct shash *port_groups,
> -                     bool (*lookup_port)(const void *aux,
> -                                         const char *port_name,
> -                                         unsigned int *portp),
> -                     const void *aux, struct flow *uflow)
> -{
> -    struct lexer lexer;
> -    lexer_init(&lexer, s);
> -    lexer_get(&lexer);
> -
> -    struct expr *e = expr_parse(&lexer, symtab, addr_sets, port_groups,
> NULL);
> -    lexer_force_end(&lexer);
> -
> -    if (e) {
> -        e = expr_parse_microflow__(&lexer, symtab, lookup_port, aux, e,
> uflow);
> -    }
> -
> -    char *error = lexer_steal_error(&lexer);
> -    lexer_destroy(&lexer);
> -    expr_destroy(e);
> -
> -    if (error) {
> -        memset(uflow, 0, sizeof *uflow);
> -    }
> -    return error;
> -}
> diff --git a/ovn/lib/extend-table.c b/ovn/lib/extend-table.c
> deleted file mode 100644
> index ccf70ca72..000000000
> --- a/ovn/lib/extend-table.c
> +++ /dev/null
> @@ -1,208 +0,0 @@
> -/*
> - * Copyright (c) 2017 DtDream Technology Co.,Ltd.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include <string.h>
> -
> -#include "bitmap.h"
> -#include "hash.h"
> -#include "lib/uuid.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/extend-table.h"
> -
> -VLOG_DEFINE_THIS_MODULE(extend_table);
> -
> -void
> -ovn_extend_table_init(struct ovn_extend_table *table)
> -{
> -    table->table_ids = bitmap_allocate(MAX_EXT_TABLE_ID);
> -    bitmap_set1(table->table_ids, 0); /* table id 0 is invalid. */
> -    hmap_init(&table->desired);
> -    hmap_init(&table->existing);
> -}
> -
> -static void
> -ovn_extend_table_info_destroy(struct hmap *target)
> -{
> -    struct ovn_extend_table_info *e, *next;
> -    HMAP_FOR_EACH_SAFE (e, next, hmap_node, target) {
> -        hmap_remove(target, &e->hmap_node);
> -        free(e->name);
> -        free(e);
> -    }
> -    hmap_destroy(target);
> -}
> -
> -void
> -ovn_extend_table_destroy(struct ovn_extend_table *table)
> -{
> -    bitmap_free(table->table_ids);
> -
> -    ovn_extend_table_info_destroy(&table->desired);
> -    ovn_extend_table_info_destroy(&table->existing);
> -}
> -
> -/* Finds and returns a group_info in 'existing' whose key is identical
> - * to 'target''s key, or NULL if there is none. */
> -struct ovn_extend_table_info *
> -ovn_extend_table_lookup(struct hmap *exisiting,
> -                        const struct ovn_extend_table_info *target)
> -{
> -    struct ovn_extend_table_info *e;
> -
> -    HMAP_FOR_EACH_WITH_HASH (e, hmap_node, target->hmap_node.hash,
> -                             exisiting) {
> -        if (e->table_id == target->table_id) {
> -            return e;
> -        }
> -   }
> -    return NULL;
> -}
> -
> -/* Clear either desired or existing in ovn_extend_table. */
> -void
> -ovn_extend_table_clear(struct ovn_extend_table *table, bool existing)
> -{
> -    struct ovn_extend_table_info *g, *next;
> -    struct hmap *target = existing ? &table->existing : &table->desired;
> -
> -    HMAP_FOR_EACH_SAFE (g, next, hmap_node, target) {
> -        hmap_remove(target, &g->hmap_node);
> -        /* Don't unset bitmap for desired group_info if the group_id
> -         * was not freshly reserved. */
> -        if (existing || g->new_table_id) {
> -            bitmap_set0(table->table_ids, g->table_id);
> -        }
> -        free(g->name);
> -        free(g);
> -    }
> -}
> -
> -/* Remove an entry from existing table */
> -void
> -ovn_extend_table_remove_existing(struct ovn_extend_table *table,
> -                                 struct ovn_extend_table_info *existing)
> -{
> -    /* Remove 'existing' from 'groups->existing' */
> -    hmap_remove(&table->existing, &existing->hmap_node);
> -    free(existing->name);
> -
> -    /* Dealloc group_id. */
> -    bitmap_set0(table->table_ids, existing->table_id);
> -    free(existing);
> -}
> -
> -/* Remove entries in desired table that are created by the lflow_uuid */
> -void
> -ovn_extend_table_remove_desired(struct ovn_extend_table *table,
> -                                const struct uuid *lflow_uuid)
> -{
> -    struct ovn_extend_table_info *e, *next_e;
> -    HMAP_FOR_EACH_SAFE (e, next_e, hmap_node, &table->desired) {
> -        if (uuid_equals(&e->lflow_uuid, lflow_uuid)) {
> -            hmap_remove(&table->desired, &e->hmap_node);
> -            free(e->name);
> -            if (e->new_table_id) {
> -                bitmap_set0(table->table_ids, e->table_id);
> -            }
> -            free(e);
> -        }
> -    }
> -
> -}
> -
> -static struct ovn_extend_table_info*
> -ovn_extend_info_clone(struct ovn_extend_table_info *source)
> -{
> -    struct ovn_extend_table_info *clone = xmalloc(sizeof *clone);
> -    clone->name = xstrdup(source->name);
> -    clone->table_id = source->table_id;
> -    clone->new_table_id = source->new_table_id;
> -    clone->hmap_node.hash = source->hmap_node.hash;
> -    clone->lflow_uuid = source->lflow_uuid;
> -    return clone;
> -}
> -
> -void
> -ovn_extend_table_sync(struct ovn_extend_table *table)
> -{
> -    struct ovn_extend_table_info *desired, *next;
> -
> -    /* Copy the contents of desired to existing. */
> -    HMAP_FOR_EACH_SAFE (desired, next, hmap_node, &table->desired) {
> -        if (!ovn_extend_table_lookup(&table->existing, desired)) {
> -            desired->new_table_id = false;
> -            struct ovn_extend_table_info *clone =
> -                ovn_extend_info_clone(desired);
> -            hmap_insert(&table->existing, &clone->hmap_node,
> -                        clone->hmap_node.hash);
> -        }
> -    }
> -}
> -
> -/* Assign a new table ID for the table information from the bitmap.
> - * If it already exists, return the old ID. */
> -uint32_t
> -ovn_extend_table_assign_id(struct ovn_extend_table *table, const char
> *name,
> -                           struct uuid lflow_uuid)
> -{
> -    uint32_t table_id = 0, hash;
> -    struct ovn_extend_table_info *table_info;
> -
> -    hash = hash_string(name, 0);
> -
> -    /* Check whether we have non installed but allocated group_id. */
> -    HMAP_FOR_EACH_WITH_HASH (table_info, hmap_node, hash,
> &table->desired) {
> -        if (!strcmp(table_info->name, name) &&
> -            table_info->new_table_id) {
> -            return table_info->table_id;
> -        }
> -    }
> -
> -    /* Check whether we already have an installed entry for this
> -     * combination. */
> -    HMAP_FOR_EACH_WITH_HASH (table_info, hmap_node, hash,
> &table->existing) {
> -        if (!strcmp(table_info->name, name)) {
> -            table_id = table_info->table_id;
> -        }
> -    }
> -
> -    bool new_table_id = false;
> -    if (!table_id) {
> -        /* Reserve a new group_id. */
> -        table_id = bitmap_scan(table->table_ids, 0, 1, MAX_EXT_TABLE_ID +
> 1);
> -        new_table_id = true;
> -    }
> -
> -    if (table_id == MAX_EXT_TABLE_ID + 1) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_ERR_RL(&rl, "%"PRIu32" out of table ids.", table_id);
> -        return EXT_TABLE_ID_INVALID;
> -    }
> -    bitmap_set1(table->table_ids, table_id);
> -
> -    table_info = xmalloc(sizeof *table_info);
> -    table_info->name = xstrdup(name);
> -    table_info->table_id = table_id;
> -    table_info->hmap_node.hash = hash;
> -    table_info->new_table_id = new_table_id;
> -    table_info->lflow_uuid = lflow_uuid;
> -
> -    hmap_insert(&table->desired,
> -                &table_info->hmap_node, table_info->hmap_node.hash);
> -
> -    return table_id;
> -}
> diff --git a/ovn/lib/extend-table.h b/ovn/lib/extend-table.h
> deleted file mode 100644
> index 5be13fee1..000000000
> --- a/ovn/lib/extend-table.h
> +++ /dev/null
> @@ -1,82 +0,0 @@
> -/*
> - * Copyright (c) 2017 DtDream Technology Co.,Ltd.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef EXTEND_TABLE_H
> -#define EXTEND_TABLE_H 1
> -
> -#define MAX_EXT_TABLE_ID 65535
> -#define EXT_TABLE_ID_INVALID 0
> -
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/list.h"
> -#include "openvswitch/uuid.h"
> -
> -/* Used to manage expansion tables associated with Flow table,
> - * such as the Group Table or Meter Table. */
> -struct ovn_extend_table {
> -    unsigned long *table_ids;  /* Used as a bitmap with value set
> -                                * for allocated group ids in either
> -                                * desired or existing. */
> -    struct hmap desired;
> -    struct hmap existing;
> -};
> -
> -struct ovn_extend_table_info {
> -    struct hmap_node hmap_node;
> -    char *name;         /* Name for the table entity. */
> -    struct uuid lflow_uuid;
> -    uint32_t table_id;
> -    bool new_table_id;  /* 'True' if 'table_id' was reserved from
> -                         * ovn_extend_table's 'table_ids' bitmap. */
> -};
> -
> -void ovn_extend_table_init(struct ovn_extend_table *);
> -
> -void ovn_extend_table_destroy(struct ovn_extend_table *);
> -
> -struct ovn_extend_table_info *ovn_extend_table_lookup(
> -    struct hmap *, const struct ovn_extend_table_info *);
> -
> -void ovn_extend_table_clear(struct ovn_extend_table *, bool);
> -
> -void ovn_extend_table_remove_existing(struct ovn_extend_table *,
> -                                      struct ovn_extend_table_info *);
> -
> -void ovn_extend_table_remove_desired(struct ovn_extend_table *,
> -                                     const struct uuid *lflow_uuid);
> -
> -/* Copy the contents of desired to existing. */
> -void ovn_extend_table_sync(struct ovn_extend_table *);
> -
> -uint32_t ovn_extend_table_assign_id(struct ovn_extend_table *,
> -                                    const char *name,
> -                                    struct uuid lflow_uuid);
> -
> -/* Iterates 'DESIRED' through all of the 'ovn_extend_table_info's in
> - * 'TABLE'->desired that are not in 'TABLE'->existing.  (The loop body
> - * presumably adds them.) */
> -#define EXTEND_TABLE_FOR_EACH_UNINSTALLED(DESIRED, TABLE) \
> -    HMAP_FOR_EACH (DESIRED, hmap_node, &(TABLE)->desired) \
> -        if (!ovn_extend_table_lookup(&(TABLE)->existing, DESIRED))
> -
> -/* Iterates 'EXISTING' through all of the 'ovn_extend_table_info's in
> - * 'TABLE'->existing that are not in 'TABLE'->desired.  (The loop body
> - * presumably removes them.) */
> -#define EXTEND_TABLE_FOR_EACH_INSTALLED(EXISTING, NEXT, TABLE)         \
> -    HMAP_FOR_EACH_SAFE (EXISTING, NEXT, hmap_node, &(TABLE)->existing) \
> -        if (!ovn_extend_table_lookup(&(TABLE)->desired, EXISTING))
> -
> -#endif /* ovn/lib/extend-table.h */
> diff --git a/ovn/lib/inc-proc-eng.c b/ovn/lib/inc-proc-eng.c
> deleted file mode 100644
> index 1ddea1a85..000000000
> --- a/ovn/lib/inc-proc-eng.c
> +++ /dev/null
> @@ -1,201 +0,0 @@
> -/*
> - * Copyright (c) 2018 eBay Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <errno.h>
> -#include <getopt.h>
> -#include <signal.h>
> -#include <stdlib.h>
> -#include <string.h>
> -
> -#include "lib/util.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/vlog.h"
> -#include "inc-proc-eng.h"
> -
> -VLOG_DEFINE_THIS_MODULE(inc_proc_eng);
> -
> -static bool engine_force_recompute = false;
> -static const struct engine_context *engine_context;
> -
> -void
> -engine_set_force_recompute(bool val)
> -{
> -    engine_force_recompute = val;
> -}
> -
> -const struct engine_context *
> -engine_get_context(void)
> -{
> -    return engine_context;
> -}
> -
> -void
> -engine_set_context(const struct engine_context *ctx)
> -{
> -    engine_context = ctx;
> -}
> -
> -void
> -engine_init(struct engine_node *node)
> -{
> -    for (size_t i = 0; i < node->n_inputs; i++) {
> -        engine_init(node->inputs[i].node);
> -    }
> -    if (node->init) {
> -        node->init(node);
> -    }
> -}
> -
> -void
> -engine_cleanup(struct engine_node *node)
> -{
> -    for (size_t i = 0; i < node->n_inputs; i++) {
> -        engine_cleanup(node->inputs[i].node);
> -    }
> -    if (node->cleanup) {
> -        node->cleanup(node);
> -    }
> -}
> -
> -struct engine_node *
> -engine_get_input(const char *input_name, struct engine_node *node)
> -{
> -    size_t i;
> -    for (i = 0; i < node->n_inputs; i++) {
> -        if (!strcmp(node->inputs[i].node->name, input_name)) {
> -            return node->inputs[i].node;
> -        }
> -    }
> -    OVS_NOT_REACHED();
> -    return NULL;
> -}
> -
> -void
> -engine_add_input(struct engine_node *node, struct engine_node *input,
> -                 bool (*change_handler)(struct engine_node *))
> -{
> -    ovs_assert(node->n_inputs < ENGINE_MAX_INPUT);
> -    node->inputs[node->n_inputs].node = input;
> -    node->inputs[node->n_inputs].change_handler = change_handler;
> -    node->n_inputs ++;
> -}
> -
> -struct ovsdb_idl_index *
> -engine_ovsdb_node_get_index(struct engine_node *node, const char *name)
> -{
> -    struct ed_type_ovsdb_table *ed = (struct ed_type_ovsdb_table
> *)node->data;
> -    for (size_t i = 0; i < ed->n_indexes; i++) {
> -        if (!strcmp(ed->indexes[i].name, name)) {
> -            return ed->indexes[i].index;
> -        }
> -    }
> -    OVS_NOT_REACHED();
> -    return NULL;
> -}
> -
> -void
> -engine_ovsdb_node_add_index(struct engine_node *node, const char *name,
> -                            struct ovsdb_idl_index *index)
> -{
> -    struct ed_type_ovsdb_table *ed = (struct ed_type_ovsdb_table
> *)node->data;
> -    ovs_assert(ed->n_indexes < ENGINE_MAX_OVSDB_INDEX);
> -
> -    ed->indexes[ed->n_indexes].name = name;
> -    ed->indexes[ed->n_indexes].index = index;
> -    ed->n_indexes ++;
> -}
> -
> -void
> -engine_run(struct engine_node *node, uint64_t run_id)
> -{
> -    if (node->run_id == run_id) {
> -        return;
> -    }
> -    node->run_id = run_id;
> -
> -    node->changed = false;
> -    if (!node->n_inputs) {
> -        node->run(node);
> -        VLOG_DBG("node: %s, changed: %d", node->name, node->changed);
> -        return;
> -    }
> -
> -    for (size_t i = 0; i < node->n_inputs; i++) {
> -        engine_run(node->inputs[i].node, run_id);
> -    }
> -
> -    bool need_compute = false;
> -    bool need_recompute = false;
> -
> -    if (engine_force_recompute) {
> -        need_recompute = true;
> -    } else {
> -        for (size_t i = 0; i < node->n_inputs; i++) {
> -            if (node->inputs[i].node->changed) {
> -                need_compute = true;
> -                if (!node->inputs[i].change_handler) {
> -                    need_recompute = true;
> -                    break;
> -                }
> -            }
> -        }
> -    }
> -
> -    if (need_recompute) {
> -        VLOG_DBG("node: %s, recompute (%s)", node->name,
> -                 engine_force_recompute ? "forced" : "triggered");
> -        node->run(node);
> -    } else if (need_compute) {
> -        for (size_t i = 0; i < node->n_inputs; i++) {
> -            if (node->inputs[i].node->changed) {
> -                VLOG_DBG("node: %s, handle change for input %s",
> -                         node->name, node->inputs[i].node->name);
> -                if (!node->inputs[i].change_handler(node)) {
> -                    VLOG_DBG("node: %s, can't handle change for input %s,
> "
> -                             "fall back to recompute",
> -                             node->name, node->inputs[i].node->name);
> -                    node->run(node);
> -                    break;
> -                }
> -            }
> -        }
> -    }
> -
> -    VLOG_DBG("node: %s, changed: %d", node->name, node->changed);
> -}
> -
> -bool
> -engine_need_run(struct engine_node *node)
> -{
> -    size_t i;
> -
> -    if (!node->n_inputs) {
> -        node->run(node);
> -        VLOG_DBG("input node: %s, changed: %d", node->name,
> node->changed);
> -        return node->changed;
> -    }
> -
> -    for (i = 0; i < node->n_inputs; i++) {
> -        if (engine_need_run(node->inputs[i].node)) {
> -            return true;
> -        }
> -    }
> -
> -    return false;
> -}
> diff --git a/ovn/lib/inc-proc-eng.h b/ovn/lib/inc-proc-eng.h
> deleted file mode 100644
> index aab899e13..000000000
> --- a/ovn/lib/inc-proc-eng.h
> +++ /dev/null
> @@ -1,234 +0,0 @@
> -/*
> - * Copyright (c) 2018 eBay Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef INC_PROC_ENG_H
> -#define INC_PROC_ENG_H 1
> -
> -/* The Incremental Processing Engine is a framework for incrementally
> - * processing changes from different inputs. The main user is
> ovn-controller.
> - * To compute desired states (e.g. openflow rules) based on many inputs
> (e.g.
> - * south-bound DB tables, local OVSDB interfaces, etc.), it is
> straightforward
> - * to recompute everything when there is any change in any inputs, but it
> - * is inefficient when the size of the input data becomes large. Instead,
> - * tracking the changes and update the desired states based on what's
> changed
> - * is more efficient and scalable. However, it is not straightforward to
> - * implement the change-based processing when there are a big number of
> - * inputs. In addition, what makes it more complicated is that
> intermediate
> - * results needs to be computed, which needs to be reused in different
> part
> - * of the processing and finally generates the final desired states. It is
> - * proved to be difficult and error-prone to implement this kind of
> complex
> - * processing by ad-hoc implementation.
> - *
> - * This framework is to provide a generic way to solve the above problem.
> - * It does not understand the processing logic, but provides a unified way
> - * to describe the inputs and dependencies clearly, with interfaces for
> - * users to implement the processing logic for how to handle each input
> - * changes.
> - *
> - * The engine is composed of engine_nodes. Each engine_node is either
> - * an input, an output or both (intermediate result). Each engine node
> - * maintains its own data, which is persistent across interactions. Each
> node
> - * has zero to ENGINE_MAX_INPUT inputs, which creates a DAG (directed
> - * acyclic graph). For each input of each engine_node, there is a
> - * change_handler to process changes of that input, and update the data
> - * of the engine_node. Then the user can simply call the run() method
> - * of the engine so that the processing will happen in the order according
> - * to the dependencies defined and handle the changes incrementally.
> - *
> - * While the more fine-grained dependencies and change-handlers are
> - * implemented, the more efficient the processing will be, it is not
> - * realistic to implement all change-processing for all inputs (and
> - * intermediate results). The engine doesn't require change-handler to be
> - * implemented for every input of every node. Users can choose to
> implement
> - * the most important change-handlers (for the changes happens most
> - * frequently) for overall performance. When there is no change_handler
> - * defined for a certain input on a certain engine_node, the run() method
> - * of the engine_node will be called to fall-back to a full recompute
> - * against all its inputs.
> - */
> -
> -#define ENGINE_MAX_INPUT 256
> -#define ENGINE_MAX_OVSDB_INDEX 256
> -
> -struct engine_context {
> -    struct ovsdb_idl_txn *ovs_idl_txn;
> -    struct ovsdb_idl_txn *ovnsb_idl_txn;
> -};
> -
> -struct engine_node;
> -
> -struct engine_node_input {
> -    /* The input node. */
> -    struct engine_node *node;
> -
> -    /* Change handler for changes of the input node. The changes may need
> to be
> -     * evaluated against all the other inputs. Returns:
> -     *  - true: if change can be handled
> -     *  - false: if change cannot be handled (indicating full recompute
> needed)
> -     */
> -    bool (*change_handler)(struct engine_node *node);
> -};
> -
> -struct engine_node {
> -    /* A unique id to distinguish each iteration of the engine_run(). */
> -    uint64_t run_id;
> -
> -    /* A unique name for each node. */
> -    char *name;
> -
> -    /* Number of inputs of this node. */
> -    size_t n_inputs;
> -
> -    /* Inputs of this node. */
> -    struct engine_node_input inputs[ENGINE_MAX_INPUT];
> -
> -    /* Data of this node. It is vague and interpreted by the related
> functions.
> -     * The content of the data should be changed only by the
> change_handlers
> -     * and run() function of the current node. Users should ensure that
> the
> -     * data is read-only in change-handlers of the nodes that depends on
> this
> -     * node. */
> -    void *data;
> -
> -    /* Whether the data changed in the last engine run. */
> -    bool changed;
> -
> -    /* Method to initialize data. It may be NULL. */
> -    void (*init)(struct engine_node *);
> -
> -    /* Method to clean up data. It may be NULL. */
> -    void (*cleanup)(struct engine_node *);
> -
> -    /* Fully processes all inputs of this node and regenerates the data
> -     * of this node */
> -    void (*run)(struct engine_node *);
> -};
> -
> -/* Initialize the data for the engine nodes recursively. It calls each
> node's
> - * init() method if not NULL. It should be called before the main loop. */
> -void engine_init(struct engine_node *);
> -
> -/* Execute the processing recursively, which should be called in the main
> - * loop. */
> -void engine_run(struct engine_node *, uint64_t run_id);
> -
> -/* Clean up the data for the engine nodes recursively. It calls each
> node's
> - * cleanup() method if not NULL. It should be called before the program
> - * terminates. */
> -void engine_cleanup(struct engine_node *);
> -
> -/* Check if engine needs to run, i.e. any change to be processed. */
> -bool
> -engine_need_run(struct engine_node *);
> -
> -/* Get the input node with <name> for <node> */
> -struct engine_node * engine_get_input(const char *input_name,
> -                                      struct engine_node *);
> -
> -/* Add an input (dependency) for <node>, with corresponding
> change_handler,
> - * which can be NULL. If the change_handler is NULL, the engine will not
> - * be able to process the change incrementally, and will fall back to call
> - * the run method to recompute. */
> -void engine_add_input(struct engine_node *node, struct engine_node *input,
> -                      bool (*change_handler)(struct engine_node *));
> -
> -/* Force the engine to recompute everything if set to true. It is used
> - * in circumstances when we are not sure there is change or not, or
> - * when there is change but the engine couldn't be executed in that
> - * iteration, and the change can't be tracked across iterations */
> -void engine_set_force_recompute(bool val);
> -
> -const struct engine_context * engine_get_context(void);
> -
> -void engine_set_context(const struct engine_context *);
> -
> -struct ed_ovsdb_index {
> -    const char *name;
> -    struct ovsdb_idl_index *index;
> -};
> -
> -struct ed_type_ovsdb_table {
> -    const void *table;
> -    size_t n_indexes;
> -    struct ed_ovsdb_index indexes[ENGINE_MAX_OVSDB_INDEX];
> -};
> -
> -#define EN_OVSDB_GET(NODE) \
> -    (((struct ed_type_ovsdb_table *)NODE->data)->table)
> -
> -struct ovsdb_idl_index * engine_ovsdb_node_get_index(struct engine_node *,
> -                                                     const char *name);
> -
> -void engine_ovsdb_node_add_index(struct engine_node *, const char *name,
> -                                 struct ovsdb_idl_index *);
> -
> -/* Macro to define an engine node. */
> -#define ENGINE_NODE(NAME, NAME_STR) \
> -    struct engine_node en_##NAME = { \
> -        .name = NAME_STR, \
> -        .data = &ed_##NAME, \
> -        .init = en_##NAME##_init, \
> -        .run = en_##NAME##_run, \
> -        .cleanup = en_##NAME##_cleanup, \
> -    };
> -
> -/* Macro to define member functions of an engine node which represents
> - * a table of OVSDB */
> -#define ENGINE_FUNC_OVSDB(DB_NAME, TBL_NAME) \
> -static void \
> -en_##DB_NAME##_##TBL_NAME##_run(struct engine_node *node) \
> -{ \
> -    const struct DB_NAME##rec_##TBL_NAME##_table *table = \
> -        EN_OVSDB_GET(node); \
> -    if (DB_NAME##rec_##TBL_NAME##_table_track_get_first(table)) { \
> -        node->changed = true; \
> -        return; \
> -    } \
> -    node->changed = false; \
> -} \
> -static void (*en_##DB_NAME##_##TBL_NAME##_init)(struct engine_node *node)
> \
> -            = NULL; \
> -static void (*en_##DB_NAME##_##TBL_NAME##_cleanup)(struct engine_node
> *node) \
> -            = NULL;
> -
> -/* Macro to define member functions of an engine node which represents
> - * a table of OVN SB DB */
> -#define ENGINE_FUNC_SB(TBL_NAME) \
> -    ENGINE_FUNC_OVSDB(sb, TBL_NAME)
> -
> -/* Macro to define member functions of an engine node which represents
> - * a table of open_vswitch DB */
> -#define ENGINE_FUNC_OVS(TBL_NAME) \
> -    ENGINE_FUNC_OVSDB(ovs, TBL_NAME)
> -
> -/* Macro to define an engine node which represents a table of OVSDB */
> -#define ENGINE_NODE_OVSDB(DB_NAME, DB_NAME_STR, TBL_NAME, TBL_NAME_STR,
> IDL) \
> -    struct ed_type_ovsdb_table ed_##DB_NAME##_##TBL_NAME; \
> -    memset(&ed_##DB_NAME##_##TBL_NAME, 0, sizeof
> ed_##DB_NAME##_##TBL_NAME); \
> -    ovs_assert(IDL); \
> -    ed_##DB_NAME##_##TBL_NAME.table = \
> -        DB_NAME##rec_##TBL_NAME##_table_get(IDL); \
> -    ENGINE_NODE(DB_NAME##_##TBL_NAME, DB_NAME_STR"_"TBL_NAME_STR)
> -
> -/* Macro to define an engine node which represents a table of OVN SB DB */
> -#define ENGINE_NODE_SB(TBL_NAME, TBL_NAME_STR) \
> -    ENGINE_NODE_OVSDB(sb, "SB", TBL_NAME, TBL_NAME_STR,
> ovnsb_idl_loop.idl);
> -
> -/* Macro to define an engine node which represents a table of open_vswitch
> - * DB */
> -#define ENGINE_NODE_OVS(TBL_NAME, TBL_NAME_STR) \
> -    ENGINE_NODE_OVSDB(ovs, "OVS", TBL_NAME, TBL_NAME_STR,
> ovs_idl_loop.idl);
> -
> -#endif /* ovn/lib/inc-proc-eng.h */
> diff --git a/ovn/lib/ip-mcast-index.c b/ovn/lib/ip-mcast-index.c
> deleted file mode 100644
> index 1f6ebc4ae..000000000
> --- a/ovn/lib/ip-mcast-index.c
> +++ /dev/null
> @@ -1,40 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "ovn/lib/ip-mcast-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -struct ovsdb_idl_index *
> -ip_mcast_index_create(struct ovsdb_idl *idl)
> -{
> -    return ovsdb_idl_index_create1(idl, &sbrec_ip_multicast_col_datapath);
> -}
> -
> -const struct sbrec_ip_multicast *
> -ip_mcast_lookup(struct ovsdb_idl_index *ip_mcast_index,
> -                const struct sbrec_datapath_binding *datapath)
> -{
> -    struct sbrec_ip_multicast *target =
> -        sbrec_ip_multicast_index_init_row(ip_mcast_index);
> -    sbrec_ip_multicast_index_set_datapath(target, datapath);
> -
> -    struct sbrec_ip_multicast *ip_mcast =
> -        sbrec_ip_multicast_index_find(ip_mcast_index, target);
> -    sbrec_ip_multicast_index_destroy_row(target);
> -
> -    return ip_mcast;
> -}
> diff --git a/ovn/lib/ip-mcast-index.h b/ovn/lib/ip-mcast-index.h
> deleted file mode 100644
> index a23b4a7e6..000000000
> --- a/ovn/lib/ip-mcast-index.h
> +++ /dev/null
> @@ -1,36 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_IP_MCAST_INDEX_H
> -#define OVN_IP_MCAST_INDEX_H 1
> -
> -struct ovsdb_idl;
> -
> -struct sbrec_datapath_binding;
> -
> -#define OVN_MCAST_MIN_IDLE_TIMEOUT_S           15
> -#define OVN_MCAST_MAX_IDLE_TIMEOUT_S           3600
> -#define OVN_MCAST_DEFAULT_IDLE_TIMEOUT_S       300
> -#define OVN_MCAST_MIN_QUERY_INTERVAL_S         1
> -#define OVN_MCAST_MAX_QUERY_INTERVAL_S
>  OVN_MCAST_MAX_IDLE_TIMEOUT_S
> -#define OVN_MCAST_DEFAULT_QUERY_MAX_RESPONSE_S 1
> -#define OVN_MCAST_DEFAULT_MAX_ENTRIES          2048
> -
> -struct ovsdb_idl_index *ip_mcast_index_create(struct ovsdb_idl *);
> -const struct sbrec_ip_multicast *ip_mcast_lookup(
> -    struct ovsdb_idl_index *ip_mcast_index,
> -    const struct sbrec_datapath_binding *datapath);
> -
> -#endif /* ovn/lib/ip-mcast-index.h */
> diff --git a/ovn/lib/lex.c b/ovn/lib/lex.c
> deleted file mode 100644
> index 7a2ab4111..000000000
> --- a/ovn/lib/lex.c
> +++ /dev/null
> @@ -1,1023 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include <ctype.h>
> -#include <errno.h>
> -#include <stdarg.h>
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/json.h"
> -#include "ovn/lex.h"
> -#include "packets.h"
> -#include "util.h"
> -
> -/* Returns a string that represents 'format'. */
> -const char *
> -lex_format_to_string(enum lex_format format)
> -{
> -    switch (format) {
> -    case LEX_F_DECIMAL:
> -        return "decimal";
> -    case LEX_F_HEXADECIMAL:
> -        return "hexadecimal";
> -    case LEX_F_IPV4:
> -        return "IPv4";
> -    case LEX_F_IPV6:
> -        return "IPv6";
> -    case LEX_F_ETHERNET:
> -        return "Ethernet";
> -    default:
> -        abort();
> -    }
> -}
> -
> -/* Initializes 'token'. */
> -void
> -lex_token_init(struct lex_token *token)
> -{
> -    token->type = LEX_T_END;
> -    token->s = NULL;
> -}
> -
> -/* Frees memory owned by 'token'. */
> -void
> -lex_token_destroy(struct lex_token *token)
> -{
> -    if (token->s != token->buffer) {
> -        free(token->s);
> -    }
> -    token->s = NULL;
> -}
> -
> -/* Exchanges 'a' and 'b'. */
> -void
> -lex_token_swap(struct lex_token *a, struct lex_token *b)
> -{
> -    struct lex_token tmp = *a;
> -    *a = *b;
> -    *b = tmp;
> -
> -    /* Before swap, if 's' was pointed to 'buffer', its value shall be
> changed
> -     * to point to the 'buffer' with the copied value. */
> -    if (a->s == b->buffer) {
> -        a->s = a->buffer;
> -    }
> -    if (b->s == a->buffer) {
> -        b->s = b->buffer;
> -    }
> -}
> -
> -/* The string 's' need not be null-terminated at 'length'. */
> -void
> -lex_token_strcpy(struct lex_token *token, const char *s, size_t length)
> -{
> -    lex_token_destroy(token);
> -    token->s = (length + 1 <= sizeof token->buffer
> -                ? token->buffer
> -                : xmalloc(length + 1));
> -    memcpy(token->s, s, length);
> -    token->s[length] = '\0';
> -}
> -
> -void
> -lex_token_strset(struct lex_token *token, char *s)
> -{
> -    lex_token_destroy(token);
> -    token->s = s;
> -}
> -
> -void
> -lex_token_vsprintf(struct lex_token *token, const char *format, va_list
> args)
> -{
> -    lex_token_destroy(token);
> -
> -    va_list args2;
> -    va_copy(args2, args);
> -    token->s = (vsnprintf(token->buffer, sizeof token->buffer, format,
> args)
> -                < sizeof token->buffer
> -                ? token->buffer
> -                : xvasprintf(format, args2));
> -    va_end(args2);
> -}
> -
> -/* lex_token_format(). */
> -
> -static size_t
> -lex_token_n_zeros(enum lex_format format)
> -{
> -    switch (format) {
> -    case LEX_F_DECIMAL:     return offsetof(union mf_subvalue, integer);
> -    case LEX_F_HEXADECIMAL: return 0;
> -    case LEX_F_IPV4:        return offsetof(union mf_subvalue, ipv4);
> -    case LEX_F_IPV6:        return offsetof(union mf_subvalue, ipv6);
> -    case LEX_F_ETHERNET:    return offsetof(union mf_subvalue, mac);
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -/* Returns the effective format for 'token', that is, the format in which
> it
> - * should actually be printed.  This is ordinarily the same as
> 'token->format',
> - * but it's always possible that someone sets up a token with a format
> that
> - * won't work for a value, e.g. 'token->value' is wider than 32 bits but
> the
> - * format is LEX_F_IPV4.  (The lexer itself won't do that; this is an
> attempt
> - * to avoid confusion in the future.) */
> -static enum lex_format
> -lex_token_get_format(const struct lex_token *token)
> -{
> -    size_t n_zeros = lex_token_n_zeros(token->format);
> -    return (is_all_zeros(&token->value, n_zeros)
> -            && (token->type != LEX_T_MASKED_INTEGER
> -                || is_all_zeros(&token->mask, n_zeros))
> -            ? token->format
> -            : LEX_F_HEXADECIMAL);
> -}
> -
> -static void
> -lex_token_format_value(const union mf_subvalue *value,
> -                       enum lex_format format, struct ds *s)
> -{
> -    switch (format) {
> -    case LEX_F_DECIMAL:
> -        ds_put_format(s, "%"PRIu64, ntohll(value->integer));
> -        break;
> -
> -    case LEX_F_HEXADECIMAL:
> -        mf_format_subvalue(value, s);
> -        break;
> -
> -    case LEX_F_IPV4:
> -        ds_put_format(s, IP_FMT, IP_ARGS(value->ipv4));
> -        break;
> -
> -    case LEX_F_IPV6:
> -        ipv6_format_addr(&value->ipv6, s);
> -        break;
> -
> -    case LEX_F_ETHERNET:
> -        ds_put_format(s, ETH_ADDR_FMT, ETH_ADDR_ARGS(value->mac));
> -        break;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -}
> -
> -static void
> -lex_token_format_masked_integer(const struct lex_token *token, struct ds
> *s)
> -{
> -    enum lex_format format = lex_token_get_format(token);
> -
> -    lex_token_format_value(&token->value, format, s);
> -    ds_put_char(s, '/');
> -
> -    const union mf_subvalue *mask = &token->mask;
> -    if (format == LEX_F_IPV4 && ip_is_cidr(mask->ipv4)) {
> -        ds_put_format(s, "%d", ip_count_cidr_bits(mask->ipv4));
> -    } else if (token->format == LEX_F_IPV6 && ipv6_is_cidr(&mask->ipv6)) {
> -        ds_put_format(s, "%d", ipv6_count_cidr_bits(&mask->ipv6));
> -    } else {
> -        lex_token_format_value(&token->mask, format, s);
> -    }
> -}
> -
> -/* Appends a string representation of 'token' to 's', in a format that
> can be
> - * losslessly parsed back by the lexer.  (LEX_T_END and LEX_T_ERROR can't
> be
> - * parsed back.) */
> -void
> -lex_token_format(const struct lex_token *token, struct ds *s)
> -{
> -    switch (token->type) {
> -    case LEX_T_END:
> -        ds_put_cstr(s, "$");
> -        break;
> -
> -    case LEX_T_ID:
> -        ds_put_cstr(s, token->s);
> -        break;
> -
> -    case LEX_T_ERROR:
> -        ds_put_cstr(s, "error(");
> -        json_string_escape(token->s, s);
> -        ds_put_char(s, ')');
> -        break;
> -
> -    case LEX_T_STRING:
> -        json_string_escape(token->s, s);
> -        break;
> -
> -    case LEX_T_INTEGER:
> -        lex_token_format_value(&token->value,
> lex_token_get_format(token), s);
> -        break;
> -
> -    case LEX_T_MASKED_INTEGER:
> -        lex_token_format_masked_integer(token, s);
> -        break;
> -
> -    case LEX_T_MACRO:
> -        ds_put_format(s, "$%s", token->s);
> -        break;
> -
> -    case LEX_T_PORT_GROUP:
> -        ds_put_format(s, "@%s", token->s);
> -        break;
> -
> -    case LEX_T_LPAREN:
> -        ds_put_cstr(s, "(");
> -        break;
> -    case LEX_T_RPAREN:
> -        ds_put_cstr(s, ")");
> -        break;
> -    case LEX_T_LCURLY:
> -        ds_put_cstr(s, "{");
> -        break;
> -    case LEX_T_RCURLY:
> -        ds_put_cstr(s, "}");
> -        break;
> -    case LEX_T_LSQUARE:
> -        ds_put_cstr(s, "[");
> -        break;
> -    case LEX_T_RSQUARE:
> -        ds_put_cstr(s, "]");
> -        break;
> -    case LEX_T_EQ:
> -        ds_put_cstr(s, "==");
> -        break;
> -    case LEX_T_NE:
> -        ds_put_cstr(s, "!=");
> -        break;
> -    case LEX_T_LT:
> -        ds_put_cstr(s, "<");
> -        break;
> -    case LEX_T_LE:
> -        ds_put_cstr(s, "<=");
> -        break;
> -    case LEX_T_GT:
> -        ds_put_cstr(s, ">");
> -        break;
> -    case LEX_T_GE:
> -        ds_put_cstr(s, ">=");
> -        break;
> -    case LEX_T_LOG_NOT:
> -        ds_put_cstr(s, "!");
> -        break;
> -    case LEX_T_LOG_AND:
> -        ds_put_cstr(s, "&&");
> -        break;
> -    case LEX_T_LOG_OR:
> -        ds_put_cstr(s, "||");
> -        break;
> -    case LEX_T_ELLIPSIS:
> -        ds_put_cstr(s, "..");
> -        break;
> -    case LEX_T_COMMA:
> -        ds_put_cstr(s, ",");
> -        break;
> -    case LEX_T_SEMICOLON:
> -        ds_put_cstr(s, ";");
> -        break;
> -    case LEX_T_EQUALS:
> -        ds_put_cstr(s, "=");
> -        break;
> -    case LEX_T_EXCHANGE:
> -        ds_put_cstr(s, "<->");
> -        break;
> -    case LEX_T_DECREMENT:
> -        ds_put_cstr(s, "--");
> -        break;
> -    case LEX_T_COLON:
> -        ds_put_char(s, ':');
> -        break;
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -}
> -
> -/* lex_token_parse(). */
> -
> -static void OVS_PRINTF_FORMAT(2, 3)
> -lex_error(struct lex_token *token, const char *message, ...)
> -{
> -    ovs_assert(!token->s);
> -    token->type = LEX_T_ERROR;
> -
> -    va_list args;
> -    va_start(args, message);
> -    lex_token_vsprintf(token, message, args);
> -    va_end(args);
> -}
> -
> -static void
> -lex_parse_hex_integer(const char *start, size_t len, struct lex_token
> *token)
> -{
> -    const char *in = start + (len - 1);
> -    uint8_t *out = token->value.u8 + (sizeof token->value.u8 - 1);
> -
> -    for (int i = 0; i < len; i++) {
> -        int hexit = hexit_value(in[-i]);
> -        if (hexit < 0) {
> -            lex_error(token, "Invalid syntax in hexadecimal constant.");
> -            return;
> -        } else if (hexit) {
> -            /* Check within loop to ignore any number of leading zeros. */
> -            if (i / 2 >= sizeof token->value.u8) {
> -                lex_error(token, "Hexadecimal constant requires more than
> "
> -                          "%"PRIuSIZE" bits.", 8 * sizeof
> token->value.u8);
> -                return;
> -            }
> -            out[-(i / 2)] |= i % 2 ? hexit << 4 : hexit;
> -        }
> -    }
> -    token->format = LEX_F_HEXADECIMAL;
> -}
> -
> -static const char *
> -lex_parse_integer__(const char *p, struct lex_token *token)
> -{
> -    lex_token_init(token);
> -    token->type = LEX_T_INTEGER;
> -    memset(&token->value, 0, sizeof token->value);
> -
> -    /* Find the extent of an "integer" token, which can be in decimal or
> -     * hexadecimal, or an Ethernet address or IPv4 or IPv6 address, as
> 'start'
> -     * through 'end'.
> -     *
> -     * Special cases we handle here are:
> -     *
> -     *     - The ellipsis token "..", used as e.g. 123..456.  A doubled
> dot
> -     *       is never valid syntax as part of an "integer", so we stop if
> -     *       we encounter two dots in a row.
> -     *
> -     *     - Syntax like 1.2.3.4:1234 to indicate an IPv4 address
> followed by a
> -     *       port number should be considered three tokens: 1.2.3.4 :
> 1234.
> -     *       The obvious approach is to allow just dots or just colons
> within a
> -     *       given integer, but that would disallow IPv4-mapped IPv6
> addresses,
> -     *       e.g. ::ffff:192.0.2.128.  However, even in those addresses, a
> -     *       colon never follows a dot, so we stop if we encounter a colon
> -     *       after a dot.
> -     *
> -     *       (There is no corresponding way to parse an IPv6 address
> followed
> -     *       by a port number: ::1:2:3:4:1234 is unavoidably ambiguous.)
> -     */
> -    const char *start = p;
> -    const char *end = start;
> -    bool saw_dot = false;
> -    while (isalnum((unsigned char) *end)
> -           || (*end == ':' && !saw_dot)
> -           || (*end == '.' && end[1] != '.')) {
> -        if (*end == '.') {
> -            saw_dot = true;
> -        }
> -        end++;
> -    }
> -    size_t len = end - start;
> -
> -    int n;
> -    struct eth_addr mac;
> -
> -    if (!len) {
> -        lex_error(token, "Integer constant expected.");
> -    } else if (len == 17
> -               && ovs_scan(start, ETH_ADDR_SCAN_FMT"%n",
> -                           ETH_ADDR_SCAN_ARGS(mac), &n)
> -               && n == len) {
> -        token->value.mac = mac;
> -        token->format = LEX_F_ETHERNET;
> -    } else if (start + strspn(start, "0123456789") == end) {
> -        if (p[0] == '0' && len > 1) {
> -            lex_error(token, "Decimal constants must not have leading
> zeros.");
> -        } else {
> -            unsigned long long int integer;
> -            char *tail;
> -
> -            errno = 0;
> -            integer = strtoull(p, &tail, 10);
> -            if (tail != end || errno == ERANGE) {
> -                lex_error(token, "Decimal constants must be less than
> 2**64.");
> -            } else {
> -                token->value.integer = htonll(integer);
> -                token->format = LEX_F_DECIMAL;
> -            }
> -        }
> -    } else if (p[0] == '0' && (p[1] == 'x' || p[1] == 'X')) {
> -        if (len > 2) {
> -            lex_parse_hex_integer(start + 2, len - 2, token);
> -        } else {
> -            lex_error(token, "Hex digits expected following 0%c.", p[1]);
> -        }
> -    } else if (len < INET6_ADDRSTRLEN) {
> -        char copy[INET6_ADDRSTRLEN];
> -        memcpy(copy, p, len);
> -        copy[len] = '\0';
> -
> -        if (ip_parse(copy, &token->value.ipv4)) {
> -            token->format = LEX_F_IPV4;
> -        } else if (ipv6_parse(copy, &token->value.ipv6)) {
> -            token->format = LEX_F_IPV6;
> -        } else {
> -            lex_error(token, "Invalid numeric constant.");
> -        }
> -    } else {
> -        lex_error(token, "Invalid numeric constant.");
> -    }
> -
> -    ovs_assert(token->type == LEX_T_INTEGER || token->type ==
> LEX_T_ERROR);
> -    return end;
> -}
> -
> -static const char *
> -lex_parse_mask(const char *p, struct lex_token *token)
> -{
> -    struct lex_token mask;
> -
> -    /* Parse just past the '/' as a second integer.  Handle errors. */
> -    p = lex_parse_integer__(p + 1, &mask);
> -    if (mask.type == LEX_T_ERROR) {
> -        lex_token_swap(&mask, token);
> -        lex_token_destroy(&mask);
> -        return p;
> -    }
> -    ovs_assert(mask.type == LEX_T_INTEGER);
> -
> -    /* Now convert the value and mask into a masked integer token.
> -     * We have a few special cases. */
> -    token->type = LEX_T_MASKED_INTEGER;
> -    memset(&token->mask, 0, sizeof token->mask);
> -    uint32_t prefix_bits = ntohll(mask.value.integer);
> -    if (token->format == mask.format) {
> -        /* Same format value and mask is always OK. */
> -        token->mask = mask.value;
> -    } else if (token->format == LEX_F_IPV4
> -               && mask.format == LEX_F_DECIMAL
> -               && prefix_bits <= 32) {
> -        /* IPv4 address with decimal mask is a CIDR prefix. */
> -        token->mask.integer =
> htonll(ntohl(be32_prefix_mask(prefix_bits)));
> -    } else if (token->format == LEX_F_IPV6
> -               && mask.format == LEX_F_DECIMAL
> -               && prefix_bits <= 128) {
> -        /* IPv6 address with decimal mask is a CIDR prefix. */
> -        token->mask.ipv6 = ipv6_create_mask(prefix_bits);
> -    } else if (token->format == LEX_F_DECIMAL
> -               && mask.format == LEX_F_HEXADECIMAL
> -               && token->value.integer == 0) {
> -        /* Special case for e.g. 0/0x1234. */
> -        token->format = LEX_F_HEXADECIMAL;
> -        token->mask = mask.value;
> -    } else {
> -        lex_error(token, "Value and mask have incompatible formats.");
> -        return p;
> -    }
> -
> -    /* Check invariant that a 1-bit in the value corresponds to a 1-bit
> in the
> -     * mask. */
> -    for (int i = 0; i < ARRAY_SIZE(token->mask.be32); i++) {
> -        ovs_be32 v = token->value.be32[i];
> -        ovs_be32 m = token->mask.be32[i];
> -
> -        if (v & ~m) {
> -            lex_error(token, "Value contains unmasked 1-bits.");
> -            break;
> -        }
> -    }
> -
> -    /* Done! */
> -    lex_token_destroy(&mask);
> -    return p;
> -}
> -
> -static const char *
> -lex_parse_integer(const char *p, struct lex_token *token)
> -{
> -    p = lex_parse_integer__(p, token);
> -    if (token->type == LEX_T_INTEGER && *p == '/') {
> -        p = lex_parse_mask(p, token);
> -    }
> -    return p;
> -}
> -
> -static const char *
> -lex_parse_string(const char *p, struct lex_token *token)
> -{
> -    const char *start = ++p;
> -    char * s = NULL;
> -    for (;;) {
> -        switch (*p) {
> -        case '\0':
> -            lex_error(token, "Input ends inside quoted string.");
> -            return p;
> -
> -        case '"':
> -            token->type = (json_string_unescape(start, p - start, &s)
> -                           ? LEX_T_STRING : LEX_T_ERROR);
> -            lex_token_strset(token, s);
> -            return p + 1;
> -
> -        case '\\':
> -            p++;
> -            if (*p) {
> -                p++;
> -            }
> -            break;
> -
> -        default:
> -            p++;
> -            break;
> -        }
> -    }
> -}
> -
> -static bool
> -lex_is_id1(unsigned char c)
> -{
> -    return ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')
> -            || c == '_' || c == '.');
> -}
> -
> -static bool
> -lex_is_idn(unsigned char c)
> -{
> -    return lex_is_id1(c) || (c >= '0' && c <= '9');
> -}
> -
> -static const char *
> -lex_parse_id(const char *p, enum lex_type type, struct lex_token *token)
> -{
> -    const char *start = p;
> -
> -    do {
> -        p++;
> -    } while (lex_is_idn(*p));
> -
> -    token->type = type;
> -    lex_token_strcpy(token, start, p - start);
> -    return p;
> -}
> -
> -static const char *
> -lex_parse_addr_set(const char *p, struct lex_token *token)
> -{
> -    p++;
> -    if (!lex_is_id1(*p)) {
> -        lex_error(token, "`$' must be followed by a valid identifier.");
> -        return p;
> -    }
> -
> -    return lex_parse_id(p, LEX_T_MACRO, token);
> -}
> -
> -static const char *
> -lex_parse_port_group(const char *p, struct lex_token *token)
> -{
> -    p++;
> -    if (!lex_is_id1(*p)) {
> -        lex_error(token, "`@' must be followed by a valid identifier.");
> -        return p;
> -    }
> -
> -    return lex_parse_id(p, LEX_T_PORT_GROUP, token);
> -}
> -
> -/* Initializes 'token' and parses the first token from the beginning of
> - * null-terminated string 'p' into 'token'.  Stores a pointer to the
> start of
> - * the token (after skipping white space and comments, if any) into
> '*startp'.
> - * Returns the character position at which to begin parsing the next
> token. */
> -const char *
> -lex_token_parse(struct lex_token *token, const char *p, const char
> **startp)
> -{
> -    lex_token_init(token);
> -
> -next:
> -    *startp = p;
> -    switch (*p) {
> -    case '\0':
> -        token->type = LEX_T_END;
> -        return p;
> -
> -    case ' ': case '\t': case '\n': case '\r': case '\v': case '\f':
> -        p++;
> -        goto next;
> -
> -    case '/':
> -        p++;
> -        if (*p == '/') {
> -            do {
> -                p++;
> -            } while (*p != '\0' && *p != '\n');
> -            goto next;
> -        } else if (*p == '*') {
> -            p++;
> -            for (;;) {
> -                if (*p == '*' && p[1] == '/') {
> -                    p += 2;
> -                    goto next;
> -                } else if (*p == '\0' || *p == '\n') {
> -                    lex_error(token, "`/*' without matching `*/'.");
> -                    return p;
> -                } else {
> -                    p++;
> -                }
> -            }
> -            goto next;
> -        } else {
> -            lex_error(token,
> -                      "`/' is only valid as part of `//' or `/*'.");
> -        }
> -        break;
> -
> -    case '(':
> -        token->type = LEX_T_LPAREN;
> -        p++;
> -        break;
> -
> -    case ')':
> -        token->type = LEX_T_RPAREN;
> -        p++;
> -        break;
> -
> -    case '{':
> -        token->type = LEX_T_LCURLY;
> -        p++;
> -        break;
> -
> -    case '}':
> -        token->type = LEX_T_RCURLY;
> -        p++;
> -        break;
> -
> -    case '[':
> -        token->type = LEX_T_LSQUARE;
> -        p++;
> -        break;
> -
> -    case ']':
> -        token->type = LEX_T_RSQUARE;
> -        p++;
> -        break;
> -
> -    case '=':
> -        p++;
> -        if (*p == '=') {
> -            token->type = LEX_T_EQ;
> -            p++;
> -        } else {
> -            token->type = LEX_T_EQUALS;
> -        }
> -        break;
> -
> -    case '!':
> -        p++;
> -        if (*p == '=') {
> -            token->type = LEX_T_NE;
> -            p++;
> -        } else {
> -            token->type = LEX_T_LOG_NOT;
> -        }
> -        break;
> -
> -    case '&':
> -        p++;
> -        if (*p == '&') {
> -            token->type = LEX_T_LOG_AND;
> -            p++;
> -        } else {
> -            lex_error(token, "`&' is only valid as part of `&&'.");
> -        }
> -        break;
> -
> -    case '|':
> -        p++;
> -        if (*p == '|') {
> -            token->type = LEX_T_LOG_OR;
> -            p++;
> -        } else {
> -            lex_error(token, "`|' is only valid as part of `||'.");
> -        }
> -        break;
> -
> -    case '<':
> -        p++;
> -        if (*p == '=') {
> -            token->type = LEX_T_LE;
> -            p++;
> -        } else if (*p == '-' && p[1] == '>') {
> -            token->type = LEX_T_EXCHANGE;
> -            p += 2;
> -        } else {
> -            token->type = LEX_T_LT;
> -        }
> -        break;
> -
> -    case '>':
> -        p++;
> -        if (*p == '=') {
> -            token->type = LEX_T_GE;
> -            p++;
> -        } else {
> -            token->type = LEX_T_GT;
> -        }
> -        break;
> -
> -    case '.':
> -        p++;
> -        if (*p == '.') {
> -            token->type = LEX_T_ELLIPSIS;
> -            p++;
> -        } else {
> -            lex_error(token, "`.' is only valid as part of `..' or a
> number.");
> -        }
> -        break;
> -
> -    case ',':
> -        p++;
> -        token->type = LEX_T_COMMA;
> -        break;
> -
> -    case ';':
> -        p++;
> -        token->type = LEX_T_SEMICOLON;
> -        break;
> -
> -    case '-':
> -        p++;
> -        if (*p == '-') {
> -            token->type = LEX_T_DECREMENT;
> -            p++;
> -        } else {
> -            lex_error(token, "`-' is only valid as part of `--'.");
> -        }
> -        break;
> -
> -    case '$':
> -        p = lex_parse_addr_set(p, token);
> -        break;
> -
> -    case '@':
> -        p = lex_parse_port_group(p, token);
> -        break;
> -
> -    case ':':
> -        if (p[1] != ':') {
> -            token->type = LEX_T_COLON;
> -            p++;
> -            break;
> -        }
> -        /* IPv6 address beginning with "::". */
> -        /* fall through */
> -    case '0': case '1': case '2': case '3': case '4':
> -    case '5': case '6': case '7': case '8': case '9':
> -        p = lex_parse_integer(p, token);
> -        break;
> -
> -    case '"':
> -        p = lex_parse_string(p, token);
> -        break;
> -
> -    case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
> -    case 'A': case 'B': case 'C': case 'D': case 'E': case 'F':
> -        /* We need to distinguish an Ethernet address or IPv6 address
> from an
> -         * identifier.  Fortunately, Ethernet addresses and IPv6
> addresses that
> -         * are ambiguous based on the first character, always start with
> hex
> -         * digits followed by a colon, but identifiers never do. */
> -        p = (p[strspn(p, "0123456789abcdefABCDEF")] == ':'
> -             ? lex_parse_integer(p, token)
> -             : lex_parse_id(p, LEX_T_ID, token));
> -        break;
> -
> -    default:
> -        if (lex_is_id1(*p)) {
> -            p = lex_parse_id(p, LEX_T_ID, token);
> -        } else {
> -            if (isprint((unsigned char) *p)) {
> -                lex_error(token, "Invalid character `%c' in input.", *p);
> -            } else {
> -                lex_error(token, "Invalid byte 0x%d in input.", *p);
> -            }
> -            p++;
> -        }
> -        break;
> -    }
> -
> -    return p;
> -}
> -
> -/* Initializes 'lexer' for parsing 'input'.
> - *
> - * While the lexer is in use, 'input' must remain available, but the
> caller
> - * otherwise retains ownership of 'input'.
> - *
> - * The caller must call lexer_get() to obtain the first token. */
> -void
> -lexer_init(struct lexer *lexer, const char *input)
> -{
> -    lexer->input = input;
> -    lexer->start = NULL;
> -    lex_token_init(&lexer->token);
> -    lexer->error = NULL;
> -}
> -
> -/* Frees storage associated with 'lexer'. */
> -void
> -lexer_destroy(struct lexer *lexer)
> -{
> -    lex_token_destroy(&lexer->token);
> -    free(lexer->error);
> -}
> -
> -/* Obtains the next token from 'lexer' into 'lexer->token', and returns
> the
> - * token's type.  The caller may examine 'lexer->token' directly to
> obtain full
> - * information about the token. */
> -enum lex_type
> -lexer_get(struct lexer *lexer)
> -{
> -    lex_token_destroy(&lexer->token);
> -    lexer->input = lex_token_parse(&lexer->token, lexer->input,
> &lexer->start);
> -    return lexer->token.type;
> -}
> -
> -/* Returns the type of the next token that will be fetched by lexer_get(),
> - * without advancing 'lexer->token' to that token. */
> -enum lex_type
> -lexer_lookahead(const struct lexer *lexer)
> -{
> -    struct lex_token next;
> -    enum lex_type type;
> -    const char *start;
> -
> -    lex_token_parse(&next, lexer->input, &start);
> -    type = next.type;
> -    lex_token_destroy(&next);
> -    return type;
> -}
> -
> -/* If 'lexer''s current token has the given 'type', advances 'lexer' to
> the
> - * next token and returns true.  Otherwise returns false. */
> -bool
> -lexer_match(struct lexer *lexer, enum lex_type type)
> -{
> -    if (lexer->token.type == type) {
> -        lexer_get(lexer);
> -        return true;
> -    } else {
> -        return false;
> -    }
> -}
> -
> -bool
> -lexer_force_match(struct lexer *lexer, enum lex_type t)
> -{
> -    if (t == LEX_T_END) {
> -        return lexer_force_end(lexer);
> -    } else if (lexer_match(lexer, t)) {
> -        return true;
> -    } else {
> -        struct lex_token token = { .type = t };
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        lex_token_format(&token, &s);
> -
> -        lexer_syntax_error(lexer, "expecting `%s'", ds_cstr(&s));
> -
> -        ds_destroy(&s);
> -
> -        return false;
> -    }
> -}
> -
> -/* If 'lexer''s current token is the identifier given in 'id', advances
> 'lexer'
> - * to the next token and returns true.  Otherwise returns false.  */
> -bool
> -lexer_match_id(struct lexer *lexer, const char *id)
> -{
> -    if (lexer->token.type == LEX_T_ID && !strcmp(lexer->token.s, id)) {
> -        lexer_get(lexer);
> -        return true;
> -    } else {
> -        return false;
> -    }
> -}
> -
> -bool
> -lexer_is_int(const struct lexer *lexer)
> -{
> -    return (lexer->token.type == LEX_T_INTEGER
> -            && lexer->token.format == LEX_F_DECIMAL
> -            && ntohll(lexer->token.value.integer) <= INT_MAX);
> -}
> -
> -bool
> -lexer_get_int(struct lexer *lexer, int *value)
> -{
> -    if (lexer_is_int(lexer)) {
> -        *value = ntohll(lexer->token.value.integer);
> -        lexer_get(lexer);
> -        return true;
> -    } else {
> -        *value = 0;
> -        return false;
> -    }
> -}
> -
> -bool
> -lexer_force_int(struct lexer *lexer, int *value)
> -{
> -    bool ok = lexer_get_int(lexer, value);
> -    if (!ok) {
> -        lexer_syntax_error(lexer, "expecting small integer");
> -    }
> -    return ok;
> -}
> -
> -bool
> -lexer_force_end(struct lexer *lexer)
> -{
> -    if (lexer->token.type == LEX_T_END) {
> -        return true;
> -    } else {
> -        lexer_syntax_error(lexer, "expecting end of input");
> -        return false;
> -    }
> -}
> -
> -static bool
> -lexer_error_handle_common(struct lexer *lexer)
> -{
> -    if (lexer->error) {
> -        /* Already have an error, suppress this one since the cascade
> seems
> -         * unlikely to be useful. */
> -        return true;
> -    } else if (lexer->token.type == LEX_T_ERROR) {
> -        /* The lexer signaled an error.  Nothing at a higher level
> accepts an
> -         * error token, so we'll inevitably end up here with some
> meaningless
> -         * parse error.  Report the lexical error instead. */
> -        lexer->error = xstrdup(lexer->token.s);
> -        return true;
> -    } else {
> -        return false;
> -    }
> -}
> -
> -void OVS_PRINTF_FORMAT(2, 3)
> -lexer_error(struct lexer *lexer, const char *message, ...)
> -{
> -    if (lexer_error_handle_common(lexer)) {
> -        return;
> -    }
> -
> -    va_list args;
> -    va_start(args, message);
> -    lexer->error = xvasprintf(message, args);
> -    va_end(args);
> -}
> -
> -void OVS_PRINTF_FORMAT(2, 3)
> -lexer_syntax_error(struct lexer *lexer, const char *message, ...)
> -{
> -    if (lexer_error_handle_common(lexer)) {
> -        return;
> -    }
> -
> -    struct ds s;
> -
> -    ds_init(&s);
> -    ds_put_cstr(&s, "Syntax error");
> -    if (lexer->token.type == LEX_T_END) {
> -        ds_put_cstr(&s, " at end of input");
> -    } else if (lexer->start) {
> -        ds_put_format(&s, " at `%.*s'",
> -                      (int) (lexer->input - lexer->start),
> -                      lexer->start);
> -    }
> -
> -    if (message) {
> -        ds_put_char(&s, ' ');
> -
> -        va_list args;
> -        va_start(args, message);
> -        ds_put_format_valist(&s, message, args);
> -        va_end(args);
> -    }
> -    ds_put_char(&s, '.');
> -
> -    lexer->error = ds_steal_cstr(&s);
> -}
> -
> -char *
> -lexer_steal_error(struct lexer *lexer)
> -{
> -    char *error = lexer->error;
> -    lexer->error = NULL;
> -    return error;
> -}
> diff --git a/ovn/lib/libovn.sym.in b/ovn/lib/libovn.sym.in
> deleted file mode 100644
> index 360de0fe8..000000000
> --- a/ovn/lib/libovn.sym.in
> +++ /dev/null
> @@ -1,4 +0,0 @@
> -libovn_ at LT_CURRENT@ {
> -global:
> -        *;
> -};
> diff --git a/ovn/lib/logical-fields.c b/ovn/lib/logical-fields.c
> deleted file mode 100644
> index 4ad5bf481..000000000
> --- a/ovn/lib/logical-fields.c
> +++ /dev/null
> @@ -1,261 +0,0 @@
> -/* Copyright (c) 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "openvswitch/shash.h"
> -#include "ovn/expr.h"
> -#include "ovn/logical-fields.h"
> -#include "ovs-thread.h"
> -#include "packets.h"
> -
> -/* Silence a warning. */
> -extern const struct ovn_field ovn_fields[OVN_FIELD_N_IDS];
> -
> -const struct ovn_field ovn_fields[OVN_FIELD_N_IDS] = {
> -    {
> -        OVN_ICMP4_FRAG_MTU,
> -        "icmp4.frag_mtu",
> -        2, 16,
> -    },
> -};
> -
> -static struct shash ovnfield_by_name;
> -
> -static void
> -add_subregister(const char *name,
> -                const char *parent_name, int parent_idx,
> -                int width, int idx,
> -                struct shash *symtab)
> -{
> -    int lsb = width * idx;
> -    int msb = lsb + (width - 1);
> -    char *expansion = xasprintf("%s%d[%d..%d]",
> -                                parent_name, parent_idx, lsb, msb);
> -    expr_symtab_add_subfield(symtab, name, NULL, expansion);
> -    free(expansion);
> -}
> -
> -static void
> -add_ct_bit(const char *name, int index, struct shash *symtab)
> -{
> -    char *expansion = xasprintf("ct_state[%d]", index);
> -    const char *prereqs = index == CS_TRACKED_BIT ? NULL : "ct.trk";
> -    expr_symtab_add_subfield(symtab, name, prereqs, expansion);
> -    free(expansion);
> -}
> -
> -void
> -ovn_init_symtab(struct shash *symtab)
> -{
> -    shash_init(symtab);
> -
> -    /* Reserve a pair of registers for the logical inport and outport.  A
> full
> -     * 32-bit register each is bigger than we need, but the expression
> code
> -     * doesn't yet support string fields that occupy less than a full
> OXM. */
> -    expr_symtab_add_string(symtab, "inport", MFF_LOG_INPORT, NULL);
> -    expr_symtab_add_string(symtab, "outport", MFF_LOG_OUTPORT, NULL);
> -
> -    /* Logical registers:
> -     *     128-bit xxregs
> -     *     64-bit xregs
> -     *     32-bit regs
> -     *
> -     * The expression language doesn't handle overlapping fields properly
> -     * unless they're formally defined as subfields.  It's a little
> awkward. */
> -    for (int xxi = 0; xxi < MFF_N_LOG_REGS / 4; xxi++) {
> -        char *xxname = xasprintf("xxreg%d", xxi);
> -        expr_symtab_add_field(symtab, xxname, MFF_XXREG0 + xxi, NULL,
> false);
> -        free(xxname);
> -    }
> -    for (int xi = 0; xi < MFF_N_LOG_REGS / 2; xi++) {
> -        char *xname = xasprintf("xreg%d", xi);
> -        int xxi = xi / 2;
> -        if (xxi < MFF_N_LOG_REGS / 4) {
> -            add_subregister(xname, "xxreg", xxi, 64, 1 - xi % 2, symtab);
> -        } else {
> -            expr_symtab_add_field(symtab, xname, MFF_XREG0 + xi, NULL,
> false);
> -        }
> -        free(xname);
> -    }
> -    for (int i = 0; i < MFF_N_LOG_REGS; i++) {
> -        char *name = xasprintf("reg%d", i);
> -        int xxi = i / 4;
> -        int xi = i / 2;
> -        if (xxi < MFF_N_LOG_REGS / 4) {
> -            add_subregister(name, "xxreg", xxi, 32, 3 - i % 4, symtab);
> -        } else if (xi < MFF_N_LOG_REGS / 2) {
> -            add_subregister(name, "xreg", xi, 32, 1 - i % 2, symtab);
> -        } else {
> -            expr_symtab_add_field(symtab, name, MFF_REG0 + i, NULL,
> false);
> -        }
> -        free(name);
> -    }
> -
> -    /* Flags used in logical to physical transformation. */
> -    expr_symtab_add_field(symtab, "flags", MFF_LOG_FLAGS, NULL, false);
> -    char flags_str[16];
> -    snprintf(flags_str, sizeof flags_str, "flags[%d]",
> MLF_ALLOW_LOOPBACK_BIT);
> -    expr_symtab_add_subfield(symtab, "flags.loopback", NULL, flags_str);
> -    snprintf(flags_str, sizeof flags_str, "flags[%d]",
> -             MLF_FORCE_SNAT_FOR_DNAT_BIT);
> -    expr_symtab_add_subfield(symtab, "flags.force_snat_for_dnat", NULL,
> -                             flags_str);
> -    snprintf(flags_str, sizeof flags_str, "flags[%d]",
> -             MLF_FORCE_SNAT_FOR_LB_BIT);
> -    expr_symtab_add_subfield(symtab, "flags.force_snat_for_lb", NULL,
> -                             flags_str);
> -
> -    /* Connection tracking state. */
> -    expr_symtab_add_field(symtab, "ct_mark", MFF_CT_MARK, NULL, false);
> -
> -    expr_symtab_add_field(symtab, "ct_label", MFF_CT_LABEL, NULL, false);
> -    expr_symtab_add_subfield(symtab, "ct_label.blocked", NULL,
> "ct_label[0]");
> -
> -    expr_symtab_add_field(symtab, "ct_state", MFF_CT_STATE, NULL, false);
> -
> -#define CS_STATE(ENUM, INDEX, NAME) \
> -    add_ct_bit("ct."NAME, CS_##ENUM##_BIT, symtab);
> -    CS_STATES
> -#undef CS_STATE
> -
> -    /* Data fields. */
> -    expr_symtab_add_field(symtab, "eth.src", MFF_ETH_SRC, NULL, false);
> -    expr_symtab_add_field(symtab, "eth.dst", MFF_ETH_DST, NULL, false);
> -    expr_symtab_add_field(symtab, "eth.type", MFF_ETH_TYPE, NULL, true);
> -    expr_symtab_add_predicate(symtab, "eth.bcast",
> -                              "eth.dst == ff:ff:ff:ff:ff:ff");
> -    expr_symtab_add_subfield(symtab, "eth.mcast", NULL, "eth.dst[40]");
> -
> -    expr_symtab_add_field(symtab, "vlan.tci", MFF_VLAN_TCI, NULL, false);
> -    expr_symtab_add_predicate(symtab, "vlan.present", "vlan.tci[12]");
> -    expr_symtab_add_subfield(symtab, "vlan.pcp", "vlan.present",
> -                             "vlan.tci[13..15]");
> -    expr_symtab_add_subfield(symtab, "vlan.vid", "vlan.present",
> -                             "vlan.tci[0..11]");
> -
> -    expr_symtab_add_predicate(symtab, "ip4", "eth.type == 0x800");
> -    expr_symtab_add_predicate(symtab, "ip6", "eth.type == 0x86dd");
> -    expr_symtab_add_predicate(symtab, "ip", "ip4 || ip6");
> -    expr_symtab_add_field(symtab, "ip.proto", MFF_IP_PROTO, "ip", true);
> -    expr_symtab_add_field(symtab, "ip.dscp", MFF_IP_DSCP_SHIFTED, "ip",
> false);
> -    expr_symtab_add_field(symtab, "ip.ecn", MFF_IP_ECN, "ip", false);
> -    expr_symtab_add_field(symtab, "ip.ttl", MFF_IP_TTL, "ip", false);
> -
> -    expr_symtab_add_field(symtab, "ip4.src", MFF_IPV4_SRC, "ip4", false);
> -    expr_symtab_add_field(symtab, "ip4.dst", MFF_IPV4_DST, "ip4", false);
> -    expr_symtab_add_predicate(symtab, "ip4.mcast", "ip4.dst[28..31] ==
> 0xe");
> -
> -    expr_symtab_add_predicate(symtab, "icmp4", "ip4 && ip.proto == 1");
> -    expr_symtab_add_field(symtab, "icmp4.type", MFF_ICMPV4_TYPE, "icmp4",
> -              false);
> -    expr_symtab_add_field(symtab, "icmp4.code", MFF_ICMPV4_CODE, "icmp4",
> -              false);
> -
> -    expr_symtab_add_predicate(symtab, "igmp", "ip4 && ip.proto == 2");
> -
> -    expr_symtab_add_field(symtab, "ip6.src", MFF_IPV6_SRC, "ip6", false);
> -    expr_symtab_add_field(symtab, "ip6.dst", MFF_IPV6_DST, "ip6", false);
> -    expr_symtab_add_field(symtab, "ip6.label", MFF_IPV6_LABEL, "ip6",
> false);
> -
> -    expr_symtab_add_predicate(symtab, "icmp6", "ip6 && ip.proto == 58");
> -    expr_symtab_add_field(symtab, "icmp6.type", MFF_ICMPV6_TYPE, "icmp6",
> -                          true);
> -    expr_symtab_add_field(symtab, "icmp6.code", MFF_ICMPV6_CODE, "icmp6",
> -                          true);
> -
> -    expr_symtab_add_predicate(symtab, "icmp", "icmp4 || icmp6");
> -
> -    expr_symtab_add_field(symtab, "ip.frag", MFF_IP_FRAG, "ip", false);
> -    expr_symtab_add_predicate(symtab, "ip.is_frag", "ip.frag[0]");
> -    expr_symtab_add_predicate(symtab, "ip.later_frag", "ip.frag[1]");
> -    expr_symtab_add_predicate(symtab, "ip.first_frag",
> -                              "ip.is_frag && !ip.later_frag");
> -
> -    expr_symtab_add_predicate(symtab, "arp", "eth.type == 0x806");
> -    expr_symtab_add_field(symtab, "arp.op", MFF_ARP_OP, "arp", false);
> -    expr_symtab_add_field(symtab, "arp.spa", MFF_ARP_SPA, "arp", false);
> -    expr_symtab_add_field(symtab, "arp.sha", MFF_ARP_SHA, "arp", false);
> -    expr_symtab_add_field(symtab, "arp.tpa", MFF_ARP_TPA, "arp", false);
> -    expr_symtab_add_field(symtab, "arp.tha", MFF_ARP_THA, "arp", false);
> -
> -    expr_symtab_add_predicate(symtab, "nd",
> -              "icmp6.type == {135, 136} && icmp6.code == 0 && ip.ttl ==
> 255");
> -    expr_symtab_add_predicate(symtab, "nd_ns",
> -              "icmp6.type == 135 && icmp6.code == 0 && ip.ttl == 255");
> -    expr_symtab_add_predicate(symtab, "nd_na",
> -              "icmp6.type == 136 && icmp6.code == 0 && ip.ttl == 255");
> -    expr_symtab_add_predicate(symtab, "nd_rs",
> -              "icmp6.type == 133 && icmp6.code == 0 && ip.ttl == 255");
> -    expr_symtab_add_predicate(symtab, "nd_ra",
> -              "icmp6.type == 134 && icmp6.code == 0 && ip.ttl == 255");
> -    expr_symtab_add_field(symtab, "nd.target", MFF_ND_TARGET, "nd",
> false);
> -    expr_symtab_add_field(symtab, "nd.sll", MFF_ND_SLL, "nd_ns", false);
> -    expr_symtab_add_field(symtab, "nd.tll", MFF_ND_TLL, "nd_na", false);
> -
> -    expr_symtab_add_predicate(symtab, "tcp", "ip.proto == 6");
> -    expr_symtab_add_field(symtab, "tcp.src", MFF_TCP_SRC, "tcp", false);
> -    expr_symtab_add_field(symtab, "tcp.dst", MFF_TCP_DST, "tcp", false);
> -    expr_symtab_add_field(symtab, "tcp.flags", MFF_TCP_FLAGS, "tcp",
> false);
> -
> -    expr_symtab_add_predicate(symtab, "udp", "ip.proto == 17");
> -    expr_symtab_add_field(symtab, "udp.src", MFF_UDP_SRC, "udp", false);
> -    expr_symtab_add_field(symtab, "udp.dst", MFF_UDP_DST, "udp", false);
> -
> -    expr_symtab_add_predicate(symtab, "sctp", "ip.proto == 132");
> -    expr_symtab_add_field(symtab, "sctp.src", MFF_SCTP_SRC, "sctp",
> false);
> -    expr_symtab_add_field(symtab, "sctp.dst", MFF_SCTP_DST, "sctp",
> false);
> -
> -    shash_init(&ovnfield_by_name);
> -    for (int i = 0; i < OVN_FIELD_N_IDS; i++) {
> -        const struct ovn_field *of = &ovn_fields[i];
> -        ovs_assert(of->id == i); /* Fields must be in the enum order. */
> -        shash_add_once(&ovnfield_by_name, of->name, of);
> -    }
> -    expr_symtab_add_ovn_field(symtab, "icmp4.frag_mtu",
> OVN_ICMP4_FRAG_MTU);
> -}
> -
> -const char *
> -event_to_string(enum ovn_controller_event event)
> -{
> -    switch (event) {
> -    case OVN_EVENT_EMPTY_LB_BACKENDS:
> -        return "empty_lb_backends";
> -    case OVN_EVENT_MAX:
> -    default:
> -        return "";
> -    }
> -}
> -
> -int
> -string_to_event(const char *s)
> -{
> -    if (!strcmp(s, "empty_lb_backends")) {
> -        return OVN_EVENT_EMPTY_LB_BACKENDS;
> -    }
> -    return -1;
> -}
> -
> -const struct ovn_field *
> -ovn_field_from_name(const char *name)
> -{
> -    return shash_find_data(&ovnfield_by_name, name);
> -}
> -
> -void
> -ovn_destroy_ovnfields(void)
> -{
> -    shash_destroy(&ovnfield_by_name);
> -}
> diff --git a/ovn/lib/mcast-group-index.c b/ovn/lib/mcast-group-index.c
> deleted file mode 100644
> index 740311e00..000000000
> --- a/ovn/lib/mcast-group-index.c
> +++ /dev/null
> @@ -1,43 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include "ovn/lib/mcast-group-index.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -struct ovsdb_idl_index *
> -mcast_group_index_create(struct ovsdb_idl *idl)
> -{
> -    return ovsdb_idl_index_create2(idl, &sbrec_multicast_group_col_name,
> -                                   &sbrec_multicast_group_col_datapath);
> -}
> -
> -const struct sbrec_multicast_group *
> -mcast_group_lookup(struct ovsdb_idl_index *mcgroup_index,
> -                   const char *name,
> -                   const struct sbrec_datapath_binding *datapath)
> -{
> -    struct sbrec_multicast_group *target =
> -        sbrec_multicast_group_index_init_row(mcgroup_index);
> -    sbrec_multicast_group_index_set_name(target, name);
> -    sbrec_multicast_group_index_set_datapath(target, datapath);
> -
> -    struct sbrec_multicast_group *mcgroup =
> -        sbrec_multicast_group_index_find(mcgroup_index, target);
> -    sbrec_multicast_group_index_destroy_row(target);
> -
> -    return mcgroup;
> -}
> diff --git a/ovn/lib/mcast-group-index.h b/ovn/lib/mcast-group-index.h
> deleted file mode 100644
> index 859e6a72f..000000000
> --- a/ovn/lib/mcast-group-index.h
> +++ /dev/null
> @@ -1,32 +0,0 @@
> -/* Copyright (c) 2019, Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_MCAST_GROUP_INDEX_H
> -#define OVN_MCAST_GROUP_INDEX_H 1
> -
> -struct ovsdb_idl;
> -
> -struct sbrec_datapath_binding;
> -
> -#define OVN_MCAST_FLOOD_TUNNEL_KEY   65535
> -#define OVN_MCAST_UNKNOWN_TUNNEL_KEY (OVN_MCAST_FLOOD_TUNNEL_KEY - 1)
> -
> -struct ovsdb_idl_index *mcast_group_index_create(struct ovsdb_idl *);
> -const struct sbrec_multicast_group *
> -mcast_group_lookup(struct ovsdb_idl_index *mcgroup_index,
> -                   const char *name,
> -                   const struct sbrec_datapath_binding *datapath);
> -
> -#endif /* ovn/lib/mcast-group-index.h */
> diff --git a/ovn/lib/ovn-l7.h b/ovn/lib/ovn-l7.h
> deleted file mode 100644
> index c93def450..000000000
> --- a/ovn/lib/ovn-l7.h
> +++ /dev/null
> @@ -1,322 +0,0 @@
> -/*
> - * Copyright (c) 2016 Red Hat, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#ifndef OVN_DHCP_H
> -#define OVN_DHCP_H 1
> -
> -#include <sys/types.h>
> -#include <netinet/in.h>
> -#include <netinet/icmp6.h>
> -#include "openvswitch/hmap.h"
> -#include "hash.h"
> -#include "ovn/logical-fields.h"
> -
> -/* Generic options map which is used to store dhcpv4 opts and dhcpv6
> opts. */
> -struct gen_opts_map {
> -    struct hmap_node hmap_node;
> -    char *name;
> -    char *type;
> -    size_t code;
> -};
> -
> -#define DHCP_OPTION(NAME, CODE, TYPE) \
> -    {.name = NAME, .code = CODE, .type = TYPE}
> -
> -#define OFFERIP              DHCP_OPTION("offerip", 0, "ipv4")
> -#define DHCP_OPT_NETMASK     DHCP_OPTION("netmask", 1, "ipv4")
> -#define DHCP_OPT_ROUTER      DHCP_OPTION("router", 3, "ipv4")
> -#define DHCP_OPT_DNS_SERVER  DHCP_OPTION("dns_server", 6, "ipv4")
> -#define DHCP_OPT_LOG_SERVER  DHCP_OPTION("log_server", 7, "ipv4")
> -#define DHCP_OPT_LPR_SERVER  DHCP_OPTION("lpr_server", 9, "ipv4")
> -#define DHCP_OPT_DOMAIN_NAME DHCP_OPTION("domain_name", 15, "str")
> -#define DHCP_OPT_SWAP_SERVER DHCP_OPTION("swap_server", 16, "ipv4")
> -
> -#define DHCP_OPT_POLICY_FILTER \
> -    DHCP_OPTION("policy_filter", 21, "ipv4")
> -
> -#define DHCP_OPT_ROUTER_SOLICITATION \
> -    DHCP_OPTION("router_solicitation", 32, "ipv4")
> -
> -#define DHCP_OPT_NIS_SERVER  DHCP_OPTION("nis_server", 41, "ipv4")
> -#define DHCP_OPT_NTP_SERVER  DHCP_OPTION("ntp_server", 42, "ipv4")
> -#define DHCP_OPT_SERVER_ID   DHCP_OPTION("server_id", 54, "ipv4")
> -#define DHCP_OPT_TFTP_SERVER DHCP_OPTION("tftp_server", 66, "ipv4")
> -
> -#define DHCP_OPT_CLASSLESS_STATIC_ROUTE \
> -    DHCP_OPTION("classless_static_route", 121, "static_routes")
> -#define DHCP_OPT_MS_CLASSLESS_STATIC_ROUTE \
> -    DHCP_OPTION("ms_classless_static_route", 249, "static_routes")
> -
> -#define DHCP_OPT_IP_FORWARD_ENABLE DHCP_OPTION("ip_forward_enable", 19,
> "bool")
> -#define DHCP_OPT_ROUTER_DISCOVERY DHCP_OPTION("router_discovery", 31,
> "bool")
> -#define DHCP_OPT_ETHERNET_ENCAP DHCP_OPTION("ethernet_encap", 36, "bool")
> -
> -#define DHCP_OPT_DEFAULT_TTL DHCP_OPTION("default_ttl", 23, "uint8")
> -
> -#define DHCP_OPT_TCP_TTL  DHCP_OPTION("tcp_ttl", 37, "uint8")
> -#define DHCP_OPT_MTU      DHCP_OPTION("mtu", 26, "uint16")
> -#define DHCP_OPT_LEASE_TIME DHCP_OPTION("lease_time", 51, "uint32")
> -#define DHCP_OPT_T1 DHCP_OPTION("T1", 58, "uint32")
> -#define DHCP_OPT_T2 DHCP_OPTION("T2", 59, "uint32")
> -
> -#define DHCP_OPT_BOOTFILE DHCP_OPTION("bootfile_name", 67, "str")
> -#define DHCP_OPT_WPAD DHCP_OPTION("wpad", 252, "str")
> -#define DHCP_OPT_PATH_PREFIX DHCP_OPTION("path_prefix", 210, "str")
> -#define DHCP_OPT_TFTP_SERVER_ADDRESS \
> -    DHCP_OPTION("tftp_server_address", 150, "ipv4")
> -
> -static inline uint32_t
> -gen_opt_hash(char *opt_name)
> -{
> -    return hash_string(opt_name, 0);
> -}
> -
> -static inline uint32_t
> -dhcp_opt_hash(char *opt_name)
> -{
> -    return gen_opt_hash(opt_name);
> -}
> -
> -static inline struct gen_opts_map *
> -gen_opts_find(const struct hmap *gen_opts, char *opt_name)
> -{
> -    struct gen_opts_map *gen_opt;
> -    HMAP_FOR_EACH_WITH_HASH (gen_opt, hmap_node, gen_opt_hash(opt_name),
> -                             gen_opts) {
> -        if (!strcmp(gen_opt->name, opt_name)) {
> -            return gen_opt;
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static inline struct gen_opts_map *
> -dhcp_opts_find(const struct hmap *dhcp_opts, char *opt_name)
> -{
> -    return gen_opts_find(dhcp_opts, opt_name);
> -}
> -
> -static inline void
> -gen_opt_add(struct hmap *gen_opts, char *opt_name, size_t code, char
> *type)
> -{
> -    struct gen_opts_map *gen_opt = xzalloc(sizeof *gen_opt);
> -    gen_opt->name = xstrdup(opt_name);
> -    gen_opt->code = code;
> -    gen_opt->type = xstrdup(type);
> -    hmap_insert(gen_opts, &gen_opt->hmap_node, gen_opt_hash(opt_name));
> -}
> -
> -static inline void
> -dhcp_opt_add(struct hmap *dhcp_opts, char *opt_name, size_t code, char
> *type)
> -{
> -    gen_opt_add(dhcp_opts, opt_name, code, type);
> -}
> -
> -static inline void
> -gen_opts_destroy(struct hmap *gen_opts)
> -{
> -    struct gen_opts_map *gen_opt;
> -    HMAP_FOR_EACH_POP (gen_opt, hmap_node, gen_opts) {
> -        free(gen_opt->name);
> -        free(gen_opt->type);
> -        free(gen_opt);
> -    }
> -    hmap_destroy(gen_opts);
> -}
> -
> -static inline void
> -dhcp_opts_destroy(struct hmap *dhcp_opts)
> -{
> -    gen_opts_destroy(dhcp_opts);
> -}
> -
> -OVS_PACKED(
> -struct dhcp_opt_header {
> -    uint8_t code;
> -    uint8_t len;
> -});
> -
> -#define DHCP_OPT_PAYLOAD(hdr) \
> -    (void *)((char *)hdr + sizeof(struct dhcp_opt_header))
> -
> -/* Used in the OpenFlow PACKET_IN userdata */
> -struct dhcp_opt6_header {
> -    ovs_be16 opt_code;
> -    ovs_be16 size;
> -};
> -
> -/* Supported DHCPv6 Message Types */
> -#define DHCPV6_MSG_TYPE_SOLICIT     1
> -#define DHCPV6_MSG_TYPE_ADVT        2
> -#define DHCPV6_MSG_TYPE_REQUEST     3
> -#define DHCPV6_MSG_TYPE_CONFIRM     4
> -#define DHCPV6_MSG_TYPE_REPLY       7
> -#define DHCPV6_MSG_TYPE_DECLINE     9
> -#define DHCPV6_MSG_TYPE_INFO_REQ    11
> -
> -
> -/* DHCPv6 Option codes */
> -#define DHCPV6_OPT_CLIENT_ID_CODE        1
> -#define DHCPV6_OPT_SERVER_ID_CODE        2
> -#define DHCPV6_OPT_IA_NA_CODE            3
> -#define DHCPV6_OPT_IA_ADDR_CODE          5
> -#define DHCPV6_OPT_DNS_SERVER_CODE       23
> -#define DHCPV6_OPT_DOMAIN_SEARCH_CODE    24
> -
> -#define DHCPV6_OPT_SERVER_ID \
> -    DHCP_OPTION("server_id", DHCPV6_OPT_SERVER_ID_CODE, "mac")
> -
> -#define DHCPV6_OPT_IA_ADDR  \
> -    DHCP_OPTION("ia_addr", DHCPV6_OPT_IA_ADDR_CODE, "ipv6")
> -
> -#define DHCPV6_OPT_DNS_SERVER  \
> -    DHCP_OPTION("dns_server", DHCPV6_OPT_DNS_SERVER_CODE, "ipv6")
> -
> -#define DHCPV6_OPT_DOMAIN_SEARCH \
> -    DHCP_OPTION("domain_search", DHCPV6_OPT_DOMAIN_SEARCH_CODE, "str")
> -
> -OVS_PACKED(
> -struct dhcpv6_opt_header {
> -    ovs_be16 code;
> -    ovs_be16 len;
> -});
> -
> -OVS_PACKED(
> -struct dhcpv6_opt_server_id {
> -    struct dhcpv6_opt_header opt;
> -    ovs_be16 duid_type;
> -    ovs_be16 hw_type;
> -    struct eth_addr mac;
> -});
> -
> -
> -OVS_PACKED(
> -struct dhcpv6_opt_ia_addr {
> -    struct dhcpv6_opt_header opt;
> -    struct in6_addr ipv6;
> -    ovs_be32 t1;
> -    ovs_be32 t2;
> -});
> -
> -OVS_PACKED(
> -struct dhcpv6_opt_ia_na {
> -    struct dhcpv6_opt_header opt;
> -    ovs_be32 iaid;
> -    ovs_be32 t1;
> -    ovs_be32 t2;
> -});
> -
> -#define DHCPV6_DUID_LL      3
> -#define DHCPV6_HW_TYPE_ETH  1
> -
> -#define DHCPV6_OPT_PAYLOAD(opt) \
> -    (void *)((char *)opt + sizeof(struct dhcpv6_opt_header))
> -
> -static inline struct gen_opts_map *
> -nd_ra_opts_find(const struct hmap *nd_ra_opts, char *opt_name)
> -{
> -    return gen_opts_find(nd_ra_opts, opt_name);
> -}
> -
> -static inline void
> -nd_ra_opt_add(struct hmap *nd_ra_opts, char *opt_name, size_t code,
> -               char *type)
> -{
> -    gen_opt_add(nd_ra_opts, opt_name, code, type);
> -}
> -
> -static inline void
> -nd_ra_opts_destroy(struct hmap *nd_ra_opts)
> -{
> -    gen_opts_destroy(nd_ra_opts);
> -}
> -
> -
> -#define ND_RA_FLAG_ADDR_MODE    0
> -
> -
> -/* Default values of various IPv6 Neighbor Discovery protocol options and
> - * flags. See RFC 4861 for more information.
> - * */
> -#define IPV6_ND_RA_FLAG_MANAGED_ADDR_CONFIG         0x80
> -#define IPV6_ND_RA_FLAG_OTHER_ADDR_CONFIG           0x40
> -
> -#define IPV6_ND_RA_CUR_HOP_LIMIT                    255
> -#define IPV6_ND_RA_LIFETIME                         0xffff
> -#define IPV6_ND_RA_REACHABLE_TIME                   0
> -#define IPV6_ND_RA_RETRANSMIT_TIMER                 0
> -
> -#define IPV6_ND_RA_OPT_PREFIX_ON_LINK               0x80
> -#define IPV6_ND_RA_OPT_PREFIX_AUTONOMOUS            0x40
> -#define IPV6_ND_RA_OPT_PREFIX_VALID_LIFETIME        0xffffffff
> -#define IPV6_ND_RA_OPT_PREFIX_PREFERRED_LIFETIME    0xffffffff
> -
> -static inline void
> -nd_ra_opts_init(struct hmap *nd_ra_opts)
> -{
> -    nd_ra_opt_add(nd_ra_opts, "addr_mode", ND_RA_FLAG_ADDR_MODE, "str");
> -    nd_ra_opt_add(nd_ra_opts, "slla", ND_OPT_SOURCE_LINKADDR, "mac");
> -    nd_ra_opt_add(nd_ra_opts, "prefix", ND_OPT_PREFIX_INFORMATION,
> "ipv6");
> -    nd_ra_opt_add(nd_ra_opts, "mtu", ND_OPT_MTU, "uint32");
> -}
> -
> -#define EMPTY_LB_VIP           1
> -#define EMPTY_LB_PROTOCOL      2
> -#define EMPTY_LB_LOAD_BALANCER 3
> -
> -/* Used in the OpenFlow PACKET_IN userdata */
> -struct controller_event_opt_header {
> -    ovs_be16 opt_code;
> -    ovs_be16 size;
> -};
> -
> -struct controller_event_options {
> -    struct hmap event_opts[OVN_EVENT_MAX];
> -};
> -
> -static inline void
> -controller_event_opt_add(struct controller_event_options *event_opts,
> -                         enum ovn_controller_event event_type, char
> *opt_name,
> -                         size_t opt_code, char *opt_type)
> -{
> -    gen_opt_add(&event_opts->event_opts[event_type], opt_name, opt_code,
> -                opt_type);
> -}
> -
> -static inline void
> -controller_event_opts_init(struct controller_event_options *opts)
> -{
> -    for (size_t i = 0; i < OVN_EVENT_MAX; i++) {
> -        hmap_init(&opts->event_opts[i]);
> -    }
> -    controller_event_opt_add(opts, OVN_EVENT_EMPTY_LB_BACKENDS, "vip",
> -                             EMPTY_LB_VIP, "str");
> -    controller_event_opt_add(opts, OVN_EVENT_EMPTY_LB_BACKENDS,
> "protocol",
> -                             EMPTY_LB_PROTOCOL, "str");
> -    controller_event_opt_add(opts, OVN_EVENT_EMPTY_LB_BACKENDS,
> -                             "load_balancer", EMPTY_LB_LOAD_BALANCER,
> "str");
> -}
> -
> -static inline void
> -controller_event_opts_destroy(struct controller_event_options *opts)
> -{
> -    for (size_t i = 0; i < OVN_EVENT_MAX; i++) {
> -        gen_opts_destroy(&opts->event_opts[i]);
> -    }
> -}
> -
> -#endif /* OVN_DHCP_H */
> diff --git a/ovn/lib/ovn-nb-idl.ann b/ovn/lib/ovn-nb-idl.ann
> deleted file mode 100644
> index 76d7384fc..000000000
> --- a/ovn/lib/ovn-nb-idl.ann
> +++ /dev/null
> @@ -1,9 +0,0 @@
> -# -*- python -*-
> -
> -# This code, when invoked by "ovsdb-idlc annotate" (by the build
> -# process), annotates vswitch.ovsschema with additional data that give
> -# the ovsdb-idl engine information about the types involved, so that
> -# it can generate more programmer-friendly data structures.
> -
> -s["idlPrefix"] = "nbrec_"
> -s["idlHeader"] = "\"ovn/lib/ovn-nb-idl.h\""
> diff --git a/ovn/lib/ovn-sb-idl.ann b/ovn/lib/ovn-sb-idl.ann
> deleted file mode 100644
> index e51238b92..000000000
> --- a/ovn/lib/ovn-sb-idl.ann
> +++ /dev/null
> @@ -1,29 +0,0 @@
> -# -*- python -*-
> -
> -# This code, when invoked by "ovsdb-idlc annotate" (by the build
> -# process), annotates vswitch.ovsschema with additional data that give
> -# the ovsdb-idl engine information about the types involved, so that
> -# it can generate more programmer-friendly data structures.
> -
> -s["idlPrefix"] = "sbrec_"
> -s["idlHeader"] = "\"ovn/lib/ovn-sb-idl.h\""
> -
> -s["hDecls"] = '#include "ovn/lib/ovn-util.h"'
> -
> -# Adds an integer column named 'column' to 'table' in 's'.  The column
> -# values is calculated with 'expression' based on the values of the
> columns
> -# named in the array 'dependencies'.
> -def synthesize_integer_column(s, table, column, dependencies, expression):
> -    s["tables"][table]["columns"][column] = {
> -        "type": "integer",
> -        "extensions": {
> -            "dependencies": dependencies,
> -            "parse": "row->%s = %s;" % (column, expression),
> -            "synthetic": True
> -        }
> -    }
> -
> -synthesize_integer_column(s, "Logical_Flow", "hash",
> -                          ["logical_datapath", "table_id", "pipeline",
> -                           "priority", "match", "actions"],
> -                          "sbrec_logical_flow_hash(row)")
> diff --git a/ovn/lib/ovn-util.c b/ovn/lib/ovn-util.c
> deleted file mode 100644
> index 0f07d80ac..000000000
> --- a/ovn/lib/ovn-util.c
> +++ /dev/null
> @@ -1,373 +0,0 @@
> -/*
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include "ovn-util.h"
> -#include "dirs.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-nb-idl.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ovn_util);
> -
> -static void
> -add_ipv4_netaddr(struct lport_addresses *laddrs, ovs_be32 addr,
> -                 unsigned int plen)
> -{
> -    laddrs->n_ipv4_addrs++;
> -    laddrs->ipv4_addrs = xrealloc(laddrs->ipv4_addrs,
> -        laddrs->n_ipv4_addrs * sizeof *laddrs->ipv4_addrs);
> -
> -    struct ipv4_netaddr *na = &laddrs->ipv4_addrs[laddrs->n_ipv4_addrs -
> 1];
> -
> -    na->addr = addr;
> -    na->mask = be32_prefix_mask(plen);
> -    na->network = addr & na->mask;
> -    na->plen = plen;
> -
> -    ovs_be32 bcast = addr | ~na->mask;
> -    inet_ntop(AF_INET, &addr, na->addr_s, sizeof na->addr_s);
> -    inet_ntop(AF_INET, &na->network, na->network_s, sizeof na->network_s);
> -    inet_ntop(AF_INET, &bcast, na->bcast_s, sizeof na->bcast_s);
> -}
> -
> -static void
> -add_ipv6_netaddr(struct lport_addresses *laddrs, struct in6_addr addr,
> -                 unsigned int plen)
> -{
> -    laddrs->n_ipv6_addrs++;
> -    laddrs->ipv6_addrs = xrealloc(laddrs->ipv6_addrs,
> -        laddrs->n_ipv6_addrs * sizeof *laddrs->ipv6_addrs);
> -
> -    struct ipv6_netaddr *na = &laddrs->ipv6_addrs[laddrs->n_ipv6_addrs -
> 1];
> -
> -    memcpy(&na->addr, &addr, sizeof na->addr);
> -    na->mask = ipv6_create_mask(plen);
> -    na->network = ipv6_addr_bitand(&addr, &na->mask);
> -    na->plen = plen;
> -    in6_addr_solicited_node(&na->sn_addr, &addr);
> -
> -    inet_ntop(AF_INET6, &addr, na->addr_s, sizeof na->addr_s);
> -    inet_ntop(AF_INET6, &na->sn_addr, na->sn_addr_s, sizeof
> na->sn_addr_s);
> -    inet_ntop(AF_INET6, &na->network, na->network_s, sizeof
> na->network_s);
> -}
> -
> -/* Returns true if specified address specifies a dynamic address,
> - * supporting the following formats:
> - *
> - *    "dynamic":
> - *        Both MAC and IP are to be allocated dynamically.
> - *
> - *    "xx:xx:xx:xx:xx:xx dynamic":
> - *        Use specified MAC address, but allocate an IP address
> - *        dynamically.
> - *
> - *    "dynamic x.x.x.x":
> - *        Use specified IP address, but allocate a MAC address
> - *        dynamically.
> - */
> -bool
> -is_dynamic_lsp_address(const char *address)
> -{
> -    char ipv6_s[IPV6_SCAN_LEN + 1];
> -    struct eth_addr ea;
> -    ovs_be32 ip;
> -    int n;
> -    return (!strcmp(address, "dynamic")
> -            || (ovs_scan(address, "dynamic "IP_SCAN_FMT"%n",
> -                         IP_SCAN_ARGS(&ip), &n)
> -                         && address[n] == '\0')
> -            || (ovs_scan(address, "dynamic "IP_SCAN_FMT"
> "IPV6_SCAN_FMT"%n",
> -                         IP_SCAN_ARGS(&ip), ipv6_s, &n)
> -                         && address[n] == '\0')
> -            || (ovs_scan(address, "dynamic "IPV6_SCAN_FMT"%n",
> -                         ipv6_s, &n) && address[n] == '\0')
> -            || (ovs_scan(address, ETH_ADDR_SCAN_FMT" dynamic%n",
> -                         ETH_ADDR_SCAN_ARGS(ea), &n) && address[n] ==
> '\0'));
> -}
> -
> -static bool
> -parse_and_store_addresses(const char *address, struct lport_addresses
> *laddrs,
> -                          int *ofs, bool extract_eth_addr)
> -{
> -    memset(laddrs, 0, sizeof *laddrs);
> -
> -    const char *buf = address;
> -    const char *const start = buf;
> -    int buf_index = 0;
> -    const char *buf_end = buf + strlen(address);
> -
> -    if (extract_eth_addr) {
> -        if (!ovs_scan_len(buf, &buf_index, ETH_ADDR_SCAN_FMT,
> -                          ETH_ADDR_SCAN_ARGS(laddrs->ea))) {
> -            laddrs->ea = eth_addr_zero;
> -            *ofs = 0;
> -            return false;
> -        }
> -
> -        snprintf(laddrs->ea_s, sizeof laddrs->ea_s, ETH_ADDR_FMT,
> -                 ETH_ADDR_ARGS(laddrs->ea));
> -    }
> -
> -    ovs_be32 ip4;
> -    struct in6_addr ip6;
> -    unsigned int plen;
> -    char *error;
> -
> -    /* Loop through the buffer and extract the IPv4/IPv6 addresses
> -     * and store in the 'laddrs'. Break the loop if invalid data is found.
> -     */
> -    buf += buf_index;
> -    while (buf < buf_end) {
> -        buf_index = 0;
> -        error = ip_parse_cidr_len(buf, &buf_index, &ip4, &plen);
> -        if (!error) {
> -            add_ipv4_netaddr(laddrs, ip4, plen);
> -            buf += buf_index;
> -            continue;
> -        }
> -        free(error);
> -        error = ipv6_parse_cidr_len(buf, &buf_index, &ip6, &plen);
> -        if (!error) {
> -            add_ipv6_netaddr(laddrs, ip6, plen);
> -        } else {
> -            free(error);
> -            break;
> -        }
> -        buf += buf_index;
> -    }
> -
> -    *ofs = buf - start;
> -    return true;
> -}
> -
> -/* Extracts the mac, IPv4 and IPv6 addresses from * 'address' which
> - * should be of the format "MAC [IP1 IP2 ..] .." where IPn should be a
> - * valid IPv4 or IPv6 address and stores them in the 'ipv4_addrs' and
> - * 'ipv6_addrs' fields of 'laddrs'.  There may be additional content in
> - * 'address' after "MAC [IP1 IP2 .. ]".  The value of 'ofs' that is
> - * returned indicates the offset where that additional content begins.
> - *
> - * Returns true if at least 'MAC' is found in 'address', false otherwise.
> - *
> - * The caller must call destroy_lport_addresses(). */
> -bool
> -extract_addresses(const char *address, struct lport_addresses *laddrs,
> -                  int *ofs)
> -{
> -    return parse_and_store_addresses(address, laddrs, ofs, true);
> -}
> -
> -/* Extracts the mac, IPv4 and IPv6 addresses from * 'address' which
> - * should be of the format 'MAC [IP1 IP2 ..]" where IPn should be a
> - * valid IPv4 or IPv6 address and stores them in the 'ipv4_addrs' and
> - * 'ipv6_addrs' fields of 'laddrs'.
> - *
> - * Return true if at least 'MAC' is found in 'address', false otherwise.
> - *
> - * The caller must call destroy_lport_addresses(). */
> -bool
> -extract_lsp_addresses(const char *address, struct lport_addresses *laddrs)
> -{
> -    int ofs;
> -    bool success = extract_addresses(address, laddrs, &ofs);
> -
> -    if (success && ofs < strlen(address)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_INFO_RL(&rl, "invalid syntax '%s' in address", address);
> -    }
> -
> -    return success;
> -}
> -
> -/* Extracts the IPv4 and IPv6 addresses from * 'address' which
> - * should be of the format 'IP1 IP2 .." where IPn should be a
> - * valid IPv4 or IPv6 address and stores them in the 'ipv4_addrs' and
> - * 'ipv6_addrs' fields of 'laddrs'.
> - *
> - * Return true if at least one IP address is found in 'address',
> - * false otherwise.
> - *
> - * The caller must call destroy_lport_addresses(). */
> -bool
> -extract_ip_addresses(const char *address, struct lport_addresses *laddrs)
> -{
> -    int ofs;
> -    if (parse_and_store_addresses(address, laddrs, &ofs, false)) {
> -        return (laddrs->n_ipv4_addrs || laddrs->n_ipv6_addrs);
> -    }
> -
> -    return false;
> -}
> -
> -/* Extracts the mac, IPv4 and IPv6 addresses from the
> - * "nbrec_logical_router_port" parameter 'lrp'.  Stores the IPv4 and
> - * IPv6 addresses in the 'ipv4_addrs' and 'ipv6_addrs' fields of
> - * 'laddrs', respectively.  In addition, a link local IPv6 address
> - * based on the 'mac' member of 'lrp' is added to the 'ipv6_addrs'
> - * field.
> - *
> - * Return true if a valid 'mac' address is found in 'lrp', false
> otherwise.
> - *
> - * The caller must call destroy_lport_addresses(). */
> -bool
> -extract_lrp_networks(const struct nbrec_logical_router_port *lrp,
> -                     struct lport_addresses *laddrs)
> -{
> -    memset(laddrs, 0, sizeof *laddrs);
> -
> -    if (!eth_addr_from_string(lrp->mac, &laddrs->ea)) {
> -        laddrs->ea = eth_addr_zero;
> -        return false;
> -    }
> -    snprintf(laddrs->ea_s, sizeof laddrs->ea_s, ETH_ADDR_FMT,
> -             ETH_ADDR_ARGS(laddrs->ea));
> -
> -    for (int i = 0; i < lrp->n_networks; i++) {
> -        ovs_be32 ip4;
> -        struct in6_addr ip6;
> -        unsigned int plen;
> -        char *error;
> -
> -        error = ip_parse_cidr(lrp->networks[i], &ip4, &plen);
> -        if (!error) {
> -            if (!ip4) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(5, 1);
> -                VLOG_WARN_RL(&rl, "bad 'networks' %s", lrp->networks[i]);
> -                continue;
> -            }
> -
> -            add_ipv4_netaddr(laddrs, ip4, plen);
> -            continue;
> -        }
> -        free(error);
> -
> -        error = ipv6_parse_cidr(lrp->networks[i], &ip6, &plen);
> -        if (!error) {
> -            add_ipv6_netaddr(laddrs, ip6, plen);
> -        } else {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_INFO_RL(&rl, "invalid syntax '%s' in networks",
> -                         lrp->networks[i]);
> -            free(error);
> -        }
> -    }
> -
> -    /* Always add the IPv6 link local address. */
> -    struct in6_addr lla;
> -    in6_generate_lla(laddrs->ea, &lla);
> -    add_ipv6_netaddr(laddrs, lla, 64);
> -
> -    return true;
> -}
> -
> -void
> -destroy_lport_addresses(struct lport_addresses *laddrs)
> -{
> -    free(laddrs->ipv4_addrs);
> -    free(laddrs->ipv6_addrs);
> -}
> -
> -/* Allocates a key for NAT conntrack zone allocation for a provided
> - * 'key' record and a 'type'.
> - *
> - * It is the caller's responsibility to free the allocated memory. */
> -char *
> -alloc_nat_zone_key(const struct uuid *key, const char *type)
> -{
> -    return xasprintf(UUID_FMT"_%s", UUID_ARGS(key), type);
> -}
> -
> -const char *
> -default_nb_db(void)
> -{
> -    static char *def;
> -    if (!def) {
> -        def = getenv("OVN_NB_DB");
> -        if (!def) {
> -            def = xasprintf("unix:%s/ovnnb_db.sock", ovs_rundir());
> -        }
> -    }
> -    return def;
> -}
> -
> -const char *
> -default_sb_db(void)
> -{
> -    static char *def;
> -    if (!def) {
> -        def = getenv("OVN_SB_DB");
> -        if (!def) {
> -            def = xasprintf("unix:%s/ovnsb_db.sock", ovs_rundir());
> -        }
> -    }
> -    return def;
> -}
> -
> -/* l3gateway, chassisredirect, and patch
> - * are not in this list since they are
> - * only set in the SB DB by northd
> - */
> -static const char *OVN_NB_LSP_TYPES[] = {
> -    "l2gateway",
> -    "localnet",
> -    "localport",
> -    "router",
> -    "vtep",
> -    "external",
> -};
> -
> -bool
> -ovn_is_known_nb_lsp_type(const char *type)
> -{
> -    int i;
> -
> -    if (!type || !type[0]) {
> -        return true;
> -    }
> -
> -    for (i = 0; i < ARRAY_SIZE(OVN_NB_LSP_TYPES); ++i) {
> -        if (!strcmp(OVN_NB_LSP_TYPES[i], type)) {
> -            return true;
> -        }
> -    }
> -
> -    return false;
> -}
> -
> -uint32_t
> -sbrec_logical_flow_hash(const struct sbrec_logical_flow *lf)
> -{
> -    const struct sbrec_datapath_binding *ld = lf->logical_datapath;
> -    if (!ld) {
> -        return 0;
> -    }
> -
> -    return ovn_logical_flow_hash(&ld->header_.uuid,
> -                                 lf->table_id, lf->pipeline,
> -                                 lf->priority, lf->match, lf->actions);
> -}
> -
> -uint32_t
> -ovn_logical_flow_hash(const struct uuid *logical_datapath,
> -                      uint8_t table_id, const char *pipeline,
> -                      uint16_t priority,
> -                      const char *match, const char *actions)
> -{
> -    size_t hash = uuid_hash(logical_datapath);
> -    hash = hash_2words((table_id << 16) | priority, hash);
> -    hash = hash_string(pipeline, hash);
> -    hash = hash_string(match, hash);
> -    return hash_string(actions, hash);
> -}
> diff --git a/ovn/lib/ovn-util.h b/ovn/lib/ovn-util.h
> deleted file mode 100644
> index 6d5e1dfb5..000000000
> --- a/ovn/lib/ovn-util.h
> +++ /dev/null
> @@ -1,84 +0,0 @@
> -/*
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -
> -#ifndef OVN_UTIL_H
> -#define OVN_UTIL_H 1
> -
> -#include "lib/packets.h"
> -
> -struct nbrec_logical_router_port;
> -struct sbrec_logical_flow;
> -struct uuid;
> -
> -struct ipv4_netaddr {
> -    ovs_be32 addr;            /* 192.168.10.123 */
> -    ovs_be32 mask;            /* 255.255.255.0 */
> -    ovs_be32 network;         /* 192.168.10.0 */
> -    unsigned int plen;        /* CIDR Prefix: 24. */
> -
> -    char addr_s[INET_ADDRSTRLEN + 1];     /* "192.168.10.123" */
> -    char network_s[INET_ADDRSTRLEN + 1];  /* "192.168.10.0" */
> -    char bcast_s[INET_ADDRSTRLEN + 1];    /* "192.168.10.255" */
> -};
> -
> -struct ipv6_netaddr {
> -    struct in6_addr addr;     /* fc00::1 */
> -    struct in6_addr mask;     /* ffff:ffff:ffff:ffff:: */
> -    struct in6_addr sn_addr;  /* ff02:1:ff00::1 */
> -    struct in6_addr network;  /* fc00:: */
> -    unsigned int plen;        /* CIDR Prefix: 64 */
> -
> -    char addr_s[INET6_ADDRSTRLEN + 1];    /* "fc00::1" */
> -    char sn_addr_s[INET6_ADDRSTRLEN + 1]; /* "ff02:1:ff00::1" */
> -    char network_s[INET6_ADDRSTRLEN + 1]; /* "fc00::" */
> -};
> -
> -struct lport_addresses {
> -    char ea_s[ETH_ADDR_STRLEN + 1];
> -    struct eth_addr ea;
> -    size_t n_ipv4_addrs;
> -    struct ipv4_netaddr *ipv4_addrs;
> -    size_t n_ipv6_addrs;
> -    struct ipv6_netaddr *ipv6_addrs;
> -};
> -
> -bool is_dynamic_lsp_address(const char *address);
> -bool extract_addresses(const char *address, struct lport_addresses *,
> -                       int *ofs);
> -bool extract_lsp_addresses(const char *address, struct lport_addresses *);
> -bool extract_ip_addresses(const char *address, struct lport_addresses *);
> -bool extract_lrp_networks(const struct nbrec_logical_router_port *,
> -                          struct lport_addresses *);
> -void destroy_lport_addresses(struct lport_addresses *);
> -
> -char *alloc_nat_zone_key(const struct uuid *key, const char *type);
> -
> -const char *default_nb_db(void);
> -const char *default_sb_db(void);
> -
> -struct ovsdb_idl_table_class;
> -const char *db_table_usage(struct ds *tables,
> -                           const struct ovsdb_idl_table_class *class,
> -                           int n_tables);
> -
> -bool ovn_is_known_nb_lsp_type(const char *type);
> -
> -uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *);
> -uint32_t ovn_logical_flow_hash(const struct uuid *logical_datapath,
> -                               uint8_t table_id, const char *pipeline,
> -                               uint16_t priority,
> -                               const char *match, const char *actions);
> -
> -#endif
> diff --git a/ovn/northd/.gitignore b/ovn/northd/.gitignore
> deleted file mode 100644
> index 97a59801b..000000000
> --- a/ovn/northd/.gitignore
> +++ /dev/null
> @@ -1,2 +0,0 @@
> -/ovn-northd
> -/ovn-northd.8
> diff --git a/ovn/northd/automake.mk b/ovn/northd/automake.mk
> deleted file mode 100644
> index 93aebe8b1..000000000
> --- a/ovn/northd/automake.mk
> +++ /dev/null
> @@ -1,10 +0,0 @@
> -# ovn-northd
> -bin_PROGRAMS += ovn/northd/ovn-northd
> -ovn_northd_ovn_northd_SOURCES = ovn/northd/ovn-northd.c
> -ovn_northd_ovn_northd_LDADD = \
> -       ovn/lib/libovn.la \
> -       ovsdb/libovsdb.la \
> -       lib/libopenvswitch.la
> -man_MANS += ovn/northd/ovn-northd.8
> -EXTRA_DIST += ovn/northd/ovn-northd.8.xml
> -CLEANFILES += ovn/northd/ovn-northd.8
> diff --git a/ovn/northd/ovn-northd.8.xml b/ovn/northd/ovn-northd.8.xml
> deleted file mode 100644
> index d2267de0e..000000000
> --- a/ovn/northd/ovn-northd.8.xml
> +++ /dev/null
> @@ -1,2544 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-northd" section="8" title="ovn-northd">
> -    <h1>Name</h1>
> -    <p>ovn-northd -- Open Virtual Network central control daemon</p>
> -
> -    <h1>Synopsis</h1>
> -    <p><code>ovn-northd</code> [<var>options</var>]</p>
> -
> -    <h1>Description</h1>
> -    <p>
> -      <code>ovn-northd</code> is a centralized daemon responsible for
> -      translating the high-level OVN configuration into logical
> -      configuration consumable by daemons such as
> -      <code>ovn-controller</code>.  It translates the logical network
> -      configuration in terms of conventional network concepts, taken
> -      from the OVN Northbound Database (see <code>ovn-nb</code>(5)),
> -      into logical datapath flows in the OVN Southbound Database (see
> -      <code>ovn-sb</code>(5)) below it.
> -    </p>
> -
> -    <h1>Options</h1>
> -    <dl>
> -      <dt><code>--ovnnb-db=<var>database</var></code></dt>
> -      <dd>
> -        The OVSDB database containing the OVN Northbound Database.  If the
> -        <env>OVN_NB_DB</env> environment variable is set, its value is
> used
> -        as the default.  Otherwise, the default is
> -        <code>unix:@RUNDIR@/ovnnb_db.sock</code>.
> -      </dd>
> -      <dt><code>--ovnsb-db=<var>database</var></code></dt>
> -      <dd>
> -        The OVSDB database containing the OVN Southbound Database.  If the
> -        <env>OVN_SB_DB</env> environment variable is set, its value is
> used
> -        as the default.  Otherwise, the default is
> -        <code>unix:@RUNDIR@/ovnsb_db.sock</code>.
> -      </dd>
> -    </dl>
> -    <p>
> -      <var>database</var> in the above options must be an OVSDB active or
> -      passive connection method, as described in <code>ovsdb</code>(7).
> -    </p>
> -
> -    <h2>Daemon Options</h2>
> -    <xi:include href="lib/daemon.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>Logging Options</h2>
> -    <xi:include href="lib/vlog.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>PKI Options</h2>
> -    <p>
> -      PKI configuration is required in order to use SSL for the
> connections to
> -      the Northbound and Southbound databases.
> -    </p>
> -    <xi:include href="lib/ssl.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>Other Options</h2>
> -    <xi:include href="lib/unixctl.xml"
> -     xmlns:xi="http://www.w3.org/2003/XInclude"/>
> -    <h3></h3>
> -    <xi:include href="lib/common.xml"
> -     xmlns:xi="http://www.w3.org/2003/XInclude"/>
> -
> -    <h1>Runtime Management Commands</h1>
> -    <p>
> -      <code>ovs-appctl</code> can send commands to a running
> -      <code>ovn-northd</code> process.  The currently supported commands
> -      are described below.
> -      <dl>
> -      <dt><code>exit</code></dt>
> -      <dd>
> -        Causes <code>ovn-northd</code> to gracefully terminate.
> -      </dd>
> -      </dl>
> -    </p>
> -
> -    <h1>Active-Standby for High Availability</h1>
> -    <p>
> -      You may run <code>ovn-northd</code> more than once in an OVN
> deployment.
> -      OVN will automatically ensure that only one of them is active at a
> time.
> -      If multiple instances of <code>ovn-northd</code> are running and the
> -      active <code>ovn-northd</code> fails, one of the hot standby
> instances
> -      of <code>ovn-northd</code> will automatically take over.
> -    </p>
> -
> -    <h1>Logical Flow Table Structure</h1>
> -
> -    <p>
> -      One of the main purposes of <code>ovn-northd</code> is to populate
> the
> -      <code>Logical_Flow</code> table in the <code>OVN_Southbound</code>
> -      database.  This section describes how <code>ovn-northd</code> does
> this
> -      for switch and router logical datapaths.
> -    </p>
> -
> -    <h2>Logical Switch Datapaths</h2>
> -
> -    <h3>Ingress Table 0: Admission Control and Ingress Port Security -
> L2</h3>
> -
> -    <p>
> -      Ingress table 0 contains these logical flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        Priority 100 flows to drop packets with VLAN tags or multicast
> Ethernet
> -        source addresses.
> -      </li>
> -
> -      <li>
> -        Priority 50 flows that implement ingress port security for each
> enabled
> -        logical port.  For logical ports on which port security is
> enabled,
> -        these match the <code>inport</code> and the valid
> <code>eth.src</code>
> -        address(es) and advance only those packets to the next flow
> table.  For
> -        logical ports on which port security is not enabled, these
> advance all
> -        packets that match the <code>inport</code>.
> -      </li>
> -    </ul>
> -
> -    <p>
> -      There are no flows for disabled logical ports because the
> default-drop
> -      behavior of logical flow tables causes packets that ingress from
> them to
> -      be dropped.
> -    </p>
> -
> -    <h3>Ingress Table 1: Ingress Port Security - IP</h3>
> -
> -    <p>
> -      Ingress table 1 contains these logical flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each element in the port security set having one or more
> IPv4 or
> -          IPv6 addresses (or both),
> -        </p>
> -
> -        <ul>
> -          <li>
> -            Priority 90 flow to allow IPv4 traffic if it has IPv4
> addresses
> -            which match the <code>inport</code>, valid
> <code>eth.src</code>
> -            and valid <code>ip4.src</code> address(es).
> -          </li>
> -
> -          <li>
> -            Priority 90 flow to allow IPv4 DHCP discovery traffic if it
> has a
> -            valid <code>eth.src</code>. This is necessary since DHCP
> discovery
> -            messages are sent from the unspecified IPv4 address (0.0.0.0)
> since
> -            the IPv4 address has not yet been assigned.
> -          </li>
> -
> -          <li>
> -            Priority 90 flow to allow IPv6 traffic if it has IPv6
> addresses
> -            which match the <code>inport</code>, valid
> <code>eth.src</code> and
> -            valid <code>ip6.src</code> address(es).
> -          </li>
> -
> -          <li>
> -            Priority 90 flow to allow IPv6 DAD (Duplicate Address
> Detection)
> -            traffic if it has a valid <code>eth.src</code>. This is is
> -            necessary since DAD include requires joining an multicast
> group and
> -            sending neighbor solicitations for the newly assigned
> address. Since
> -            no address is yet assigned, these are sent from the
> unspecified
> -            IPv6 address (::).
> -          </li>
> -
> -          <li>
> -            Priority 80 flow to drop IP (both IPv4 and IPv6) traffic which
> -            match the <code>inport</code> and valid <code>eth.src</code>.
> -          </li>
> -        </ul>
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and
> advances to
> -        the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 2: Ingress Port Security - Neighbor discovery</h3>
> -
> -    <p>
> -      Ingress table 2 contains these logical flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each element in the port security set,
> -        </p>
> -
> -        <ul>
> -          <li>
> -            Priority 90 flow to allow ARP traffic which match the
> -            <code>inport</code> and valid <code>eth.src</code> and
> -            <code>arp.sha</code>. If the element has one or more
> -            IPv4 addresses, then it also matches the valid
> -            <code>arp.spa</code>.
> -          </li>
> -
> -          <li>
> -            Priority 90 flow to allow IPv6 Neighbor Solicitation and
> -            Advertisement traffic which match the <code>inport</code>,
> -            valid <code>eth.src</code> and
> -            <code>nd.sll</code>/<code>nd.tll</code>.
> -            If the element has one or more IPv6 addresses, then it also
> -            matches the valid <code>nd.target</code> address(es) for
> Neighbor
> -            Advertisement traffic.
> -          </li>
> -
> -          <li>
> -            Priority 80 flow to drop ARP and IPv6 Neighbor Solicitation
> and
> -            Advertisement traffic which match the <code>inport</code> and
> -            valid <code>eth.src</code>.
> -          </li>
> -        </ul>
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and
> advances to
> -        the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 3: <code>from-lport</code> Pre-ACLs</h3>
> -
> -    <p>
> -      This table prepares flows for possible stateful ACL processing in
> -      ingress table <code>ACLs</code>.  It contains a priority-0 flow that
> -      simply moves traffic to the next table.  If stateful ACLs are used
> in the
> -      logical datapath, a priority-100 flow is added that sets a hint
> -      (with <code>reg0[0] = 1; next;</code>) for table
> -      <code>Pre-stateful</code> to send IP packets to the connection
> tracker
> -      before eventually advancing to ingress table <code>ACLs</code>. If
> -      special ports such as route ports or localnet ports can't use ct(),
> a
> -      priority-110 flow is added to skip over stateful ACLs.
> -    </p>
> -
> -    <h3>Ingress Table 4: Pre-LB</h3>
> -
> -    <p>
> -      This table prepares flows for possible stateful load balancing
> processing
> -      in ingress table <code>LB</code> and <code>Stateful</code>.  It
> contains
> -      a priority-0 flow that simply moves traffic to the next table.
> Moreover
> -      it contains a priority-110 flow to move IPv6 Neighbor Discovery
> traffic
> -      to the next table. If load balancing rules with virtual IP addresses
> -      (and ports) are configured in <code>OVN_Northbound</code> database
> for a
> -      logical switch datapath, a priority-100 flow is added for each
> configured
> -      virtual IP address <var>VIP</var>. For IPv4 <var>VIPs</var>, the
> match is
> -      <code>ip &amp;&amp; ip4.dst == <var>VIP</var></code>. For IPv6
> -      <var>VIPs</var>, the match is <code>ip &amp;&amp;
> -      ip6.dst == <var>VIP</var></code>. The flow sets an action
> -      <code>reg0[0] = 1; next;</code> to act as a hint for table
> -      <code>Pre-stateful</code> to send IP packets to the connection
> tracker
> -      for packet de-fragmentation before eventually advancing to ingress
> table
> -      <code>LB</code>.
> -    </p>
> -
> -    <h3>Ingress Table 5: Pre-stateful</h3>
> -
> -    <p>
> -      This table prepares flows for all possible stateful processing
> -      in next tables.  It contains a priority-0 flow that simply moves
> -      traffic to the next table.  A priority-100 flow sends the packets to
> -      connection tracker based on a hint provided by the previous tables
> -      (with a match for <code>reg0[0] == 1</code>) by using the
> -      <code>ct_next;</code> action.
> -    </p>
> -
> -    <h3>Ingress table 6: <code>from-lport</code> ACLs</h3>
> -
> -    <p>
> -      Logical flows in this table closely reproduce those in the
> -      <code>ACL</code> table in the <code>OVN_Northbound</code> database
> -      for the <code>from-lport</code> direction. The <code>priority</code>
> -      values from the <code>ACL</code> table have a limited range and have
> -      1000 added to them to leave room for OVN default flows at both
> -      higher and lower priorities.
> -    </p>
> -    <ul>
> -      <li>
> -        <code>allow</code> ACLs translate into logical flows with
> -        the <code>next;</code> action.  If there are any stateful ACLs
> -        on this datapath, then <code>allow</code> ACLs translate to
> -        <code>ct_commit; next;</code> (which acts as a hint for the next
> tables
> -        to commit the connection to conntrack),
> -      </li>
> -      <li>
> -        <code>allow-related</code> ACLs translate into logical
> -        flows with the <code>ct_commit(ct_label=0/1); next;</code> actions
> -        for new connections and <code>reg0[1] = 1; next;</code> for
> existing
> -        connections.
> -      </li>
> -      <li>
> -        Other ACLs translate to <code>drop;</code> for new or untracked
> -        connections and <code>ct_commit(ct_label=1/1);</code> for known
> -        connections.  Setting <code>ct_label</code> marks a connection
> -        as one that was previously allowed, but should no longer be
> -        allowed due to a policy change.
> -      </li>
> -    </ul>
> -
> -    <p>
> -      This table also contains a priority 0 flow with action
> -      <code>next;</code>, so that ACLs allow packets by default.  If the
> -      logical datapath has a statetful ACL, the following flows will
> -      also be added:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        A priority-1 flow that sets the hint to commit IP traffic to the
> -        connection tracker (with action <code>reg0[1] = 1;
> next;</code>).  This
> -        is needed for the default allow policy because, while the
> initiator's
> -        direction may not have any stateful rules, the server's may and
> then
> -        its return traffic would not be known and marked as invalid.
> -      </li>
> -
> -      <li>
> -        A priority-65535 flow that allows any traffic in the reply
> -        direction for a connection that has been committed to the
> -        connection tracker (i.e., established flows), as long as
> -        the committed flow does not have <code>ct_label.blocked</code>
> set.
> -        We only handle traffic in the reply direction here because
> -        we want all packets going in the request direction to still
> -        go through the flows that implement the currently defined
> -        policy based on ACLs.  If a connection is no longer allowed by
> -        policy, <code>ct_label.blocked</code> will get set and packets in
> the
> -        reply direction will no longer be allowed, either.
> -      </li>
> -
> -      <li>
> -        A priority-65535 flow that allows any traffic that is considered
> -        related to a committed flow in the connection tracker (e.g., an
> -        ICMP Port Unreachable from a non-listening UDP port), as long
> -        as the committed flow does not have <code>ct_label.blocked</code>
> set.
> -      </li>
> -
> -      <li>
> -        A priority-65535 flow that drops all traffic marked by the
> -        connection tracker as invalid.
> -      </li>
> -
> -      <li>
> -        A priority-65535 flow that drops all traffic in the reply
> direction
> -        with <code>ct_label.blocked</code> set meaning that the connection
> -        should no longer be allowed due to a policy change.  Packets
> -        in the request direction are skipped here to let a newly created
> -        ACL re-allow this connection.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 7: <code>from-lport</code> QoS Marking</h3>
> -
> -    <p>
> -      Logical flows in this table closely reproduce those in the
> -      <code>QoS</code> table with the <code>action</code> column set in
> -      the <code>OVN_Northbound</code> database for the
> -      <code>from-lport</code> direction.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        For every qos_rules entry in a logical switch with DSCP marking
> -        enabled, a flow will be added at the priority mentioned in the
> -        QoS table.
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and
> advances to
> -        the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 8: <code>from-lport</code> QoS Meter</h3>
> -
> -    <p>
> -      Logical flows in this table closely reproduce those in the
> -      <code>QoS</code> table with the  <code>bandwidth</code> column set
> -      in the <code>OVN_Northbound</code> database for the
> -      <code>from-lport</code> direction.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        For every qos_rules entry in a logical switch with metering
> -        enabled, a flow will be added at the priorirty mentioned in the
> -        QoS table.
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and
> advances to
> -        the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 9: LB</h3>
> -
> -    <p>
> -      It contains a priority-0 flow that simply moves traffic to the next
> -      table.  For established connections a priority 100 flow matches on
> -      <code>ct.est &amp;&amp; !ct.rel &amp;&amp; !ct.new &amp;&amp;
> -      !ct.inv</code> and sets an action <code>reg0[2] = 1; next;</code>
> to act
> -      as a hint for table <code>Stateful</code> to send packets through
> -      connection tracker to NAT the packets.  (The packet will
> automatically
> -      get DNATed to the same IP address as the first packet in that
> -      connection.)
> -    </p>
> -
> -    <h3>Ingress Table 10: Stateful</h3>
> -
> -    <ul>
> -      <li>
> -        For all the configured load balancing rules for a switch in
> -        <code>OVN_Northbound</code> database that includes a L4 port
> -        <var>PORT</var> of protocol <var>P</var> and IP address
> -        <var>VIP</var>, a priority-120 flow is added.  For IPv4 <var>VIPs
> -        </var>, the flow matches <code>ct.new &amp;&amp; ip &amp;&amp;
> -        ip4.dst == <var>VIP</var> &amp;&amp; <var>P</var> &amp;&amp;
> -        <var>P</var>.dst == <var>PORT</var></code>.  For IPv6
> <var>VIPs</var>,
> -        the flow matches <code>ct.new &amp;&amp; ip &amp;&amp; ip6.dst ==
> <var>
> -        VIP </var>&amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst ==
> <var>
> -        PORT</var></code>. The flow's action is
> <code>ct_lb(<var>args</var>)
> -        </code>, where <var>args</var> contains comma separated IP
> addresses
> -        (and optional port numbers) to load balance to.  The address
> family of
> -        the IP addresses of <var>args</var> is the same as the address
> family
> -        of <var>VIP</var>
> -      </li>
> -      <li>
> -        For all the configured load balancing rules for a switch in
> -        <code>OVN_Northbound</code> database that includes just an IP
> address
> -        <var>VIP</var> to match on, OVN adds a priority-110 flow.  For
> IPv4
> -        <var>VIPs</var>, the flow matches <code>ct.new &amp;&amp; ip
> &amp;&amp;
> -        ip4.dst == <var>VIP</var></code>. For IPv6 <var>VIPs</var>,
> -        the flow matches <code>ct.new &amp;&amp; ip &amp;&amp; ip6.dst ==
> <var>
> -        VIP</var></code>. The action on this flow is <code>
> -        ct_lb(<var>args</var>)</code>, where <var>args</var> contains
> comma
> -        separated IP addresses of the same address family as
> <var>VIP</var>.
> -      </li>
> -      <li>
> -        A priority-100 flow commits packets to connection tracker using
> -        <code>ct_commit; next;</code> action based on a hint provided by
> -        the previous tables (with a match for <code>reg0[1] == 1</code>).
> -      </li>
> -      <li>
> -        A priority-100 flow sends the packets to connection tracker using
> -        <code>ct_lb;</code> as the action based on a hint provided by the
> -        previous tables (with a match for <code>reg0[2] == 1</code>).
> -      </li>
> -      <li>
> -        A priority-0 flow that simply moves traffic to the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 11: ARP/ND responder</h3>
> -
> -    <p>
> -      This table implements ARP/ND responder in a logical switch for known
> -      IPs.  The advantage of the ARP responder flow is to limit ARP
> -      broadcasts by locally responding to ARP requests without the need to
> -      send to other hypervisors.  One common case is when the inport is a
> -      logical port associated with a VIF and the broadcast is responded to
> -      on the local hypervisor rather than broadcast across the whole
> -      network and responded to by the destination VM.  This behavior is
> -      proxy ARP.
> -    </p>
> -
> -    <p>
> -      ARP requests arrive from VMs from a logical switch inport of type
> -      default.  For this case, the logical switch proxy ARP rules can be
> -      for other VMs or logical router ports.  Logical switch proxy ARP
> -      rules may be programmed both for mac binding of IP addresses on
> -      other logical switch VIF ports (which are of the default logical
> -      switch port type, representing connectivity to VMs or containers),
> -      and for mac binding of IP addresses on logical switch router type
> -      ports, representing their logical router port peers.  In order to
> -      support proxy ARP for logical router ports, an IP address must be
> -      configured on the logical switch router type port, with the same
> -      value as the peer logical router port.  The configured MAC addresses
> -      must match as well.  When a VM sends an ARP request for a
> distributed
> -      logical router port and if the peer router type port of the attached
> -      logical switch does not have an IP address configured, the ARP
> request
> -      will be broadcast on the logical switch.  One of the copies of the
> ARP
> -      request will go through the logical switch router type port to the
> -      logical router datapath, where the logical router ARP responder will
> -      generate a reply.  The MAC binding of a distributed logical router,
> -      once learned by an associated VM, is used for all that VM's
> -      communication needing routing.  Hence, the action of a VM re-arping
> for
> -      the mac binding of the logical router port should be rare.
> -    </p>
> -
> -    <p>
> -      Logical switch ARP responder proxy ARP rules can also be hit when
> -      receiving ARP requests externally on a L2 gateway port.  In this
> case,
> -      the hypervisor acting as an L2 gateway, responds to the ARP request
> on
> -      behalf of a destination VM.
> -    </p>
> -
> -    <p>
> -      Note that ARP requests received from <code>localnet</code> or
> -      <code>vtep</code> logical inports can either go directly to VMs, in
> -      which case the VM responds or can hit an ARP responder for a logical
> -      router port if the packet is used to resolve a logical router port
> -      next hop address.  In either case, logical switch ARP responder
> rules
> -      will not be hit.  It contains these logical flows:
> -     </p>
> -
> -    <ul>
> -      <li>
> -        Priority-100 flows to skip the ARP responder if inport is of type
> -        <code>localnet</code> or <code>vtep</code> and advances directly
> -        to the next table.  ARP requests sent to <code>localnet</code> or
> -        <code>vtep</code> ports can be received by multiple hypervisors.
> -        Now, because the same mac binding rules are downloaded to all
> -        hypervisors, each of the multiple hypervisors will respond.  This
> -        will confuse L2 learning on the source of the ARP requests.  ARP
> -        requests received on an inport of type <code>router</code> are not
> -        expected to hit any logical switch ARP responder flows.  However,
> -        no skip flows are installed for these packets, as there would be
> -        some additional flow cost for this and the value appears limited.
> -      </li>
> -
> -      <li>
> -        <p>
> -          Priority-50 flows that match ARP requests to each known IP
> address
> -          <var>A</var> of every logical switch port, and respond with ARP
> -          replies directly with corresponding Ethernet address
> <var>E</var>:
> -        </p>
> -
> -        <pre>
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -arp.op = 2; /* ARP reply. */
> -arp.tha = arp.sha;
> -arp.sha = <var>E</var>;
> -arp.tpa = arp.spa;
> -arp.spa = <var>A</var>;
> -outport = inport;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          These flows are omitted for logical ports (other than router
> ports or
> -          <code>localport</code> ports) that are down.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Priority-50 flows that match IPv6 ND neighbor solicitations to
> -          each known IP address <var>A</var> (and <var>A</var>'s
> -          solicited node address) of every logical switch port except of
> type
> -          router, and respond with neighbor advertisements directly with
> -          corresponding Ethernet address <var>E</var>:
> -        </p>
> -
> -        <pre>
> -nd_na {
> -    eth.src = <var>E</var>;
> -    ip6.src = <var>A</var>;
> -    nd.target = <var>A</var>;
> -    nd.tll = <var>E</var>;
> -    outport = inport;
> -    flags.loopback = 1;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          Priority-50 flows that match IPv6 ND neighbor solicitations to
> -          each known IP address <var>A</var> (and <var>A</var>'s
> -          solicited node address) of logical switch port of type router,
> and
> -          respond with neighbor advertisements directly with
> -          corresponding Ethernet address <var>E</var>:
> -        </p>
> -
> -        <pre>
> -nd_na_router {
> -    eth.src = <var>E</var>;
> -    ip6.src = <var>A</var>;
> -    nd.target = <var>A</var>;
> -    nd.tll = <var>E</var>;
> -    outport = inport;
> -    flags.loopback = 1;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          These flows are omitted for logical ports (other than router
> ports or
> -          <code>localport</code> ports) that are down.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Priority-100 flows with match criteria like the ARP and ND flows
> -          above, except that they only match packets from the
> -          <code>inport</code> that owns the IP addresses in question, with
> -          action <code>next;</code>.  These flows prevent OVN from
> replying to,
> -          for example, an ARP request emitted by a VM for its own IP
> address.
> -          A VM only makes this kind of request to attempt to detect a
> duplicate
> -          IP address assignment, so sending a reply will prevent the VM
> from
> -          accepting the IP address that it owns.
> -        </p>
> -
> -        <p>
> -          In place of <code>next;</code>, it would be reasonable to use
> -          <code>drop;</code> for the flows' actions.  If everything is
> working
> -          as it is configured, then this would produce equivalent results,
> -          since no host should reply to the request.  But ARPing for
> one's own
> -          IP address is intended to detect situations where the network
> is not
> -          working as configured, so dropping the request would frustrate
> that
> -          intent.
> -        </p>
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and
> advances to
> -        the next table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 12: DHCP option processing</h3>
> -
> -    <p>
> -      This table adds the DHCPv4 options to a DHCPv4 packet from the
> -      logical ports configured with IPv4 address(es) and DHCPv4 options,
> -      and similarly for DHCPv6 options. This table also adds flows for the
> -      logical ports of type <code>external</code>.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-100 logical flow is added for these logical ports
> -          which matches the IPv4 packet with <code>udp.src</code> = 68 and
> -          <code>udp.dst</code> = 67 and applies the action
> -          <code>put_dhcp_opts</code> and advances the packet to the next
> table.
> -        </p>
> -
> -        <pre>
> -reg0[3] = put_dhcp_opts(offer_ip = <var>ip</var>, <var>options</var>...);
> -next;
> -        </pre>
> -
> -        <p>
> -          For DHCPDISCOVER and DHCPREQUEST, this transforms the packet
> into a
> -          DHCP reply, adds the DHCP offer IP <var>ip</var> and options to
> the
> -          packet, and stores 1 into reg0[3].  For other kinds of packets,
> it
> -          just stores 0 into reg0[3].  Either way, it continues to the
> next
> -          table.
> -        </p>
> -
> -      </li>
> -
> -      <li>
> -        <p>
> -          A priority-100 logical flow is added for these logical ports
> -          which matches the IPv6 packet with <code>udp.src</code> = 546
> and
> -          <code>udp.dst</code> = 547 and applies the action
> -          <code>put_dhcpv6_opts</code> and advances the packet to the next
> -          table.
> -        </p>
> -
> -        <pre>
> -reg0[3] = put_dhcpv6_opts(ia_addr = <var>ip</var>, <var>options</var>...);
> -next;
> -        </pre>
> -
> -        <p>
> -          For DHCPv6 Solicit/Request/Confirm packets, this transforms the
> -          packet into a DHCPv6 Advertise/Reply, adds the DHCPv6 offer IP
> -          <var>ip</var> and options to the packet, and stores 1 into
> reg0[3].
> -          For other kinds of packets, it just stores 0 into reg0[3].
> Either
> -          way, it continues to the next table.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 flow that matches all packets to advances to table
> 11.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 13: DHCP responses</h3>
> -
> -    <p>
> -      This table implements DHCP responder for the DHCP replies generated
> by
> -      the previous table.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority 100 logical flow is added for the logical ports
> configured
> -          with DHCPv4 options which matches IPv4 packets with
> <code>udp.src == 68
> -          &amp;&amp; udp.dst == 67 &amp;&amp; reg0[3] == 1</code> and
> -          responds back to the <code>inport</code> after applying these
> -          actions.  If <code>reg0[3]</code> is set to 1, it means that the
> -          action <code>put_dhcp_opts</code> was successful.
> -        </p>
> -
> -        <pre>
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -ip4.dst = <var>A</var>;
> -ip4.src = <var>S</var>;
> -udp.src = 67;
> -udp.dst = 68;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          where <var>E</var> is the server MAC address and <var>S</var>
> is the
> -          server IPv4 address defined in the DHCPv4 options and
> <var>A</var> is
> -          the IPv4 address defined in the logical port's addresses column.
> -        </p>
> -
> -        <p>
> -          (This terminates ingress packet processing; the packet does not
> go
> -           to the next ingress table.)
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          A priority 100 logical flow is added for the logical ports
> configured
> -          with DHCPv6 options which matches IPv6 packets with
> <code>udp.src == 546
> -          &amp;&amp; udp.dst == 547 &amp;&amp; reg0[3] == 1</code> and
> -          responds back to the <code>inport</code> after applying these
> -          actions.  If <code>reg0[3]</code> is set to 1, it means that the
> -          action <code>put_dhcpv6_opts</code> was successful.
> -        </p>
> -
> -        <pre>
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -ip6.dst = <var>A</var>;
> -ip6.src = <var>S</var>;
> -udp.src = 547;
> -udp.dst = 546;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          where <var>E</var> is the server MAC address and <var>S</var>
> is the
> -          server IPv6 LLA address  generated from the
> <code>server_id</code>
> -          defined in the DHCPv6 options and <var>A</var> is
> -          the IPv6 address defined in the logical port's addresses column.
> -        </p>
> -
> -        <p>
> -          (This terminates packet processing; the packet does not go on
> the
> -          next ingress table.)
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 flow that matches all packets to advances to table
> 12.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 14 DNS Lookup</h3>
> -
> -    <p>
> -      This table looks up and resolves the DNS names to the corresponding
> -      configured IP address(es).
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-100 logical flow for each logical switch datapath
> -          if it is configured with DNS records, which matches the IPv4
> and IPv6
> -          packets with <code>udp.dst</code> = 53 and applies the action
> -          <code>dns_lookup</code> and advances the packet to the next
> table.
> -        </p>
> -
> -        <pre>
> -reg0[4] = dns_lookup(); next;
> -        </pre>
> -
> -        <p>
> -          For valid DNS packets, this transforms the packet into a DNS
> -          reply if the DNS name can be resolved, and stores 1 into
> reg0[4].
> -          For failed DNS resolution or other kinds of packets, it just
> stores
> -          0 into reg0[4]. Either way, it continues to the next table.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 15 DNS Responses</h3>
> -
> -    <p>
> -      This table implements DNS responder for the DNS replies generated by
> -      the previous table.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-100 logical flow for each logical switch datapath
> -          if it is configured with DNS records, which matches the IPv4
> and IPv6
> -          packets with <code>udp.dst = 53 &amp;&amp; reg0[4] == 1</code>
> -          and responds back to the <code>inport</code> after applying
> these
> -          actions.  If <code>reg0[4]</code> is set to 1, it means that the
> -          action <code>dns_lookup</code> was successful.
> -        </p>
> -
> -        <pre>
> -eth.dst &lt;-&gt; eth.src;
> -ip4.src &lt;-&gt; ip4.dst;
> -udp.dst = udp.src;
> -udp.src = 53;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          (This terminates ingress packet processing; the packet does not
> go
> -           to the next ingress table.)
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress table 16 External ports</h3>
> -
> -    <p>
> -      Traffic from the <code>external</code> logical ports enter the
> ingress
> -      datapath pipeline via the <code>localnet</code> port. This table
> adds the
> -      below logical flows to handle the traffic from these ports.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-100 flow is added for each <code>external</code>
> logical
> -          port which doesn't reside on a chassis to drop the ARP/IPv6 NS
> -          request to the router IP(s) (of the logical switch) which
> matches
> -          on the <code>inport</code> of the <code>external</code> logical
> port
> -          and the valid <code>eth.src</code> address(es) of the
> -          <code>external</code> logical port.
> -        </p>
> -
> -        <p>
> -          This flow guarantees that the ARP/NS request to the router IP
> -          address from the external ports is responded by only the chassis
> -          which has claimed these external ports. All the other chassis,
> -          drops these packets.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 flow that matches all packets to advances to table
> 17.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 17 Destination Lookup</h3>
> -
> -    <p>
> -      This table implements switching behavior.  It contains these logical
> -      flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        A priority-100 flow that outputs all packets with an Ethernet
> broadcast
> -        or multicast <code>eth.dst</code> to the <code>MC_FLOOD</code>
> -        multicast group, which <code>ovn-northd</code> populates with all
> -        enabled logical ports.
> -      </li>
> -
> -      <li>
> -        <p>
> -          One priority-50 flow that matches each known Ethernet address
> against
> -          <code>eth.dst</code> and outputs the packet to the single
> associated
> -          output port.
> -        </p>
> -
> -        <p>
> -          For the Ethernet address on a logical switch port of type
> -          <code>router</code>, when that logical switch port's
> -          <ref column="addresses" table="Logical_Switch_Port"
> -          db="OVN_Northbound"/> column is set to <code>router</code> and
> -          the connected logical router port specifies a
> -          <code>redirect-chassis</code>:
> -        </p>
> -
> -        <ul>
> -          <li>
> -            The flow for the connected logical router port's Ethernet
> -            address is only programmed on the
> <code>redirect-chassis</code>.
> -          </li>
> -
> -          <li>
> -            If the logical router has rules specified in
> -            <ref column="nat" table="Logical_Router"
> db="OVN_Northbound"/> with
> -            <ref column="external_mac" table="NAT" db="OVN_Northbound"/>,
> then
> -            those addresses are also used to populate the switch's
> destination
> -            lookup on the chassis where
> -            <ref column="logical_port" table="NAT" db="OVN_Northbound"/>
> is
> -            resident.
> -          </li>
> -        </ul>
> -
> -        <p>
> -          For the Ethernet address on a logical switch port of type
> -          <code>router</code>, when that logical switch port's
> -          <ref column="addresses" table="Logical_Switch_Port"
> -          db="OVN_Northbound"/> column is set to <code>router</code> and
> -          the connected logical router port specifies a
> -          <code>reside-on-redirect-chassis</code> and the logical router
> -          to which the connected logical router port belongs to has a
> -          <code>redirect-chassis</code> distributed gateway logical router
> -          port:
> -        </p>
> -
> -        <ul>
> -          <li>
> -            The flow for the connected logical router port's Ethernet
> -            address is only programmed on the
> <code>redirect-chassis</code>.
> -          </li>
> -        </ul>
> -      </li>
> -
> -      <li>
> -        One priority-0 fallback flow that matches all packets and outputs
> them
> -        to the <code>MC_UNKNOWN</code> multicast group, which
> -        <code>ovn-northd</code> populates with all enabled logical ports
> that
> -        accept unknown destination packets.  As a small optimization, if
> no
> -        logical ports accept unknown destination packets,
> -        <code>ovn-northd</code> omits this multicast group and logical
> flow.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 0: Pre-LB</h3>
> -
> -    <p>
> -      This table is similar to ingress table <code>Pre-LB</code>.  It
> -      contains a priority-0 flow that simply moves traffic to the next
> table.
> -      Moreover it contains a priority-110 flow to move IPv6 Neighbor
> Discovery
> -      traffic to the next table. If any load balancing rules exist for the
> -      datapath, a priority-100 flow is added with a match of
> <code>ip</code>
> -      and action of <code>reg0[0] = 1; next;</code> to act as a hint for
> -      table <code>Pre-stateful</code> to send IP packets to the connection
> -      tracker for packet de-fragmentation.
> -    </p>
> -
> -    <h3>Egress Table 1: <code>to-lport</code> Pre-ACLs</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>Pre-ACLs</code> except for
> -     <code>to-lport</code> traffic.
> -    </p>
> -
> -    <h3>Egress Table 2: Pre-stateful</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>Pre-stateful</code>.
> -    </p>
> -
> -    <h3>Egress Table 3: LB</h3>
> -    <p>
> -      This is similar to ingress table <code>LB</code>.
> -    </p>
> -
> -    <h3>Egress Table 4: <code>to-lport</code> ACLs</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>ACLs</code> except for
> -      <code>to-lport</code> ACLs.
> -    </p>
> -
> -    <p>
> -      In addition, the following flows are added.
> -    </p>
> -    <ul>
> -      <li>
> -        A priority 34000 logical flow is added for each logical port which
> -        has DHCPv4 options defined to allow the DHCPv4 reply packet and
> which has
> -        DHCPv6 options defined to allow the DHCPv6 reply packet from the
> -        <code>Ingress Table 13: DHCP responses</code>.
> -      </li>
> -
> -      <li>
> -        A priority 34000 logical flow is added for each logical switch
> datapath
> -        configured with DNS records with the match <code>udp.dst =
> 53</code>
> -        to allow the DNS reply packet from the
> -        <code>Ingress Table 15:DNS responses</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 5: <code>to-lport</code> QoS Marking</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>QoS marking</code> except
> -      they apply to <code>to-lport</code> QoS rules.
> -    </p>
> -
> -    <h3>Egress Table 6: <code>to-lport</code> QoS Meter</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>QoS meter</code> except
> -      they apply to <code>to-lport</code> QoS rules.
> -    </p>
> -
> -    <h3>Egress Table 7: Stateful</h3>
> -
> -    <p>
> -      This is similar to ingress table <code>Stateful</code> except that
> -      there are no rules added for load balancing new connections.
> -    </p>
> -
> -    <h3>Egress Table 8: Egress Port Security - IP</h3>
> -
> -    <p>
> -      This is similar to the port security logic in table
> -      <code>Ingress Port Security - IP</code> except that
> <code>outport</code>,
> -      <code>eth.dst</code>, <code>ip4.dst</code> and <code>ip6.dst</code>
> -      are checked instead of <code>inport</code>, <code>eth.src</code>,
> -      <code>ip4.src</code> and <code>ip6.src</code>
> -    </p>
> -
> -    <h3>Egress Table 9: Egress Port Security - L2</h3>
> -
> -    <p>
> -      This is similar to the ingress port security logic in ingress table
> -      <code>Admission Control and Ingress Port Security - L2</code>,
> -      but with important differences.  Most obviously,
> <code>outport</code> and
> -      <code>eth.dst</code> are checked instead of <code>inport</code> and
> -      <code>eth.src</code>.  Second, packets directed to broadcast or
> multicast
> -      <code>eth.dst</code> are always accepted instead of being subject
> to the
> -      port security rules; this is implemented through a priority-100
> flow that
> -      matches on <code>eth.mcast</code> with action <code>output;</code>.
> -      Finally, to ensure that even broadcast and multicast packets are not
> -      delivered to disabled logical ports, a priority-150 flow for each
> -      disabled logical <code>outport</code> overrides the priority-100
> flow
> -      with a <code>drop;</code> action.
> -    </p>
> -
> -    <h2>Logical Router Datapaths</h2>
> -
> -    <p>
> -      Logical router datapaths will only exist for <ref
> table="Logical_Router"
> -      db="OVN_Northbound"/> rows in the <ref db="OVN_Northbound"/>
> database
> -      that do not have <ref column="enabled" table="Logical_Router"
> -      db="OVN_Northbound"/> set to <code>false</code>
> -    </p>
> -
> -    <h3>Ingress Table 0: L2 Admission Control</h3>
> -
> -    <p>
> -      This table drops packets that the router shouldn't see at all based
> on
> -      their Ethernet headers.  It contains the following flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        Priority-100 flows to drop packets with VLAN tags or multicast
> Ethernet
> -        source addresses.
> -      </li>
> -
> -      <li>
> -        <p>
> -          For each enabled router port <var>P</var> with Ethernet address
> -          <var>E</var>, a priority-50 flow that matches <code>inport ==
> -          <var>P</var> &amp;&amp; (eth.mcast || eth.dst ==
> -          <var>E</var></code>), with action <code>next;</code>.
> -        </p>
> -
> -        <p>
> -          For the gateway port on a distributed logical router (where
> -          one of the logical router ports specifies a
> -          <code>redirect-chassis</code>), the above flow matching
> -          <code>eth.dst == <var>E</var></code> is only programmed on
> -          the gateway port instance on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          For each <code>dnat_and_snat</code> NAT rule on a distributed
> -          router that specifies an external Ethernet address <var>E</var>,
> -          a priority-50 flow that matches <code>inport == <var>GW</var>
> -          &amp;&amp; eth.dst == <var>E</var></code>, where <var>GW</var>
> -          is the logical router gateway port, with action
> -          <code>next;</code>.
> -        </p>
> -
> -        <p>
> -          This flow is only programmed on the gateway port instance on
> -          the chassis where the <code>logical_port</code> specified in
> -          the NAT rule resides.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <p>
> -      Other packets are implicitly dropped.
> -    </p>
> -
> -    <h3>Ingress Table 1: IP Input</h3>
> -
> -    <p>
> -      This table is the core of the logical router datapath
> functionality.  It
> -      contains the following flows to implement very basic IP host
> -      functionality.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          L3 admission control: A priority-100 flow drops packets that
> match
> -          any of the following:
> -        </p>
> -
> -        <ul>
> -          <li>
> -            <code>ip4.src[28..31] == 0xe</code> (multicast source)
> -          </li>
> -          <li>
> -            <code>ip4.src == 255.255.255.255</code> (broadcast source)
> -          </li>
> -          <li>
> -            <code>ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8</code>
> -            (localhost source or destination)
> -          </li>
> -          <li>
> -            <code>ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8</code>
> (zero
> -            network source or destination)
> -          </li>
> -          <li>
> -            <code>ip4.src</code> or <code>ip6.src</code> is any IP
> -            address owned by the router, unless the packet was
> recirculated
> -            due to egress loopback as indicated by
> -            <code>REGBIT_EGRESS_LOOPBACK</code>.
> -          </li>
> -          <li>
> -            <code>ip4.src</code> is the broadcast address of any IP
> network
> -            known to the router.
> -          </li>
> -        </ul>
> -      </li>
> -
> -      <li>
> -        <p>
> -          ICMP echo reply.  These flows reply to ICMP echo requests
> received
> -          for the router's IP address.  Let <var>A</var> be an IP address
> -          owned by a router port.  Then, for each <var>A</var> that is
> -          an IPv4 address, a priority-90 flow matches on
> -          <code>ip4.dst == <var>A</var></code> and
> -          <code>icmp4.type == 8 &amp;&amp; icmp4.code == 0</code>
> -          (ICMP echo request).  For each <var>A</var> that is an IPv6
> -          address, a priority-90 flow matches on
> -          <code>ip6.dst == <var>A</var></code> and
> -          <code>icmp6.type == 128 &amp;&amp; icmp6.code == 0</code>
> -          (ICMPv6 echo request).  The port of the router that receives the
> -          echo request does not matter. Also, the <code>ip.ttl</code> of
> -          the echo request packet is not checked, so it complies with
> -          RFC 1812, section 4.2.2.9. Flows for ICMPv4 echo requests use
> the
> -          following actions:
> -        </p>
> -
> -        <pre>
> -ip4.dst &lt;-&gt; ip4.src;
> -ip.ttl = 255;
> -icmp4.type = 0;
> -flags.loopback = 1;
> -next;
> -        </pre>
> -
> -        <p>
> -          Flows for ICMPv6 echo requests use the following actions:
> -        </p>
> -
> -        <pre>
> -ip6.dst &lt;-&gt; ip6.src;
> -ip.ttl = 255;
> -icmp6.type = 129;
> -flags.loopback = 1;
> -next;
> -        </pre>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Reply to ARP requests.
> -        </p>
> -
> -        <p>
> -          These flows reply to ARP requests for the router's own IP
> address
> -          and populates mac binding table of the logical router port.
> -          The ARP requests are handled only if the requestor's IP belongs
> -          to the same subnets of the logical router port.
> -          For each router port <var>P</var> that owns IP address
> <var>A</var>,
> -          which belongs to subnet <var>S</var> with prefix length
> <var>L</var>,
> -          and Ethernet address <var>E</var>, a priority-90 flow matches
> -          <code>inport == <var>P</var> &amp;&amp;
> -          arp.spa == <var>S</var>/<var>L</var> &amp;&amp; arp.op == 1
> -          &amp;&amp; arp.tpa == <var>A</var></code> (ARP request) with the
> -          following actions:
> -        </p>
> -
> -        <pre>
> -put_arp(inport, arp.spa, arp.sha);
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -arp.op = 2; /* ARP reply. */
> -arp.tha = arp.sha;
> -arp.sha = <var>E</var>;
> -arp.tpa = arp.spa;
> -arp.spa = <var>A</var>;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          For the gateway port on a distributed logical router (where
> -          one of the logical router ports specifies a
> -          <code>redirect-chassis</code>), the above flows are only
> -          programmed on the gateway port instance on the
> -          <code>redirect-chassis</code>.  This behavior avoids generation
> -          of multiple ARP responses from different chassis, and allows
> -          upstream MAC learning to point to the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          For the logical router port with the option
> -          <code>reside-on-redirect-chassis</code> set (which is
> centralized),
> -          the above flows are only programmed on the gateway port
> instance on
> -          the <code>redirect-chassis</code> (if the logical router has a
> -          distributed gateway port). This behavior avoids generation
> -          of multiple ARP responses from different chassis, and allows
> -          upstream MAC learning to point to the
> -          <code>redirect-chassis</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          These flows handles ARP requests not for router's own IP
> address.
> -          They use the SPA and SHA to populate the logical router port's
> -          mac binding table, with priority 80.  The typical use case of
> -          these flows are GARP requests handling.  For the gateway port
> -          on a distributed logical router, these flows are only programmed
> -          on the gateway port instance on the
> <code>redirect-chassis</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          These flows reply to ARP requests for the virtual IP addresses
> -          configured in the router for DNAT or load balancing.  For a
> -          configured DNAT IP address or a load balancer IPv4 VIP
> <var>A</var>,
> -          for each router port <var>P</var> with Ethernet
> -          address <var>E</var>, a priority-90 flow matches
> -          <code>inport == <var>P</var> &amp;&amp; arp.op == 1 &amp;&amp;
> -          arp.tpa == <var>A</var></code> (ARP request)
> -          with the following actions:
> -        </p>
> -
> -        <pre>
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -arp.op = 2; /* ARP reply. */
> -arp.tha = arp.sha;
> -arp.sha = <var>E</var>;
> -arp.tpa = arp.spa;
> -arp.spa = <var>A</var>;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          For the gateway port on a distributed logical router with NAT
> -          (where one of the logical router ports specifies a
> -          <code>redirect-chassis</code>):
> -        </p>
> -
> -        <ul>
> -          <li>
> -            If the corresponding NAT rule cannot be handled in a
> -            distributed manner, then this flow is only programmed on
> -            the gateway port instance on the
> -            <code>redirect-chassis</code>.  This behavior avoids
> -            generation of multiple ARP responses from different chassis,
> -            and allows upstream MAC learning to point to the
> -            <code>redirect-chassis</code>.
> -          </li>
> -
> -          <li>
> -            <p>
> -              If the corresponding NAT rule can be handled in a
> distributed
> -              manner, then this flow is only programmed on the gateway
> port
> -              instance where the <code>logical_port</code> specified in
> the
> -              NAT rule resides.
> -            </p>
> -
> -            <p>
> -              Some of the actions are different for this case, using the
> -              <code>external_mac</code> specified in the NAT rule rather
> -              than the gateway port's Ethernet address <var>E</var>:
> -            </p>
> -
> -            <pre>
> -eth.src = <var>external_mac</var>;
> -arp.sha = <var>external_mac</var>;
> -            </pre>
> -
> -            <p>
> -              This behavior avoids generation of multiple ARP responses
> -              from different chassis, and allows upstream MAC learning to
> -              point to the correct chassis.
> -            </p>
> -          </li>
> -        </ul>
> -      </li>
> -
> -      <li>
> -        ARP reply handling.  This flow uses ARP replies to populate the
> -        logical router's ARP table.  A priority-90 flow with match
> <code>arp.op
> -        == 2</code> has actions <code>put_arp(inport, arp.spa,
> -        arp.sha);</code>.
> -      </li>
> -
> -      <li>
> -        <p>
> -          Reply to IPv6 Neighbor Solicitations.  These flows reply to
> -          Neighbor Solicitation requests for the router's own IPv6
> -          address and load balancing IPv6 VIPs and populate the logical
> -          router's mac binding table.
> -        </p>
> -
> -        <p>
> -          For each router port <var>P</var> that
> -          owns IPv6 address <var>A</var>, solicited node address
> <var>S</var>,
> -          and Ethernet address <var>E</var>, a priority-90 flow matches
> -          <code>inport == <var>P</var> &amp;&amp;
> -          nd_ns &amp;&amp; ip6.dst == {<var>A</var>, <var>E</var>}
> &amp;&amp;
> -          nd.target == <var>A</var></code> with the following actions:
> -        </p>
> -
> -        <pre>
> -put_nd(inport, ip6.src, nd.sll);
> -nd_na_router {
> -    eth.src = <var>E</var>;
> -    ip6.src = <var>A</var>;
> -    nd.target = <var>A</var>;
> -    nd.tll = <var>E</var>;
> -    outport = inport;
> -    flags.loopback = 1;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          For each router port <var>P</var> that has load balancing VIP
> -          <var>A</var>, solicited node address <var>S</var>, and Ethernet
> -          address <var>E</var>, a priority-90 flow matches
> -          <code>inport == <var>P</var> &amp;&amp;
> -          nd_ns &amp;&amp; ip6.dst == {<var>A</var>, <var>E</var>}
> &amp;&amp;
> -          nd.target == <var>A</var></code> with the following actions:
> -        </p>
> -
> -        <pre>
> -put_nd(inport, ip6.src, nd.sll);
> -nd_na {
> -    eth.src = <var>E</var>;
> -    ip6.src = <var>A</var>;
> -    nd.target = <var>A</var>;
> -    nd.tll = <var>E</var>;
> -    outport = inport;
> -    flags.loopback = 1;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          For the gateway port on a distributed logical router (where
> -          one of the logical router ports specifies a
> -          <code>redirect-chassis</code>), the above flows replying to
> -          IPv6 Neighbor Solicitations are only programmed on the
> -          gateway port instance on the <code>redirect-chassis</code>.
> -          This behavior avoids generation of multiple replies from
> -          different chassis, and allows upstream MAC learning to point
> -          to the <code>redirect-chassis</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        IPv6 neighbor advertisement handling.  This flow uses neighbor
> -        advertisements to populate the logical router's mac binding
> -        table.  A priority-90 flow with match <code>nd_na</code>
> -        has actions <code>put_nd(inport, nd.target, nd.tll);</code>.
> -      </li>
> -
> -      <li>
> -        IPv6 neighbor solicitation for non-hosted addresses handling.
> -        This flow uses neighbor solicitations to populate the logical
> -        router's mac binding table (ones that were directed at the
> -        logical router would have matched the priority-90 neighbor
> -        solicitation flow already).  A priority-80 flow with match
> -        <code>nd_ns</code> has actions
> -        <code>put_nd(inport, ip6.src, nd.sll);</code>.
> -      </li>
> -
> -      <li>
> -        <p>
> -          UDP port unreachable.  Priority-80 flows generate ICMP port
> -          unreachable messages in reply to UDP datagrams directed to the
> -          router's IP address, except in the special case of gateways,
> -          which accept traffic directed to a router IP for load balancing
> -          and NAT purposes.
> -        </p>
> -
> -        <p>
> -          These flows should not match IP fragments with nonzero offset.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          TCP reset.  Priority-80 flows generate TCP reset messages in
> reply
> -          to TCP datagrams directed to the router's IP address, except in
> -          the special case of gateways, which accept traffic directed to a
> -          router IP for load balancing and NAT purposes.
> -        </p>
> -
> -        <p>
> -          These flows should not match IP fragments with nonzero offset.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Protocol or address unreachable. Priority-70 flows generate ICMP
> -          protocol or address unreachable messages for IPv4 and IPv6
> -          respectively in reply to packets directed to the router's IP
> -          address on IP protocols other than UDP, TCP, and ICMP, except
> in the
> -          special case of gateways, which accept traffic directed to a
> router
> -          IP for load balancing purposes.
> -        </p>
> -
> -        <p>
> -          These flows should not match IP fragments with nonzero offset.
> -        </p>
> -      </li>
> -
> -      <li>
> -        Drop other IP traffic to this router.  These flows drop any other
> -        traffic destined to an IP address of this router that is not
> already
> -        handled by one of the flows above, which amounts to ICMP (other
> than
> -        echo requests) and fragments with nonzero offsets.  For each IP
> address
> -        <var>A</var> owned by the router, a priority-60 flow matches
> -        <code>ip4.dst == <var>A</var></code> and drops the traffic.  An
> -        exception is made and the above flow is not added if the router
> -        port's own IP address is used to SNAT packets passing through that
> -        router.
> -      </li>
> -    </ul>
> -
> -    <p>
> -      The flows above handle all of the traffic that might be directed to
> the
> -      router itself.  The following flows (with lower priorities) handle
> the
> -      remaining traffic, potentially for forwarding:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        Drop Ethernet local broadcast.  A priority-50 flow with match
> -        <code>eth.bcast</code> drops traffic destined to the local
> Ethernet
> -        broadcast address.  By definition this traffic should not be
> forwarded.
> -      </li>
> -
> -      <li>
> -        <p>
> -          ICMP time exceeded.  For each router port <var>P</var>, whose IP
> -          address is <var>A</var>, a priority-40 flow with match
> <code>inport
> -          == <var>P</var> &amp;&amp; ip.ttl == {0, 1} &amp;&amp;
> -          !ip.later_frag</code> matches packets whose TTL has expired,
> with the
> -          following actions to send an ICMP time exceeded reply for IPv4
> and
> -          IPv6 respectively:
> -        </p>
> -
> -        <pre>
> -icmp4 {
> -    icmp4.type = 11; /* Time exceeded. */
> -    icmp4.code = 0;  /* TTL exceeded in transit. */
> -    ip4.dst = ip4.src;
> -    ip4.src = <var>A</var>;
> -    ip.ttl = 255;
> -    next;
> -};
> -
> -icmp6 {
> -    icmp6.type = 3; /* Time exceeded. */
> -    icmp6.code = 0;  /* TTL exceeded in transit. */
> -    ip6.dst = ip6.src;
> -    ip6.src = <var>A</var>;
> -    ip.ttl = 255;
> -    next;
> -};
> -        </pre>
> -      </li>
> -
> -      <li>
> -        TTL discard.  A priority-30 flow with match <code>ip.ttl == {0,
> -        1}</code> and actions <code>drop;</code> drops other packets
> whose TTL
> -        has expired, that should not receive a ICMP error reply (i.e.
> fragments
> -        with nonzero offset).
> -      </li>
> -
> -      <li>
> -        Next table.  A priority-0 flows match all packets that aren't
> already
> -        handled and uses actions <code>next;</code> to feed them to the
> next
> -        table.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 2: DEFRAG</h3>
> -
> -    <p>
> -      This is to send packets to connection tracker for tracking and
> -      defragmentation.  It contains a priority-0 flow that simply moves
> traffic
> -      to the next table.  If load balancing rules with virtual IP
> addresses
> -      (and ports) are configured in <code>OVN_Northbound</code> database
> for a
> -      Gateway router, a priority-100 flow is added for each configured
> virtual
> -      IP address <var>VIP</var>. For IPv4 <var>VIPs</var> the flow matches
> -      <code>ip &amp;&amp; ip4.dst == <var>VIP</var></code>.  For IPv6
> -      <var>VIPs</var>, the flow matches <code>ip &amp;&amp; ip6.dst ==
> -      <var>VIP</var></code>.  The flow uses the action
> <code>ct_next;</code>
> -      to send IP packets to the connection tracker for packet
> de-fragmentation
> -      and tracking before sending it to the next table.
> -    </p>
> -
> -    <h3>Ingress Table 3: UNSNAT</h3>
> -
> -    <p>
> -      This is for already established connections' reverse traffic.
> -      i.e., SNAT has already been done in egress pipeline and now the
> -      packet has entered the ingress pipeline as part of a reply.  It is
> -      unSNATted here.
> -    </p>
> -
> -    <p>Ingress Table 3: UNSNAT on Gateway Routers</p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          If the Gateway router has been configured to force SNAT any
> -          previously DNATted packets to <var>B</var>, a priority-110 flow
> -          matches <code>ip &amp;&amp; ip4.dst == <var>B</var></code> with
> -          an action <code>ct_snat; </code>.
> -        </p>
> -
> -        <p>
> -          If the Gateway router has been configured to force SNAT any
> -          previously load-balanced packets to <var>B</var>, a
> priority-100 flow
> -          matches <code>ip &amp;&amp; ip4.dst == <var>B</var></code> with
> -          an action <code>ct_snat; </code>.
> -        </p>
> -
> -        <p>
> -          For each NAT configuration in the OVN Northbound database, that
> asks
> -          to change the source IP address of a packet from <var>A</var> to
> -          <var>B</var>, a priority-90 flow matches <code>ip &amp;&amp;
> -          ip4.dst == <var>B</var></code> with an action
> -          <code>ct_snat; </code>.
> -        </p>
> -
> -        <p>
> -          A priority-0 logical flow with match <code>1</code> has actions
> -          <code>next;</code>.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <p>Ingress Table 3: UNSNAT on Distributed Routers</p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the source IP address of a packet from <var>A</var> to
> -          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
> -          ip4.dst == <var>B</var> &amp;&amp; inport ==
> <var>GW</var></code>,
> -          where <var>GW</var> is the logical router gateway port, with an
> -          action <code>ct_snat;</code>.
> -        </p>
> -
> -        <p>
> -          If the NAT rule cannot be handled in a distributed manner, then
> -          the priority-100 flow above is only programmed on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the source IP address of a packet from <var>A</var> to
> -          <var>B</var>, a priority-50 flow matches <code>ip &amp;&amp;
> -          ip4.dst == <var>B</var></code> with an action
> -          <code>REGBIT_NAT_REDIRECT = 1; next;</code>.  This flow is for
> -          east/west traffic to a NAT destination IPv4 address.  By
> -          setting the <code>REGBIT_NAT_REDIRECT</code> flag, in the
> -          ingress table <code>Gateway Redirect</code> this will trigger a
> -          redirect to the instance of the gateway port on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          A priority-0 logical flow with match <code>1</code> has actions
> -          <code>next;</code>.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 4: DNAT</h3>
> -
> -    <p>
> -      Packets enter the pipeline with destination IP address that needs to
> -      be DNATted from a virtual IP address to a real IP address.  Packets
> -      in the reverse direction needs to be unDNATed.
> -    </p>
> -
> -    <p>Ingress Table 4: Load balancing DNAT rules</p>
> -
> -    <p>
> -      Following load balancing DNAT flows are added for Gateway router or
> -      Router with gateway port. These flows are programmed only on the
> -      <code>redirect-chassis</code>.  These flows do not get programmed
> for
> -      load balancers with IPv6 <var>VIPs</var>.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        For all the configured load balancing rules for a Gateway router
> or
> -        Router with gateway port in <code>OVN_Northbound</code> database
> that
> -        includes a L4 port <var>PORT</var> of protocol <var>P</var> and
> IPv4
> -        address <var>VIP</var>, a priority-120 flow that matches on
> -        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var>
> -        &amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
> -        </var></code> with an action of
> <code>ct_lb(<var>args</var>)</code>,
> -        where <var>args</var> contains comma separated IPv4 addresses (and
> -        optional port numbers) to load balance to.  If the router is
> configured
> -        to force SNAT any load-balanced packets, the above action will be
> -        replaced by <code>flags.force_snat_for_lb = 1;
> -        ct_lb(<var>args</var>);</code>.
> -      </li>
> -
> -      <li>
> -        For all the configured load balancing rules for a router in
> -        <code>OVN_Northbound</code> database that includes a L4 port
> -        <var>PORT</var> of protocol <var>P</var> and IPv4 address
> -        <var>VIP</var>, a priority-120 flow that matches on
> -        <code>ct.est &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var>
> -        &amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
> -        </var></code> with an action of <code>ct_dnat;</code>. If the
> router is
> -        configured to force SNAT any load-balanced packets, the above
> action
> -        will be replaced by <code>flags.force_snat_for_lb = 1;
> ct_dnat;</code>.
> -      </li>
> -
> -      <li>
> -        For all the configured load balancing rules for a router in
> -        <code>OVN_Northbound</code> database that includes just an IP
> address
> -        <var>VIP</var> to match on, a priority-110 flow that matches on
> -        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst ==
> -        <var>VIP</var></code> with an action of
> -        <code>ct_lb(<var>args</var>)</code>, where <var>args</var>
> contains
> -        comma separated IPv4 addresses.  If the router is configured to
> force
> -        SNAT any load-balanced packets, the above action will be replaced
> by
> -        <code>flags.force_snat_for_lb = 1; ct_lb(<var>args</var>);</code>.
> -      </li>
> -
> -      <li>
> -        For all the configured load balancing rules for a router in
> -        <code>OVN_Northbound</code> database that includes just an IP
> address
> -        <var>VIP</var> to match on, a priority-110 flow that matches on
> -        <code>ct.est &amp;&amp; ip &amp;&amp; ip4.dst ==
> -        <var>VIP</var></code> with an action of <code>ct_dnat;</code>.
> -        If the router is configured to force SNAT any load-balanced
> -        packets, the above action will be replaced by
> -        <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
> -      </li>
> -    </ul>
> -
> -    <p>Ingress Table 4: DNAT on Gateway Routers</p>
> -
> -    <ul>
> -      <li>
> -        For each configuration in the OVN Northbound database, that asks
> -        to change the destination IP address of a packet from
> <var>A</var> to
> -        <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
> -        ip4.dst == <var>A</var></code> with an action
> -        <code>flags.loopback = 1; ct_dnat(<var>B</var>);</code>.  If the
> -        Gateway router is configured to force SNAT any DNATed packet,
> -        the above action will be replaced by
> -        <code>flags.force_snat_for_dnat = 1; flags.loopback = 1;
> -        ct_dnat(<var>B</var>);</code>.
> -      </li>
> -
> -      <li>
> -        For all IP packets of a Gateway router, a priority-50 flow with an
> -        action <code>flags.loopback = 1; ct_dnat;</code>.
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <p>Ingress Table 4: DNAT on Distributed Routers</p>
> -
> -    <p>
> -      On distributed routers, the DNAT table only handles packets
> -      with destination IP address that needs to be DNATted from a
> -      virtual IP address to a real IP address.  The unDNAT processing
> -      in the reverse direction is handled in a separate table in the
> -      egress pipeline.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the destination IP address of a packet from
> <var>A</var> to
> -          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
> -          ip4.dst == <var>B</var> &amp;&amp; inport ==
> <var>GW</var></code>,
> -          where <var>GW</var> is the logical router gateway port, with an
> -          action <code>ct_dnat(<var>B</var>);</code>.
> -        </p>
> -
> -        <p>
> -          If the NAT rule cannot be handled in a distributed manner, then
> -          the priority-100 flow above is only programmed on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the destination IP address of a packet from
> <var>A</var> to
> -          <var>B</var>, a priority-50 flow matches <code>ip &amp;&amp;
> -          ip4.dst == <var>B</var></code> with an action
> -          <code>REGBIT_NAT_REDIRECT = 1; next;</code>.  This flow is for
> -          east/west traffic to a NAT destination IPv4 address.  By
> -          setting the <code>REGBIT_NAT_REDIRECT</code> flag, in the
> -          ingress table <code>Gateway Redirect</code> this will trigger a
> -          redirect to the instance of the gateway port on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          A priority-0 logical flow with match <code>1</code> has actions
> -          <code>next;</code>.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 5: IPv6 ND RA option processing</h3>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-50 logical flow is added for each logical router port
> -          configured with IPv6 ND RA options which matches IPv6 ND Router
> -          Solicitation packet and applies the action
> -          <code>put_nd_ra_opts</code> and advances the packet to the next
> -          table.
> -        </p>
> -
> -        <pre>
> -reg0[5] = put_nd_ra_opts(<var>options</var>);next;
> -        </pre>
> -
> -        <p>
> -          For a valid IPv6 ND RS packet, this transforms the packet into
> an
> -          IPv6 ND RA reply and sets the RA options to the packet and
> stores 1
> -          into reg0[5]. For other kinds of packets, it just stores 0 into
> -          reg0[5]. Either way, it continues to the next table.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 6: IPv6 ND RA responder</h3>
> -
> -    <p>
> -      This table implements IPv6 ND RA responder for the IPv6 ND RA
> replies
> -      generated by the previous table.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          A priority-50 logical flow is added for each logical router port
> -          configured with IPv6 ND RA options which matches IPv6 ND RA
> -          packets and <code>reg0[5] == 1</code> and responds back to the
> -          <code>inport</code> after applying these actions.
> -          If <code>reg0[5]</code> is set to 1, it means that the action
> -          <code>put_nd_ra_opts</code> was successful.
> -        </p>
> -
> -        <pre>
> -eth.dst = eth.src;
> -eth.src = <var>E</var>;
> -ip6.dst = ip6.src;
> -ip6.src = <var>I</var>;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -output;
> -        </pre>
> -
> -        <p>
> -          where <var>E</var> is the MAC address and <var>I</var> is the
> IPv6
> -          link local address of the logical router port.
> -        </p>
> -
> -        <p>
> -          (This terminates packet processing in ingress pipeline; the
> packet
> -          does not go to the next ingress table.)
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 7: IP Routing</h3>
> -
> -    <p>
> -      A packet that arrives at this table is an IP packet that should be
> -      routed to the address in <code>ip4.dst</code> or
> -      <code>ip6.dst</code>.  This table implements IP routing, setting
> -      <code>reg0</code> (or <code>xxreg0</code> for IPv6) to the next-hop
> IP
> -      address (leaving <code>ip4.dst</code> or <code>ip6.dst</code>, the
> -      packet's final destination, unchanged) and advances to the next
> -      table for ARP resolution.  It also sets <code>reg1</code> (or
> -      <code>xxreg1</code>) to the IP address owned by the selected router
> -      port (ingress table <code>ARP Request</code> will generate an ARP
> -      request, if needed, with <code>reg0</code> as the target protocol
> -      address and <code>reg1</code> as the source protocol address).
> -    </p>
> -
> -    <p>
> -      This table contains the following logical flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For distributed logical routers where one of the logical router
> -          ports specifies a <code>redirect-chassis</code>, a priority-400
> -          logical flow for each ip source/destination couple that matches
> the
> -          <code>dnat_and_snat</code> NAT rules configured. These flows
> will
> -          allow to properly forward traffic to the external connections if
> -          available and avoid sending it through the tunnel.
> -          Assuming the two following NAT rules have been configured:
> -        </p>
> -
> -        <pre>
> -external_ip{0,1} = <var>EIP{0,1}</var>;
> -external_mac{0,1} = <var>MAC{0,1}</var>;
> -logical_ip{0,1} = <var>LIP{0,1}</var>;
> -        </pre>
> -
> -        <p>
> -            the following action will be applied:
> -        </p>
> -
> -        <pre>
> -eth.dst = <var>MAC0</var>;
> -eth.src = <var>MAC1</var>;
> -reg0 = ip4.dst;
> -reg1 = <var>EIP1</var>;
> -outport = <code>redirect-chassis-port</code>;
> -<code>REGBIT_DISTRIBUTED_NAT = 1; next;</code>.
> -        </pre>
> -
> -        <p>
> -            Morover a priority-400 logical flow is configured for each
> -            <code>dnat_and_snat</code> NAT rule configured in order to
> -            not send traffic for local FIP through the overlay tunnels
> -            but manage it in the local hypervisor
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          For distributed logical routers where one of the logical router
> -          ports specifies a <code>redirect-chassis</code>, a priority-300
> -          logical flow with match <code>REGBIT_NAT_REDIRECT == 1</code>
> has
> -          actions <code>ip.ttl--; next;</code>.  The <code>outport</code>
> -          will be set later in the Gateway Redirect table.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          IPv4 routing table.  For each route to IPv4 network
> <var>N</var> with
> -          netmask <var>M</var>, on router port <var>P</var> with IP
> address
> -          <var>A</var> and Ethernet
> -          address <var>E</var>, a logical flow with match <code>ip4.dst ==
> -          <var>N</var>/<var>M</var></code>, whose priority is the number
> of
> -          1-bits in <var>M</var>, has the following actions:
> -        </p>
> -
> -        <pre>
> -ip.ttl--;
> -reg0 = <var>G</var>;
> -reg1 = <var>A</var>;
> -eth.src = <var>E</var>;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -next;
> -        </pre>
> -
> -        <p>
> -          (Ingress table 1 already verified that <code>ip.ttl--;</code>
> will
> -          not yield a TTL exceeded error.)
> -        </p>
> -
> -        <p>
> -          If the route has a gateway, <var>G</var> is the gateway IP
> address.
> -          Instead, if the route is from a configured static route,
> <var>G</var>
> -          is the next hop IP address.  Else it is <code>ip4.dst</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          IPv6 routing table.  For each route to IPv6 network
> -          <var>N</var> with netmask <var>M</var>, on router port
> -          <var>P</var> with IP address <var>A</var> and Ethernet address
> -          <var>E</var>, a logical flow with match in CIDR notation
> -          <code>ip6.dst == <var>N</var>/<var>M</var></code>,
> -          whose priority is the integer value of <var>M</var>, has the
> -          following actions:
> -        </p>
> -
> -        <pre>
> -ip.ttl--;
> -xxreg0 = <var>G</var>;
> -xxreg1 = <var>A</var>;
> -eth.src = <var>E</var>;
> -outport = <var>P</var>;
> -flags.loopback = 1;
> -next;
> -        </pre>
> -
> -        <p>
> -          (Ingress table 1 already verified that <code>ip.ttl--;</code>
> will
> -          not yield a TTL exceeded error.)
> -        </p>
> -
> -        <p>
> -          If the route has a gateway, <var>G</var> is the gateway IP
> address.
> -          Instead, if the route is from a configured static route,
> <var>G</var>
> -          is the next hop IP address.  Else it is <code>ip6.dst</code>.
> -        </p>
> -
> -        <p>
> -          If the address <var>A</var> is in the link-local scope, the
> -          route will be limited to sending on the ingress port.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 8: ARP/ND Resolution</h3>
> -
> -    <p>
> -      Any packet that reaches this table is an IP packet whose next-hop
> -      IPv4 address is in <code>reg0</code> or IPv6 address is in
> -      <code>xxreg0</code>.  (<code>ip4.dst</code> or
> -      <code>ip6.dst</code> contains the final destination.)  This table
> -      resolves the IP address in <code>reg0</code> (or
> -      <code>xxreg0</code>) into an output port in <code>outport</code>
> -      and an Ethernet address in <code>eth.dst</code>, using the
> -      following flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For distributed logical routers where one of the logical router
> -          ports specifies a <code>redirect-chassis</code>, a priority-400
> -          logical flow with match <code>REGBIT_DISTRIBUTED_NAT == 1</code>
> -          has action <code>next;</code>
> -        </p>
> -        <p>
> -          For distributed logical routers where one of the logical router
> -          ports specifies a <code>redirect-chassis</code>, a priority-200
> -          logical flow with match <code>REGBIT_NAT_REDIRECT == 1</code>
> has
> -          actions <code>eth.dst = <var>E</var>; next;</code>, where
> -          <var>E</var> is the ethernet address of the router's distributed
> -          gateway port.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Static MAC bindings.  MAC bindings can be known statically
> based on
> -          data in the <code>OVN_Northbound</code> database.  For router
> ports
> -          connected to logical switches, MAC bindings can be known
> statically
> -          from the <code>addresses</code> column in the
> -          <code>Logical_Switch_Port</code> table.  For router ports
> -          connected to other logical routers, MAC bindings can be known
> -          statically from the <code>mac</code> and <code>networks</code>
> -          column in the <code>Logical_Router_Port</code> table.
> -        </p>
> -
> -        <p>
> -          For each IPv4 address <var>A</var> whose host is known to have
> -          Ethernet address <var>E</var> on router port <var>P</var>, a
> -          priority-100 flow with match <code>outport === <var>P</var>
> -          &amp;&amp; reg0 == <var>A</var></code> has actions
> -          <code>eth.dst = <var>E</var>; next;</code>.
> -        </p>
> -
> -        <p>
> -          For each IPv6 address <var>A</var> whose host is known to have
> -          Ethernet address <var>E</var> on router port <var>P</var>, a
> -          priority-100 flow with match <code>outport === <var>P</var>
> -          &amp;&amp; xxreg0 == <var>A</var></code> has actions
> -          <code>eth.dst = <var>E</var>; next;</code>.
> -        </p>
> -
> -        <p>
> -          For each logical router port with an IPv4 address <var>A</var>
> and
> -          a mac address of <var>E</var> that is reachable via a different
> -          logical router port <var>P</var>, a priority-100 flow with
> -          match <code>outport === <var>P</var> &amp;&amp; reg0 ==
> -          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
> -          next;</code>.
> -        </p>
> -
> -        <p>
> -          For each logical router port with an IPv6 address <var>A</var>
> and
> -          a mac address of <var>E</var> that is reachable via a different
> -          logical router port <var>P</var>, a priority-100 flow with
> -          match <code>outport === <var>P</var> &amp;&amp; xxreg0 ==
> -          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
> -          next;</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          Dynamic MAC bindings.  These flows resolve MAC-to-IP bindings
> -          that have become known dynamically through ARP or neighbor
> -          discovery.  (The ingress table <code>ARP Request</code> will
> -          issue an ARP or neighbor solicitation request for cases where
> -          the binding is not yet known.)
> -        </p>
> -
> -        <p>
> -          A priority-0 logical flow with match <code>ip4</code> has
> actions
> -          <code>get_arp(outport, reg0); next;</code>.
> -        </p>
> -
> -        <p>
> -          A priority-0 logical flow with match <code>ip6</code> has
> actions
> -          <code>get_nd(outport, xxreg0); next;</code>.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 9: Check packet length</h3>
> -
> -    <p>
> -      For distributed logical routers with distributed gateway port
> configured
> -      with <code>options:gateway_mtu</code> to a valid integer value, this
> -      table adds a priority-50 logical flow with the match
> -      <code>ip4 &amp;&amp; outport == <var>GW_PORT</var></code> where
> -      <var>GW_PORT</var> is the distributed gateway router port and
> applies the
> -      action <code>check_pkt_larger</code> and advances the packet to the
> -      next table.
> -    </p>
> -
> -    <pre>
> -REGBIT_PKT_LARGER = check_pkt_larger(<var>L</var>); next;
> -    </pre>
> -
> -    <p>
> -      where <var>L</var> is the packet length to check for. If the packet
> -      is larger than <var>L</var>, it stores 1 in the register bit
> -      <code>REGBIT_PKT_LARGER</code>. The value of
> -      <var>L</var> is taken from <ref column="options:gateway_mtu"
> -      table="Logical_Router_Port" db="OVN_Northbound"/> column of
> -      <ref table="Logical_Router_Port" db="OVN_Northbound"/> row.
> -    </p>
> -
> -    <p>
> -      This table adds one priority-0 fallback flow that matches all
> packets
> -      and advances to the next table.
> -    </p>
> -
> -    <h3>Ingress Table 10: Handle larger packets</h3>
> -
> -    <p>
> -      For distributed logical routers with distributed gateway port
> configured
> -      with <code>options:gateway_mtu</code> to a valid integer value, this
> -      table adds the following priority-50 logical flow for each
> -      logical router port with the match <code>ip4 &amp;&amp;
> -      inport == <var>LRP</var> &amp;&amp; outport == <var>GW_PORT</var>
> -      &amp;&amp; REGBIT_PKT_LARGER</code>, where <var>LRP</var> is the
> logical
> -      router port and <var>GW_PORT</var> is the distributed gateway
> router port
> -      and applies the following action
> -    </p>
> -
> -    <pre>
> -icmp4 {
> -    icmp4.type = 3; /* Destination Unreachable. */
> -    icmp4.code = 4;  /* Frag Needed and DF was Set. */
> -    icmp4.frag_mtu = <var>M</var>;
> -    eth.dst = <var>E</var>;
> -    ip4.dst = ip4.src;
> -    ip4.src = <var>I</var>;
> -    ip.ttl = 255;
> -    REGBIT_EGRESS_LOOPBACK = 1;
> -    next(pipeline=ingress, table=0);
> -};
> -    </pre>
> -
> -    <ul>
> -      <li>
> -        Where <var>M</var> is the (fragment MTU - 58) whose value is
> taken from
> -        <ref column="options:gateway_mtu" table="Logical_Router_Port"
> -        db="OVN_Northbound"/> column of
> -        <ref table="Logical_Router_Port" db="OVN_Northbound"/> row.
> -      </li>
> -
> -      <li>
> -        <var>E</var> is the Ethernet address of the logical router port.
> -      </li>
> -
> -      <li>
> -        <var>I</var> is the IPv4 address of the logical router port.
> -      </li>
> -    </ul>
> -
> -    <p>
> -      This table adds one priority-0 fallback flow that matches all
> packets
> -      and advances to the next table.
> -    </p>
> -
> -    <h3>Ingress Table 11: Gateway Redirect</h3>
> -
> -    <p>
> -      For distributed logical routers where one of the logical router
> -      ports specifies a <code>redirect-chassis</code>, this table
> redirects
> -      certain packets to the distributed gateway port instance on the
> -      <code>redirect-chassis</code>.  This table has the following flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        A priority-300 logical flow with match
> -        <code>REGBIT_DISTRIBUTED_NAT == 1</code> has action
> -        <code>next;</code>
> -      </li>
> -      <li>
> -        A priority-200 logical flow with match
> -        <code>REGBIT_NAT_REDIRECT == 1</code> has actions
> -        <code>outport = <var>CR</var>; next;</code>, where <var>CR</var>
> -        is the <code>chassisredirect</code> port representing the instance
> -        of the logical router distributed gateway port on the
> -        <code>redirect-chassis</code>.
> -      </li>
> -
> -      <li>
> -        A priority-150 logical flow with match
> -        <code>outport == <var>GW</var> &amp;&amp;
> -        eth.dst == 00:00:00:00:00:00</code> has actions
> -        <code>outport = <var>CR</var>; next;</code>, where
> -        <var>GW</var> is the logical router distributed gateway
> -        port and <var>CR</var> is the <code>chassisredirect</code>
> -        port representing the instance of the logical router
> -        distributed gateway port on the
> -        <code>redirect-chassis</code>.
> -      </li>
> -
> -      <li>
> -        For each NAT rule in the OVN Northbound database that can
> -        be handled in a distributed manner, a priority-100 logical
> -        flow with match <code>ip4.src == <var>B</var> &amp;&amp;
> -        outport == <var>GW</var></code>, where <var>GW</var> is
> -        the logical router distributed gateway port, with actions
> -        <code>next;</code>.
> -      </li>
> -
> -      <li>
> -        A priority-50 logical flow with match
> -        <code>outport == <var>GW</var></code> has actions
> -        <code>outport = <var>CR</var>; next;</code>, where
> -        <var>GW</var> is the logical router distributed gateway
> -        port and <var>CR</var> is the <code>chassisredirect</code>
> -        port representing the instance of the logical router
> -        distributed gateway port on the
> -        <code>redirect-chassis</code>.
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Ingress Table 12: ARP Request</h3>
> -
> -    <p>
> -      In the common case where the Ethernet destination has been
> resolved, this
> -      table outputs the packet.  Otherwise, it composes and sends an ARP
> or
> -      IPv6 Neighbor Solicitation request.  It holds the following flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          Unknown MAC address.  A priority-100 flow for IPv4 packets with
> match
> -          <code>eth.dst == 00:00:00:00:00:00</code> has the following
> actions:
> -        </p>
> -
> -        <pre>
> -arp {
> -    eth.dst = ff:ff:ff:ff:ff:ff;
> -    arp.spa = reg1;
> -    arp.tpa = reg0;
> -    arp.op = 1;  /* ARP request. */
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          Unknown MAC address.  For each IPv6 static route associated
> with the
> -          router with the nexthop IP: <var>G</var>, a priority-200 flow
> -          for IPv6 packets with match
> -          <code>eth.dst == 00:00:00:00:00:00 &amp;&amp;
> -          xxreg0 == <var>G</var></code>
> -          with the following actions is added:
> -        </p>
> -
> -        <pre>
> -nd_ns {
> -    eth.dst = <var>E</var>;
> -    ip6.dst = <var>I</var>
> -    nd.target = <var>G</var>;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          Where <var>E</var> is the multicast mac derived from the
> Gateway IP,
> -          <var>I</var> is the solicited-node multicast address
> corresponding
> -          to the target address <var>G</var>.
> -        </p>
> -
> -        <p>
> -          Unknown MAC address.  A priority-100 flow for IPv6 packets with
> match
> -          <code>eth.dst == 00:00:00:00:00:00</code> has the following
> actions:
> -        </p>
> -
> -        <pre>
> -nd_ns {
> -    nd.target = xxreg0;
> -    output;
> -};
> -        </pre>
> -
> -        <p>
> -          (Ingress table <code>IP Routing</code> initialized
> <code>reg1</code>
> -          with the IP address owned by <code>outport</code> and
> -          <code>(xx)reg0</code> with the next-hop IP address)
> -        </p>
> -
> -        <p>
> -          The IP packet that triggers the ARP/IPv6 NS request is dropped.
> -        </p>
> -      </li>
> -
> -      <li>
> -        Known MAC address.  A priority-0 flow with match <code>1</code>
> has
> -        actions <code>output;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 0: UNDNAT</h3>
> -
> -    <p>
> -      This is for already established connections' reverse traffic.
> -      i.e., DNAT has already been done in ingress pipeline and now the
> -      packet has entered the egress pipeline as part of a reply.  For
> -      NAT on a distributed router, it is unDNATted here.  For Gateway
> -      routers, the unDNAT processing is carried out in the ingress DNAT
> -      table.
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For all the configured load balancing rules for a router with
> gateway
> -          port in <code>OVN_Northbound</code> database that includes an
> IPv4
> -          address <code>VIP</code>, for every backend IPv4 address
> <var>B</var>
> -          defined for the <code>VIP</code> a priority-120 flow is
> programmed on
> -          <code>redirect-chassis</code> that matches
> -          <code>ip &amp;&amp; ip4.src == <var>B</var> &amp;&amp;
> -          outport == <var>GW</var></code>, where <var>GW</var> is the
> logical
> -          router gateway port with an action <code>ct_dnat;</code>. If the
> -          backend IPv4 address <var>B</var> is also configured with L4
> port
> -          <var>PORT</var> of protocol <var>P</var>, then the
> -          match also includes <code>P.src</code> == <var>PORT</var>.
> These
> -          flows are not added for load balancers with IPv6
> <var>VIPs</var>.
> -        </p>
> -
> -        <p>
> -          If the router is configured to force SNAT  any load-balanced
> packets,
> -          above action will be replaced by
> -          <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
> -        </p>
> -      </li>
> -
> -      <li>
> -        <p>
> -          For each configuration in the OVN Northbound database that asks
> -          to change the destination IP address of a packet from an IP
> -          address of <var>A</var> to <var>B</var>, a priority-100 flow
> -          matches <code>ip &amp;&amp; ip4.src == <var>B</var>
> -          &amp;&amp; outport == <var>GW</var></code>, where <var>GW</var>
> -          is the logical router gateway port, with an action
> -          <code>ct_dnat;</code>.
> -        </p>
> -
> -        <p>
> -          If the NAT rule cannot be handled in a distributed manner, then
> -          the priority-100 flow above is only programmed on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          If the NAT rule can be handled in a distributed manner, then
> -          there is an additional action
> -          <code>eth.src = <var>EA</var>;</code>, where <var>EA</var>
> -          is the ethernet address associated with the IP address
> -          <var>A</var> in the NAT rule.  This allows upstream MAC
> -          learning to point to the correct chassis.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 1: SNAT</h3>
> -
> -    <p>
> -      Packets that are configured to be SNATed get their source IP address
> -      changed based on the configuration in the OVN Northbound database.
> -    </p>
> -
> -    <p>Egress Table 1: SNAT on Gateway Routers</p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          If the Gateway router in the OVN Northbound database has been
> -          configured to force SNAT a packet (that has been previously
> DNATted)
> -          to <var>B</var>, a priority-100 flow matches
> -          <code>flags.force_snat_for_dnat == 1 &amp;&amp; ip</code> with
> an
> -          action <code>ct_snat(<var>B</var>);</code>.
> -        </p>
> -        <p>
> -          If the Gateway router in the OVN Northbound database has been
> -          configured to force SNAT a packet (that has been previously
> -          load-balanced) to <var>B</var>, a priority-100 flow matches
> -          <code>flags.force_snat_for_lb == 1 &amp;&amp; ip</code> with an
> -          action <code>ct_snat(<var>B</var>);</code>.
> -        </p>
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the source IP address of a packet from an IP address
> of
> -          <var>A</var> or to change the source IP address of a packet that
> -          belongs to network <var>A</var> to <var>B</var>, a flow matches
> -          <code>ip &amp;&amp; ip4.src == <var>A</var></code> with an
> action
> -          <code>ct_snat(<var>B</var>);</code>.  The priority of the flow
> -          is calculated based on the mask of <var>A</var>, with matches
> -          having larger masks getting higher priorities.
> -        </p>
> -        <p>
> -          A priority-0 logical flow with match <code>1</code> has actions
> -          <code>next;</code>.
> -        </p>
> -      </li>
> -    </ul>
> -
> -    <p>Egress Table 1: SNAT on Distributed Routers</p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each configuration in the OVN Northbound database, that asks
> -          to change the source IP address of a packet from an IP address
> of
> -          <var>A</var> or to change the source IP address of a packet that
> -          belongs to network <var>A</var> to <var>B</var>, a flow matches
> -          <code>ip &amp;&amp; ip4.src == <var>A</var> &amp;&amp;
> -          outport == <var>GW</var></code>, where <var>GW</var> is the
> -          logical router gateway port, with an action
> -          <code>ct_snat(<var>B</var>);</code>.  The priority of the flow
> -          is calculated based on the mask of <var>A</var>, with matches
> -          having larger masks getting higher priorities.
> -        </p>
> -
> -        <p>
> -          If the NAT rule cannot be handled in a distributed manner, then
> -          the flow above is only programmed on the
> -          <code>redirect-chassis</code> increasing flow priority by 128 in
> -          order to be run first
> -        </p>
> -
> -        <p>
> -          If the NAT rule can be handled in a distributed manner, then
> -          there is an additional action
> -          <code>eth.src = <var>EA</var>;</code>, where <var>EA</var>
> -          is the ethernet address associated with the IP address
> -          <var>A</var> in the NAT rule.  This allows upstream MAC
> -          learning to point to the correct chassis.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 2: Egress Loopback</h3>
> -
> -    <p>
> -      For distributed logical routers where one of the logical router
> -      ports specifies a <code>redirect-chassis</code>.
> -    </p>
> -
> -    <p>
> -      Earlier in the ingress pipeline, some east-west traffic was
> -      redirected to the <code>chassisredirect</code> port, based on
> -      flows in the <code>UNSNAT</code> and <code>DNAT</code> ingress
> -      tables setting the <code>REGBIT_NAT_REDIRECT</code> flag, which
> -      then triggered a match to a flow in the
> -      <code>Gateway Redirect</code> ingress table.  The intention was
> -      not to actually send traffic out the distributed gateway port
> -      instance on the <code>redirect-chassis</code>.  This traffic was
> -      sent to the distributed gateway port instance in order for DNAT
> -      and/or SNAT processing to be applied.
> -    </p>
> -
> -    <p>
> -      While UNDNAT and SNAT processing have already occurred by this
> -      point, this traffic needs to be forced through egress loopback on
> -      this distributed gateway port instance, in order for UNSNAT and
> -      DNAT processing to be applied, and also for IP routing and ARP
> -      resolution after all of the NAT processing, so that the packet can
> -      be forwarded to the destination.
> -    </p>
> -
> -    <p>
> -      This table has the following flows:
> -    </p>
> -
> -    <ul>
> -      <li>
> -        <p>
> -          For each <code>dnat_and_snat</code> NAT rule couple in the
> -          OVN Northbound database on a distributed router,
> -          a priority-200 logical with match
> -          <code>ip4.dst == <var>external_ip0</var> &amp;&amp;
> -          ip4.src == <var>external_ip1</var></code>, has action
> -          <code>next;</code>
> -        </p>
> -
> -        <p>
> -          For each NAT rule in the OVN Northbound database on a
> -          distributed router, a priority-100 logical flow with match
> -          <code>ip4.dst == <var>E</var> &amp;&amp;
> -          outport == <var>GW</var></code>, where <var>E</var> is the
> -          external IP address specified in the NAT rule, and <var>GW</var>
> -          is the logical router distributed gateway port, with the
> -          following actions:
> -        </p>
> -
> -        <pre>
> -clone {
> -    ct_clear;
> -    inport = outport;
> -    outport = "";
> -    flags = 0;
> -    flags.loopback = 1;
> -    reg0 = 0;
> -    reg1 = 0;
> -    ...
> -    reg9 = 0;
> -    REGBIT_EGRESS_LOOPBACK = 1;
> -    next(pipeline=ingress, table=0);
> -};
> -        </pre>
> -
> -        <p>
> -          <code>flags.loopback</code> is set since in_port is unchanged
> -          and the packet may return back to that port after NAT
> processing.
> -          <code>REGBIT_EGRESS_LOOPBACK</code> is set to indicate that
> -          egress loopback has occurred, in order to skip the source IP
> -          address check against the router address.
> -        </p>
> -      </li>
> -
> -      <li>
> -        A priority-0 logical flow with match <code>1</code> has actions
> -        <code>next;</code>.
> -      </li>
> -    </ul>
> -
> -    <h3>Egress Table 3: Delivery</h3>
> -
> -    <p>
> -      Packets that reach this table are ready for delivery.  It contains
> -      priority-100 logical flows that match packets on each enabled
> logical
> -      router port, with action <code>output;</code>.
> -    </p>
> -
> -</manpage>
> diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
> deleted file mode 100644
> index eb6c47cad..000000000
> --- a/ovn/northd/ovn-northd.c
> +++ /dev/null
> @@ -1,9447 +0,0 @@
> -/*
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <getopt.h>
> -#include <stdlib.h>
> -#include <stdio.h>
> -
> -#include "bitmap.h"
> -#include "command-line.h"
> -#include "daemon.h"
> -#include "dirs.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "fatal-signal.h"
> -#include "hash.h"
> -#include "openvswitch/hmap.h"
> -#include "openvswitch/json.h"
> -#include "ovn/lex.h"
> -#include "ovn/lib/chassis-index.h"
> -#include "ovn/lib/ip-mcast-index.h"
> -#include "ovn/lib/mcast-group-index.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/ovn-nb-idl.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "ovn/actions.h"
> -#include "ovn/logical-fields.h"
> -#include "packets.h"
> -#include "openvswitch/poll-loop.h"
> -#include "smap.h"
> -#include "sset.h"
> -#include "svec.h"
> -#include "stream.h"
> -#include "stream-ssl.h"
> -#include "unixctl.h"
> -#include "util.h"
> -#include "uuid.h"
> -#include "openvswitch/vlog.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ovn_northd);
> -
> -static unixctl_cb_func ovn_northd_exit;
> -
> -struct northd_context {
> -    struct ovsdb_idl *ovnnb_idl;
> -    struct ovsdb_idl *ovnsb_idl;
> -    struct ovsdb_idl_txn *ovnnb_txn;
> -    struct ovsdb_idl_txn *ovnsb_txn;
> -    struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name;
> -    struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp;
> -    struct ovsdb_idl_index *sbrec_ip_mcast_by_dp;
> -};
> -
> -static const char *ovnnb_db;
> -static const char *ovnsb_db;
> -static const char *unixctl_path;
> -
> -#define MAC_ADDR_SPACE 0xffffff
> -
> -/* MAC address management (macam) table of "struct eth_addr"s, that holds
> the
> - * MAC addresses allocated by the OVN ipam module. */
> -static struct hmap macam = HMAP_INITIALIZER(&macam);
> -static struct eth_addr mac_prefix;
> -
> -static bool controller_event_en;
> -
> -#define MAX_OVN_TAGS 4096
> -
> -/* Pipeline stages. */
> -
> -/* The two pipelines in an OVN logical flow table. */
> -enum ovn_pipeline {
> -    P_IN,                       /* Ingress pipeline. */
> -    P_OUT                       /* Egress pipeline. */
> -};
> -
> -/* The two purposes for which ovn-northd uses OVN logical datapaths. */
> -enum ovn_datapath_type {
> -    DP_SWITCH,                  /* OVN logical switch. */
> -    DP_ROUTER                   /* OVN logical router. */
> -};
> -
> -/* Returns an "enum ovn_stage" built from the arguments.
> - *
> - * (It's better to use ovn_stage_build() for type-safety reasons, but
> inline
> - * functions can't be used in enums or switch cases.) */
> -#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \
> -    (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE))
> -
> -/* A stage within an OVN logical switch or router.
> - *
> - * An "enum ovn_stage" indicates whether the stage is part of a logical
> switch
> - * or router, whether the stage is part of the ingress or egress
> pipeline, and
> - * the table within that pipeline.  The first three components are
> combined to
> - * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2,
> - * S_ROUTER_OUT_DELIVERY. */
> -enum ovn_stage {
> -#define PIPELINE_STAGES
>  \
> -    /* Logical switch ingress stages. */
> \
> -    PIPELINE_STAGE(SWITCH, IN,  PORT_SEC_L2,    0, "ls_in_port_sec_l2")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  PORT_SEC_IP,    1, "ls_in_port_sec_ip")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  PORT_SEC_ND,    2, "ls_in_port_sec_nd")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  PRE_ACL,        3, "ls_in_pre_acl")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  PRE_LB,         4, "ls_in_pre_lb")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  PRE_STATEFUL,   5, "ls_in_pre_stateful")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  ACL,            6, "ls_in_acl")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  QOS_MARK,       7, "ls_in_qos_mark")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  QOS_METER,      8, "ls_in_qos_meter")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  LB,             9, "ls_in_lb")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  STATEFUL,      10, "ls_in_stateful")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  ARP_ND_RSP,    11, "ls_in_arp_rsp")
>  \
> -    PIPELINE_STAGE(SWITCH, IN,  DHCP_OPTIONS,  12, "ls_in_dhcp_options")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  DHCP_RESPONSE, 13, "ls_in_dhcp_response")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  DNS_LOOKUP,    14, "ls_in_dns_lookup")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  DNS_RESPONSE,  15, "ls_in_dns_response")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  EXTERNAL_PORT, 16, "ls_in_external_port")
> \
> -    PIPELINE_STAGE(SWITCH, IN,  L2_LKUP,       17, "ls_in_l2_lkup")
>  \
> -
> \
> -    /* Logical switch egress stages. */
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, PRE_LB,       0, "ls_out_pre_lb")
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, PRE_ACL,      1, "ls_out_pre_acl")
> \
> -    PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful")
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, LB,           3, "ls_out_lb")
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, ACL,          4, "ls_out_acl")
> \
> -    PIPELINE_STAGE(SWITCH, OUT, QOS_MARK,     5, "ls_out_qos_mark")
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, QOS_METER,    6, "ls_out_qos_meter")
> \
> -    PIPELINE_STAGE(SWITCH, OUT, STATEFUL,     7, "ls_out_stateful")
>  \
> -    PIPELINE_STAGE(SWITCH, OUT, PORT_SEC_IP,  8, "ls_out_port_sec_ip")
> \
> -    PIPELINE_STAGE(SWITCH, OUT, PORT_SEC_L2,  9, "ls_out_port_sec_l2")
> \
> -                                                                      \
> -    /* Logical router ingress stages. */                              \
> -    PIPELINE_STAGE(ROUTER, IN,  ADMISSION,      0, "lr_in_admission")    \
> -    PIPELINE_STAGE(ROUTER, IN,  IP_INPUT,       1, "lr_in_ip_input")     \
> -    PIPELINE_STAGE(ROUTER, IN,  DEFRAG,         2, "lr_in_defrag")       \
> -    PIPELINE_STAGE(ROUTER, IN,  UNSNAT,         3, "lr_in_unsnat")       \
> -    PIPELINE_STAGE(ROUTER, IN,  DNAT,           4, "lr_in_dnat")         \
> -    PIPELINE_STAGE(ROUTER, IN,  ND_RA_OPTIONS,  5, "lr_in_nd_ra_options")
> \
> -    PIPELINE_STAGE(ROUTER, IN,  ND_RA_RESPONSE, 6,
> "lr_in_nd_ra_response") \
> -    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING,     7, "lr_in_ip_routing")   \
> -    PIPELINE_STAGE(ROUTER, IN,  POLICY,         8, "lr_in_policy")       \
> -    PIPELINE_STAGE(ROUTER, IN,  ARP_RESOLVE,    9, "lr_in_arp_resolve")  \
> -    PIPELINE_STAGE(ROUTER, IN,  CHK_PKT_LEN   , 10, "lr_in_chk_pkt_len")
>  \
> -    PIPELINE_STAGE(ROUTER, IN,  LARGER_PKTS,    11,"lr_in_larger_pkts")
>  \
> -    PIPELINE_STAGE(ROUTER, IN,  GW_REDIRECT,    12, "lr_in_gw_redirect")
> \
> -    PIPELINE_STAGE(ROUTER, IN,  ARP_REQUEST,    13, "lr_in_arp_request")
> \
> -                                                                      \
> -    /* Logical router egress stages. */                               \
> -    PIPELINE_STAGE(ROUTER, OUT, UNDNAT,    0, "lr_out_undnat")        \
> -    PIPELINE_STAGE(ROUTER, OUT, SNAT,      1, "lr_out_snat")          \
> -    PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP,  2, "lr_out_egr_loop")      \
> -    PIPELINE_STAGE(ROUTER, OUT, DELIVERY,  3, "lr_out_delivery")
> -
> -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME)   \
> -    S_##DP_TYPE##_##PIPELINE##_##STAGE                          \
> -        = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE),
> -    PIPELINE_STAGES
> -#undef PIPELINE_STAGE
> -};
> -
> -/* Due to various hard-coded priorities need to implement ACLs, the
> - * northbound database supports a smaller range of ACL priorities than
> - * are available to logical flows.  This value is added to an ACL
> - * priority to determine the ACL's logical flow priority. */
> -#define OVN_ACL_PRI_OFFSET 1000
> -
> -/* Register definitions specific to switches. */
> -#define REGBIT_CONNTRACK_DEFRAG  "reg0[0]"
> -#define REGBIT_CONNTRACK_COMMIT  "reg0[1]"
> -#define REGBIT_CONNTRACK_NAT     "reg0[2]"
> -#define REGBIT_DHCP_OPTS_RESULT  "reg0[3]"
> -#define REGBIT_DNS_LOOKUP_RESULT "reg0[4]"
> -#define REGBIT_ND_RA_OPTS_RESULT "reg0[5]"
> -
> -/* Register definitions for switches and routers. */
> -#define REGBIT_NAT_REDIRECT     "reg9[0]"
> -/* Indicate that this packet has been recirculated using egress
> - * loopback.  This allows certain checks to be bypassed, such as a
> - * logical router dropping packets with source IP address equals
> - * one of the logical router's own IP addresses. */
> -#define REGBIT_EGRESS_LOOPBACK  "reg9[1]"
> -#define REGBIT_DISTRIBUTED_NAT  "reg9[2]"
> -/* Register to store the result of check_pkt_larger action. */
> -#define REGBIT_PKT_LARGER        "reg9[3]"
> -
> -/* Returns an "enum ovn_stage" built from the arguments. */
> -static enum ovn_stage
> -ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline
> pipeline,
> -                uint8_t table)
> -{
> -    return OVN_STAGE_BUILD(dp_type, pipeline, table);
> -}
> -
> -/* Returns the pipeline to which 'stage' belongs. */
> -static enum ovn_pipeline
> -ovn_stage_get_pipeline(enum ovn_stage stage)
> -{
> -    return (stage >> 8) & 1;
> -}
> -
> -/* Returns the pipeline name to which 'stage' belongs. */
> -static const char *
> -ovn_stage_get_pipeline_name(enum ovn_stage stage)
> -{
> -    return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress";
> -}
> -
> -/* Returns the table to which 'stage' belongs. */
> -static uint8_t
> -ovn_stage_get_table(enum ovn_stage stage)
> -{
> -    return stage & 0xff;
> -}
> -
> -/* Returns a string name for 'stage'. */
> -static const char *
> -ovn_stage_to_str(enum ovn_stage stage)
> -{
> -    switch (stage) {
> -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME)       \
> -        case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME;
> -    PIPELINE_STAGES
> -#undef PIPELINE_STAGE
> -        default: return "<unknown>";
> -    }
> -}
> -
> -/* Returns the type of the datapath to which a flow with the given
> 'stage' may
> - * be added. */
> -static enum ovn_datapath_type
> -ovn_stage_to_datapath_type(enum ovn_stage stage)
> -{
> -    switch (stage) {
> -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME)       \
> -        case S_##DP_TYPE##_##PIPELINE##_##STAGE: return DP_##DP_TYPE;
> -    PIPELINE_STAGES
> -#undef PIPELINE_STAGE
> -    default: OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN northbound management daemon\n\
> -usage: %s [OPTIONS]\n\
> -\n\
> -Options:\n\
> -  --ovnnb-db=DATABASE       connect to ovn-nb database at DATABASE\n\
> -                            (default: %s)\n\
> -  --ovnsb-db=DATABASE       connect to ovn-sb database at DATABASE\n\
> -                            (default: %s)\n\
> -  --unixctl=SOCKET          override default control socket name\n\
> -  -h, --help                display this help message\n\
> -  -o, --options             list available options\n\
> -  -V, --version             display version information\n\
> -", program_name, program_name, default_nb_db(), default_sb_db());
> -    daemon_usage();
> -    vlog_usage();
> -    stream_usage("database", true, true, false);
> -}
> -
> -struct tnlid_node {
> -    struct hmap_node hmap_node;
> -    uint32_t tnlid;
> -};
> -
> -static void
> -destroy_tnlids(struct hmap *tnlids)
> -{
> -    struct tnlid_node *node;
> -    HMAP_FOR_EACH_POP (node, hmap_node, tnlids) {
> -        free(node);
> -    }
> -    hmap_destroy(tnlids);
> -}
> -
> -static void
> -add_tnlid(struct hmap *set, uint32_t tnlid)
> -{
> -    struct tnlid_node *node = xmalloc(sizeof *node);
> -    hmap_insert(set, &node->hmap_node, hash_int(tnlid, 0));
> -    node->tnlid = tnlid;
> -}
> -
> -static bool
> -tnlid_in_use(const struct hmap *set, uint32_t tnlid)
> -{
> -    const struct tnlid_node *node;
> -    HMAP_FOR_EACH_IN_BUCKET (node, hmap_node, hash_int(tnlid, 0), set) {
> -        if (node->tnlid == tnlid) {
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -static uint32_t
> -next_tnlid(uint32_t tnlid, uint32_t min, uint32_t max)
> -{
> -    return tnlid + 1 <= max ? tnlid + 1 : min;
> -}
> -
> -static uint32_t
> -allocate_tnlid(struct hmap *set, const char *name, uint32_t min, uint32_t
> max,
> -               uint32_t *hint)
> -{
> -    for (uint32_t tnlid = next_tnlid(*hint, min, max); tnlid != *hint;
> -         tnlid = next_tnlid(tnlid, min, max)) {
> -        if (!tnlid_in_use(set, tnlid)) {
> -            add_tnlid(set, tnlid);
> -            *hint = tnlid;
> -            return tnlid;
> -        }
> -    }
> -
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -    VLOG_WARN_RL(&rl, "all %s tunnel ids exhausted", name);
> -    return 0;
> -}
> -
> -struct ovn_chassis_qdisc_queues {
> -    struct hmap_node key_node;
> -    uint32_t queue_id;
> -    struct uuid chassis_uuid;
> -};
> -
> -static uint32_t
> -hash_chassis_queue(const struct uuid *chassis_uuid, uint32_t queue_id)
> -{
> -    return hash_2words(uuid_hash(chassis_uuid), queue_id);
> -}
> -
> -static void
> -destroy_chassis_queues(struct hmap *set)
> -{
> -    struct ovn_chassis_qdisc_queues *node;
> -    HMAP_FOR_EACH_POP (node, key_node, set) {
> -        free(node);
> -    }
> -    hmap_destroy(set);
> -}
> -
> -static void
> -add_chassis_queue(struct hmap *set, struct uuid *chassis_uuid,
> -                  uint32_t queue_id)
> -{
> -    struct ovn_chassis_qdisc_queues *node = xmalloc(sizeof *node);
> -    node->queue_id = queue_id;
> -    node->chassis_uuid = *chassis_uuid;
> -    hmap_insert(set, &node->key_node,
> -                hash_chassis_queue(chassis_uuid, queue_id));
> -}
> -
> -static bool
> -chassis_queueid_in_use(const struct hmap *set, struct uuid *chassis_uuid,
> -                       uint32_t queue_id)
> -{
> -    const struct ovn_chassis_qdisc_queues *node;
> -    HMAP_FOR_EACH_WITH_HASH (node, key_node,
> -                             hash_chassis_queue(chassis_uuid, queue_id),
> set) {
> -        if (uuid_equals(chassis_uuid, &node->chassis_uuid)
> -            && node->queue_id == queue_id) {
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -static uint32_t
> -allocate_chassis_queueid(struct hmap *set, struct sbrec_chassis *chassis)
> -{
> -    for (uint32_t queue_id = QDISC_MIN_QUEUE_ID + 1;
> -         queue_id <= QDISC_MAX_QUEUE_ID;
> -         queue_id++) {
> -        if (!chassis_queueid_in_use(set, &chassis->header_.uuid,
> queue_id)) {
> -            add_chassis_queue(set, &chassis->header_.uuid, queue_id);
> -            return queue_id;
> -        }
> -    }
> -
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -    VLOG_WARN_RL(&rl, "all %s queue ids exhausted", chassis->name);
> -    return 0;
> -}
> -
> -static void
> -free_chassis_queueid(struct hmap *set, struct sbrec_chassis *chassis,
> -                     uint32_t queue_id)
> -{
> -    const struct uuid *chassis_uuid = &chassis->header_.uuid;
> -    struct ovn_chassis_qdisc_queues *node;
> -    HMAP_FOR_EACH_WITH_HASH (node, key_node,
> -                             hash_chassis_queue(chassis_uuid, queue_id),
> set) {
> -        if (uuid_equals(chassis_uuid, &node->chassis_uuid)
> -            && node->queue_id == queue_id) {
> -            hmap_remove(set, &node->key_node);
> -            free(node);
> -            break;
> -        }
> -    }
> -}
> -
> -static inline bool
> -port_has_qos_params(const struct smap *opts)
> -{
> -    return (smap_get(opts, "qos_max_rate") ||
> -            smap_get(opts, "qos_burst"));
> -}
> -
> -
> -struct ipam_info {
> -    uint32_t start_ipv4;
> -    size_t total_ipv4s;
> -    unsigned long *allocated_ipv4s; /* A bitmap of allocated IPv4s */
> -    bool ipv6_prefix_set;
> -    struct in6_addr ipv6_prefix;
> -    bool mac_only;
> -};
> -
> -#define OVN_MIN_MULTICAST 32768
> -#define OVN_MAX_MULTICAST OVN_MCAST_FLOOD_TUNNEL_KEY
> -BUILD_ASSERT_DECL(OVN_MIN_MULTICAST < OVN_MAX_MULTICAST);
> -
> -#define OVN_MIN_IP_MULTICAST OVN_MIN_MULTICAST
> -#define OVN_MAX_IP_MULTICAST (OVN_MCAST_UNKNOWN_TUNNEL_KEY - 1)
> -BUILD_ASSERT_DECL(OVN_MAX_IP_MULTICAST >= OVN_MIN_MULTICAST);
> -
> -/*
> - * Multicast snooping and querier per datapath configuration.
> - */
> -struct mcast_info {
> -    bool enabled;
> -    bool querier;
> -    bool flood_unregistered;
> -
> -    int64_t table_size;
> -    int64_t idle_timeout;
> -    int64_t query_interval;
> -    char *eth_src;
> -    char *ipv4_src;
> -    int64_t  query_max_response;
> -
> -    struct hmap group_tnlids;
> -    uint32_t group_tnlid_hint;
> -    uint32_t active_flows;
> -};
> -
> -static uint32_t
> -ovn_mcast_group_allocate_key(struct mcast_info *mcast_info)
> -{
> -    return allocate_tnlid(&mcast_info->group_tnlids, "multicast group",
> -                          OVN_MIN_IP_MULTICAST, OVN_MAX_IP_MULTICAST,
> -                          &mcast_info->group_tnlid_hint);
> -}
> -
> -/* The 'key' comes from nbs->header_.uuid or nbr->header_.uuid or
> - * sb->external_ids:logical-switch. */
> -struct ovn_datapath {
> -    struct hmap_node key_node;  /* Index on 'key'. */
> -    struct uuid key;            /* (nbs/nbr)->header_.uuid. */
> -
> -    const struct nbrec_logical_switch *nbs;  /* May be NULL. */
> -    const struct nbrec_logical_router *nbr;  /* May be NULL. */
> -    const struct sbrec_datapath_binding *sb; /* May be NULL. */
> -
> -    struct ovs_list list;       /* In list of similar records. */
> -
> -    /* Logical switch data. */
> -    struct ovn_port **router_ports;
> -    size_t n_router_ports;
> -
> -    struct hmap port_tnlids;
> -    uint32_t port_key_hint;
> -
> -    bool has_unknown;
> -
> -    /* IPAM data. */
> -    struct ipam_info ipam_info;
> -
> -    /* Multicast data. */
> -    struct mcast_info mcast_info;
> -
> -    /* OVN northd only needs to know about the logical router gateway
> port for
> -     * NAT on a distributed router.  This "distributed gateway port" is
> -     * populated only when there is a "redirect-chassis" specified for
> one of
> -     * the ports on the logical router.  Otherwise this will be NULL. */
> -    struct ovn_port *l3dgw_port;
> -    /* The "derived" OVN port representing the instance of l3dgw_port on
> -     * the "redirect-chassis". */
> -    struct ovn_port *l3redirect_port;
> -    struct ovn_port *localnet_port;
> -
> -    struct ovs_list lr_list; /* In list of logical router datapaths. */
> -    /* The logical router group to which this datapath belongs.
> -     * Valid only if it is logical router datapath. NULL otherwise. */
> -    struct lrouter_group *lr_group;
> -
> -    /* Port groups related to the datapath, used only when nbs is NOT
> NULL. */
> -    struct hmap nb_pgs;
> -};
> -
> -/* A group of logical router datapaths which are connected - either
> - * directly or indirectly.
> - * Each logical router can belong to only one group. */
> -struct lrouter_group {
> -    struct ovn_datapath **router_dps;
> -    int n_router_dps;
> -    /* Set of ha_chassis_groups which are associated with the router dps.
> */
> -    struct sset ha_chassis_groups;
> -};
> -
> -struct macam_node {
> -    struct hmap_node hmap_node;
> -    struct eth_addr mac_addr; /* Allocated MAC address. */
> -};
> -
> -static void
> -cleanup_macam(struct hmap *macam_)
> -{
> -    struct macam_node *node;
> -    HMAP_FOR_EACH_POP (node, hmap_node, macam_) {
> -        free(node);
> -    }
> -}
> -
> -static struct ovn_datapath *
> -ovn_datapath_create(struct hmap *datapaths, const struct uuid *key,
> -                    const struct nbrec_logical_switch *nbs,
> -                    const struct nbrec_logical_router *nbr,
> -                    const struct sbrec_datapath_binding *sb)
> -{
> -    struct ovn_datapath *od = xzalloc(sizeof *od);
> -    od->key = *key;
> -    od->sb = sb;
> -    od->nbs = nbs;
> -    od->nbr = nbr;
> -    hmap_init(&od->port_tnlids);
> -    hmap_init(&od->nb_pgs);
> -    od->port_key_hint = 0;
> -    hmap_insert(datapaths, &od->key_node, uuid_hash(&od->key));
> -    od->lr_group = NULL;
> -    return od;
> -}
> -
> -static void ovn_ls_port_group_destroy(struct hmap *nb_pgs);
> -
> -static void
> -ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od)
> -{
> -    if (od) {
> -        /* Don't remove od->list.  It is used within build_datapaths() as
> a
> -         * private list and once we've exited that function it is not
> safe to
> -         * use it. */
> -        hmap_remove(datapaths, &od->key_node);
> -        destroy_tnlids(&od->port_tnlids);
> -        bitmap_free(od->ipam_info.allocated_ipv4s);
> -        free(od->router_ports);
> -        ovn_ls_port_group_destroy(&od->nb_pgs);
> -
> -        if (od->nbs) {
> -            free(od->mcast_info.eth_src);
> -            free(od->mcast_info.ipv4_src);
> -            destroy_tnlids(&od->mcast_info.group_tnlids);
> -        }
> -
> -        free(od);
> -    }
> -}
> -
> -/* Returns 'od''s datapath type. */
> -static enum ovn_datapath_type
> -ovn_datapath_get_type(const struct ovn_datapath *od)
> -{
> -    return od->nbs ? DP_SWITCH : DP_ROUTER;
> -}
> -
> -static struct ovn_datapath *
> -ovn_datapath_find(struct hmap *datapaths, const struct uuid *uuid)
> -{
> -    struct ovn_datapath *od;
> -
> -    HMAP_FOR_EACH_WITH_HASH (od, key_node, uuid_hash(uuid), datapaths) {
> -        if (uuid_equals(uuid, &od->key)) {
> -            return od;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static struct ovn_datapath *
> -ovn_datapath_from_sbrec(struct hmap *datapaths,
> -                        const struct sbrec_datapath_binding *sb)
> -{
> -    struct uuid key;
> -
> -    if (!smap_get_uuid(&sb->external_ids, "logical-switch", &key) &&
> -        !smap_get_uuid(&sb->external_ids, "logical-router", &key)) {
> -        return NULL;
> -    }
> -    return ovn_datapath_find(datapaths, &key);
> -}
> -
> -static bool
> -lrouter_is_enabled(const struct nbrec_logical_router *lrouter)
> -{
> -    return !lrouter->enabled || *lrouter->enabled;
> -}
> -
> -static void
> -init_ipam_info_for_datapath(struct ovn_datapath *od)
> -{
> -    if (!od->nbs) {
> -        return;
> -    }
> -
> -    const char *subnet_str = smap_get(&od->nbs->other_config, "subnet");
> -    const char *ipv6_prefix = smap_get(&od->nbs->other_config,
> "ipv6_prefix");
> -
> -    if (ipv6_prefix) {
> -        od->ipam_info.ipv6_prefix_set = ipv6_parse(
> -            ipv6_prefix, &od->ipam_info.ipv6_prefix);
> -    }
> -
> -    if (!subnet_str) {
> -        if (!ipv6_prefix) {
> -            od->ipam_info.mac_only = smap_get_bool(&od->nbs->other_config,
> -                                                   "mac_only", false);
> -        }
> -        return;
> -    }
> -
> -    ovs_be32 subnet, mask;
> -    char *error = ip_parse_masked(subnet_str, &subnet, &mask);
> -    if (error || mask == OVS_BE32_MAX || !ip_is_cidr(mask)) {
> -        static struct vlog_rate_limit rl
> -            = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "bad 'subnet' %s", subnet_str);
> -        free(error);
> -        return;
> -    }
> -
> -    od->ipam_info.start_ipv4 = ntohl(subnet) + 1;
> -    od->ipam_info.total_ipv4s = ~ntohl(mask);
> -    od->ipam_info.allocated_ipv4s =
> -        bitmap_allocate(od->ipam_info.total_ipv4s);
> -
> -    /* Mark first IP as taken */
> -    bitmap_set1(od->ipam_info.allocated_ipv4s, 0);
> -
> -    /* Check if there are any reserver IPs (list) to be excluded from
> IPAM */
> -    const char *exclude_ip_list = smap_get(&od->nbs->other_config,
> -                                           "exclude_ips");
> -    if (!exclude_ip_list) {
> -        return;
> -    }
> -
> -    struct lexer lexer;
> -    lexer_init(&lexer, exclude_ip_list);
> -    /* exclude_ip_list could be in the format -
> -    *  "10.0.0.4 10.0.0.10 10.0.0.20..10.0.0.50 10.0.0.100..10.0.0.110".
> -    */
> -    lexer_get(&lexer);
> -    while (lexer.token.type != LEX_T_END) {
> -        if (lexer.token.type != LEX_T_INTEGER) {
> -            lexer_syntax_error(&lexer, "expecting address");
> -            break;
> -        }
> -        uint32_t start = ntohl(lexer.token.value.ipv4);
> -        lexer_get(&lexer);
> -
> -        uint32_t end = start + 1;
> -        if (lexer_match(&lexer, LEX_T_ELLIPSIS)) {
> -            if (lexer.token.type != LEX_T_INTEGER) {
> -                lexer_syntax_error(&lexer, "expecting address range");
> -                break;
> -            }
> -            end = ntohl(lexer.token.value.ipv4) + 1;
> -            lexer_get(&lexer);
> -        }
> -
> -        /* Clamp start...end to fit the subnet. */
> -        start = MAX(od->ipam_info.start_ipv4, start);
> -        end = MIN(od->ipam_info.start_ipv4 + od->ipam_info.total_ipv4s,
> end);
> -        if (end > start) {
> -            bitmap_set_multiple(od->ipam_info.allocated_ipv4s,
> -                                start - od->ipam_info.start_ipv4,
> -                                end - start, 1);
> -        } else {
> -            lexer_error(&lexer, "excluded addresses not in subnet");
> -        }
> -    }
> -    if (lexer.error) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "logical switch "UUID_FMT": bad exclude_ips
> (%s)",
> -                     UUID_ARGS(&od->key), lexer.error);
> -    }
> -    lexer_destroy(&lexer);
> -}
> -
> -static void
> -init_mcast_info_for_datapath(struct ovn_datapath *od)
> -{
> -    if (!od->nbs) {
> -        return;
> -    }
> -
> -    struct mcast_info *mcast_info = &od->mcast_info;
> -
> -    mcast_info->enabled =
> -        smap_get_bool(&od->nbs->other_config, "mcast_snoop", false);
> -    mcast_info->querier =
> -        smap_get_bool(&od->nbs->other_config, "mcast_querier", true);
> -    mcast_info->flood_unregistered =
> -        smap_get_bool(&od->nbs->other_config, "mcast_flood_unregistered",
> -                      false);
> -
> -    mcast_info->table_size =
> -        smap_get_ullong(&od->nbs->other_config, "mcast_table_size",
> -                        OVN_MCAST_DEFAULT_MAX_ENTRIES);
> -
> -    uint32_t idle_timeout =
> -        smap_get_ullong(&od->nbs->other_config, "mcast_idle_timeout",
> -                        OVN_MCAST_DEFAULT_IDLE_TIMEOUT_S);
> -    if (idle_timeout < OVN_MCAST_MIN_IDLE_TIMEOUT_S) {
> -        idle_timeout = OVN_MCAST_MIN_IDLE_TIMEOUT_S;
> -    } else if (idle_timeout > OVN_MCAST_MAX_IDLE_TIMEOUT_S) {
> -        idle_timeout = OVN_MCAST_MAX_IDLE_TIMEOUT_S;
> -    }
> -    mcast_info->idle_timeout = idle_timeout;
> -
> -    uint32_t query_interval =
> -        smap_get_ullong(&od->nbs->other_config, "mcast_query_interval",
> -                        mcast_info->idle_timeout / 2);
> -    if (query_interval < OVN_MCAST_MIN_QUERY_INTERVAL_S) {
> -        query_interval = OVN_MCAST_MIN_QUERY_INTERVAL_S;
> -    } else if (query_interval > OVN_MCAST_MAX_QUERY_INTERVAL_S) {
> -        query_interval = OVN_MCAST_MAX_QUERY_INTERVAL_S;
> -    }
> -    mcast_info->query_interval = query_interval;
> -
> -    mcast_info->eth_src =
> -        nullable_xstrdup(smap_get(&od->nbs->other_config,
> "mcast_eth_src"));
> -    mcast_info->ipv4_src =
> -        nullable_xstrdup(smap_get(&od->nbs->other_config,
> "mcast_ip4_src"));
> -
> -    mcast_info->query_max_response =
> -        smap_get_ullong(&od->nbs->other_config,
> "mcast_query_max_response",
> -                        OVN_MCAST_DEFAULT_QUERY_MAX_RESPONSE_S);
> -
> -    hmap_init(&mcast_info->group_tnlids);
> -    mcast_info->group_tnlid_hint = OVN_MIN_IP_MULTICAST;
> -    mcast_info->active_flows = 0;
> -}
> -
> -static void
> -store_mcast_info_for_datapath(const struct sbrec_ip_multicast *sb,
> -                              struct ovn_datapath *od)
> -{
> -    struct mcast_info *mcast_info = &od->mcast_info;
> -
> -    sbrec_ip_multicast_set_datapath(sb, od->sb);
> -    sbrec_ip_multicast_set_enabled(sb, &mcast_info->enabled, 1);
> -    sbrec_ip_multicast_set_querier(sb, &mcast_info->querier, 1);
> -    sbrec_ip_multicast_set_table_size(sb, &mcast_info->table_size, 1);
> -    sbrec_ip_multicast_set_idle_timeout(sb, &mcast_info->idle_timeout, 1);
> -    sbrec_ip_multicast_set_query_interval(sb,
> -                                          &mcast_info->query_interval, 1);
> -    sbrec_ip_multicast_set_query_max_resp(sb,
> -
> &mcast_info->query_max_response, 1);
> -
> -    if (mcast_info->eth_src) {
> -        sbrec_ip_multicast_set_eth_src(sb, mcast_info->eth_src);
> -    }
> -
> -    if (mcast_info->ipv4_src) {
> -        sbrec_ip_multicast_set_ip4_src(sb, mcast_info->ipv4_src);
> -    }
> -}
> -
> -static void
> -ovn_datapath_update_external_ids(struct ovn_datapath *od)
> -{
> -    /* Get the logical-switch or logical-router UUID to set in
> -     * external-ids. */
> -    char uuid_s[UUID_LEN + 1];
> -    sprintf(uuid_s, UUID_FMT, UUID_ARGS(&od->key));
> -    const char *key = od->nbs ? "logical-switch" : "logical-router";
> -
> -    /* Get names to set in external-ids. */
> -    const char *name = od->nbs ? od->nbs->name : od->nbr->name;
> -    const char *name2 = (od->nbs
> -                         ? smap_get(&od->nbs->external_ids,
> -                                    "neutron:network_name")
> -                         : smap_get(&od->nbr->external_ids,
> -                                    "neutron:router_name"));
> -
> -    /* Set external-ids. */
> -    struct smap ids = SMAP_INITIALIZER(&ids);
> -    smap_add(&ids, key, uuid_s);
> -    smap_add(&ids, "name", name);
> -    if (name2 && name2[0]) {
> -        smap_add(&ids, "name2", name2);
> -    }
> -    sbrec_datapath_binding_set_external_ids(od->sb, &ids);
> -    smap_destroy(&ids);
> -}
> -
> -static void
> -join_datapaths(struct northd_context *ctx, struct hmap *datapaths,
> -               struct ovs_list *sb_only, struct ovs_list *nb_only,
> -               struct ovs_list *both, struct ovs_list *lr_list)
> -{
> -    ovs_list_init(sb_only);
> -    ovs_list_init(nb_only);
> -    ovs_list_init(both);
> -
> -    const struct sbrec_datapath_binding *sb, *sb_next;
> -    SBREC_DATAPATH_BINDING_FOR_EACH_SAFE (sb, sb_next, ctx->ovnsb_idl) {
> -        struct uuid key;
> -        if (!smap_get_uuid(&sb->external_ids, "logical-switch", &key) &&
> -            !smap_get_uuid(&sb->external_ids, "logical-router", &key)) {
> -            ovsdb_idl_txn_add_comment(
> -                ctx->ovnsb_txn,
> -                "deleting Datapath_Binding "UUID_FMT" that lacks "
> -                "external-ids:logical-switch and "
> -                "external-ids:logical-router",
> -                UUID_ARGS(&sb->header_.uuid));
> -            sbrec_datapath_binding_delete(sb);
> -            continue;
> -        }
> -
> -        if (ovn_datapath_find(datapaths, &key)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_INFO_RL(
> -                &rl, "deleting Datapath_Binding "UUID_FMT" with "
> -                "duplicate external-ids:logical-switch/router "UUID_FMT,
> -                UUID_ARGS(&sb->header_.uuid), UUID_ARGS(&key));
> -            sbrec_datapath_binding_delete(sb);
> -            continue;
> -        }
> -
> -        struct ovn_datapath *od = ovn_datapath_create(datapaths, &key,
> -                                                      NULL, NULL, sb);
> -        ovs_list_push_back(sb_only, &od->list);
> -    }
> -
> -    const struct nbrec_logical_switch *nbs;
> -    NBREC_LOGICAL_SWITCH_FOR_EACH (nbs, ctx->ovnnb_idl) {
> -        struct ovn_datapath *od = ovn_datapath_find(datapaths,
> -                                                    &nbs->header_.uuid);
> -        if (od) {
> -            od->nbs = nbs;
> -            ovs_list_remove(&od->list);
> -            ovs_list_push_back(both, &od->list);
> -            ovn_datapath_update_external_ids(od);
> -        } else {
> -            od = ovn_datapath_create(datapaths, &nbs->header_.uuid,
> -                                     nbs, NULL, NULL);
> -            ovs_list_push_back(nb_only, &od->list);
> -        }
> -
> -        init_ipam_info_for_datapath(od);
> -        init_mcast_info_for_datapath(od);
> -    }
> -
> -    const struct nbrec_logical_router *nbr;
> -    NBREC_LOGICAL_ROUTER_FOR_EACH (nbr, ctx->ovnnb_idl) {
> -        if (!lrouter_is_enabled(nbr)) {
> -            continue;
> -        }
> -
> -        struct ovn_datapath *od = ovn_datapath_find(datapaths,
> -                                                    &nbr->header_.uuid);
> -        if (od) {
> -            if (!od->nbs) {
> -                od->nbr = nbr;
> -                ovs_list_remove(&od->list);
> -                ovs_list_push_back(both, &od->list);
> -                ovn_datapath_update_external_ids(od);
> -            } else {
> -                /* Can't happen! */
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(5, 1);
> -                VLOG_WARN_RL(&rl,
> -                             "duplicate UUID "UUID_FMT" in
> OVN_Northbound",
> -                             UUID_ARGS(&nbr->header_.uuid));
> -                continue;
> -            }
> -        } else {
> -            od = ovn_datapath_create(datapaths, &nbr->header_.uuid,
> -                                     NULL, nbr, NULL);
> -            ovs_list_push_back(nb_only, &od->list);
> -        }
> -        ovs_list_push_back(lr_list, &od->lr_list);
> -    }
> -}
> -
> -static uint32_t
> -ovn_datapath_allocate_key(struct hmap *dp_tnlids)
> -{
> -    static uint32_t hint;
> -    return allocate_tnlid(dp_tnlids, "datapath", 1, (1u << 24) - 1,
> &hint);
> -}
> -
> -/* Updates the southbound Datapath_Binding table so that it contains the
> - * logical switches and routers specified by the northbound database.
> - *
> - * Initializes 'datapaths' to contain a "struct ovn_datapath" for every
> logical
> - * switch and router. */
> -static void
> -build_datapaths(struct northd_context *ctx, struct hmap *datapaths,
> -                struct ovs_list *lr_list)
> -{
> -    struct ovs_list sb_only, nb_only, both;
> -
> -    join_datapaths(ctx, datapaths, &sb_only, &nb_only, &both, lr_list);
> -
> -    if (!ovs_list_is_empty(&nb_only)) {
> -        /* First index the in-use datapath tunnel IDs. */
> -        struct hmap dp_tnlids = HMAP_INITIALIZER(&dp_tnlids);
> -        struct ovn_datapath *od;
> -        LIST_FOR_EACH (od, list, &both) {
> -            add_tnlid(&dp_tnlids, od->sb->tunnel_key);
> -        }
> -
> -        /* Add southbound record for each unmatched northbound record. */
> -        LIST_FOR_EACH (od, list, &nb_only) {
> -            uint16_t tunnel_key = ovn_datapath_allocate_key(&dp_tnlids);
> -            if (!tunnel_key) {
> -                break;
> -            }
> -
> -            od->sb = sbrec_datapath_binding_insert(ctx->ovnsb_txn);
> -            ovn_datapath_update_external_ids(od);
> -            sbrec_datapath_binding_set_tunnel_key(od->sb, tunnel_key);
> -        }
> -        destroy_tnlids(&dp_tnlids);
> -    }
> -
> -    /* Delete southbound records without northbound matches. */
> -    struct ovn_datapath *od, *next;
> -    LIST_FOR_EACH_SAFE (od, next, list, &sb_only) {
> -        ovs_list_remove(&od->list);
> -        sbrec_datapath_binding_delete(od->sb);
> -        ovn_datapath_destroy(datapaths, od);
> -    }
> -}
> -
> -struct ovn_port {
> -    struct hmap_node key_node;  /* Index on 'key'. */
> -    char *key;                  /* nbs->name, nbr->name,
> sb->logical_port. */
> -    char *json_key;             /* 'key', quoted for use in JSON. */
> -
> -    const struct sbrec_port_binding *sb;         /* May be NULL. */
> -
> -    /* Logical switch port data. */
> -    const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */
> -
> -    struct lport_addresses *lsp_addrs;  /* Logical switch port addresses.
> */
> -    unsigned int n_lsp_addrs;
> -
> -    struct lport_addresses *ps_addrs;   /* Port security addresses. */
> -    unsigned int n_ps_addrs;
> -
> -    /* Logical router port data. */
> -    const struct nbrec_logical_router_port *nbrp; /* May be NULL. */
> -
> -    struct lport_addresses lrp_networks;
> -
> -    bool derived; /* Indicates whether this is an additional port
> -                   * derived from nbsp or nbrp. */
> -
> -    /* The port's peer:
> -     *
> -     *     - A switch port S of type "router" has a router port R as a
> peer,
> -     *       and R in turn has S has its peer.
> -     *
> -     *     - Two connected logical router ports have each other as peer.
> */
> -    struct ovn_port *peer;
> -
> -    struct ovn_datapath *od;
> -
> -    struct ovs_list list;       /* In list of similar records. */
> -};
> -
> -static struct ovn_port *
> -ovn_port_create(struct hmap *ports, const char *key,
> -                const struct nbrec_logical_switch_port *nbsp,
> -                const struct nbrec_logical_router_port *nbrp,
> -                const struct sbrec_port_binding *sb)
> -{
> -    struct ovn_port *op = xzalloc(sizeof *op);
> -
> -    struct ds json_key = DS_EMPTY_INITIALIZER;
> -    json_string_escape(key, &json_key);
> -    op->json_key = ds_steal_cstr(&json_key);
> -
> -    op->key = xstrdup(key);
> -    op->sb = sb;
> -    op->nbsp = nbsp;
> -    op->nbrp = nbrp;
> -    op->derived = false;
> -    hmap_insert(ports, &op->key_node, hash_string(op->key, 0));
> -    return op;
> -}
> -
> -static void
> -ovn_port_destroy(struct hmap *ports, struct ovn_port *port)
> -{
> -    if (port) {
> -        /* Don't remove port->list.  It is used within build_ports() as a
> -         * private list and once we've exited that function it is not
> safe to
> -         * use it. */
> -        hmap_remove(ports, &port->key_node);
> -
> -        for (int i = 0; i < port->n_lsp_addrs; i++) {
> -            destroy_lport_addresses(&port->lsp_addrs[i]);
> -        }
> -        free(port->lsp_addrs);
> -
> -        for (int i = 0; i < port->n_ps_addrs; i++) {
> -            destroy_lport_addresses(&port->ps_addrs[i]);
> -        }
> -        free(port->ps_addrs);
> -
> -        destroy_lport_addresses(&port->lrp_networks);
> -        free(port->json_key);
> -        free(port->key);
> -        free(port);
> -    }
> -}
> -
> -static struct ovn_port *
> -ovn_port_find(const struct hmap *ports, const char *name)
> -{
> -    struct ovn_port *op;
> -
> -    HMAP_FOR_EACH_WITH_HASH (op, key_node, hash_string(name, 0), ports) {
> -        if (!strcmp(op->key, name)) {
> -            return op;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static uint32_t
> -ovn_port_allocate_key(struct ovn_datapath *od)
> -{
> -    return allocate_tnlid(&od->port_tnlids, "port",
> -                          1, (1u << 15) - 1, &od->port_key_hint);
> -}
> -
> -static char *
> -chassis_redirect_name(const char *port_name)
> -{
> -    return xasprintf("cr-%s", port_name);
> -}
> -
> -static bool
> -ipam_is_duplicate_mac(struct eth_addr *ea, uint64_t mac64, bool warn)
> -{
> -    struct macam_node *macam_node;
> -    HMAP_FOR_EACH_WITH_HASH (macam_node, hmap_node, hash_uint64(mac64),
> -                             &macam) {
> -        if (eth_addr_equals(*ea, macam_node->mac_addr)) {
> -            if (warn) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_WARN_RL(&rl, "Duplicate MAC set: "ETH_ADDR_FMT,
> -                             ETH_ADDR_ARGS(macam_node->mac_addr));
> -            }
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -static void
> -ipam_insert_mac(struct eth_addr *ea, bool check)
> -{
> -    if (!ea) {
> -        return;
> -    }
> -
> -    uint64_t mac64 = eth_addr_to_uint64(*ea);
> -    uint64_t prefix = eth_addr_to_uint64(mac_prefix);
> -
> -    /* If the new MAC was not assigned by this address management system
> or
> -     * check is true and the new MAC is a duplicate, do not insert it
> into the
> -     * macam hmap. */
> -    if (((mac64 ^ prefix) >> 24)
> -        || (check && ipam_is_duplicate_mac(ea, mac64, true))) {
> -        return;
> -    }
> -
> -    struct macam_node *new_macam_node = xmalloc(sizeof *new_macam_node);
> -    new_macam_node->mac_addr = *ea;
> -    hmap_insert(&macam, &new_macam_node->hmap_node, hash_uint64(mac64));
> -}
> -
> -static void
> -ipam_insert_ip(struct ovn_datapath *od, uint32_t ip)
> -{
> -    if (!od || !od->ipam_info.allocated_ipv4s) {
> -        return;
> -    }
> -
> -    if (ip >= od->ipam_info.start_ipv4 &&
> -        ip < (od->ipam_info.start_ipv4 + od->ipam_info.total_ipv4s)) {
> -        if (bitmap_is_set(od->ipam_info.allocated_ipv4s,
> -                          ip - od->ipam_info.start_ipv4)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_WARN_RL(&rl, "Duplicate IP set on switch %s: "IP_FMT,
> -                         od->nbs->name, IP_ARGS(htonl(ip)));
> -        }
> -        bitmap_set1(od->ipam_info.allocated_ipv4s,
> -                    ip - od->ipam_info.start_ipv4);
> -    }
> -}
> -
> -static void
> -ipam_insert_lsp_addresses(struct ovn_datapath *od, struct ovn_port *op,
> -                          char *address)
> -{
> -    if (!od || !op || !address || !strcmp(address, "unknown")
> -        || !strcmp(address, "router") || is_dynamic_lsp_address(address))
> {
> -        return;
> -    }
> -
> -    struct lport_addresses laddrs;
> -    if (!extract_lsp_addresses(address, &laddrs)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "Extract addresses failed.");
> -        return;
> -    }
> -    ipam_insert_mac(&laddrs.ea, true);
> -
> -    /* IP is only added to IPAM if the switch's subnet option
> -     * is set, whereas MAC is always added to MACAM. */
> -    if (!od->ipam_info.allocated_ipv4s) {
> -        destroy_lport_addresses(&laddrs);
> -        return;
> -    }
> -
> -    for (size_t j = 0; j < laddrs.n_ipv4_addrs; j++) {
> -        uint32_t ip = ntohl(laddrs.ipv4_addrs[j].addr);
> -        ipam_insert_ip(od, ip);
> -    }
> -
> -    destroy_lport_addresses(&laddrs);
> -}
> -
> -static void
> -ipam_add_port_addresses(struct ovn_datapath *od, struct ovn_port *op)
> -{
> -    if (!od || !op) {
> -        return;
> -    }
> -
> -    if (op->nbsp) {
> -        /* Add all the port's addresses to address data structures. */
> -        for (size_t i = 0; i < op->nbsp->n_addresses; i++) {
> -            ipam_insert_lsp_addresses(od, op, op->nbsp->addresses[i]);
> -        }
> -    } else if (op->nbrp) {
> -        struct lport_addresses lrp_networks;
> -        if (!extract_lrp_networks(op->nbrp, &lrp_networks)) {
> -            static struct vlog_rate_limit rl
> -                = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_WARN_RL(&rl, "Extract addresses failed.");
> -            return;
> -        }
> -        ipam_insert_mac(&lrp_networks.ea, true);
> -
> -        if (!op->peer || !op->peer->nbsp || !op->peer->od ||
> !op->peer->od->nbs
> -            || !smap_get(&op->peer->od->nbs->other_config, "subnet")) {
> -            destroy_lport_addresses(&lrp_networks);
> -            return;
> -        }
> -
> -        for (size_t i = 0; i < lrp_networks.n_ipv4_addrs; i++) {
> -            uint32_t ip = ntohl(lrp_networks.ipv4_addrs[i].addr);
> -            ipam_insert_ip(op->peer->od, ip);
> -        }
> -
> -        destroy_lport_addresses(&lrp_networks);
> -    }
> -}
> -
> -static uint64_t
> -ipam_get_unused_mac(ovs_be32 ip)
> -{
> -    uint32_t mac_addr_suffix, i, base_addr = ntohl(ip) & MAC_ADDR_SPACE;
> -    struct eth_addr mac;
> -    uint64_t mac64;
> -
> -    for (i = 0; i < MAC_ADDR_SPACE - 1; i++) {
> -        /* The tentative MAC's suffix will be in the interval (1,
> 0xfffffe). */
> -        mac_addr_suffix = ((base_addr + i) % (MAC_ADDR_SPACE - 1)) + 1;
> -        mac64 =  eth_addr_to_uint64(mac_prefix) | mac_addr_suffix;
> -        eth_addr_from_uint64(mac64, &mac);
> -        if (!ipam_is_duplicate_mac(&mac, mac64, true)) {
> -            break;
> -        }
> -    }
> -
> -    if (i == MAC_ADDR_SPACE) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "MAC address space exhausted.");
> -        mac64 = 0;
> -    }
> -
> -    return mac64;
> -}
> -
> -static uint32_t
> -ipam_get_unused_ip(struct ovn_datapath *od)
> -{
> -    if (!od || !od->ipam_info.allocated_ipv4s) {
> -        return 0;
> -    }
> -
> -    size_t new_ip_index = bitmap_scan(od->ipam_info.allocated_ipv4s, 0, 0,
> -                                      od->ipam_info.total_ipv4s - 1);
> -    if (new_ip_index == od->ipam_info.total_ipv4s - 1) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL( &rl, "Subnet address space has been exhausted.");
> -        return 0;
> -    }
> -
> -    return od->ipam_info.start_ipv4 + new_ip_index;
> -}
> -
> -enum dynamic_update_type {
> -    NONE,    /* No change to the address */
> -    REMOVE,  /* Address is no longer dynamic */
> -    STATIC,  /* Use static address (MAC only) */
> -    DYNAMIC, /* Assign a new dynamic address */
> -};
> -
> -struct dynamic_address_update {
> -    struct ovs_list node;       /* In build_ipam()'s list of updates. */
> -
> -    struct ovn_datapath *od;
> -    struct ovn_port *op;
> -
> -    struct lport_addresses current_addresses;
> -    struct eth_addr static_mac;
> -    ovs_be32 static_ip;
> -    struct in6_addr static_ipv6;
> -    enum dynamic_update_type mac;
> -    enum dynamic_update_type ipv4;
> -    enum dynamic_update_type ipv6;
> -};
> -
> -static enum dynamic_update_type
> -dynamic_mac_changed(const char *lsp_addresses,
> -                    struct dynamic_address_update *update)
> -{
> -   struct eth_addr ea;
> -
> -   if (ovs_scan(lsp_addresses, ETH_ADDR_SCAN_FMT,
> ETH_ADDR_SCAN_ARGS(ea))) {
> -       if (eth_addr_equals(ea, update->current_addresses.ea)) {
> -           return NONE;
> -       } else {
> -           /* MAC is still static, but it has changed */
> -           update->static_mac = ea;
> -           return STATIC;
> -       }
> -   }
> -
> -   uint64_t mac64 = eth_addr_to_uint64(update->current_addresses.ea);
> -   uint64_t prefix = eth_addr_to_uint64(mac_prefix);
> -
> -   if ((mac64 ^ prefix) >> 24) {
> -       return DYNAMIC;
> -   } else {
> -       return NONE;
> -   }
> -}
> -
> -static enum dynamic_update_type
> -dynamic_ip4_changed(const char *lsp_addrs,
> -                    struct dynamic_address_update *update)
> -{
> -    const struct ipam_info *ipam = &update->op->od->ipam_info;
> -    const struct lport_addresses *cur_addresses =
> &update->current_addresses;
> -    bool dynamic_ip4 = ipam->allocated_ipv4s != NULL;
> -
> -    if (!dynamic_ip4) {
> -        if (update->current_addresses.n_ipv4_addrs) {
> -            return REMOVE;
> -        } else {
> -            return NONE;
> -        }
> -    }
> -
> -    if (!cur_addresses->n_ipv4_addrs) {
> -        /* IPv4 was previously static but now is dynamic */
> -        return DYNAMIC;
> -    }
> -
> -    uint32_t ip4 = ntohl(cur_addresses->ipv4_addrs[0].addr);
> -    if (ip4 < ipam->start_ipv4) {
> -        return DYNAMIC;
> -    }
> -
> -    uint32_t index = ip4 - ipam->start_ipv4;
> -    if (index > ipam->total_ipv4s ||
> -        bitmap_is_set(ipam->allocated_ipv4s, index)) {
> -        /* Previously assigned dynamic IPv4 address can no longer be used.
> -         * It's either outside the subnet, conflicts with an excluded IP,
> -         * or conflicts with a statically-assigned address on the switch
> -         */
> -        return DYNAMIC;
> -    } else {
> -        char ipv6_s[IPV6_SCAN_LEN + 1];
> -        ovs_be32 new_ip;
> -        int n = 0;
> -
> -        if ((ovs_scan(lsp_addrs, "dynamic "IP_SCAN_FMT"%n",
> -                     IP_SCAN_ARGS(&new_ip), &n)
> -             && lsp_addrs[n] == '\0') ||
> -            (ovs_scan(lsp_addrs, "dynamic "IP_SCAN_FMT"
> "IPV6_SCAN_FMT"%n",
> -                      IP_SCAN_ARGS(&new_ip), ipv6_s, &n)
> -             && lsp_addrs[n] == '\0')) {
> -            index = ntohl(new_ip) - ipam->start_ipv4;
> -            if (ntohl(new_ip) < ipam->start_ipv4 ||
> -                index > ipam->total_ipv4s ||
> -                bitmap_is_set(ipam->allocated_ipv4s, index)) {
> -                /* new static ip is not valid */
> -                return DYNAMIC;
> -            } else if (cur_addresses->ipv4_addrs[0].addr != new_ip) {
> -                update->ipv4 = STATIC;
> -                update->static_ip = new_ip;
> -                return STATIC;
> -            }
> -        }
> -        return NONE;
> -    }
> -}
> -
> -static enum dynamic_update_type
> -dynamic_ip6_changed(const char *lsp_addrs,
> -                    struct dynamic_address_update *update)
> -{
> -    bool dynamic_ip6 = update->op->od->ipam_info.ipv6_prefix_set;
> -    struct eth_addr ea;
> -
> -    if (!dynamic_ip6) {
> -        if (update->current_addresses.n_ipv6_addrs) {
> -            /* IPv6 was dynamic but now is not */
> -            return REMOVE;
> -        } else {
> -            /* IPv6 has never been dynamic */
> -            return NONE;
> -        }
> -    }
> -
> -    if (!update->current_addresses.n_ipv6_addrs ||
> -        ovs_scan(lsp_addrs, ETH_ADDR_SCAN_FMT, ETH_ADDR_SCAN_ARGS(ea))) {
> -        /* IPv6 was previously static but now is dynamic */
> -        return DYNAMIC;
> -    }
> -
> -    const struct lport_addresses *cur_addresses;
> -    char ipv6_s[IPV6_SCAN_LEN + 1];
> -    ovs_be32 new_ip;
> -    int n = 0;
> -
> -    if ((ovs_scan(lsp_addrs, "dynamic "IPV6_SCAN_FMT"%n",
> -                  ipv6_s, &n) && lsp_addrs[n] == '\0') ||
> -        (ovs_scan(lsp_addrs, "dynamic "IP_SCAN_FMT" "IPV6_SCAN_FMT"%n",
> -                  IP_SCAN_ARGS(&new_ip), ipv6_s, &n)
> -         && lsp_addrs[n] == '\0')) {
> -        struct in6_addr ipv6;
> -
> -        if (!ipv6_parse(ipv6_s, &ipv6)) {
> -            return DYNAMIC;
> -        }
> -
> -        struct in6_addr masked = ipv6_addr_bitand(&ipv6,
> -                &update->op->od->ipam_info.ipv6_prefix);
> -        if (!IN6_ARE_ADDR_EQUAL(&masked,
> -                                &update->op->od->ipam_info.ipv6_prefix)) {
> -            return DYNAMIC;
> -        }
> -
> -        cur_addresses = &update->current_addresses;
> -
> -        if (!IN6_ARE_ADDR_EQUAL(&cur_addresses->ipv6_addrs[0].addr,
> -                                &ipv6)) {
> -            update->static_ipv6 = ipv6;
> -            return STATIC;
> -        }
> -    } else if (update->mac != NONE) {
> -        return DYNAMIC;
> -    }
> -
> -    return NONE;
> -}
> -
> -/* Check previously assigned dynamic addresses for validity. This will
> - * check if the assigned addresses need to change.
> - *
> - * Returns true if any changes to dynamic addresses are required
> - */
> -static bool
> -dynamic_addresses_check_for_updates(const char *lsp_addrs,
> -                                    struct dynamic_address_update *update)
> -{
> -    update->mac = dynamic_mac_changed(lsp_addrs, update);
> -    update->ipv4 = dynamic_ip4_changed(lsp_addrs, update);
> -    update->ipv6 = dynamic_ip6_changed(lsp_addrs, update);
> -    if (update->mac == NONE &&
> -        update->ipv4 == NONE &&
> -        update->ipv6 == NONE) {
> -        return false;
> -    } else {
> -        return true;
> -    }
> -}
> -
> -/* For addresses that do not need to be updated, go ahead and insert them
> - * into IPAM. This way, their addresses will be claimed and cannot be
> assigned
> - * elsewhere later.
> - */
> -static void
> -update_unchanged_dynamic_addresses(struct dynamic_address_update *update)
> -{
> -    if (update->mac == NONE) {
> -        ipam_insert_mac(&update->current_addresses.ea, false);
> -    }
> -    if (update->ipv4 == NONE && update->current_addresses.n_ipv4_addrs) {
> -        ipam_insert_ip(update->op->od,
> -
>  ntohl(update->current_addresses.ipv4_addrs[0].addr));
> -    }
> -}
> -
> -static void
> -set_lsp_dynamic_addresses(const char *dynamic_addresses, struct ovn_port
> *op)
> -{
> -    extract_lsp_addresses(dynamic_addresses,
> &op->lsp_addrs[op->n_lsp_addrs]);
> -    op->n_lsp_addrs++;
> -}
> -
> -/* Determines which components (MAC, IPv4, and IPv6) of dynamic
> - * addresses need to be assigned. This is used exclusively for
> - * ports that do not have dynamic addresses already assigned.
> - */
> -static void
> -set_dynamic_updates(const char *addrspec,
> -                    struct dynamic_address_update *update)
> -{
> -    bool has_ipv4 = false, has_ipv6 = false;
> -    char ipv6_s[IPV6_SCAN_LEN + 1];
> -    struct eth_addr mac;
> -    ovs_be32 ip;
> -    int n = 0;
> -    if (ovs_scan(addrspec, ETH_ADDR_SCAN_FMT" dynamic%n",
> -                 ETH_ADDR_SCAN_ARGS(mac), &n)
> -        && addrspec[n] == '\0') {
> -        update->mac = STATIC;
> -        update->static_mac = mac;
> -    } else {
> -        update->mac = DYNAMIC;
> -    }
> -
> -    if ((ovs_scan(addrspec, "dynamic "IP_SCAN_FMT"%n",
> -                 IP_SCAN_ARGS(&ip), &n) && addrspec[n] == '\0')) {
> -        has_ipv4 = true;
> -    } else if ((ovs_scan(addrspec, "dynamic "IPV6_SCAN_FMT"%n",
> -                         ipv6_s, &n) && addrspec[n] == '\0')) {
> -        has_ipv6 = true;
> -    } else if ((ovs_scan(addrspec, "dynamic "IP_SCAN_FMT"
> "IPV6_SCAN_FMT"%n",
> -                         IP_SCAN_ARGS(&ip), ipv6_s, &n)
> -               && addrspec[n] == '\0')) {
> -        has_ipv4 = has_ipv6 = true;
> -    }
> -
> -    if (has_ipv4) {
> -        update->ipv4 = STATIC;
> -        update->static_ip = ip;
> -    } else if (update->op->od->ipam_info.allocated_ipv4s) {
> -        update->ipv4 = DYNAMIC;
> -    } else {
> -        update->ipv4 = NONE;
> -    }
> -
> -    if (has_ipv6 && ipv6_parse(ipv6_s, &update->static_ipv6)) {
> -        update->ipv6 = STATIC;
> -    } else if (update->op->od->ipam_info.ipv6_prefix_set) {
> -        update->ipv6 = DYNAMIC;
> -    } else {
> -        update->ipv6 = NONE;
> -    }
> -}
> -
> -static void
> -update_dynamic_addresses(struct dynamic_address_update *update)
> -{
> -    ovs_be32 ip4 = 0;
> -    switch (update->ipv4) {
> -    case NONE:
> -        if (update->current_addresses.n_ipv4_addrs) {
> -            ip4 = update->current_addresses.ipv4_addrs[0].addr;
> -        }
> -        break;
> -    case REMOVE:
> -        break;
> -    case STATIC:
> -        ip4 = update->static_ip;
> -        break;
> -    case DYNAMIC:
> -        ip4 = htonl(ipam_get_unused_ip(update->od));
> -    }
> -
> -    struct eth_addr mac;
> -    switch (update->mac) {
> -    case NONE:
> -        mac = update->current_addresses.ea;
> -        break;
> -    case REMOVE:
> -        OVS_NOT_REACHED();
> -    case STATIC:
> -        mac = update->static_mac;
> -        break;
> -    case DYNAMIC:
> -        eth_addr_from_uint64(ipam_get_unused_mac(ip4), &mac);
> -        break;
> -    }
> -
> -    struct in6_addr ip6 = in6addr_any;
> -    switch (update->ipv6) {
> -    case NONE:
> -        if (update->current_addresses.n_ipv6_addrs) {
> -            ip6 = update->current_addresses.ipv6_addrs[0].addr;
> -        }
> -        break;
> -    case REMOVE:
> -        break;
> -    case STATIC:
> -        ip6 = update->static_ipv6;
> -        break;
> -    case DYNAMIC:
> -        in6_generate_eui64(mac, &update->od->ipam_info.ipv6_prefix, &ip6);
> -        break;
> -    }
> -
> -    struct ds new_addr = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&new_addr, ETH_ADDR_FMT, ETH_ADDR_ARGS(mac));
> -    ipam_insert_mac(&mac, true);
> -
> -    if (ip4) {
> -        ipam_insert_ip(update->od, ntohl(ip4));
> -        ds_put_format(&new_addr, " "IP_FMT, IP_ARGS(ip4));
> -    }
> -    if (!IN6_ARE_ADDR_EQUAL(&ip6, &in6addr_any)) {
> -        char ip6_s[INET6_ADDRSTRLEN + 1];
> -        ipv6_string_mapped(ip6_s, &ip6);
> -        ds_put_format(&new_addr, " %s", ip6_s);
> -    }
> -    nbrec_logical_switch_port_set_dynamic_addresses(update->op->nbsp,
> -                                                    ds_cstr(&new_addr));
> -    set_lsp_dynamic_addresses(ds_cstr(&new_addr), update->op);
> -    ds_destroy(&new_addr);
> -}
> -
> -static void
> -build_ipam(struct hmap *datapaths, struct hmap *ports)
> -{
> -    /* IPAM generally stands for IP address management.  In
> non-virtualized
> -     * world, MAC addresses come with the hardware.  But, with virtualized
> -     * workloads, they need to be assigned and managed.  This function
> -     * does both IP address management (ipam) and MAC address management
> -     * (macam). */
> -
> -    /* If the switch's other_config:subnet is set, allocate new addresses
> for
> -     * ports that have the "dynamic" keyword in their addresses column. */
> -    struct ovn_datapath *od;
> -    struct ovs_list updates;
> -
> -    ovs_list_init(&updates);
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        for (size_t i = 0; i < od->nbs->n_ports; i++) {
> -            const struct nbrec_logical_switch_port *nbsp =
> od->nbs->ports[i];
> -
> -            if (!od->ipam_info.allocated_ipv4s &&
> -                !od->ipam_info.ipv6_prefix_set &&
> -                !od->ipam_info.mac_only) {
> -                if (nbsp->dynamic_addresses) {
> -                    nbrec_logical_switch_port_set_dynamic_addresses(nbsp,
> -                                                                    NULL);
> -                }
> -                continue;
> -            }
> -
> -            struct ovn_port *op = ovn_port_find(ports, nbsp->name);
> -            if (!op || op->nbsp != nbsp || op->peer) {
> -                /* Do not allocate addresses for logical switch ports that
> -                 * have a peer. */
> -                continue;
> -            }
> -
> -            int num_dynamic_addresses = 0;
> -            for (size_t j = 0; j < nbsp->n_addresses; j++) {
> -                if (!is_dynamic_lsp_address(nbsp->addresses[j])) {
> -                    continue;
> -                }
> -                if (num_dynamic_addresses) {
> -                    static struct vlog_rate_limit rl
> -                        = VLOG_RATE_LIMIT_INIT(1, 1);
> -                    VLOG_WARN_RL(&rl, "More than one dynamic address "
> -                                 "configured for logical switch port
> '%s'",
> -                                 nbsp->name);
> -                    continue;
> -                }
> -                num_dynamic_addresses++;
> -                struct dynamic_address_update *update
> -                    = xzalloc(sizeof *update);
> -                update->op = op;
> -                update->od = od;
> -                if (nbsp->dynamic_addresses) {
> -                    bool any_changed;
> -                    extract_lsp_addresses(nbsp->dynamic_addresses,
> -                                          &update->current_addresses);
> -                    any_changed = dynamic_addresses_check_for_updates(
> -                        nbsp->addresses[j], update);
> -                    update_unchanged_dynamic_addresses(update);
> -                    if (any_changed) {
> -                        ovs_list_push_back(&updates, &update->node);
> -                    } else {
> -                        /* No changes to dynamic addresses */
> -
> set_lsp_dynamic_addresses(nbsp->dynamic_addresses, op);
> -
> destroy_lport_addresses(&update->current_addresses);
> -                        free(update);
> -                    }
> -                } else {
> -                    set_dynamic_updates(nbsp->addresses[j], update);
> -                    ovs_list_push_back(&updates, &update->node);
> -                }
> -            }
> -
> -            if (!num_dynamic_addresses && nbsp->dynamic_addresses) {
> -                nbrec_logical_switch_port_set_dynamic_addresses(nbsp,
> NULL);
> -            }
> -        }
> -
> -    }
> -
> -    /* After retaining all unchanged dynamic addresses, now assign
> -     * new ones.
> -     */
> -    struct dynamic_address_update *update;
> -    LIST_FOR_EACH_POP (update, node, &updates) {
> -        update_dynamic_addresses(update);
> -        destroy_lport_addresses(&update->current_addresses);
> -        free(update);
> -    }
> -}
> -
> -/* Tag allocation for nested containers.
> - *
> - * For a logical switch port with 'parent_name' and a request to allocate
> tags,
> - * keeps a track of all allocated tags. */
> -struct tag_alloc_node {
> -    struct hmap_node hmap_node;
> -    char *parent_name;
> -    unsigned long *allocated_tags;  /* A bitmap to track allocated tags.
> */
> -};
> -
> -static void
> -tag_alloc_destroy(struct hmap *tag_alloc_table)
> -{
> -    struct tag_alloc_node *node;
> -    HMAP_FOR_EACH_POP (node, hmap_node, tag_alloc_table) {
> -        bitmap_free(node->allocated_tags);
> -        free(node->parent_name);
> -        free(node);
> -    }
> -    hmap_destroy(tag_alloc_table);
> -}
> -
> -static struct tag_alloc_node *
> -tag_alloc_get_node(struct hmap *tag_alloc_table, const char *parent_name)
> -{
> -    /* If a node for the 'parent_name' exists, return it. */
> -    struct tag_alloc_node *tag_alloc_node;
> -    HMAP_FOR_EACH_WITH_HASH (tag_alloc_node, hmap_node,
> -                             hash_string(parent_name, 0),
> -                             tag_alloc_table) {
> -        if (!strcmp(tag_alloc_node->parent_name, parent_name)) {
> -            return tag_alloc_node;
> -        }
> -    }
> -
> -    /* Create a new node. */
> -    tag_alloc_node = xmalloc(sizeof *tag_alloc_node);
> -    tag_alloc_node->parent_name = xstrdup(parent_name);
> -    tag_alloc_node->allocated_tags = bitmap_allocate(MAX_OVN_TAGS);
> -    /* Tag 0 is invalid for nested containers. */
> -    bitmap_set1(tag_alloc_node->allocated_tags, 0);
> -    hmap_insert(tag_alloc_table, &tag_alloc_node->hmap_node,
> -                hash_string(parent_name, 0));
> -
> -    return tag_alloc_node;
> -}
> -
> -static void
> -tag_alloc_add_existing_tags(struct hmap *tag_alloc_table,
> -                            const struct nbrec_logical_switch_port *nbsp)
> -{
> -    /* Add the tags of already existing nested containers.  If there is no
> -     * 'nbsp->parent_name' or no 'nbsp->tag' set, there is nothing to do.
> */
> -    if (!nbsp->parent_name || !nbsp->parent_name[0] || !nbsp->tag) {
> -        return;
> -    }
> -
> -    struct tag_alloc_node *tag_alloc_node;
> -    tag_alloc_node = tag_alloc_get_node(tag_alloc_table,
> nbsp->parent_name);
> -    bitmap_set1(tag_alloc_node->allocated_tags, *nbsp->tag);
> -}
> -
> -static void
> -tag_alloc_create_new_tag(struct hmap *tag_alloc_table,
> -                         const struct nbrec_logical_switch_port *nbsp)
> -{
> -    if (!nbsp->tag_request) {
> -        return;
> -    }
> -
> -    if (nbsp->parent_name && nbsp->parent_name[0]
> -        && *nbsp->tag_request == 0) {
> -        /* For nested containers that need allocation, do the allocation.
> */
> -
> -        if (nbsp->tag) {
> -            /* This has already been allocated. */
> -            return;
> -        }
> -
> -        struct tag_alloc_node *tag_alloc_node;
> -        int64_t tag;
> -        tag_alloc_node = tag_alloc_get_node(tag_alloc_table,
> -                                            nbsp->parent_name);
> -        tag = bitmap_scan(tag_alloc_node->allocated_tags, 0, 1,
> MAX_OVN_TAGS);
> -        if (tag == MAX_OVN_TAGS) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -            VLOG_ERR_RL(&rl, "out of vlans for logical switch ports with "
> -                        "parent %s", nbsp->parent_name);
> -            return;
> -        }
> -        bitmap_set1(tag_alloc_node->allocated_tags, tag);
> -        nbrec_logical_switch_port_set_tag(nbsp, &tag, 1);
> -    } else if (*nbsp->tag_request != 0) {
> -        /* For everything else, copy the contents of 'tag_request' to
> 'tag'. */
> -        nbrec_logical_switch_port_set_tag(nbsp, nbsp->tag_request, 1);
> -    }
> -}
> -
> -
> -static void
> -join_logical_ports(struct northd_context *ctx,
> -                   struct hmap *datapaths, struct hmap *ports,
> -                   struct hmap *chassis_qdisc_queues,
> -                   struct hmap *tag_alloc_table, struct ovs_list *sb_only,
> -                   struct ovs_list *nb_only, struct ovs_list *both)
> -{
> -    ovs_list_init(sb_only);
> -    ovs_list_init(nb_only);
> -    ovs_list_init(both);
> -
> -    const struct sbrec_port_binding *sb;
> -    SBREC_PORT_BINDING_FOR_EACH (sb, ctx->ovnsb_idl) {
> -        struct ovn_port *op = ovn_port_create(ports, sb->logical_port,
> -                                              NULL, NULL, sb);
> -        ovs_list_push_back(sb_only, &op->list);
> -    }
> -
> -    struct ovn_datapath *od;
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (od->nbs) {
> -            for (size_t i = 0; i < od->nbs->n_ports; i++) {
> -                const struct nbrec_logical_switch_port *nbsp
> -                    = od->nbs->ports[i];
> -                struct ovn_port *op = ovn_port_find(ports, nbsp->name);
> -                if (op) {
> -                    if (op->nbsp || op->nbrp) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(5, 1);
> -                        VLOG_WARN_RL(&rl, "duplicate logical port %s",
> -                                     nbsp->name);
> -                        continue;
> -                    }
> -                    op->nbsp = nbsp;
> -                    ovs_list_remove(&op->list);
> -
> -                    uint32_t queue_id = smap_get_int(&op->sb->options,
> -                                                     "qdisc_queue_id", 0);
> -                    if (queue_id && op->sb->chassis) {
> -                        add_chassis_queue(
> -                             chassis_qdisc_queues,
> &op->sb->chassis->header_.uuid,
> -                             queue_id);
> -                    }
> -
> -                    ovs_list_push_back(both, &op->list);
> -
> -                    /* This port exists due to a SB binding, but should
> -                     * not have been initialized fully. */
> -                    ovs_assert(!op->n_lsp_addrs && !op->n_ps_addrs);
> -                } else {
> -                    op = ovn_port_create(ports, nbsp->name, nbsp, NULL,
> NULL);
> -                    ovs_list_push_back(nb_only, &op->list);
> -                }
> -
> -                if (!strcmp(nbsp->type, "localnet")) {
> -                   od->localnet_port = op;
> -                }
> -
> -                op->lsp_addrs
> -                    = xmalloc(sizeof *op->lsp_addrs * nbsp->n_addresses);
> -                for (size_t j = 0; j < nbsp->n_addresses; j++) {
> -                    if (!strcmp(nbsp->addresses[j], "unknown")
> -                        || !strcmp(nbsp->addresses[j], "router")) {
> -                        continue;
> -                    }
> -                    if (is_dynamic_lsp_address(nbsp->addresses[j])) {
> -                        continue;
> -                    } else if (!extract_lsp_addresses(nbsp->addresses[j],
> -
>  &op->lsp_addrs[op->n_lsp_addrs])) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(1, 1);
> -                        VLOG_INFO_RL(&rl, "invalid syntax '%s' in logical
> "
> -                                          "switch port addresses. No MAC "
> -                                          "address found",
> -                                          op->nbsp->addresses[j]);
> -                        continue;
> -                    }
> -                    op->n_lsp_addrs++;
> -                }
> -
> -                op->ps_addrs
> -                    = xmalloc(sizeof *op->ps_addrs *
> nbsp->n_port_security);
> -                for (size_t j = 0; j < nbsp->n_port_security; j++) {
> -                    if (!extract_lsp_addresses(nbsp->port_security[j],
> -
>  &op->ps_addrs[op->n_ps_addrs])) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(1, 1);
> -                        VLOG_INFO_RL(&rl, "invalid syntax '%s' in port "
> -                                          "security. No MAC address
> found",
> -                                          op->nbsp->port_security[j]);
> -                        continue;
> -                    }
> -                    op->n_ps_addrs++;
> -                }
> -
> -                op->od = od;
> -                tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> -            }
> -        } else {
> -            for (size_t i = 0; i < od->nbr->n_ports; i++) {
> -                const struct nbrec_logical_router_port *nbrp
> -                    = od->nbr->ports[i];
> -
> -                struct lport_addresses lrp_networks;
> -                if (!extract_lrp_networks(nbrp, &lrp_networks)) {
> -                    static struct vlog_rate_limit rl
> -                        = VLOG_RATE_LIMIT_INIT(5, 1);
> -                    VLOG_WARN_RL(&rl, "bad 'mac' %s", nbrp->mac);
> -                    continue;
> -                }
> -
> -                if (!lrp_networks.n_ipv4_addrs &&
> !lrp_networks.n_ipv6_addrs) {
> -                    continue;
> -                }
> -
> -                struct ovn_port *op = ovn_port_find(ports, nbrp->name);
> -                if (op) {
> -                    if (op->nbsp || op->nbrp) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(5, 1);
> -                        VLOG_WARN_RL(&rl, "duplicate logical router port
> %s",
> -                                     nbrp->name);
> -                        continue;
> -                    }
> -                    op->nbrp = nbrp;
> -                    ovs_list_remove(&op->list);
> -                    ovs_list_push_back(both, &op->list);
> -
> -                    /* This port exists but should not have been
> -                     * initialized fully. */
> -                    ovs_assert(!op->lrp_networks.n_ipv4_addrs
> -                               && !op->lrp_networks.n_ipv6_addrs);
> -                } else {
> -                    op = ovn_port_create(ports, nbrp->name, NULL, nbrp,
> NULL);
> -                    ovs_list_push_back(nb_only, &op->list);
> -                }
> -
> -                op->lrp_networks = lrp_networks;
> -                op->od = od;
> -
> -                const char *redirect_chassis =
> smap_get(&op->nbrp->options,
> -
> "redirect-chassis");
> -                if (op->nbrp->ha_chassis_group || redirect_chassis ||
> -                    op->nbrp->n_gateway_chassis) {
> -                    /* Additional "derived" ovn_port crp represents the
> -                     * instance of op on the "redirect-chassis". */
> -                    const char *gw_chassis =
> smap_get(&op->od->nbr->options,
> -                                                   "chassis");
> -                    if (gw_chassis) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(1, 1);
> -                        VLOG_WARN_RL(&rl, "Bad configuration: "
> -                                     "redirect-chassis configured on port
> %s "
> -                                     "on L3 gateway router", nbrp->name);
> -                        continue;
> -                    }
> -                    if (od->l3dgw_port || od->l3redirect_port) {
> -                        static struct vlog_rate_limit rl
> -                            = VLOG_RATE_LIMIT_INIT(1, 1);
> -                        VLOG_WARN_RL(&rl, "Bad configuration: multiple
> ports "
> -                                     "with redirect-chassis on same
> logical "
> -                                     "router %s", od->nbr->name);
> -                        continue;
> -                    }
> -
> -                    char *redirect_name =
> chassis_redirect_name(nbrp->name);
> -                    struct ovn_port *crp = ovn_port_find(ports,
> redirect_name);
> -                    if (crp) {
> -                        crp->derived = true;
> -                        crp->nbrp = nbrp;
> -                        ovs_list_remove(&crp->list);
> -                        ovs_list_push_back(both, &crp->list);
> -                    } else {
> -                        crp = ovn_port_create(ports, redirect_name,
> -                                              NULL, nbrp, NULL);
> -                        crp->derived = true;
> -                        ovs_list_push_back(nb_only, &crp->list);
> -                    }
> -                    crp->od = od;
> -                    free(redirect_name);
> -
> -                    /* Set l3dgw_port and l3redirect_port in od, for later
> -                     * use during flow creation. */
> -                    od->l3dgw_port = op;
> -                    od->l3redirect_port = crp;
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Connect logical router ports, and logical switch ports of type
> "router",
> -     * to their peers. */
> -    struct ovn_port *op;
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (op->nbsp && !strcmp(op->nbsp->type, "router") &&
> !op->derived) {
> -            const char *peer_name = smap_get(&op->nbsp->options,
> "router-port");
> -            if (!peer_name) {
> -                continue;
> -            }
> -
> -            struct ovn_port *peer = ovn_port_find(ports, peer_name);
> -            if (!peer || !peer->nbrp) {
> -                continue;
> -            }
> -
> -            peer->peer = op;
> -            op->peer = peer;
> -            op->od->router_ports = xrealloc(
> -                op->od->router_ports,
> -                sizeof *op->od->router_ports * (op->od->n_router_ports +
> 1));
> -            op->od->router_ports[op->od->n_router_ports++] = op;
> -
> -            /* Fill op->lsp_addrs for op->nbsp->addresses[] with
> -             * contents "router", which was skipped in the loop above. */
> -            for (size_t j = 0; j < op->nbsp->n_addresses; j++) {
> -                if (!strcmp(op->nbsp->addresses[j], "router")) {
> -                    if (extract_lrp_networks(peer->nbrp,
> -
> &op->lsp_addrs[op->n_lsp_addrs])) {
> -                        op->n_lsp_addrs++;
> -                    }
> -                    break;
> -                }
> -            }
> -        } else if (op->nbrp && op->nbrp->peer && !op->derived) {
> -            struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> -            if (peer) {
> -                if (peer->nbrp) {
> -                    op->peer = peer;
> -                } else if (peer->nbsp) {
> -                    /* An ovn_port for a switch port of type "router"
> does have
> -                     * a router port as its peer (see the case above for
> -                     * "router" ports), but this is set via
> options:router-port
> -                     * in Logical_Switch_Port and does not involve the
> -                     * Logical_Router_Port's 'peer' column. */
> -                    static struct vlog_rate_limit rl =
> -                            VLOG_RATE_LIMIT_INIT(5, 1);
> -                    VLOG_WARN_RL(&rl, "Bad configuration: The peer of
> router "
> -                                 "port %s is a switch port", op->key);
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Wait until all ports have been connected to add to IPAM since
> -     * it relies on proper peers to be set
> -     */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        ipam_add_port_addresses(op->od, op);
> -    }
> -}
> -
> -static void
> -ip_address_and_port_from_lb_key(const char *key, char **ip_address,
> -                                uint16_t *port, int *addr_family);
> -
> -static void
> -get_router_load_balancer_ips(const struct ovn_datapath *od,
> -                             struct sset *all_ips, int *addr_family)
> -{
> -    if (!od->nbr) {
> -        return;
> -    }
> -
> -    for (int i = 0; i < od->nbr->n_load_balancer; i++) {
> -        struct nbrec_load_balancer *lb = od->nbr->load_balancer[i];
> -        struct smap *vips = &lb->vips;
> -        struct smap_node *node;
> -
> -        SMAP_FOR_EACH (node, vips) {
> -            /* node->key contains IP:port or just IP. */
> -            char *ip_address = NULL;
> -            uint16_t port;
> -
> -            ip_address_and_port_from_lb_key(node->key, &ip_address, &port,
> -                                            addr_family);
> -            if (!ip_address) {
> -                continue;
> -            }
> -
> -            if (!sset_contains(all_ips, ip_address)) {
> -                sset_add(all_ips, ip_address);
> -            }
> -
> -            free(ip_address);
> -        }
> -    }
> -}
> -
> -/* Returns an array of strings, each consisting of a MAC address followed
> - * by one or more IP addresses, and if the port is a distributed gateway
> - * port, followed by 'is_chassis_resident("LPORT_NAME")', where the
> - * LPORT_NAME is the name of the L3 redirect port or the name of the
> - * logical_port specified in a NAT rule.  These strings include the
> - * external IP addresses of all NAT rules defined on that router, and all
> - * of the IP addresses used in load balancer VIPs defined on that router.
> - *
> - * The caller must free each of the n returned strings with free(),
> - * and must free the returned array when it is no longer needed. */
> -static char **
> -get_nat_addresses(const struct ovn_port *op, size_t *n)
> -{
> -    size_t n_nats = 0;
> -    struct eth_addr mac;
> -    if (!op->nbrp || !op->od || !op->od->nbr
> -        || (!op->od->nbr->n_nat && !op->od->nbr->n_load_balancer)
> -        || !eth_addr_from_string(op->nbrp->mac, &mac)) {
> -        *n = n_nats;
> -        return NULL;
> -    }
> -
> -    struct ds c_addresses = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&c_addresses, ETH_ADDR_FMT, ETH_ADDR_ARGS(mac));
> -    bool central_ip_address = false;
> -
> -    char **addresses;
> -    addresses = xmalloc(sizeof *addresses * (op->od->nbr->n_nat + 1));
> -
> -    /* Get NAT IP addresses. */
> -    for (size_t i = 0; i < op->od->nbr->n_nat; i++) {
> -        const struct nbrec_nat *nat = op->od->nbr->nat[i];
> -        ovs_be32 ip, mask;
> -
> -        char *error = ip_parse_masked(nat->external_ip, &ip, &mask);
> -        if (error || mask != OVS_BE32_MAX) {
> -            free(error);
> -            continue;
> -        }
> -
> -        /* Determine whether this NAT rule satisfies the conditions for
> -         * distributed NAT processing. */
> -        if (op->od->l3redirect_port && !strcmp(nat->type, "dnat_and_snat")
> -            && nat->logical_port && nat->external_mac) {
> -            /* Distributed NAT rule. */
> -            if (eth_addr_from_string(nat->external_mac, &mac)) {
> -                struct ds address = DS_EMPTY_INITIALIZER;
> -                ds_put_format(&address, ETH_ADDR_FMT, ETH_ADDR_ARGS(mac));
> -                ds_put_format(&address, " %s", nat->external_ip);
> -                ds_put_format(&address, " is_chassis_resident(\"%s\")",
> -                              nat->logical_port);
> -                addresses[n_nats++] = ds_steal_cstr(&address);
> -            }
> -        } else {
> -            /* Centralized NAT rule, either on gateway router or
> distributed
> -             * router.
> -             * Check if external_ip is same as router ip. If so, then
> there
> -             * is no need to add this to the nat_addresses. The router IPs
> -             * will be added separately. */
> -            bool is_router_ip = false;
> -            for (size_t j = 0; j < op->lrp_networks.n_ipv4_addrs; j++) {
> -                if (!strcmp(nat->external_ip,
> -                            op->lrp_networks.ipv4_addrs[j].addr_s)) {
> -                    is_router_ip = true;
> -                    break;
> -                }
> -            }
> -
> -            if (!is_router_ip) {
> -                ds_put_format(&c_addresses, " %s", nat->external_ip);
> -                central_ip_address = true;
> -            }
> -        }
> -    }
> -
> -    /* A set to hold all load-balancer vips. */
> -    struct sset all_ips = SSET_INITIALIZER(&all_ips);
> -    int addr_family;
> -    get_router_load_balancer_ips(op->od, &all_ips, &addr_family);
> -
> -    const char *ip_address;
> -    SSET_FOR_EACH (ip_address, &all_ips) {
> -        ds_put_format(&c_addresses, " %s", ip_address);
> -        central_ip_address = true;
> -    }
> -    sset_destroy(&all_ips);
> -
> -    if (central_ip_address) {
> -        /* Gratuitous ARP for centralized NAT rules on distributed gateway
> -         * ports should be restricted to the "redirect-chassis". */
> -        if (op->od->l3redirect_port) {
> -            ds_put_format(&c_addresses, " is_chassis_resident(%s)",
> -                          op->od->l3redirect_port->json_key);
> -        }
> -
> -        addresses[n_nats++] = ds_steal_cstr(&c_addresses);
> -    }
> -
> -    *n = n_nats;
> -
> -    return addresses;
> -}
> -
> -static bool
> -sbpb_gw_chassis_needs_update(
> -    const struct sbrec_port_binding *pb,
> -    const struct nbrec_logical_router_port *lrp,
> -    struct ovsdb_idl_index *sbrec_chassis_by_name)
> -{
> -    if (!lrp || !pb) {
> -        return false;
> -    }
> -
> -    if (lrp->n_gateway_chassis && !pb->ha_chassis_group) {
> -        /* If there are gateway chassis in the NB DB, but there is
> -         * no corresponding HA chassis group in SB DB we need to
> -         * create the HA chassis group in SB DB for this lrp. */
> -        return true;
> -    }
> -
> -    if (strcmp(pb->ha_chassis_group->name, lrp->name)) {
> -        /* Name doesn't match. */
> -        return true;
> -    }
> -
> -    if (lrp->n_gateway_chassis != pb->ha_chassis_group->n_ha_chassis) {
> -        return true;
> -    }
> -
> -    for (size_t i = 0; i < lrp->n_gateway_chassis; i++) {
> -        struct nbrec_gateway_chassis *nbgw_ch = lrp->gateway_chassis[i];
> -        bool found = false;
> -        for (size_t j = 0; j < pb->ha_chassis_group->n_ha_chassis; j++) {
> -            struct sbrec_ha_chassis *sbha_ch =
> -                pb->ha_chassis_group->ha_chassis[j];
> -            const char *chassis_name = smap_get(&sbha_ch->external_ids,
> -                                                "chassis-name");
> -            if (!chassis_name) {
> -                return true;
> -            }
> -
> -            if (strcmp(chassis_name, nbgw_ch->chassis_name)) {
> -                continue;
> -            }
> -
> -            found = true;
> -
> -            if (nbgw_ch->priority != sbha_ch->priority) {
> -                return true;
> -            }
> -
> -            if (sbha_ch->chassis &&
> -                strcmp(nbgw_ch->chassis_name, sbha_ch->chassis->name)) {
> -                /* sbha_ch->chassis's name is different from the one
> -                 * in sbha_ch->external_ids:chassis-name. */
> -                return true;
> -            }
> -
> -            if (!sbha_ch->chassis &&
> -                chassis_lookup_by_name(sbrec_chassis_by_name,
> -                                       nbgw_ch->chassis_name)) {
> -                /* sbha_ch->chassis is NULL, but the chassis is
> -                 * present in Chassis table. */
> -                return true;
> -            }
> -        }
> -
> -        if (!found) {
> -            return true;
> -        }
> -    }
> -
> -    /* No need to update SB DB. Its in sync. */
> -    return false;
> -}
> -
> -static struct sbrec_ha_chassis *
> -create_sb_ha_chassis(struct northd_context *ctx,
> -                     const struct sbrec_chassis *chassis,
> -                     const char *chassis_name, int priority)
> -{
> -    struct sbrec_ha_chassis *sb_ha_chassis =
> -        sbrec_ha_chassis_insert(ctx->ovnsb_txn);
> -    sbrec_ha_chassis_set_chassis(sb_ha_chassis, chassis);
> -    sbrec_ha_chassis_set_priority(sb_ha_chassis, priority);
> -    /* Store the chassis_name in external_ids. If the chassis
> -     * entry doesn't exist in the Chassis table then we can
> -     * figure out the chassis to which this ha_chassis
> -     * maps to. */
> -    const struct smap external_ids =
> -        SMAP_CONST1(&external_ids, "chassis-name", chassis_name);
> -    sbrec_ha_chassis_set_external_ids(sb_ha_chassis, &external_ids);
> -    return sb_ha_chassis;
> -}
> -
> -static bool
> -chassis_group_list_changed(
> -    const struct nbrec_ha_chassis_group *nb_ha_grp,
> -    const struct sbrec_ha_chassis_group *sb_ha_grp,
> -    struct ovsdb_idl_index *sbrec_chassis_by_name)
> -{
> -    if (nb_ha_grp->n_ha_chassis != sb_ha_grp->n_ha_chassis) {
> -        return true;
> -    }
> -
> -    struct shash nb_ha_chassis_list =
> SHASH_INITIALIZER(&nb_ha_chassis_list);
> -    for (size_t i = 0; i < nb_ha_grp->n_ha_chassis; i++) {
> -        shash_add(&nb_ha_chassis_list,
> -                  nb_ha_grp->ha_chassis[i]->chassis_name,
> -                  nb_ha_grp->ha_chassis[i]);
> -    }
> -
> -    bool changed = false;
> -    const struct sbrec_ha_chassis *sb_ha_chassis;
> -    const struct nbrec_ha_chassis *nb_ha_chassis;
> -    for (size_t i = 0; i < sb_ha_grp->n_ha_chassis; i++) {
> -        sb_ha_chassis = sb_ha_grp->ha_chassis[i];
> -        const char *chassis_name = smap_get(&sb_ha_chassis->external_ids,
> -                                            "chassis-name");
> -
> -        if (!chassis_name) {
> -            changed = true;
> -            break;
> -        }
> -
> -        nb_ha_chassis = shash_find_and_delete(&nb_ha_chassis_list,
> -                                              chassis_name);
> -        if (!nb_ha_chassis ||
> -            nb_ha_chassis->priority != sb_ha_chassis->priority) {
> -            changed = true;
> -            break;
> -        }
> -
> -        if (sb_ha_chassis->chassis &&
> -            strcmp(sb_ha_chassis->chassis->name, chassis_name)) {
> -            /* sb_ha_chassis->chassis's name is different from the one
> -             * in sb_ha_chassis->external_ids:chassis-name. */
> -            changed = true;
> -            break;
> -        }
> -
> -        if (!sb_ha_chassis->chassis &&
> -            chassis_lookup_by_name(sbrec_chassis_by_name,
> -                                   chassis_name)) {
> -            /* sb_ha_chassis->chassis is NULL, but the chassis is
> -             * present in Chassis table. */
> -            changed = true;
> -            break;
> -        }
> -    }
> -
> -    struct shash_node *node, *next;
> -    SHASH_FOR_EACH_SAFE (node, next, &nb_ha_chassis_list) {
> -        shash_delete(&nb_ha_chassis_list, node);
> -        changed = true;
> -    }
> -    shash_destroy(&nb_ha_chassis_list);
> -
> -    return changed;
> -}
> -
> -static void
> -sync_ha_chassis_group_for_sbpb(struct northd_context *ctx,
> -                               const struct nbrec_ha_chassis_group
> *nb_ha_grp,
> -                               struct ovsdb_idl_index
> *sbrec_chassis_by_name,
> -                               const struct sbrec_port_binding *pb)
> -{
> -    bool new_sb_chassis_group = false;
> -    const struct sbrec_ha_chassis_group *sb_ha_grp =
> -        ha_chassis_group_lookup_by_name(
> -            ctx->sbrec_ha_chassis_grp_by_name, nb_ha_grp->name);
> -
> -    if (!sb_ha_grp) {
> -        sb_ha_grp = sbrec_ha_chassis_group_insert(ctx->ovnsb_txn);
> -        sbrec_ha_chassis_group_set_name(sb_ha_grp, nb_ha_grp->name);
> -        new_sb_chassis_group = true;
> -    }
> -
> -    if (new_sb_chassis_group ||
> -        chassis_group_list_changed(nb_ha_grp, sb_ha_grp,
> -                                   sbrec_chassis_by_name)) {
> -        struct sbrec_ha_chassis **sb_ha_chassis = NULL;
> -        size_t n_ha_chassis = nb_ha_grp->n_ha_chassis;
> -        sb_ha_chassis = xcalloc(n_ha_chassis, sizeof *sb_ha_chassis);
> -        for (size_t i = 0; i < nb_ha_grp->n_ha_chassis; i++) {
> -            const struct nbrec_ha_chassis *nb_ha_chassis
> -                = nb_ha_grp->ha_chassis[i];
> -            const struct sbrec_chassis *chassis =
> -                chassis_lookup_by_name(sbrec_chassis_by_name,
> -                                       nb_ha_chassis->chassis_name);
> -            sb_ha_chassis[i] = sbrec_ha_chassis_insert(ctx->ovnsb_txn);
> -            /* It's perfectly ok if the chassis is NULL. This could
> -             * happen when ovn-controller exits and removes its row
> -             * from the chassis table in OVN SB DB. */
> -            sbrec_ha_chassis_set_chassis(sb_ha_chassis[i], chassis);
> -            sbrec_ha_chassis_set_priority(sb_ha_chassis[i],
> -                                          nb_ha_chassis->priority);
> -            const struct smap external_ids =
> -                SMAP_CONST1(&external_ids, "chassis-name",
> -                            nb_ha_chassis->chassis_name);
> -            sbrec_ha_chassis_set_external_ids(sb_ha_chassis[i],
> &external_ids);
> -        }
> -        sbrec_ha_chassis_group_set_ha_chassis(sb_ha_grp, sb_ha_chassis,
> -                                              n_ha_chassis);
> -        free(sb_ha_chassis);
> -    }
> -
> -    sbrec_port_binding_set_ha_chassis_group(pb, sb_ha_grp);
> -}
> -
> -/* This functions translates the gw chassis on the nb database
> - * to HA chassis group in the sb database entries.
> - */
> -static void
> -copy_gw_chassis_from_nbrp_to_sbpb(
> -        struct northd_context *ctx,
> -        struct ovsdb_idl_index *sbrec_chassis_by_name,
> -        const struct nbrec_logical_router_port *lrp,
> -        const struct sbrec_port_binding *port_binding)
> -{
> -
> -    /* Make use of the new HA chassis group table to support HA
> -     * for the distributed gateway router port. */
> -    const struct sbrec_ha_chassis_group *sb_ha_chassis_group =
> -        ha_chassis_group_lookup_by_name(
> -            ctx->sbrec_ha_chassis_grp_by_name, lrp->name);
> -    if (!sb_ha_chassis_group) {
> -        sb_ha_chassis_group =
> sbrec_ha_chassis_group_insert(ctx->ovnsb_txn);
> -        sbrec_ha_chassis_group_set_name(sb_ha_chassis_group, lrp->name);
> -    }
> -
> -    struct sbrec_ha_chassis **sb_ha_chassis =
> xcalloc(lrp->n_gateway_chassis,
> -                                                      sizeof
> *sb_ha_chassis);
> -    size_t n_sb_ha_ch = 0;
> -    for (size_t n = 0; n < lrp->n_gateway_chassis; n++) {
> -        struct nbrec_gateway_chassis *lrp_gwc = lrp->gateway_chassis[n];
> -        if (!lrp_gwc->chassis_name) {
> -            continue;
> -        }
> -
> -        const struct sbrec_chassis *chassis =
> -            chassis_lookup_by_name(sbrec_chassis_by_name,
> -                                   lrp_gwc->chassis_name);
> -
> -        sb_ha_chassis[n_sb_ha_ch] =
> -            create_sb_ha_chassis(ctx, chassis, lrp_gwc->chassis_name,
> -                                 lrp_gwc->priority);
> -        n_sb_ha_ch++;
> -    }
> -
> -    sbrec_ha_chassis_group_set_ha_chassis(sb_ha_chassis_group,
> -                                          sb_ha_chassis, n_sb_ha_ch);
> -    sbrec_port_binding_set_ha_chassis_group(port_binding,
> sb_ha_chassis_group);
> -    free(sb_ha_chassis);
> -}
> -
> -static void
> -ovn_port_update_sbrec(struct northd_context *ctx,
> -                      struct ovsdb_idl_index *sbrec_chassis_by_name,
> -                      const struct ovn_port *op,
> -                      struct hmap *chassis_qdisc_queues,
> -                      struct sset *active_ha_chassis_grps)
> -{
> -    sbrec_port_binding_set_datapath(op->sb, op->od->sb);
> -    if (op->nbrp) {
> -        /* If the router is for l3 gateway, it resides on a chassis
> -         * and its port type is "l3gateway". */
> -        const char *chassis_name = smap_get(&op->od->nbr->options,
> "chassis");
> -        if (op->derived) {
> -            sbrec_port_binding_set_type(op->sb, "chassisredirect");
> -        } else if (chassis_name) {
> -            sbrec_port_binding_set_type(op->sb, "l3gateway");
> -        } else {
> -            sbrec_port_binding_set_type(op->sb, "patch");
> -        }
> -
> -        struct smap new;
> -        smap_init(&new);
> -        if (op->derived) {
> -            const char *redirect_chassis = smap_get(&op->nbrp->options,
> -                                                    "redirect-chassis");
> -            int n_gw_options_set = 0;
> -            if (op->nbrp->ha_chassis_group) {
> -                n_gw_options_set++;
> -            }
> -            if (op->nbrp->n_gateway_chassis) {
> -                n_gw_options_set++;
> -            }
> -            if (redirect_chassis) {
> -                n_gw_options_set++;
> -            }
> -            if (n_gw_options_set > 1) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_WARN_RL(
> -                    &rl, "Multiple gatway options set for the logical
> router "
> -                         "port %s. The first preferred option is "
> -                         "ha_chassis_group; the second is
> gateway_chassis; "
> -                         "and the last is redirect-chassis.",
> op->nbrp->name);
> -            }
> -
> -            if (op->nbrp->ha_chassis_group) {
> -                /* HA Chassis group is set. Ignore 'gateway_chassis'
> -                 * column and redirect-chassis option. */
> -                sync_ha_chassis_group_for_sbpb(ctx,
> op->nbrp->ha_chassis_group,
> -                                               sbrec_chassis_by_name,
> op->sb);
> -                sset_add(active_ha_chassis_grps,
> -                         op->nbrp->ha_chassis_group->name);
> -            } else if (op->nbrp->n_gateway_chassis) {
> -                /* Legacy gateway_chassis support.
> -                 * Create ha_chassis_group for the Northbound
> gateway_chassis
> -                 * associated with the lrp. */
> -                if (sbpb_gw_chassis_needs_update(op->sb, op->nbrp,
> -                                                 sbrec_chassis_by_name)) {
> -                    copy_gw_chassis_from_nbrp_to_sbpb(ctx,
> -
> sbrec_chassis_by_name,
> -                                                      op->nbrp, op->sb);
> -                }
> -
> -                sset_add(active_ha_chassis_grps, op->nbrp->name);
> -            } else if (redirect_chassis) {
> -                /* Handle ports that had redirect-chassis option attached
> -                 * to them, and for backwards compatibility convert them
> -                 * to a single HA Chassis group entry */
> -                const struct sbrec_chassis *chassis =
> -                    chassis_lookup_by_name(sbrec_chassis_by_name,
> -                                           redirect_chassis);
> -                if (chassis) {
> -                    /* If we found the chassis, and the gw chassis on
> record
> -                     * differs from what we expect go ahead and update */
> -                    char *gwc_name = xasprintf("%s_%s", op->nbrp->name,
> -                                chassis->name);
> -                    const struct sbrec_ha_chassis_group *sb_ha_ch_grp;
> -                    sb_ha_ch_grp = ha_chassis_group_lookup_by_name(
> -                        ctx->sbrec_ha_chassis_grp_by_name, gwc_name);
> -                    if (!sb_ha_ch_grp) {
> -                        sb_ha_ch_grp =
> -                            sbrec_ha_chassis_group_insert(ctx->ovnsb_txn);
> -                        sbrec_ha_chassis_group_set_name(sb_ha_ch_grp,
> -                                                        gwc_name);
> -                    }
> -
> -                    if (sb_ha_ch_grp->n_ha_chassis != 1) {
> -                        struct sbrec_ha_chassis **sb_ha_ch =
> -                            xcalloc(1, sizeof *sb_ha_ch);
> -                        sb_ha_ch[0] = create_sb_ha_chassis(ctx, chassis,
> -                                                           chassis->name,
> 0);
> -
> sbrec_ha_chassis_group_set_ha_chassis(sb_ha_ch_grp,
> -                                                              sb_ha_ch,
> 1);
> -                    }
> -                    sbrec_port_binding_set_ha_chassis_group(op->sb,
> -                                                            sb_ha_ch_grp);
> -                    sset_add(active_ha_chassis_grps, gwc_name);
> -                    free(gwc_name);
> -                } else {
> -                    VLOG_WARN("chassis name '%s' from redirect from
> logical "
> -                              " router port '%s' redirect-chassis not
> found",
> -                              redirect_chassis, op->nbrp->name);
> -                    if (op->sb->ha_chassis_group) {
> -                        sbrec_port_binding_set_ha_chassis_group(op->sb,
> NULL);
> -                    }
> -                }
> -            } else {
> -                /* Nothing is set. Clear ha_chassis_group  from pb. */
> -                if (op->sb->ha_chassis_group) {
> -                    sbrec_port_binding_set_ha_chassis_group(op->sb, NULL);
> -                }
> -            }
> -
> -            if (op->sb->n_gateway_chassis) {
> -                /* Delete the legacy gateway_chassis from the pb. */
> -                sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0);
> -            }
> -            smap_add(&new, "distributed-port", op->nbrp->name);
> -        } else {
> -            if (op->peer) {
> -                smap_add(&new, "peer", op->peer->key);
> -            }
> -            if (chassis_name) {
> -                smap_add(&new, "l3gateway-chassis", chassis_name);
> -            }
> -        }
> -        sbrec_port_binding_set_options(op->sb, &new);
> -        smap_destroy(&new);
> -
> -        sbrec_port_binding_set_parent_port(op->sb, NULL);
> -        sbrec_port_binding_set_tag(op->sb, NULL, 0);
> -
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        ds_put_cstr(&s, op->nbrp->mac);
> -        for (int i = 0; i < op->nbrp->n_networks; ++i) {
> -            ds_put_format(&s, " %s", op->nbrp->networks[i]);
> -        }
> -        const char *addresses = ds_cstr(&s);
> -        sbrec_port_binding_set_mac(op->sb, &addresses, 1);
> -        ds_destroy(&s);
> -
> -        struct smap ids = SMAP_INITIALIZER(&ids);
> -        sbrec_port_binding_set_external_ids(op->sb, &ids);
> -
> -        sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0);
> -    } else {
> -        if (strcmp(op->nbsp->type, "router")) {
> -            uint32_t queue_id = smap_get_int(
> -                    &op->sb->options, "qdisc_queue_id", 0);
> -            bool has_qos = port_has_qos_params(&op->nbsp->options);
> -            struct smap options;
> -
> -            if (op->sb->chassis && has_qos && !queue_id) {
> -                queue_id = allocate_chassis_queueid(chassis_qdisc_queues,
> -                                                    op->sb->chassis);
> -            } else if (!has_qos && queue_id) {
> -                free_chassis_queueid(chassis_qdisc_queues,
> -                                     op->sb->chassis,
> -                                     queue_id);
> -                queue_id = 0;
> -            }
> -
> -            smap_clone(&options, &op->nbsp->options);
> -            if (queue_id) {
> -                smap_add_format(&options,
> -                                "qdisc_queue_id", "%d", queue_id);
> -            }
> -            sbrec_port_binding_set_options(op->sb, &options);
> -            smap_destroy(&options);
> -            if (ovn_is_known_nb_lsp_type(op->nbsp->type)) {
> -                sbrec_port_binding_set_type(op->sb, op->nbsp->type);
> -            } else {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_WARN_RL(
> -                    &rl, "Unknown port type '%s' set on logical switch
> '%s'.",
> -                    op->nbsp->type, op->nbsp->name);
> -            }
> -
> -            sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0);
> -
> -            if (!strcmp(op->nbsp->type, "external")) {
> -                if (op->nbsp->ha_chassis_group) {
> -                    sync_ha_chassis_group_for_sbpb(
> -                        ctx, op->nbsp->ha_chassis_group,
> -                        sbrec_chassis_by_name, op->sb);
> -                    sset_add(active_ha_chassis_grps,
> -                             op->nbsp->ha_chassis_group->name);
> -                } else {
> -                    sbrec_port_binding_set_ha_chassis_group(op->sb, NULL);
> -                }
> -            }
> -        } else {
> -            const char *chassis = NULL;
> -            if (op->peer && op->peer->od && op->peer->od->nbr) {
> -                chassis = smap_get(&op->peer->od->nbr->options,
> "chassis");
> -            }
> -
> -            /* A switch port connected to a gateway router is also of
> -             * type "l3gateway". */
> -            if (chassis) {
> -                sbrec_port_binding_set_type(op->sb, "l3gateway");
> -            } else {
> -                sbrec_port_binding_set_type(op->sb, "patch");
> -            }
> -
> -            const char *router_port = smap_get(&op->nbsp->options,
> -                                               "router-port");
> -            if (router_port || chassis) {
> -                struct smap new;
> -                smap_init(&new);
> -                if (router_port) {
> -                    smap_add(&new, "peer", router_port);
> -                }
> -                if (chassis) {
> -                    smap_add(&new, "l3gateway-chassis", chassis);
> -                }
> -                sbrec_port_binding_set_options(op->sb, &new);
> -                smap_destroy(&new);
> -            } else {
> -                sbrec_port_binding_set_options(op->sb, NULL);
> -            }
> -
> -            const char *nat_addresses = smap_get(&op->nbsp->options,
> -                                           "nat-addresses");
> -            size_t n_nats = 0;
> -            char **nats = NULL;
> -            if (nat_addresses && !strcmp(nat_addresses, "router")) {
> -                if (op->peer && op->peer->od
> -                    && (chassis || op->peer->od->l3redirect_port)) {
> -                    nats = get_nat_addresses(op->peer, &n_nats);
> -                }
> -            /* Only accept manual specification of ethernet address
> -             * followed by IPv4 addresses on type "l3gateway" ports. */
> -            } else if (nat_addresses && chassis) {
> -                struct lport_addresses laddrs;
> -                if (!extract_lsp_addresses(nat_addresses, &laddrs)) {
> -                    static struct vlog_rate_limit rl =
> -                        VLOG_RATE_LIMIT_INIT(1, 1);
> -                    VLOG_WARN_RL(&rl, "Error extracting nat-addresses.");
> -                } else {
> -                    destroy_lport_addresses(&laddrs);
> -                    n_nats = 1;
> -                    nats = xcalloc(1, sizeof *nats);
> -                    nats[0] = xstrdup(nat_addresses);
> -                }
> -            }
> -
> -            /* Add the router mac and IPv4 addresses to
> -             * Port_Binding.nat_addresses so that GARP is sent for these
> -             * IPs by the ovn-controller on which the distributed gateway
> -             * router port resides if:
> -             *
> -             * -  op->peer has 'reside-on-gateway-chassis' set and the
> -             *    the logical router datapath has distributed router port.
> -             *
> -             * -  op->peer is distributed gateway router port.
> -             *
> -             * -  op->peer's router is a gateway router and op has a
> localnet
> -             *    port.
> -             *
> -             * Note: Port_Binding.nat_addresses column is also used for
> -             * sending the GARPs for the router port IPs.
> -             * */
> -            bool add_router_port_garp = false;
> -            if (op->peer && op->peer->nbrp && op->peer->od->l3dgw_port &&
> -                op->peer->od->l3redirect_port &&
> -                (smap_get_bool(&op->peer->nbrp->options,
> -                              "reside-on-redirect-chassis", false) ||
> -                op->peer == op->peer->od->l3dgw_port)) {
> -                add_router_port_garp = true;
> -            } else if (chassis && op->od->localnet_port) {
> -                add_router_port_garp = true;
> -            }
> -
> -            if (add_router_port_garp) {
> -                struct ds garp_info = DS_EMPTY_INITIALIZER;
> -                ds_put_format(&garp_info, "%s",
> op->peer->lrp_networks.ea_s);
> -                for (size_t i = 0; i <
> op->peer->lrp_networks.n_ipv4_addrs;
> -                     i++) {
> -                    ds_put_format(&garp_info, " %s",
> -
> op->peer->lrp_networks.ipv4_addrs[i].addr_s);
> -                }
> -
> -                if (op->peer->od->l3redirect_port) {
> -                    ds_put_format(&garp_info, " is_chassis_resident(%s)",
> -
> op->peer->od->l3redirect_port->json_key);
> -                }
> -
> -                n_nats++;
> -                nats = xrealloc(nats, (n_nats * sizeof *nats));
> -                nats[n_nats - 1] = ds_steal_cstr(&garp_info);
> -                ds_destroy(&garp_info);
> -            }
> -
> -            sbrec_port_binding_set_nat_addresses(op->sb,
> -                                                 (const char **) nats,
> n_nats);
> -            for (size_t i = 0; i < n_nats; i++) {
> -                free(nats[i]);
> -            }
> -            free(nats);
> -        }
> -
> -        sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name);
> -        sbrec_port_binding_set_tag(op->sb, op->nbsp->tag,
> op->nbsp->n_tag);
> -        sbrec_port_binding_set_mac(op->sb, (const char **)
> op->nbsp->addresses,
> -                                   op->nbsp->n_addresses);
> -
> -        struct smap ids = SMAP_INITIALIZER(&ids);
> -        smap_clone(&ids, &op->nbsp->external_ids);
> -        const char *name = smap_get(&ids, "neutron:port_name");
> -        if (name && name[0]) {
> -            smap_add(&ids, "name", name);
> -        }
> -        sbrec_port_binding_set_external_ids(op->sb, &ids);
> -        smap_destroy(&ids);
> -    }
> -}
> -
> -/* Remove mac_binding entries that refer to logical_ports which are
> - * deleted. */
> -static void
> -cleanup_mac_bindings(struct northd_context *ctx, struct hmap *ports)
> -{
> -    const struct sbrec_mac_binding *b, *n;
> -    SBREC_MAC_BINDING_FOR_EACH_SAFE (b, n, ctx->ovnsb_idl) {
> -        if (!ovn_port_find(ports, b->logical_port)) {
> -            sbrec_mac_binding_delete(b);
> -        }
> -    }
> -}
> -
> -static void
> -cleanup_sb_ha_chassis_groups(struct northd_context *ctx,
> -                             struct sset *active_ha_chassis_groups)
> -{
> -    const struct sbrec_ha_chassis_group *b, *n;
> -    SBREC_HA_CHASSIS_GROUP_FOR_EACH_SAFE (b, n, ctx->ovnsb_idl) {
> -        if (!sset_contains(active_ha_chassis_groups, b->name)) {
> -            sbrec_ha_chassis_group_delete(b);
> -        }
> -    }
> -}
> -
> -/* Updates the southbound Port_Binding table so that it contains the
> logical
> - * switch ports specified by the northbound database.
> - *
> - * Initializes 'ports' to contain a "struct ovn_port" for every logical
> port,
> - * using the "struct ovn_datapath"s in 'datapaths' to look up logical
> - * datapaths. */
> -static void
> -build_ports(struct northd_context *ctx,
> -            struct ovsdb_idl_index *sbrec_chassis_by_name,
> -            struct hmap *datapaths, struct hmap *ports)
> -{
> -    struct ovs_list sb_only, nb_only, both;
> -    struct hmap tag_alloc_table = HMAP_INITIALIZER(&tag_alloc_table);
> -    struct hmap chassis_qdisc_queues =
> HMAP_INITIALIZER(&chassis_qdisc_queues);
> -
> -    /* sset which stores the set of ha chassis group names used. */
> -    struct sset active_ha_chassis_grps =
> -        SSET_INITIALIZER(&active_ha_chassis_grps);
> -
> -    join_logical_ports(ctx, datapaths, ports, &chassis_qdisc_queues,
> -                       &tag_alloc_table, &sb_only, &nb_only, &both);
> -
> -    struct ovn_port *op, *next;
> -    /* For logical ports that are in both databases, update the southbound
> -     * record based on northbound data.  Also index the in-use
> tunnel_keys.
> -     * For logical ports that are in NB database, do any tag allocation
> -     * needed. */
> -    LIST_FOR_EACH_SAFE (op, next, list, &both) {
> -        if (op->nbsp) {
> -            tag_alloc_create_new_tag(&tag_alloc_table, op->nbsp);
> -        }
> -        ovn_port_update_sbrec(ctx, sbrec_chassis_by_name,
> -                              op, &chassis_qdisc_queues,
> -                              &active_ha_chassis_grps);
> -        add_tnlid(&op->od->port_tnlids, op->sb->tunnel_key);
> -        if (op->sb->tunnel_key > op->od->port_key_hint) {
> -            op->od->port_key_hint = op->sb->tunnel_key;
> -        }
> -    }
> -
> -    /* Add southbound record for each unmatched northbound record. */
> -    LIST_FOR_EACH_SAFE (op, next, list, &nb_only) {
> -        uint16_t tunnel_key = ovn_port_allocate_key(op->od);
> -        if (!tunnel_key) {
> -            continue;
> -        }
> -
> -        op->sb = sbrec_port_binding_insert(ctx->ovnsb_txn);
> -        ovn_port_update_sbrec(ctx, sbrec_chassis_by_name, op,
> -                              &chassis_qdisc_queues,
> -                              &active_ha_chassis_grps);
> -        sbrec_port_binding_set_logical_port(op->sb, op->key);
> -        sbrec_port_binding_set_tunnel_key(op->sb, tunnel_key);
> -    }
> -
> -    bool remove_mac_bindings = false;
> -    if (!ovs_list_is_empty(&sb_only)) {
> -        remove_mac_bindings = true;
> -    }
> -
> -    /* Delete southbound records without northbound matches. */
> -    LIST_FOR_EACH_SAFE(op, next, list, &sb_only) {
> -        ovs_list_remove(&op->list);
> -        sbrec_port_binding_delete(op->sb);
> -        ovn_port_destroy(ports, op);
> -    }
> -    if (remove_mac_bindings) {
> -        cleanup_mac_bindings(ctx, ports);
> -    }
> -
> -    tag_alloc_destroy(&tag_alloc_table);
> -    destroy_chassis_queues(&chassis_qdisc_queues);
> -    cleanup_sb_ha_chassis_groups(ctx, &active_ha_chassis_grps);
> -    sset_destroy(&active_ha_chassis_grps);
> -}
> -
> -struct multicast_group {
> -    const char *name;
> -    uint16_t key;               /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST.
> */
> -};
> -
> -#define MC_FLOOD "_MC_flood"
> -static const struct multicast_group mc_flood =
> -    { MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY };
> -
> -#define MC_UNKNOWN "_MC_unknown"
> -static const struct multicast_group mc_unknown =
> -    { MC_UNKNOWN, OVN_MCAST_UNKNOWN_TUNNEL_KEY };
> -
> -static bool
> -multicast_group_equal(const struct multicast_group *a,
> -                      const struct multicast_group *b)
> -{
> -    return !strcmp(a->name, b->name) && a->key == b->key;
> -}
> -
> -/* Multicast group entry. */
> -struct ovn_multicast {
> -    struct hmap_node hmap_node; /* Index on 'datapath' and 'key'. */
> -    struct ovn_datapath *datapath;
> -    const struct multicast_group *group;
> -
> -    struct ovn_port **ports;
> -    size_t n_ports, allocated_ports;
> -};
> -
> -static uint32_t
> -ovn_multicast_hash(const struct ovn_datapath *datapath,
> -                   const struct multicast_group *group)
> -{
> -    return hash_pointer(datapath, group->key);
> -}
> -
> -static struct ovn_multicast *
> -ovn_multicast_find(struct hmap *mcgroups, struct ovn_datapath *datapath,
> -                   const struct multicast_group *group)
> -{
> -    struct ovn_multicast *mc;
> -
> -    HMAP_FOR_EACH_WITH_HASH (mc, hmap_node,
> -                             ovn_multicast_hash(datapath, group),
> mcgroups) {
> -        if (mc->datapath == datapath
> -            && multicast_group_equal(mc->group, group)) {
> -            return mc;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -ovn_multicast_add_ports(struct hmap *mcgroups, struct ovn_datapath *od,
> -                        const struct multicast_group *group,
> -                        struct ovn_port **ports, size_t n_ports)
> -{
> -    struct ovn_multicast *mc = ovn_multicast_find(mcgroups, od, group);
> -    if (!mc) {
> -        mc = xmalloc(sizeof *mc);
> -        hmap_insert(mcgroups, &mc->hmap_node, ovn_multicast_hash(od,
> group));
> -        mc->datapath = od;
> -        mc->group = group;
> -        mc->n_ports = 0;
> -        mc->allocated_ports = 4;
> -        mc->ports = xmalloc(mc->allocated_ports * sizeof *mc->ports);
> -    }
> -
> -    size_t n_ports_total = mc->n_ports + n_ports;
> -
> -    if (n_ports_total > 2 * mc->allocated_ports) {
> -        mc->allocated_ports = n_ports_total;
> -        mc->ports = xrealloc(mc->ports,
> -                             mc->allocated_ports * sizeof *mc->ports);
> -    } else if (n_ports_total > mc->allocated_ports) {
> -        mc->ports = x2nrealloc(mc->ports, &mc->allocated_ports,
> -                               sizeof *mc->ports);
> -    }
> -
> -    memcpy(&mc->ports[mc->n_ports], &ports[0], n_ports * sizeof *ports);
> -    mc->n_ports += n_ports;
> -}
> -
> -static void
> -ovn_multicast_add(struct hmap *mcgroups, const struct multicast_group
> *group,
> -                  struct ovn_port *port)
> -{
> -    ovn_multicast_add_ports(mcgroups, port->od, group, &port, 1);
> -}
> -
> -static void
> -ovn_multicast_destroy(struct hmap *mcgroups, struct ovn_multicast *mc)
> -{
> -    if (mc) {
> -        hmap_remove(mcgroups, &mc->hmap_node);
> -        free(mc->ports);
> -        free(mc);
> -    }
> -}
> -
> -static void
> -ovn_multicast_update_sbrec(const struct ovn_multicast *mc,
> -                           const struct sbrec_multicast_group *sb)
> -{
> -    struct sbrec_port_binding **ports = xmalloc(mc->n_ports * sizeof
> *ports);
> -    for (size_t i = 0; i < mc->n_ports; i++) {
> -        ports[i] = CONST_CAST(struct sbrec_port_binding *,
> mc->ports[i]->sb);
> -    }
> -    sbrec_multicast_group_set_ports(sb, ports, mc->n_ports);
> -    free(ports);
> -}
> -
> -/*
> - * IGMP group entry (1:1 mapping to SB database).
> - */
> -struct ovn_igmp_group_entry {
> -    struct ovs_list list_node; /* Linkage in the list of entries. */
> -    const struct sbrec_igmp_group *sb;
> -};
> -
> -/*
> - * IGMP group entry (aggregate of all entries from the SB database
> - * corresponding to the multicast group).
> - */
> -struct ovn_igmp_group {
> -    struct hmap_node hmap_node; /* Index on 'datapath' and 'address'. */
> -
> -    struct ovn_datapath *datapath;
> -    struct in6_addr address; /* Multicast IPv6-mapped-IPv4 or IPv4
> address. */
> -    struct multicast_group mcgroup;
> -
> -    struct ovs_list sb_entries; /* List of SB entries for this group. */
> -};
> -
> -static uint32_t
> -ovn_igmp_group_hash(const struct ovn_datapath *datapath,
> -                    const struct in6_addr *address)
> -{
> -    return hash_pointer(datapath, hash_bytes(address, sizeof *address,
> 0));
> -}
> -
> -static struct ovn_igmp_group *
> -ovn_igmp_group_find(struct hmap *igmp_groups,
> -                    const struct ovn_datapath *datapath,
> -                    const struct in6_addr *address)
> -{
> -    struct ovn_igmp_group *group;
> -
> -    HMAP_FOR_EACH_WITH_HASH (group, hmap_node,
> -                             ovn_igmp_group_hash(datapath, address),
> -                             igmp_groups) {
> -        if (group->datapath == datapath &&
> -                ipv6_addr_equals(&group->address, address)) {
> -            return group;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -ovn_igmp_group_add(struct northd_context *ctx, struct hmap *igmp_groups,
> -                   struct ovn_datapath *datapath,
> -                   const struct sbrec_igmp_group *sb_igmp_group)
> -{
> -    struct in6_addr group_address;
> -    ovs_be32 ipv4;
> -
> -    if (ip_parse(sb_igmp_group->address, &ipv4)) {
> -        group_address = in6_addr_mapped_ipv4(ipv4);
> -    } else if (!ipv6_parse(sb_igmp_group->address, &group_address)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> -        VLOG_WARN_RL(&rl, "invalid IGMP group address: %s",
> -                     sb_igmp_group->address);
> -        return;
> -    }
> -
> -    struct ovn_igmp_group *igmp_group =
> -        ovn_igmp_group_find(igmp_groups, datapath, &group_address);
> -
> -    if (!igmp_group) {
> -        igmp_group = xmalloc(sizeof *igmp_group);
> -
> -        const struct sbrec_multicast_group *mcgroup =
> -            mcast_group_lookup(ctx->sbrec_mcast_group_by_name_dp,
> -                               sb_igmp_group->address, datapath->sb);
> -
> -        igmp_group->datapath = datapath;
> -        igmp_group->address = group_address;
> -        if (mcgroup) {
> -            igmp_group->mcgroup.key = mcgroup->tunnel_key;
> -            add_tnlid(&datapath->mcast_info.group_tnlids,
> mcgroup->tunnel_key);
> -        } else {
> -            igmp_group->mcgroup.key = 0;
> -        }
> -        igmp_group->mcgroup.name = sb_igmp_group->address;
> -        ovs_list_init(&igmp_group->sb_entries);
> -
> -        hmap_insert(igmp_groups, &igmp_group->hmap_node,
> -                    ovn_igmp_group_hash(datapath, &group_address));
> -    }
> -
> -    struct ovn_igmp_group_entry *entry = xmalloc(sizeof *entry);
> -
> -    entry->sb = sb_igmp_group;
> -    ovs_list_push_back(&igmp_group->sb_entries , &entry->list_node);
> -}
> -
> -static void
> -ovn_igmp_group_aggregate_ports(struct ovn_igmp_group *igmp_group,
> -                               struct hmap *ovn_ports,
> -                               struct hmap *mcast_groups)
> -{
> -    struct ovn_igmp_group_entry *entry;
> -
> -    LIST_FOR_EACH_POP (entry, list_node, &igmp_group->sb_entries) {
> -        size_t n_oports = 0;
> -        struct ovn_port **oports =
> -            xmalloc(entry->sb->n_ports * sizeof *oports);
> -
> -        for (size_t i = 0; i < entry->sb->n_ports; i++) {
> -            oports[n_oports] =
> -                ovn_port_find(ovn_ports,
> entry->sb->ports[i]->logical_port);
> -            if (oports[n_oports]) {
> -                n_oports++;
> -            }
> -        }
> -
> -        ovn_multicast_add_ports(mcast_groups, igmp_group->datapath,
> -                                &igmp_group->mcgroup, oports, n_oports);
> -        free(oports);
> -        free(entry);
> -    }
> -}
> -
> -static void
> -ovn_igmp_group_destroy(struct hmap *igmp_groups,
> -                       struct ovn_igmp_group *igmp_group)
> -{
> -    if (igmp_group) {
> -        struct ovn_igmp_group_entry *entry;
> -
> -        LIST_FOR_EACH_POP (entry, list_node, &igmp_group->sb_entries) {
> -            free(entry);
> -        }
> -        hmap_remove(igmp_groups, &igmp_group->hmap_node);
> -        free(igmp_group);
> -    }
> -}
> -
> -/* Logical flow generation.
> - *
> - * This code generates the Logical_Flow table in the southbound database,
> as a
> - * function of most of the northbound database.
> - */
> -
> -struct ovn_lflow {
> -    struct hmap_node hmap_node;
> -
> -    struct ovn_datapath *od;
> -    enum ovn_stage stage;
> -    uint16_t priority;
> -    char *match;
> -    char *actions;
> -    char *stage_hint;
> -    const char *where;
> -};
> -
> -static size_t
> -ovn_lflow_hash(const struct ovn_lflow *lflow)
> -{
> -    return ovn_logical_flow_hash(&lflow->od->sb->header_.uuid,
> -                                 ovn_stage_get_table(lflow->stage),
> -
>  ovn_stage_get_pipeline_name(lflow->stage),
> -                                 lflow->priority, lflow->match,
> -                                 lflow->actions);
> -}
> -
> -static bool
> -ovn_lflow_equal(const struct ovn_lflow *a, const struct ovn_lflow *b)
> -{
> -    return (a->od == b->od
> -            && a->stage == b->stage
> -            && a->priority == b->priority
> -            && !strcmp(a->match, b->match)
> -            && !strcmp(a->actions, b->actions));
> -}
> -
> -static void
> -ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od,
> -               enum ovn_stage stage, uint16_t priority,
> -               char *match, char *actions, char *stage_hint,
> -               const char *where)
> -{
> -    lflow->od = od;
> -    lflow->stage = stage;
> -    lflow->priority = priority;
> -    lflow->match = match;
> -    lflow->actions = actions;
> -    lflow->stage_hint = stage_hint;
> -    lflow->where = where;
> -}
> -
> -/* Adds a row with the specified contents to the Logical_Flow table. */
> -static void
> -ovn_lflow_add_at(struct hmap *lflow_map, struct ovn_datapath *od,
> -                 enum ovn_stage stage, uint16_t priority,
> -                 const char *match, const char *actions,
> -                 const char *stage_hint, const char *where)
> -{
> -    ovs_assert(ovn_stage_to_datapath_type(stage) ==
> ovn_datapath_get_type(od));
> -
> -    struct ovn_lflow *lflow = xmalloc(sizeof *lflow);
> -    ovn_lflow_init(lflow, od, stage, priority,
> -                   xstrdup(match), xstrdup(actions),
> -                   nullable_xstrdup(stage_hint), where);
> -    hmap_insert(lflow_map, &lflow->hmap_node, ovn_lflow_hash(lflow));
> -}
> -
> -/* Adds a row with the specified contents to the Logical_Flow table. */
> -#define ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \
> -                                ACTIONS, STAGE_HINT) \
> -    ovn_lflow_add_at(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS, \
> -                     STAGE_HINT, OVS_SOURCE_LOCATOR)
> -
> -#define ovn_lflow_add(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS) \
> -    ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \
> -                            ACTIONS, NULL)
> -
> -static struct ovn_lflow *
> -ovn_lflow_find(struct hmap *lflows, struct ovn_datapath *od,
> -               enum ovn_stage stage, uint16_t priority,
> -               const char *match, const char *actions, uint32_t hash)
> -{
> -    struct ovn_lflow target;
> -    ovn_lflow_init(&target, od, stage, priority,
> -                   CONST_CAST(char *, match), CONST_CAST(char *, actions),
> -                   NULL, NULL);
> -
> -    struct ovn_lflow *lflow;
> -    HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) {
> -        if (ovn_lflow_equal(lflow, &target)) {
> -            return lflow;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static void
> -ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow)
> -{
> -    if (lflow) {
> -        hmap_remove(lflows, &lflow->hmap_node);
> -        free(lflow->match);
> -        free(lflow->actions);
> -        free(lflow->stage_hint);
> -        free(lflow);
> -    }
> -}
> -
> -/* Appends port security constraints on L2 address field 'eth_addr_field'
> - * (e.g. "eth.src" or "eth.dst") to 'match'.  'ps_addrs', with
> 'n_ps_addrs'
> - * elements, is the collection of port_security constraints from an
> - * OVN_NB Logical_Switch_Port row generated by extract_lsp_addresses(). */
> -static void
> -build_port_security_l2(const char *eth_addr_field,
> -                       struct lport_addresses *ps_addrs,
> -                       unsigned int n_ps_addrs,
> -                       struct ds *match)
> -{
> -    if (!n_ps_addrs) {
> -        return;
> -    }
> -
> -    ds_put_format(match, " && %s == {", eth_addr_field);
> -
> -    for (size_t i = 0; i < n_ps_addrs; i++) {
> -        ds_put_format(match, "%s ", ps_addrs[i].ea_s);
> -    }
> -    ds_chomp(match, ' ');
> -    ds_put_cstr(match, "}");
> -}
> -
> -static void
> -build_port_security_ipv6_nd_flow(
> -    struct ds *match, struct eth_addr ea, struct ipv6_netaddr *ipv6_addrs,
> -    int n_ipv6_addrs)
> -{
> -    ds_put_format(match, " && ip6 && nd && ((nd.sll == "ETH_ADDR_FMT" || "
> -                  "nd.sll == "ETH_ADDR_FMT") || ((nd.tll ==
> "ETH_ADDR_FMT" || "
> -                  "nd.tll == "ETH_ADDR_FMT")",
> ETH_ADDR_ARGS(eth_addr_zero),
> -                  ETH_ADDR_ARGS(ea), ETH_ADDR_ARGS(eth_addr_zero),
> -                  ETH_ADDR_ARGS(ea));
> -    if (!n_ipv6_addrs) {
> -        ds_put_cstr(match, "))");
> -        return;
> -    }
> -
> -    char ip6_str[INET6_ADDRSTRLEN + 1];
> -    struct in6_addr lla;
> -    in6_generate_lla(ea, &lla);
> -    memset(ip6_str, 0, sizeof(ip6_str));
> -    ipv6_string_mapped(ip6_str, &lla);
> -    ds_put_format(match, " && (nd.target == %s", ip6_str);
> -
> -    for(int i = 0; i < n_ipv6_addrs; i++) {
> -        memset(ip6_str, 0, sizeof(ip6_str));
> -        ipv6_string_mapped(ip6_str, &ipv6_addrs[i].addr);
> -        ds_put_format(match, " || nd.target == %s", ip6_str);
> -    }
> -
> -    ds_put_format(match, ")))");
> -}
> -
> -static void
> -build_port_security_ipv6_flow(
> -    enum ovn_pipeline pipeline, struct ds *match, struct eth_addr ea,
> -    struct ipv6_netaddr *ipv6_addrs, int n_ipv6_addrs)
> -{
> -    char ip6_str[INET6_ADDRSTRLEN + 1];
> -
> -    ds_put_format(match, " && %s == {",
> -                  pipeline == P_IN ? "ip6.src" : "ip6.dst");
> -
> -    /* Allow link-local address. */
> -    struct in6_addr lla;
> -    in6_generate_lla(ea, &lla);
> -    ipv6_string_mapped(ip6_str, &lla);
> -    ds_put_format(match, "%s, ", ip6_str);
> -
> -    /* Allow ip6.dst=ff00::/8 for multicast packets */
> -    if (pipeline == P_OUT) {
> -        ds_put_cstr(match, "ff00::/8, ");
> -    }
> -    for(int i = 0; i < n_ipv6_addrs; i++) {
> -        ipv6_string_mapped(ip6_str, &ipv6_addrs[i].addr);
> -        ds_put_format(match, "%s, ", ip6_str);
> -    }
> -    /* Replace ", " by "}". */
> -    ds_chomp(match, ' ');
> -    ds_chomp(match, ',');
> -    ds_put_cstr(match, "}");
> -}
> -
> -/**
> - * Build port security constraints on ARP and IPv6 ND fields
> - * and add logical flows to S_SWITCH_IN_PORT_SEC_ND stage.
> - *
> - * For each port security of the logical port, following
> - * logical flows are added
> - *   - If the port security has no IP (both IPv4 and IPv6) or
> - *     if it has IPv4 address(es)
> - *      - Priority 90 flow to allow ARP packets for known MAC addresses
> - *        in the eth.src and arp.spa fields. If the port security
> - *        has IPv4 addresses, allow known IPv4 addresses in the arp.tpa
> field.
> - *
> - *   - If the port security has no IP (both IPv4 and IPv6) or
> - *     if it has IPv6 address(es)
> - *     - Priority 90 flow to allow IPv6 ND packets for known MAC addresses
> - *       in the eth.src and nd.sll/nd.tll fields. If the port security
> - *       has IPv6 addresses, allow known IPv6 addresses in the nd.target
> field
> - *       for IPv6 Neighbor Advertisement packet.
> - *
> - *   - Priority 80 flow to drop ARP and IPv6 ND packets.
> - */
> -static void
> -build_port_security_nd(struct ovn_port *op, struct hmap *lflows)
> -{
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -
> -    for (size_t i = 0; i < op->n_ps_addrs; i++) {
> -        struct lport_addresses *ps = &op->ps_addrs[i];
> -
> -        bool no_ip = !(ps->n_ipv4_addrs || ps->n_ipv6_addrs);
> -
> -        ds_clear(&match);
> -        if (ps->n_ipv4_addrs || no_ip) {
> -            ds_put_format(&match,
> -                          "inport == %s && eth.src == %s && arp.sha ==
> %s",
> -                          op->json_key, ps->ea_s, ps->ea_s);
> -
> -            if (ps->n_ipv4_addrs) {
> -                ds_put_cstr(&match, " && arp.spa == {");
> -                for (size_t j = 0; j < ps->n_ipv4_addrs; j++) {
> -                    /* When the netmask is applied, if the host portion is
> -                     * non-zero, the host can only use the specified
> -                     * address in the arp.spa.  If zero, the host is
> allowed
> -                     * to use any address in the subnet. */
> -                    if (ps->ipv4_addrs[j].plen == 32
> -                        || ps->ipv4_addrs[j].addr &
> ~ps->ipv4_addrs[j].mask) {
> -                        ds_put_cstr(&match, ps->ipv4_addrs[j].addr_s);
> -                    } else {
> -                        ds_put_format(&match, "%s/%d",
> -                                      ps->ipv4_addrs[j].network_s,
> -                                      ps->ipv4_addrs[j].plen);
> -                    }
> -                    ds_put_cstr(&match, ", ");
> -                }
> -                ds_chomp(&match, ' ');
> -                ds_chomp(&match, ',');
> -                ds_put_cstr(&match, "}");
> -            }
> -            ovn_lflow_add(lflows, op->od, S_SWITCH_IN_PORT_SEC_ND, 90,
> -                          ds_cstr(&match), "next;");
> -        }
> -
> -        if (ps->n_ipv6_addrs || no_ip) {
> -            ds_clear(&match);
> -            ds_put_format(&match, "inport == %s && eth.src == %s",
> -                          op->json_key, ps->ea_s);
> -            build_port_security_ipv6_nd_flow(&match, ps->ea,
> ps->ipv6_addrs,
> -                                             ps->n_ipv6_addrs);
> -            ovn_lflow_add(lflows, op->od, S_SWITCH_IN_PORT_SEC_ND, 90,
> -                          ds_cstr(&match), "next;");
> -        }
> -    }
> -
> -    ds_clear(&match);
> -    ds_put_format(&match, "inport == %s && (arp || nd)", op->json_key);
> -    ovn_lflow_add(lflows, op->od, S_SWITCH_IN_PORT_SEC_ND, 80,
> -                  ds_cstr(&match), "drop;");
> -    ds_destroy(&match);
> -}
> -
> -/**
> - * Build port security constraints on IPv4 and IPv6 src and dst fields
> - * and add logical flows to S_SWITCH_(IN/OUT)_PORT_SEC_IP stage.
> - *
> - * For each port security of the logical port, following
> - * logical flows are added
> - *   - If the port security has IPv4 addresses,
> - *     - Priority 90 flow to allow IPv4 packets for known IPv4 addresses
> - *
> - *   - If the port security has IPv6 addresses,
> - *     - Priority 90 flow to allow IPv6 packets for known IPv6 addresses
> - *
> - *   - If the port security has IPv4 addresses or IPv6 addresses or both
> - *     - Priority 80 flow to drop all IPv4 and IPv6 traffic
> - */
> -static void
> -build_port_security_ip(enum ovn_pipeline pipeline, struct ovn_port *op,
> -                       struct hmap *lflows)
> -{
> -    char *port_direction;
> -    enum ovn_stage stage;
> -    if (pipeline == P_IN) {
> -        port_direction = "inport";
> -        stage = S_SWITCH_IN_PORT_SEC_IP;
> -    } else {
> -        port_direction = "outport";
> -        stage = S_SWITCH_OUT_PORT_SEC_IP;
> -    }
> -
> -    for (size_t i = 0; i < op->n_ps_addrs; i++) {
> -        struct lport_addresses *ps = &op->ps_addrs[i];
> -
> -        if (!(ps->n_ipv4_addrs || ps->n_ipv6_addrs)) {
> -            continue;
> -        }
> -
> -        if (ps->n_ipv4_addrs) {
> -            struct ds match = DS_EMPTY_INITIALIZER;
> -            if (pipeline == P_IN) {
> -                /* Permit use of the unspecified address for DHCP
> discovery */
> -                struct ds dhcp_match = DS_EMPTY_INITIALIZER;
> -                ds_put_format(&dhcp_match, "inport == %s"
> -                              " && eth.src == %s"
> -                              " && ip4.src == 0.0.0.0"
> -                              " && ip4.dst == 255.255.255.255"
> -                              " && udp.src == 68 && udp.dst == 67",
> -                              op->json_key, ps->ea_s);
> -                ovn_lflow_add(lflows, op->od, stage, 90,
> -                              ds_cstr(&dhcp_match), "next;");
> -                ds_destroy(&dhcp_match);
> -                ds_put_format(&match, "inport == %s && eth.src == %s"
> -                              " && ip4.src == {", op->json_key,
> -                              ps->ea_s);
> -            } else {
> -                ds_put_format(&match, "outport == %s && eth.dst == %s"
> -                              " && ip4.dst == {255.255.255.255,
> 224.0.0.0/4, ",
> -                              op->json_key, ps->ea_s);
> -            }
> -
> -            for (int j = 0; j < ps->n_ipv4_addrs; j++) {
> -                ovs_be32 mask = ps->ipv4_addrs[j].mask;
> -                /* When the netmask is applied, if the host portion is
> -                 * non-zero, the host can only use the specified
> -                 * address.  If zero, the host is allowed to use any
> -                 * address in the subnet.
> -                 */
> -                if (ps->ipv4_addrs[j].plen == 32
> -                    || ps->ipv4_addrs[j].addr & ~mask) {
> -                    ds_put_format(&match, "%s", ps->ipv4_addrs[j].addr_s);
> -                    if (pipeline == P_OUT && ps->ipv4_addrs[j].plen !=
> 32) {
> -                        /* Host is also allowed to receive packets to the
> -                         * broadcast address in the specified subnet. */
> -                        ds_put_format(&match, ", %s",
> -                                      ps->ipv4_addrs[j].bcast_s);
> -                    }
> -                } else {
> -                    /* host portion is zero */
> -                    ds_put_format(&match, "%s/%d",
> ps->ipv4_addrs[j].network_s,
> -                                  ps->ipv4_addrs[j].plen);
> -                }
> -                ds_put_cstr(&match, ", ");
> -            }
> -
> -            /* Replace ", " by "}". */
> -            ds_chomp(&match, ' ');
> -            ds_chomp(&match, ',');
> -            ds_put_cstr(&match, "}");
> -            ovn_lflow_add(lflows, op->od, stage, 90, ds_cstr(&match),
> "next;");
> -            ds_destroy(&match);
> -        }
> -
> -        if (ps->n_ipv6_addrs) {
> -            struct ds match = DS_EMPTY_INITIALIZER;
> -            if (pipeline == P_IN) {
> -                /* Permit use of unspecified address for duplicate address
> -                 * detection */
> -                struct ds dad_match = DS_EMPTY_INITIALIZER;
> -                ds_put_format(&dad_match, "inport == %s"
> -                              " && eth.src == %s"
> -                              " && ip6.src == ::"
> -                              " && ip6.dst == ff02::/16"
> -                              " && icmp6.type == {131, 135, 143}",
> op->json_key,
> -                              ps->ea_s);
> -                ovn_lflow_add(lflows, op->od, stage, 90,
> -                              ds_cstr(&dad_match), "next;");
> -                ds_destroy(&dad_match);
> -            }
> -            ds_put_format(&match, "%s == %s && %s == %s",
> -                          port_direction, op->json_key,
> -                          pipeline == P_IN ? "eth.src" : "eth.dst",
> ps->ea_s);
> -            build_port_security_ipv6_flow(pipeline, &match, ps->ea,
> -                                          ps->ipv6_addrs,
> ps->n_ipv6_addrs);
> -            ovn_lflow_add(lflows, op->od, stage, 90,
> -                          ds_cstr(&match), "next;");
> -            ds_destroy(&match);
> -        }
> -
> -        char *match = xasprintf("%s == %s && %s == %s && ip",
> -                                port_direction, op->json_key,
> -                                pipeline == P_IN ? "eth.src" : "eth.dst",
> -                                ps->ea_s);
> -        ovn_lflow_add(lflows, op->od, stage, 80, match, "drop;");
> -        free(match);
> -    }
> -
> -}
> -
> -static bool
> -lsp_is_enabled(const struct nbrec_logical_switch_port *lsp)
> -{
> -    return !lsp->enabled || *lsp->enabled;
> -}
> -
> -static bool
> -lsp_is_up(const struct nbrec_logical_switch_port *lsp)
> -{
> -    return !lsp->up || *lsp->up;
> -}
> -
> -static bool
> -lsp_is_external(const struct nbrec_logical_switch_port *nbsp)
> -{
> -    return !strcmp(nbsp->type, "external");
> -}
> -
> -static bool
> -build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip,
> -                    struct ds *options_action, struct ds *response_action,
> -                    struct ds *ipv4_addr_match)
> -{
> -    if (!op->nbsp->dhcpv4_options) {
> -        /* CMS has disabled native DHCPv4 for this lport. */
> -        return false;
> -    }
> -
> -    ovs_be32 host_ip, mask;
> -    char *error = ip_parse_masked(op->nbsp->dhcpv4_options->cidr,
> &host_ip,
> -                                  &mask);
> -    if (error || ((offer_ip ^ host_ip) & mask)) {
> -       /* Either
> -        *  - cidr defined is invalid or
> -        *  - the offer ip of the logical port doesn't belong to the cidr
> -        *    defined in the DHCPv4 options.
> -        *  */
> -        free(error);
> -        return false;
> -    }
> -
> -    const char *server_ip = smap_get(
> -        &op->nbsp->dhcpv4_options->options, "server_id");
> -    const char *server_mac = smap_get(
> -        &op->nbsp->dhcpv4_options->options, "server_mac");
> -    const char *lease_time = smap_get(
> -        &op->nbsp->dhcpv4_options->options, "lease_time");
> -
> -    if (!(server_ip && server_mac && lease_time)) {
> -        /* "server_id", "server_mac" and "lease_time" should be
> -         * present in the dhcp_options. */
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -        VLOG_WARN_RL(&rl, "Required DHCPv4 options not defined for lport
> - %s",
> -                     op->json_key);
> -        return false;
> -    }
> -
> -    struct smap dhcpv4_options = SMAP_INITIALIZER(&dhcpv4_options);
> -    smap_clone(&dhcpv4_options, &op->nbsp->dhcpv4_options->options);
> -
> -    /* server_mac is not DHCPv4 option, delete it from the smap. */
> -    smap_remove(&dhcpv4_options, "server_mac");
> -    char *netmask = xasprintf(IP_FMT, IP_ARGS(mask));
> -    smap_add(&dhcpv4_options, "netmask", netmask);
> -    free(netmask);
> -
> -    ds_put_format(options_action,
> -                  REGBIT_DHCP_OPTS_RESULT" = put_dhcp_opts(offerip = "
> -                  IP_FMT", ", IP_ARGS(offer_ip));
> -
> -    /* We're not using SMAP_FOR_EACH because we want a consistent order
> of the
> -     * options on different architectures (big or little endian, SSE4.2)
> */
> -    const struct smap_node **sorted_opts = smap_sort(&dhcpv4_options);
> -    for (size_t i = 0; i < smap_count(&dhcpv4_options); i++) {
> -        const struct smap_node *node = sorted_opts[i];
> -        ds_put_format(options_action, "%s = %s, ", node->key,
> node->value);
> -    }
> -    free(sorted_opts);
> -
> -    ds_chomp(options_action, ' ');
> -    ds_chomp(options_action, ',');
> -    ds_put_cstr(options_action, "); next;");
> -
> -    ds_put_format(response_action, "eth.dst = eth.src; eth.src = %s; "
> -                  "ip4.dst = "IP_FMT"; ip4.src = %s; udp.src = 67; "
> -                  "udp.dst = 68; outport = inport; flags.loopback = 1; "
> -                  "output;",
> -                  server_mac, IP_ARGS(offer_ip), server_ip);
> -
> -    ds_put_format(ipv4_addr_match,
> -                  "ip4.src == "IP_FMT" && ip4.dst == {%s,
> 255.255.255.255}",
> -                  IP_ARGS(offer_ip), server_ip);
> -    smap_destroy(&dhcpv4_options);
> -    return true;
> -}
> -
> -static bool
> -build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip,
> -                    struct ds *options_action, struct ds *response_action)
> -{
> -    if (!op->nbsp->dhcpv6_options) {
> -        /* CMS has disabled native DHCPv6 for this lport. */
> -        return false;
> -    }
> -
> -    struct in6_addr host_ip, mask;
> -
> -    char *error = ipv6_parse_masked(op->nbsp->dhcpv6_options->cidr,
> &host_ip,
> -                                        &mask);
> -    if (error) {
> -        free(error);
> -        return false;
> -    }
> -    struct in6_addr ip6_mask = ipv6_addr_bitxor(offer_ip, &host_ip);
> -    ip6_mask = ipv6_addr_bitand(&ip6_mask, &mask);
> -    if (!ipv6_mask_is_any(&ip6_mask)) {
> -        /* offer_ip doesn't belongs to the cidr defined in lport's DHCPv6
> -         * options.*/
> -        return false;
> -    }
> -
> -    const struct smap *options_map = &op->nbsp->dhcpv6_options->options;
> -    /* "server_id" should be the MAC address. */
> -    const char *server_mac = smap_get(options_map, "server_id");
> -    struct eth_addr ea;
> -    if (!server_mac || !eth_addr_from_string(server_mac, &ea)) {
> -        /* "server_id" should be present in the dhcpv6_options. */
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "server_id not present in the DHCPv6 options"
> -                          " for lport %s", op->json_key);
> -        return false;
> -    }
> -
> -    /* Get the link local IP of the DHCPv6 server from the server MAC. */
> -    struct in6_addr lla;
> -    in6_generate_lla(ea, &lla);
> -
> -    char server_ip[INET6_ADDRSTRLEN + 1];
> -    ipv6_string_mapped(server_ip, &lla);
> -
> -    char ia_addr[INET6_ADDRSTRLEN + 1];
> -    ipv6_string_mapped(ia_addr, offer_ip);
> -
> -    ds_put_format(options_action,
> -                  REGBIT_DHCP_OPTS_RESULT" = put_dhcpv6_opts(");
> -
> -    /* Check whether the dhcpv6 options should be configured as stateful.
> -     * Only reply with ia_addr option for dhcpv6 stateful address mode. */
> -    if (!smap_get_bool(options_map, "dhcpv6_stateless", false)) {
> -        ipv6_string_mapped(ia_addr, offer_ip);
> -        ds_put_format(options_action, "ia_addr = %s, ", ia_addr);
> -    }
> -
> -    /* We're not using SMAP_FOR_EACH because we want a consistent order
> of the
> -     * options on different architectures (big or little endian, SSE4.2)
> */
> -    const struct smap_node **sorted_opts = smap_sort(options_map);
> -    for (size_t i = 0; i < smap_count(options_map); i++) {
> -        const struct smap_node *node = sorted_opts[i];
> -        if (strcmp(node->key, "dhcpv6_stateless")) {
> -            ds_put_format(options_action, "%s = %s, ", node->key,
> node->value);
> -        }
> -    }
> -    free(sorted_opts);
> -
> -    ds_chomp(options_action, ' ');
> -    ds_chomp(options_action, ',');
> -    ds_put_cstr(options_action, "); next;");
> -
> -    ds_put_format(response_action, "eth.dst = eth.src; eth.src = %s; "
> -                  "ip6.dst = ip6.src; ip6.src = %s; udp.src = 547; "
> -                  "udp.dst = 546; outport = inport; flags.loopback = 1; "
> -                  "output;",
> -                  server_mac, server_ip);
> -
> -    return true;
> -}
> -
> -struct ovn_port_group_ls {
> -    struct hmap_node key_node;  /* Index on 'key'. */
> -    struct uuid key;            /* nb_ls->header_.uuid. */
> -    const struct nbrec_logical_switch *nb_ls;
> -};
> -
> -struct ovn_port_group {
> -    struct hmap_node key_node;  /* Index on 'key'. */
> -    struct uuid key;            /* nb_pg->header_.uuid. */
> -    const struct nbrec_port_group *nb_pg;
> -    struct hmap nb_lswitches;   /* NB lswitches related to the port group
> */
> -};
> -
> -static void
> -ovn_port_group_ls_add(struct ovn_port_group *pg,
> -                      const struct nbrec_logical_switch *nb_ls)
> -{
> -    struct ovn_port_group_ls *pg_ls = xzalloc(sizeof *pg_ls);
> -    pg_ls->key = nb_ls->header_.uuid;
> -    pg_ls->nb_ls = nb_ls;
> -    hmap_insert(&pg->nb_lswitches, &pg_ls->key_node,
> uuid_hash(&pg_ls->key));
> -}
> -
> -static struct ovn_port_group_ls *
> -ovn_port_group_ls_find(struct ovn_port_group *pg, const struct uuid
> *ls_uuid)
> -{
> -    struct ovn_port_group_ls *pg_ls;
> -
> -    HMAP_FOR_EACH_WITH_HASH (pg_ls, key_node, uuid_hash(ls_uuid),
> -                             &pg->nb_lswitches) {
> -        if (uuid_equals(ls_uuid, &pg_ls->key)) {
> -            return pg_ls;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -struct ovn_ls_port_group {
> -    struct hmap_node key_node;  /* Index on 'key'. */
> -    struct uuid key;            /* nb_pg->header_.uuid. */
> -    const struct nbrec_port_group *nb_pg;
> -};
> -
> -static void
> -ovn_ls_port_group_add(struct hmap *nb_pgs,
> -                      const struct nbrec_port_group *nb_pg)
> -{
> -    struct ovn_ls_port_group *ls_pg = xzalloc(sizeof *ls_pg);
> -    ls_pg->key = nb_pg->header_.uuid;
> -    ls_pg->nb_pg = nb_pg;
> -    hmap_insert(nb_pgs, &ls_pg->key_node, uuid_hash(&ls_pg->key));
> -}
> -
> -static void
> -ovn_ls_port_group_destroy(struct hmap *nb_pgs)
> -{
> -    struct ovn_ls_port_group *ls_pg;
> -    HMAP_FOR_EACH_POP (ls_pg, key_node, nb_pgs) {
> -        free(ls_pg);
> -    }
> -    hmap_destroy(nb_pgs);
> -}
> -
> -static bool
> -has_stateful_acl(struct ovn_datapath *od)
> -{
> -    for (size_t i = 0; i < od->nbs->n_acls; i++) {
> -        struct nbrec_acl *acl = od->nbs->acls[i];
> -        if (!strcmp(acl->action, "allow-related")) {
> -            return true;
> -        }
> -    }
> -
> -    struct ovn_ls_port_group *ls_pg;
> -    HMAP_FOR_EACH (ls_pg, key_node, &od->nb_pgs) {
> -        for (size_t i = 0; i < ls_pg->nb_pg->n_acls; i++) {
> -            struct nbrec_acl *acl = ls_pg->nb_pg->acls[i];
> -            if (!strcmp(acl->action, "allow-related")) {
> -                return true;
> -            }
> -        }
> -    }
> -
> -    return false;
> -}
> -
> -static void
> -build_pre_acls(struct ovn_datapath *od, struct hmap *lflows)
> -{
> -    bool has_stateful = has_stateful_acl(od);
> -
> -    /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are
> -     * allowed by default. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;");
> -
> -    /* If there are any stateful ACL rules in this datapath, we must
> -     * send all IP packets through the conntrack action, which handles
> -     * defragmentation, in order to match L4 headers. */
> -    if (has_stateful) {
> -        for (size_t i = 0; i < od->n_router_ports; i++) {
> -            struct ovn_port *op = od->router_ports[i];
> -            /* Can't use ct() for router ports. Consider the
> -             * following configuration: lp1(10.0.0.2) on
> -             * hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a
> -             * ping from lp1 to lp2, First, the response will go
> -             * through ct() with a zone for lp2 in the ls2 ingress
> -             * pipeline on hostB.  That ct zone knows about this
> -             * connection. Next, it goes through ct() with the zone
> -             * for the router port in the egress pipeline of ls2 on
> -             * hostB.  This zone does not know about the connection,
> -             * as the icmp request went through the logical router
> -             * on hostA, not hostB. This would only work with
> -             * distributed conntrack state across all chassis. */
> -            struct ds match_in = DS_EMPTY_INITIALIZER;
> -            struct ds match_out = DS_EMPTY_INITIALIZER;
> -
> -            ds_put_format(&match_in, "ip && inport == %s", op->json_key);
> -            ds_put_format(&match_out, "ip && outport == %s",
> op->json_key);
> -            ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110,
> -                          ds_cstr(&match_in), "next;");
> -            ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110,
> -                          ds_cstr(&match_out), "next;");
> -
> -            ds_destroy(&match_in);
> -            ds_destroy(&match_out);
> -        }
> -        if (od->localnet_port) {
> -            struct ds match_in = DS_EMPTY_INITIALIZER;
> -            struct ds match_out = DS_EMPTY_INITIALIZER;
> -
> -            ds_put_format(&match_in, "ip && inport == %s",
> -                          od->localnet_port->json_key);
> -            ds_put_format(&match_out, "ip && outport == %s",
> -                          od->localnet_port->json_key);
> -            ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110,
> -                          ds_cstr(&match_in), "next;");
> -            ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110,
> -                          ds_cstr(&match_out), "next;");
> -
> -            ds_destroy(&match_in);
> -            ds_destroy(&match_out);
> -        }
> -
> -        /* Ingress and Egress Pre-ACL Table (Priority 110).
> -         *
> -         * Not to do conntrack on ND and ICMP destination
> -         * unreachable packets. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110,
> -                      "nd || nd_rs || nd_ra || icmp4.type == 3 || "
> -                      "icmp6.type == 1 || (tcp && tcp.flags == 4)",
> -                      "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110,
> -                      "nd || nd_rs || nd_ra || icmp4.type == 3 || "
> -                      "icmp6.type == 1 || (tcp && tcp.flags == 4)",
> -                      "next;");
> -
> -        /* Ingress and Egress Pre-ACL Table (Priority 100).
> -         *
> -         * Regardless of whether the ACL is "from-lport" or "to-lport",
> -         * we need rules in both the ingress and egress table, because
> -         * the return traffic needs to be followed.
> -         *
> -         * 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table
> send
> -         * it to conntrack for tracking and defragmentation. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip",
> -                      REGBIT_CONNTRACK_DEFRAG" = 1; next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip",
> -                      REGBIT_CONNTRACK_DEFRAG" = 1; next;");
> -    }
> -}
> -
> -/* For a 'key' of the form "IP:port" or just "IP", sets 'port' and
> - * 'ip_address'.  The caller must free() the memory allocated for
> - * 'ip_address'. */
> -static void
> -ip_address_and_port_from_lb_key(const char *key, char **ip_address,
> -                                uint16_t *port, int *addr_family)
> -{
> -    struct sockaddr_storage ss;
> -    if (!inet_parse_active(key, 0, &ss, false)) {
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "bad ip address or port for load balancer key
> %s",
> -                     key);
> -        return;
> -    }
> -
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    ss_format_address_nobracks(&ss, &s);
> -    *ip_address = ds_steal_cstr(&s);
> -
> -    *port = ss_get_port(&ss);
> -
> -    *addr_family = ss.ss_family;
> -}
> -
> -/*
> - * Returns true if logical switch is configured with DNS records, false
> - * otherwise.
> - */
> -static bool
> -ls_has_dns_records(const struct nbrec_logical_switch *nbs)
> -{
> -    for (size_t i = 0; i < nbs->n_dns_records; i++) {
> -        if (!smap_is_empty(&nbs->dns_records[i]->records)) {
> -            return true;
> -        }
> -    }
> -
> -    return false;
> -}
> -
> -static void
> -build_pre_lb(struct ovn_datapath *od, struct hmap *lflows)
> -{
> -    /* Do not send ND packets to conntrack */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110,
> -                  "nd || nd_rs || nd_ra", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110,
> -                  "nd || nd_rs || nd_ra", "next;");
> -
> -    /* Allow all packets to go to next tables by default. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;");
> -
> -    struct sset all_ips = SSET_INITIALIZER(&all_ips);
> -    bool vip_configured = false;
> -    int addr_family = AF_INET;
> -    for (int i = 0; i < od->nbs->n_load_balancer; i++) {
> -        struct nbrec_load_balancer *lb = od->nbs->load_balancer[i];
> -        struct smap *vips = &lb->vips;
> -        struct smap_node *node;
> -
> -        SMAP_FOR_EACH (node, vips) {
> -            vip_configured = true;
> -
> -            /* node->key contains IP:port or just IP. */
> -            char *ip_address = NULL;
> -            uint16_t port;
> -            ip_address_and_port_from_lb_key(node->key, &ip_address, &port,
> -                                            &addr_family);
> -            if (!ip_address) {
> -                continue;
> -            }
> -
> -            if (!sset_contains(&all_ips, ip_address)) {
> -                sset_add(&all_ips, ip_address);
> -            }
> -
> -            if (controller_event_en && !node->value[0]) {
> -                struct ds match = DS_EMPTY_INITIALIZER;
> -                char *action;
> -
> -                if (addr_family == AF_INET) {
> -                    ds_put_format(&match, "ip4.dst == %s && %s",
> -                                  ip_address, lb->protocol);
> -                } else {
> -                    ds_put_format(&match, "ip6.dst == %s && %s",
> -                                  ip_address, lb->protocol);
> -                }
> -                if (port) {
> -                    ds_put_format(&match, " && %s.dst == %u",
> lb->protocol,
> -                                  port);
> -                }
> -                action = xasprintf("trigger_event(event = \"%s\", "
> -                               "vip = \"%s\", protocol = \"%s\", "
> -                               "load_balancer = \"" UUID_FMT "\");",
> -
>  event_to_string(OVN_EVENT_EMPTY_LB_BACKENDS),
> -                               node->key, lb->protocol,
> -                               UUID_ARGS(&lb->header_.uuid));
> -                ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 120,
> -                              ds_cstr(&match), action);
> -                ds_destroy(&match);
> -                free(action);
> -                continue;
> -            }
> -
> -            free(ip_address);
> -
> -            /* Ignore L4 port information in the key because fragmented
> packets
> -             * may not have L4 information.  The pre-stateful table will
> send
> -             * the packet through ct() action to de-fragment. In stateful
> -             * table, we will eventually look at L4 information. */
> -        }
> -    }
> -
> -    /* 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table send
> -     * packet to conntrack for defragmentation. */
> -    const char *ip_address;
> -    SSET_FOR_EACH(ip_address, &all_ips) {
> -        char *match;
> -
> -        if (addr_family == AF_INET) {
> -            match = xasprintf("ip && ip4.dst == %s", ip_address);
> -        } else {
> -            match = xasprintf("ip && ip6.dst == %s", ip_address);
> -        }
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB,
> -                      100, match, REGBIT_CONNTRACK_DEFRAG" = 1; next;");
> -        free(match);
> -    }
> -
> -    sset_destroy(&all_ips);
> -
> -    if (vip_configured) {
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB,
> -                      100, "ip", REGBIT_CONNTRACK_DEFRAG" = 1; next;");
> -    }
> -}
> -
> -static void
> -build_pre_stateful(struct ovn_datapath *od, struct hmap *lflows)
> -{
> -    /* Ingress and Egress pre-stateful Table (Priority 0): Packets are
> -     * allowed by default. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;");
> -
> -    /* If REGBIT_CONNTRACK_DEFRAG is set as 1, then the packets should be
> -     * sent to conntrack for tracking and defragmentation. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;");
> -}
> -
> -static void
> -build_acl_log(struct ds *actions, const struct nbrec_acl *acl)
> -{
> -    if (!acl->log) {
> -        return;
> -    }
> -
> -    ds_put_cstr(actions, "log(");
> -
> -    if (acl->name) {
> -        ds_put_format(actions, "name=\"%s\", ", acl->name);
> -    }
> -
> -    /* If a severity level isn't specified, default to "info". */
> -    if (acl->severity) {
> -        ds_put_format(actions, "severity=%s, ", acl->severity);
> -    } else {
> -        ds_put_format(actions, "severity=info, ");
> -    }
> -
> -    if (!strcmp(acl->action, "drop")) {
> -        ds_put_cstr(actions, "verdict=drop, ");
> -    } else if (!strcmp(acl->action, "reject")) {
> -        ds_put_cstr(actions, "verdict=reject, ");
> -    } else if (!strcmp(acl->action, "allow")
> -        || !strcmp(acl->action, "allow-related")) {
> -        ds_put_cstr(actions, "verdict=allow, ");
> -    }
> -
> -    if (acl->meter) {
> -        ds_put_format(actions, "meter=\"%s\", ", acl->meter);
> -    }
> -
> -    ds_chomp(actions, ' ');
> -    ds_chomp(actions, ',');
> -    ds_put_cstr(actions, "); ");
> -}
> -
> -static void
> -build_reject_acl_rules(struct ovn_datapath *od, struct hmap *lflows,
> -                       enum ovn_stage stage, struct nbrec_acl *acl,
> -                       struct ds *extra_match, struct ds *extra_actions)
> -{
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -    bool ingress = (stage == S_SWITCH_IN_ACL);
> -
> -    /* TCP */
> -    build_acl_log(&actions, acl);
> -    if (extra_match->length > 0) {
> -        ds_put_format(&match, "(%s) && ", extra_match->string);
> -    }
> -    ds_put_format(&match, "ip4 && tcp && (%s)", acl->match);
> -    ds_put_format(&actions, "reg0 = 0; "
> -                  "eth.dst <-> eth.src; ip4.dst <-> ip4.src; "
> -                  "tcp_reset { outport <-> inport; %s };",
> -                  ingress ? "output;" :
> "next(pipeline=ingress,table=0);");
> -    ovn_lflow_add(lflows, od, stage, acl->priority + OVN_ACL_PRI_OFFSET +
> 10,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -    ds_clear(&match);
> -    ds_clear(&actions);
> -    build_acl_log(&actions, acl);
> -    if (extra_match->length > 0) {
> -        ds_put_format(&match, "(%s) && ", extra_match->string);
> -    }
> -    ds_put_format(&match, "ip6 && tcp && (%s)", acl->match);
> -    ds_put_format(&actions, "reg0 = 0; "
> -                  "eth.dst <-> eth.src; ip6.dst <-> ip6.src; "
> -                  "tcp_reset { outport <-> inport; %s };",
> -                  ingress ? "output;" :
> "next(pipeline=ingress,table=0);");
> -    ovn_lflow_add(lflows, od, stage, acl->priority + OVN_ACL_PRI_OFFSET +
> 10,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -
> -    /* IP traffic */
> -    ds_clear(&match);
> -    ds_clear(&actions);
> -    build_acl_log(&actions, acl);
> -    if (extra_match->length > 0) {
> -        ds_put_format(&match, "(%s) && ", extra_match->string);
> -    }
> -    ds_put_format(&match, "ip4 && (%s)", acl->match);
> -    if (extra_actions->length > 0) {
> -        ds_put_format(&actions, "%s ", extra_actions->string);
> -    }
> -    ds_put_format(&actions, "reg0 = 0; "
> -                  "eth.dst <-> eth.src; ip4.dst <-> ip4.src; "
> -                  "icmp4 { outport <-> inport; %s };",
> -                  ingress ? "output;" :
> "next(pipeline=ingress,table=0);");
> -    ovn_lflow_add(lflows, od, stage, acl->priority + OVN_ACL_PRI_OFFSET,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -    ds_clear(&match);
> -    ds_clear(&actions);
> -    build_acl_log(&actions, acl);
> -    if (extra_match->length > 0) {
> -        ds_put_format(&match, "(%s) && ", extra_match->string);
> -    }
> -    ds_put_format(&match, "ip6 && (%s)", acl->match);
> -    if (extra_actions->length > 0) {
> -        ds_put_format(&actions, "%s ", extra_actions->string);
> -    }
> -    ds_put_format(&actions, "reg0 = 0; icmp6 { "
> -                  "eth.dst <-> eth.src; ip6.dst <-> ip6.src; "
> -                  "outport <-> inport; %s };",
> -                  ingress ? "output;" :
> "next(pipeline=ingress,table=0);");
> -    ovn_lflow_add(lflows, od, stage, acl->priority + OVN_ACL_PRI_OFFSET,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -
> -    ds_destroy(&match);
> -    ds_destroy(&actions);
> -}
> -
> -static void
> -consider_acl(struct hmap *lflows, struct ovn_datapath *od,
> -             struct nbrec_acl *acl, bool has_stateful)
> -{
> -    bool ingress = !strcmp(acl->direction, "from-lport") ? true :false;
> -    enum ovn_stage stage = ingress ? S_SWITCH_IN_ACL : S_SWITCH_OUT_ACL;
> -
> -    char *stage_hint = xasprintf("%08x", acl->header_.uuid.parts[0]);
> -    if (!strcmp(acl->action, "allow")
> -        || !strcmp(acl->action, "allow-related")) {
> -        /* If there are any stateful flows, we must even commit "allow"
> -         * actions.  This is because, while the initiater's
> -         * direction may not have any stateful rules, the server's
> -         * may and then its return traffic would not have an
> -         * associated conntrack entry and would return "+invalid". */
> -        if (!has_stateful) {
> -            struct ds actions = DS_EMPTY_INITIALIZER;
> -            build_acl_log(&actions, acl);
> -            ds_put_cstr(&actions, "next;");
> -            ovn_lflow_add_with_hint(lflows, od, stage,
> -                                    acl->priority + OVN_ACL_PRI_OFFSET,
> -                                    acl->match, ds_cstr(&actions),
> -                                    stage_hint);
> -            ds_destroy(&actions);
> -        } else {
> -            struct ds match = DS_EMPTY_INITIALIZER;
> -            struct ds actions = DS_EMPTY_INITIALIZER;
> -
> -            /* Commit the connection tracking entry if it's a new
> -             * connection that matches this ACL.  After this commit,
> -             * the reply traffic is allowed by a flow we create at
> -             * priority 65535, defined earlier.
> -             *
> -             * It's also possible that a known connection was marked for
> -             * deletion after a policy was deleted, but the policy was
> -             * re-added while that connection is still known.  We catch
> -             * that case here and un-set ct_label.blocked (which will be
> done
> -             * by ct_commit in the "stateful" stage) to indicate that the
> -             * connection should be allowed to resume.
> -             */
> -            ds_put_format(&match, "((ct.new && !ct.est)"
> -                                  " || (!ct.new && ct.est && !ct.rpl "
> -                                       "&& ct_label.blocked == 1)) "
> -                                  "&& (%s)", acl->match);
> -            ds_put_cstr(&actions, REGBIT_CONNTRACK_COMMIT" = 1; ");
> -            build_acl_log(&actions, acl);
> -            ds_put_cstr(&actions, "next;");
> -            ovn_lflow_add_with_hint(lflows, od, stage,
> -                                    acl->priority + OVN_ACL_PRI_OFFSET,
> -                                    ds_cstr(&match),
> -                                    ds_cstr(&actions),
> -                                    stage_hint);
> -
> -            /* Match on traffic in the request direction for an
> established
> -             * connection tracking entry that has not been marked for
> -             * deletion.  There is no need to commit here, so we can just
> -             * proceed to the next table. We use this to ensure that this
> -             * connection is still allowed by the currently defined
> -             * policy. */
> -            ds_clear(&match);
> -            ds_clear(&actions);
> -            ds_put_format(&match,
> -                          "!ct.new && ct.est && !ct.rpl"
> -                          " && ct_label.blocked == 0 && (%s)",
> -                          acl->match);
> -
> -            build_acl_log(&actions, acl);
> -            ds_put_cstr(&actions, "next;");
> -            ovn_lflow_add_with_hint(lflows, od, stage,
> -                                    acl->priority + OVN_ACL_PRI_OFFSET,
> -                                    ds_cstr(&match), ds_cstr(&actions),
> -                                    stage_hint);
> -
> -            ds_destroy(&match);
> -            ds_destroy(&actions);
> -        }
> -    } else if (!strcmp(acl->action, "drop")
> -               || !strcmp(acl->action, "reject")) {
> -        struct ds match = DS_EMPTY_INITIALIZER;
> -        struct ds actions = DS_EMPTY_INITIALIZER;
> -
> -        /* The implementation of "drop" differs if stateful ACLs are in
> -         * use for this datapath.  In that case, the actions differ
> -         * depending on whether the connection was previously committed
> -         * to the connection tracker with ct_commit. */
> -        if (has_stateful) {
> -            /* If the packet is not part of an established connection,
> then
> -             * we can simply reject/drop it. */
> -            ds_put_cstr(&match,
> -                        "(!ct.est || (ct.est && ct_label.blocked == 1))");
> -            if (!strcmp(acl->action, "reject")) {
> -                build_reject_acl_rules(od, lflows, stage, acl, &match,
> -                                       &actions);
> -            } else {
> -                ds_put_format(&match, " && (%s)", acl->match);
> -                build_acl_log(&actions, acl);
> -                ds_put_cstr(&actions, "/* drop */");
> -                ovn_lflow_add(lflows, od, stage,
> -                              acl->priority + OVN_ACL_PRI_OFFSET,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            }
> -            /* For an existing connection without ct_label set, we've
> -             * encountered a policy change. ACLs previously allowed
> -             * this connection and we committed the connection tracking
> -             * entry.  Current policy says that we should drop this
> -             * connection.  First, we set bit 0 of ct_label to indicate
> -             * that this connection is set for deletion.  By not
> -             * specifying "next;", we implicitly drop the packet after
> -             * updating conntrack state.  We would normally defer
> -             * ct_commit() to the "stateful" stage, but since we're
> -             * rejecting/dropping the packet, we go ahead and do it here.
> -             */
> -            ds_clear(&match);
> -            ds_clear(&actions);
> -            ds_put_cstr(&match, "ct.est && ct_label.blocked == 0");
> -            ds_put_cstr(&actions, "ct_commit(ct_label=1/1); ");
> -            if (!strcmp(acl->action, "reject")) {
> -                build_reject_acl_rules(od, lflows, stage, acl, &match,
> -                                       &actions);
> -            } else {
> -                ds_put_format(&match, " && (%s)", acl->match);
> -                build_acl_log(&actions, acl);
> -                ds_put_cstr(&actions, "/* drop */");
> -                ovn_lflow_add(lflows, od, stage,
> -                              acl->priority + OVN_ACL_PRI_OFFSET,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            }
> -        } else {
> -            /* There are no stateful ACLs in use on this datapath,
> -             * so a "reject/drop" ACL is simply the "reject/drop"
> -             * logical flow action in all cases. */
> -            if (!strcmp(acl->action, "reject")) {
> -                build_reject_acl_rules(od, lflows, stage, acl, &match,
> -                                       &actions);
> -            } else {
> -                build_acl_log(&actions, acl);
> -                ds_put_cstr(&actions, "/* drop */");
> -                ovn_lflow_add(lflows, od, stage,
> -                              acl->priority + OVN_ACL_PRI_OFFSET,
> -                              acl->match, ds_cstr(&actions));
> -            }
> -        }
> -        ds_destroy(&match);
> -        ds_destroy(&actions);
> -    }
> -    free(stage_hint);
> -}
> -
> -static struct ovn_port_group *
> -ovn_port_group_create(struct hmap *pgs,
> -                      const struct nbrec_port_group *nb_pg)
> -{
> -    struct ovn_port_group *pg = xzalloc(sizeof *pg);
> -    pg->key = nb_pg->header_.uuid;
> -    pg->nb_pg = nb_pg;
> -    hmap_init(&pg->nb_lswitches);
> -    hmap_insert(pgs, &pg->key_node, uuid_hash(&pg->key));
> -    return pg;
> -}
> -
> -static void
> -ovn_port_group_destroy(struct hmap *pgs, struct ovn_port_group *pg)
> -{
> -    if (pg) {
> -        hmap_remove(pgs, &pg->key_node);
> -        struct ovn_port_group_ls *ls;
> -        HMAP_FOR_EACH_POP (ls, key_node, &pg->nb_lswitches) {
> -            free(ls);
> -        }
> -        hmap_destroy(&pg->nb_lswitches);
> -        free(pg);
> -    }
> -}
> -
> -static void
> -build_port_group_lswitches(struct northd_context *ctx, struct hmap *pgs,
> -                           struct hmap *ports)
> -{
> -    hmap_init(pgs);
> -
> -    const struct nbrec_port_group *nb_pg;
> -    NBREC_PORT_GROUP_FOR_EACH (nb_pg, ctx->ovnnb_idl) {
> -        struct ovn_port_group *pg = ovn_port_group_create(pgs, nb_pg);
> -        for (size_t i = 0; i < nb_pg->n_ports; i++) {
> -            struct ovn_port *op = ovn_port_find(ports,
> nb_pg->ports[i]->name);
> -            if (!op) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_ERR_RL(&rl, "lport %s in port group %s not found.",
> -                            nb_pg->ports[i]->name,
> -                            nb_pg->name);
> -                continue;
> -            }
> -
> -            if (!op->od->nbs) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -                VLOG_WARN_RL(&rl, "lport %s in port group %s has no
> lswitch.",
> -                             nb_pg->ports[i]->name,
> -                             nb_pg->name);
> -                continue;
> -            }
> -
> -            struct ovn_port_group_ls *pg_ls =
> -                ovn_port_group_ls_find(pg, &op->od->nbs->header_.uuid);
> -            if (!pg_ls) {
> -                ovn_port_group_ls_add(pg, op->od->nbs);
> -                ovn_ls_port_group_add(&op->od->nb_pgs, nb_pg);
> -            }
> -        }
> -    }
> -}
> -
> -static void
> -build_acls(struct ovn_datapath *od, struct hmap *lflows,
> -           struct hmap *port_groups)
> -{
> -    bool has_stateful = has_stateful_acl(od);
> -
> -    /* Ingress and Egress ACL Table (Priority 0): Packets are allowed by
> -     * default.  A related rule at priority 1 is added below if there
> -     * are any stateful ACLs in this datapath. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, 0, "1", "next;");
> -
> -    if (has_stateful) {
> -        /* Ingress and Egress ACL Table (Priority 1).
> -         *
> -         * By default, traffic is allowed.  This is partially handled by
> -         * the Priority 0 ACL flows added earlier, but we also need to
> -         * commit IP flows.  This is because, while the initiater's
> -         * direction may not have any stateful rules, the server's may
> -         * and then its return traffic would not have an associated
> -         * conntrack entry and would return "+invalid".
> -         *
> -         * We use "ct_commit" for a connection that is not already known
> -         * by the connection tracker.  Once a connection is committed,
> -         * subsequent packets will hit the flow at priority 0 that just
> -         * uses "next;"
> -         *
> -         * We also check for established connections that have
> ct_label.blocked
> -         * set on them.  That's a connection that was disallowed, but is
> -         * now allowed by policy again since it hit this default-allow
> flow.
> -         * We need to set ct_label.blocked=0 to let the connection
> continue,
> -         * which will be done by ct_commit() in the "stateful" stage.
> -         * Subsequent packets will hit the flow at priority 0 that just
> -         * uses "next;". */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, 1,
> -                      "ip && (!ct.est || (ct.est && ct_label.blocked ==
> 1))",
> -                       REGBIT_CONNTRACK_COMMIT" = 1; next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, 1,
> -                      "ip && (!ct.est || (ct.est && ct_label.blocked ==
> 1))",
> -                       REGBIT_CONNTRACK_COMMIT" = 1; next;");
> -
> -        /* Ingress and Egress ACL Table (Priority 65535).
> -         *
> -         * Always drop traffic that's in an invalid state.  Also drop
> -         * reply direction packets for connections that have been marked
> -         * for deletion (bit 0 of ct_label is set).
> -         *
> -         * This is enforced at a higher priority than ACLs can be
> defined. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, UINT16_MAX,
> -                      "ct.inv || (ct.est && ct.rpl && ct_label.blocked ==
> 1)",
> -                      "drop;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, UINT16_MAX,
> -                      "ct.inv || (ct.est && ct.rpl && ct_label.blocked ==
> 1)",
> -                      "drop;");
> -
> -        /* Ingress and Egress ACL Table (Priority 65535).
> -         *
> -         * Allow reply traffic that is part of an established
> -         * conntrack entry that has not been marked for deletion
> -         * (bit 0 of ct_label).  We only match traffic in the
> -         * reply direction because we want traffic in the request
> -         * direction to hit the currently defined policy from ACLs.
> -         *
> -         * This is enforced at a higher priority than ACLs can be
> defined. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, UINT16_MAX,
> -                      "ct.est && !ct.rel && !ct.new && !ct.inv "
> -                      "&& ct.rpl && ct_label.blocked == 0",
> -                      "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, UINT16_MAX,
> -                      "ct.est && !ct.rel && !ct.new && !ct.inv "
> -                      "&& ct.rpl && ct_label.blocked == 0",
> -                      "next;");
> -
> -        /* Ingress and Egress ACL Table (Priority 65535).
> -         *
> -         * Allow traffic that is related to an existing conntrack entry
> that
> -         * has not been marked for deletion (bit 0 of ct_label).
> -         *
> -         * This is enforced at a higher priority than ACLs can be defined.
> -         *
> -         * NOTE: This does not support related data sessions (eg,
> -         * a dynamically negotiated FTP data channel), but will allow
> -         * related traffic such as an ICMP Port Unreachable through
> -         * that's generated from a non-listening UDP port.  */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, UINT16_MAX,
> -                      "!ct.est && ct.rel && !ct.new && !ct.inv "
> -                      "&& ct_label.blocked == 0",
> -                      "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, UINT16_MAX,
> -                      "!ct.est && ct.rel && !ct.new && !ct.inv "
> -                      "&& ct_label.blocked == 0",
> -                      "next;");
> -
> -        /* Ingress and Egress ACL Table (Priority 65535).
> -         *
> -         * Not to do conntrack on ND packets. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL, UINT16_MAX, "nd",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL, UINT16_MAX, "nd",
> "next;");
> -    }
> -
> -    /* Ingress or Egress ACL Table (Various priorities). */
> -    for (size_t i = 0; i < od->nbs->n_acls; i++) {
> -        struct nbrec_acl *acl = od->nbs->acls[i];
> -        consider_acl(lflows, od, acl, has_stateful);
> -    }
> -    struct ovn_port_group *pg;
> -    HMAP_FOR_EACH (pg, key_node, port_groups) {
> -        if (ovn_port_group_ls_find(pg, &od->nbs->header_.uuid)) {
> -            for (size_t i = 0; i < pg->nb_pg->n_acls; i++) {
> -                consider_acl(lflows, od, pg->nb_pg->acls[i],
> has_stateful);
> -            }
> -        }
> -    }
> -
> -    /* Add 34000 priority flow to allow DHCP reply from ovn-controller to
> all
> -     * logical ports of the datapath if the CMS has configured DHCPv4
> options.
> -     * */
> -    for (size_t i = 0; i < od->nbs->n_ports; i++) {
> -        if (lsp_is_external(od->nbs->ports[i])) {
> -            continue;
> -        }
> -
> -        if (od->nbs->ports[i]->dhcpv4_options) {
> -            const char *server_id = smap_get(
> -                &od->nbs->ports[i]->dhcpv4_options->options, "server_id");
> -            const char *server_mac = smap_get(
> -                &od->nbs->ports[i]->dhcpv4_options->options,
> "server_mac");
> -            const char *lease_time = smap_get(
> -                &od->nbs->ports[i]->dhcpv4_options->options,
> "lease_time");
> -            if (server_id && server_mac && lease_time) {
> -                struct ds match = DS_EMPTY_INITIALIZER;
> -                const char *actions =
> -                    has_stateful ? "ct_commit; next;" : "next;";
> -                ds_put_format(&match, "outport == \"%s\" && eth.src == %s
> "
> -                              "&& ip4.src == %s && udp && udp.src == 67 "
> -                              "&& udp.dst == 68", od->nbs->ports[i]->name,
> -                              server_mac, server_id);
> -                ovn_lflow_add(
> -                    lflows, od, S_SWITCH_OUT_ACL, 34000, ds_cstr(&match),
> -                    actions);
> -                ds_destroy(&match);
> -            }
> -        }
> -
> -        if (od->nbs->ports[i]->dhcpv6_options) {
> -            const char *server_mac = smap_get(
> -                &od->nbs->ports[i]->dhcpv6_options->options, "server_id");
> -            struct eth_addr ea;
> -            if (server_mac && eth_addr_from_string(server_mac, &ea)) {
> -                /* Get the link local IP of the DHCPv6 server from the
> -                 * server MAC. */
> -                struct in6_addr lla;
> -                in6_generate_lla(ea, &lla);
> -
> -                char server_ip[INET6_ADDRSTRLEN + 1];
> -                ipv6_string_mapped(server_ip, &lla);
> -
> -                struct ds match = DS_EMPTY_INITIALIZER;
> -                const char *actions = has_stateful ? "ct_commit; next;" :
> -                    "next;";
> -                ds_put_format(&match, "outport == \"%s\" && eth.src == %s
> "
> -                              "&& ip6.src == %s && udp && udp.src == 547 "
> -                              "&& udp.dst == 546",
> od->nbs->ports[i]->name,
> -                              server_mac, server_ip);
> -                ovn_lflow_add(
> -                    lflows, od, S_SWITCH_OUT_ACL, 34000, ds_cstr(&match),
> -                    actions);
> -                ds_destroy(&match);
> -            }
> -        }
> -    }
> -
> -    /* Add a 34000 priority flow to advance the DNS reply from
> ovn-controller,
> -     * if the CMS has configured DNS records for the datapath.
> -     */
> -    if (ls_has_dns_records(od->nbs)) {
> -        const char *actions = has_stateful ? "ct_commit; next;" : "next;";
> -        ovn_lflow_add(
> -            lflows, od, S_SWITCH_OUT_ACL, 34000, "udp.src == 53",
> -            actions);
> -    }
> -}
> -
> -static void
> -build_qos(struct ovn_datapath *od, struct hmap *lflows) {
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;");
> -
> -    for (size_t i = 0; i < od->nbs->n_qos_rules; i++) {
> -        struct nbrec_qos *qos = od->nbs->qos_rules[i];
> -        bool ingress = !strcmp(qos->direction, "from-lport") ? true
> :false;
> -        enum ovn_stage stage = ingress ? S_SWITCH_IN_QOS_MARK :
> S_SWITCH_OUT_QOS_MARK;
> -        int64_t rate = 0;
> -        int64_t burst = 0;
> -
> -        for (size_t j = 0; j < qos->n_action; j++) {
> -            if (!strcmp(qos->key_action[j], "dscp")) {
> -                struct ds dscp_action = DS_EMPTY_INITIALIZER;
> -
> -                ds_put_format(&dscp_action, "ip.dscp = %"PRId64"; next;",
> -                              qos->value_action[j]);
> -                ovn_lflow_add(lflows, od, stage,
> -                              qos->priority,
> -                              qos->match, ds_cstr(&dscp_action));
> -                ds_destroy(&dscp_action);
> -            }
> -        }
> -
> -        for (size_t n = 0; n < qos->n_bandwidth; n++) {
> -            if (!strcmp(qos->key_bandwidth[n], "rate")) {
> -                rate = qos->value_bandwidth[n];
> -            } else if (!strcmp(qos->key_bandwidth[n], "burst")) {
> -                burst = qos->value_bandwidth[n];
> -            }
> -        }
> -        if (rate) {
> -            struct ds meter_action = DS_EMPTY_INITIALIZER;
> -            stage = ingress ? S_SWITCH_IN_QOS_METER :
> S_SWITCH_OUT_QOS_METER;
> -            if (burst) {
> -                ds_put_format(&meter_action,
> -                              "set_meter(%"PRId64", %"PRId64"); next;",
> -                              rate, burst);
> -            } else {
> -                ds_put_format(&meter_action,
> -                              "set_meter(%"PRId64"); next;",
> -                              rate);
> -            }
> -
> -            /* Ingress and Egress QoS Meter Table.
> -             *
> -             * We limit the bandwidth of this flow by adding a meter
> table.
> -             */
> -            ovn_lflow_add(lflows, od, stage,
> -                          qos->priority,
> -                          qos->match, ds_cstr(&meter_action));
> -            ds_destroy(&meter_action);
> -        }
> -    }
> -}
> -
> -static void
> -build_lb(struct ovn_datapath *od, struct hmap *lflows)
> -{
> -    /* Ingress and Egress LB Table (Priority 0): Packets are allowed by
> -     * default.  */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_LB, 0, "1", "next;");
> -
> -    if (od->nbs->load_balancer) {
> -        /* Ingress and Egress LB Table (Priority 65535).
> -         *
> -         * Send established traffic through conntrack for just NAT. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, UINT16_MAX,
> -                      "ct.est && !ct.rel && !ct.new && !ct.inv",
> -                      REGBIT_CONNTRACK_NAT" = 1; next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_LB, UINT16_MAX,
> -                      "ct.est && !ct.rel && !ct.new && !ct.inv",
> -                      REGBIT_CONNTRACK_NAT" = 1; next;");
> -    }
> -}
> -
> -static void
> -build_stateful(struct ovn_datapath *od, struct hmap *lflows)
> -{
> -    /* Ingress and Egress stateful Table (Priority 0): Packets are
> -     * allowed by default. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;");
> -
> -    /* If REGBIT_CONNTRACK_COMMIT is set as 1, then the packets should be
> -     * committed to conntrack. We always set ct_label.blocked to 0 here as
> -     * any packet that makes it this far is part of a connection we
> -     * want to allow to continue. */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_COMMIT" == 1",
> "ct_commit(ct_label=0/1); next;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_COMMIT" == 1",
> "ct_commit(ct_label=0/1); next;");
> -
> -    /* If REGBIT_CONNTRACK_NAT is set as 1, then packets should just be
> sent
> -     * through nat (without committing).
> -     *
> -     * REGBIT_CONNTRACK_COMMIT is set for new connections and
> -     * REGBIT_CONNTRACK_NAT is set for established connections. So they
> -     * don't overlap.
> -     */
> -    ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_NAT" == 1", "ct_lb;");
> -    ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100,
> -                  REGBIT_CONNTRACK_NAT" == 1", "ct_lb;");
> -
> -    /* Load balancing rules for new connections get committed to conntrack
> -     * table.  So even if REGBIT_CONNTRACK_COMMIT is set in a previous
> table
> -     * a higher priority rule for load balancing below also commits the
> -     * connection, so it is okay if we do not hit the above match on
> -     * REGBIT_CONNTRACK_COMMIT. */
> -    for (int i = 0; i < od->nbs->n_load_balancer; i++) {
> -        struct nbrec_load_balancer *lb = od->nbs->load_balancer[i];
> -        struct smap *vips = &lb->vips;
> -        struct smap_node *node;
> -
> -        SMAP_FOR_EACH (node, vips) {
> -            uint16_t port = 0;
> -            int addr_family;
> -
> -            /* node->key contains IP:port or just IP. */
> -            char *ip_address = NULL;
> -            ip_address_and_port_from_lb_key(node->key, &ip_address, &port,
> -                                            &addr_family);
> -            if (!ip_address) {
> -                continue;
> -            }
> -
> -            /* New connections in Ingress table. */
> -            char *action = xasprintf("ct_lb(%s);", node->value);
> -            struct ds match = DS_EMPTY_INITIALIZER;
> -            if (addr_family == AF_INET) {
> -                ds_put_format(&match, "ct.new && ip4.dst == %s",
> ip_address);
> -            } else {
> -                ds_put_format(&match, "ct.new && ip6.dst == %s",
> ip_address);
> -            }
> -            if (port) {
> -                if (lb->protocol && !strcmp(lb->protocol, "udp")) {
> -                    ds_put_format(&match, " && udp.dst == %d", port);
> -                } else {
> -                    ds_put_format(&match, " && tcp.dst == %d", port);
> -                }
> -                ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL,
> -                              120, ds_cstr(&match), action);
> -            } else {
> -                ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL,
> -                              110, ds_cstr(&match), action);
> -            }
> -
> -            free(ip_address);
> -            ds_destroy(&match);
> -            free(action);
> -       }
> -    }
> -}
> -
> -static void
> -build_lrouter_groups__(struct hmap *ports, struct ovn_datapath *od)
> -{
> -    ovs_assert((od && od->nbr && od->lr_group));
> -
> -    if (od->l3dgw_port && od->l3redirect_port) {
> -        /* It's a logical router with gateway port. If it
> -         * has HA_Chassis_Group associated to it in SB DB, then store the
> -         * ha chassis group name. */
> -        if (od->l3redirect_port->sb->ha_chassis_group) {
> -            sset_add(&od->lr_group->ha_chassis_groups,
> -                     od->l3redirect_port->sb->ha_chassis_group->name);
> -        }
> -    }
> -
> -    for (size_t i = 0; i < od->nbr->n_ports; i++) {
> -        struct ovn_port *router_port =
> -            ovn_port_find(ports, od->nbr->ports[i]->name);
> -
> -        if (!router_port || !router_port->peer) {
> -            continue;
> -        }
> -
> -        /* Get the peer logical switch/logical router datapath. */
> -        struct ovn_datapath *peer_dp = router_port->peer->od;
> -        if (peer_dp->nbr) {
> -            if (!peer_dp->lr_group) {
> -                peer_dp->lr_group = od->lr_group;
> -                od->lr_group->router_dps[od->lr_group->n_router_dps++]
> -                    = peer_dp;
> -                build_lrouter_groups__(ports, peer_dp);
> -            }
> -        } else {
> -            for (size_t j = 0; j < peer_dp->n_router_ports; j++) {
> -                if (!peer_dp->router_ports[j]->peer) {
> -                    /* If there is no peer port connecting to the
> -                    * router port, ignore it. */
> -                    continue;
> -                }
> -
> -                struct ovn_datapath *router_dp;
> -                router_dp = peer_dp->router_ports[j]->peer->od;
> -                if (router_dp == od) {
> -                    continue;
> -                }
> -
> -                if (router_dp->lr_group == od->lr_group) {
> -                    /* 'router_dp' and 'od' already belong to the same
> -                    * lrouter group. Nothing to be done. */
> -                    continue;
> -                }
> -
> -                router_dp->lr_group = od->lr_group;
> -                od->lr_group->router_dps[od->lr_group->n_router_dps++]
> -                    = router_dp;
> -                build_lrouter_groups__(ports, router_dp);
> -            }
> -        }
> -    }
> -}
> -
> -/* Adds each logical router into a logical router group. All the
> - * logical routers which belong to a group are connected to
> - * each other either directly or indirectly (via transit logical switches
> - * in between).
> - *
> - * Suppose if 'lr_list' has lr0, lr1, lr2, lr3, lr4, lr5
> - * and the topology is like
> - *  sw0 <-> lr0 <-> sw1 <-> lr1 <->sw2 <-> lr2
> - *  sw3 <-> lr3 <-> lr4 <-> sw5
> - *  sw6 <-> lr5 <-> sw7
> - * Then 3 groups are created.
> - * Group 1 -> lr0, lr1 and lr2
> - *            lr0, lr1 and lr2's ovn_datapath->lr_group will point to this
> - *            group. This means sw0's logical ports can send packets to
> sw2's
> - *            logical ports if proper static route's are added.
> - * Group 2 -> lr3 and lr4
> - *            lr3 and lr4's ovn_datapath->lr_group will point to this
> group.
> - * Group 3 -> lr5
> - *
> - * Each logical router can belong to only one group.
> - */
> -static void
> -build_lrouter_groups(struct hmap *ports, struct ovs_list *lr_list)
> -{
> -    struct ovn_datapath *od;
> -    size_t n_router_dps = ovs_list_size(lr_list);
> -
> -    LIST_FOR_EACH (od, lr_list, lr_list) {
> -        if (!od->lr_group) {
> -            od->lr_group = xzalloc(sizeof *od->lr_group);
> -            /* Each logical router group can have max
> -             * 'n_router_dps'. So allocate enough memory. */
> -            od->lr_group->router_dps = xcalloc(sizeof *od, n_router_dps);
> -            od->lr_group->router_dps[0] = od;
> -            od->lr_group->n_router_dps = 1;
> -            sset_init(&od->lr_group->ha_chassis_groups);
> -            build_lrouter_groups__(ports, od);
> -        }
> -    }
> -}
> -
> -static void
> -build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
> -                    struct hmap *port_groups, struct hmap *lflows,
> -                    struct hmap *mcgroups, struct hmap *igmp_groups)
> -{
> -    /* This flow table structure is documented in ovn-northd(8), so please
> -     * update ovn-northd.8.xml if you change anything. */
> -
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -
> -    /* Build pre-ACL and ACL tables for both ingress and egress.
> -     * Ingress tables 3 through 10.  Egress tables 0 through 7. */
> -    struct ovn_datapath *od;
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        build_pre_acls(od, lflows);
> -        build_pre_lb(od, lflows);
> -        build_pre_stateful(od, lflows);
> -        build_acls(od, lflows, port_groups);
> -        build_qos(od, lflows);
> -        build_lb(od, lflows);
> -        build_stateful(od, lflows);
> -    }
> -
> -    /* Logical switch ingress table 0: Admission control framework
> (priority
> -     * 100). */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        /* Logical VLANs not supported. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100,
> "vlan.present",
> -                      "drop;");
> -
> -        /* Broadcast/multicast source address is invalid. */
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100,
> "eth.src[40]",
> -                      "drop;");
> -
> -        /* Port security flows have priority 50 (see below) and will
> continue
> -         * to the next table if packet source is acceptable. */
> -    }
> -
> -    /* Logical switch ingress table 0: Ingress port security - L2
> -     *  (priority 50).
> -     *  Ingress table 1: Ingress port security - IP (priority 90 and 80)
> -     *  Ingress table 2: Ingress port security - ND (priority 90 and 80)
> -     */
> -    struct ovn_port *op;
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp) {
> -            continue;
> -        }
> -
> -        if (!lsp_is_enabled(op->nbsp)) {
> -            /* Drop packets from disabled logical ports (since logical
> flow
> -             * tables are default-drop). */
> -            continue;
> -        }
> -
> -        if (lsp_is_external(op->nbsp)) {
> -            continue;
> -        }
> -
> -        ds_clear(&match);
> -        ds_clear(&actions);
> -        ds_put_format(&match, "inport == %s", op->json_key);
> -        build_port_security_l2("eth.src", op->ps_addrs, op->n_ps_addrs,
> -                               &match);
> -
> -        const char *queue_id = smap_get(&op->sb->options,
> "qdisc_queue_id");
> -        if (queue_id) {
> -            ds_put_format(&actions, "set_queue(%s); ", queue_id);
> -        }
> -        ds_put_cstr(&actions, "next;");
> -        ovn_lflow_add(lflows, op->od, S_SWITCH_IN_PORT_SEC_L2, 50,
> -                      ds_cstr(&match), ds_cstr(&actions));
> -
> -        if (op->nbsp->n_port_security) {
> -            build_port_security_ip(P_IN, op, lflows);
> -            build_port_security_nd(op, lflows);
> -        }
> -    }
> -
> -    /* Ingress table 1 and 2: Port security - IP and ND, by default goto
> next.
> -     * (priority 0)*/
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_ND, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_IP, 0, "1",
> "next;");
> -    }
> -
> -    /* Ingress table 11: ARP/ND responder, skip requests coming from
> localnet
> -     * and vtep ports. (priority 100); see ovn-northd.8.xml for the
> -     * rationale. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp) {
> -            continue;
> -        }
> -
> -        if ((!strcmp(op->nbsp->type, "localnet")) ||
> -            (!strcmp(op->nbsp->type, "vtep"))) {
> -            ds_clear(&match);
> -            ds_put_format(&match, "inport == %s", op->json_key);
> -            ovn_lflow_add(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100,
> -                          ds_cstr(&match), "next;");
> -        }
> -    }
> -
> -    /* Ingress table 11: ARP/ND responder, reply for known IPs.
> -     * (priority 50). */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp) {
> -            continue;
> -        }
> -
> -        /*
> -         * Add ARP/ND reply flows if either the
> -         *  - port is up or
> -         *  - port type is router or
> -         *  - port type is localport
> -         */
> -        if (!lsp_is_up(op->nbsp) && strcmp(op->nbsp->type, "router") &&
> -            strcmp(op->nbsp->type, "localport")) {
> -            continue;
> -        }
> -
> -        if (lsp_is_external(op->nbsp)) {
> -            continue;
> -        }
> -
> -        for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> -            for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) {
> -                ds_clear(&match);
> -                ds_put_format(&match, "arp.tpa == %s && arp.op == 1",
> -                              op->lsp_addrs[i].ipv4_addrs[j].addr_s);
> -                ds_clear(&actions);
> -                ds_put_format(&actions,
> -                    "eth.dst = eth.src; "
> -                    "eth.src = %s; "
> -                    "arp.op = 2; /* ARP reply */ "
> -                    "arp.tha = arp.sha; "
> -                    "arp.sha = %s; "
> -                    "arp.tpa = arp.spa; "
> -                    "arp.spa = %s; "
> -                    "outport = inport; "
> -                    "flags.loopback = 1; "
> -                    "output;",
> -                    op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s,
> -                    op->lsp_addrs[i].ipv4_addrs[j].addr_s);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -
> -                /* Do not reply to an ARP request from the port that owns
> the
> -                 * address (otherwise a DHCP client that ARPs to check
> for a
> -                 * duplicate address will fail).  Instead, forward it the
> usual
> -                 * way.
> -                 *
> -                 * (Another alternative would be to simply drop the
> packet.  If
> -                 * everything is working as it is configured, then this
> would
> -                 * produce equivalent results, since no one should reply
> to the
> -                 * request.  But ARPing for one's own IP address is
> intended to
> -                 * detect situations where the network is not working as
> -                 * configured, so dropping the request would frustrate
> that
> -                 * intent.) */
> -                ds_put_format(&match, " && inport == %s", op->json_key);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100,
> -                              ds_cstr(&match), "next;");
> -            }
> -
> -            /* For ND solicitations, we need to listen for both the
> -             * unicast IPv6 address and its all-nodes multicast address,
> -             * but always respond with the unicast IPv6 address. */
> -            for (size_t j = 0; j < op->lsp_addrs[i].n_ipv6_addrs; j++) {
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                        "nd_ns && ip6.dst == {%s, %s} && nd.target == %s",
> -                        op->lsp_addrs[i].ipv6_addrs[j].addr_s,
> -                        op->lsp_addrs[i].ipv6_addrs[j].sn_addr_s,
> -                        op->lsp_addrs[i].ipv6_addrs[j].addr_s);
> -
> -                ds_clear(&actions);
> -                ds_put_format(&actions,
> -                        "%s { "
> -                        "eth.src = %s; "
> -                        "ip6.src = %s; "
> -                        "nd.target = %s; "
> -                        "nd.tll = %s; "
> -                        "outport = inport; "
> -                        "flags.loopback = 1; "
> -                        "output; "
> -                        "};",
> -                        !strcmp(op->nbsp->type, "router") ?
> -                            "nd_na_router" : "nd_na",
> -                        op->lsp_addrs[i].ea_s,
> -                        op->lsp_addrs[i].ipv6_addrs[j].addr_s,
> -                        op->lsp_addrs[i].ipv6_addrs[j].addr_s,
> -                        op->lsp_addrs[i].ea_s);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -
> -                /* Do not reply to a solicitation from the port that owns
> the
> -                 * address (otherwise DAD detection will fail). */
> -                ds_put_format(&match, " && inport == %s", op->json_key);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100,
> -                              ds_cstr(&match), "next;");
> -            }
> -        }
> -    }
> -
> -    /* Ingress table 11: ARP/ND responder, by default goto next.
> -     * (priority 0)*/
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1",
> "next;");
> -    }
> -
> -    /* Logical switch ingress table 12 and 13: DHCP options and response
> -         * priority 100 flows. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp) {
> -           continue;
> -        }
> -
> -        if (!lsp_is_enabled(op->nbsp) || !strcmp(op->nbsp->type,
> "router")) {
> -            /* Don't add the DHCP flows if the port is not enabled or if
> the
> -             * port is a router port. */
> -            continue;
> -        }
> -
> -        if (!op->nbsp->dhcpv4_options && !op->nbsp->dhcpv6_options) {
> -            /* CMS has disabled both native DHCPv4 and DHCPv6 for this
> lport.
> -             */
> -            continue;
> -        }
> -
> -        bool is_external = lsp_is_external(op->nbsp);
> -        if (is_external && (!op->od->localnet_port ||
> -                            !op->nbsp->ha_chassis_group)) {
> -            /* If it's an external port and there is no localnet port
> -             * and if it doesn't belong to an HA chassis group ignore it.
> */
> -            continue;
> -        }
> -
> -        for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> -            for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) {
> -                struct ds options_action = DS_EMPTY_INITIALIZER;
> -                struct ds response_action = DS_EMPTY_INITIALIZER;
> -                struct ds ipv4_addr_match = DS_EMPTY_INITIALIZER;
> -                if (build_dhcpv4_action(
> -                        op, op->lsp_addrs[i].ipv4_addrs[j].addr,
> -                        &options_action, &response_action,
> &ipv4_addr_match)) {
> -                    ds_clear(&match);
> -                    ds_put_format(
> -                        &match, "inport == %s && eth.src == %s && "
> -                        "ip4.src == 0.0.0.0 && ip4.dst == 255.255.255.255
> && "
> -                        "udp.src == 68 && udp.dst == 67",
> -                        is_external ? op->od->localnet_port->json_key :
> -                            op->json_key,
> -                        op->lsp_addrs[i].ea_s);
> -
> -                    if (is_external) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      op->json_key);
> -                    }
> -
> -                    ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_DHCP_OPTIONS,
> -                                  100, ds_cstr(&match),
> -                                  ds_cstr(&options_action));
> -                    ds_clear(&match);
> -                    /* Allow ip4.src = OFFER_IP and
> -                     * ip4.dst = {SERVER_IP, 255.255.255.255} for the
> below
> -                     * cases
> -                     *  -  When the client wants to renew the IP by
> sending
> -                     *     the DHCPREQUEST to the server ip.
> -                     *  -  When the client wants to renew the IP by
> -                     *     broadcasting the DHCPREQUEST.
> -                     */
> -                    ds_put_format(
> -                        &match, "inport == %s && eth.src == %s && "
> -                        "%s && udp.src == 68 && udp.dst == 67",
> -                        is_external ? op->od->localnet_port->json_key :
> -                            op->json_key,
> -                        op->lsp_addrs[i].ea_s, ds_cstr(&ipv4_addr_match));
> -
> -                    if (is_external) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      op->json_key);
> -                    }
> -
> -                    ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_DHCP_OPTIONS,
> -                                  100, ds_cstr(&match),
> -                                  ds_cstr(&options_action));
> -                    ds_clear(&match);
> -
> -                    /* If REGBIT_DHCP_OPTS_RESULT is set, it means the
> -                     * put_dhcp_opts action  is successful. */
> -                    ds_put_format(
> -                        &match, "inport == %s && eth.src == %s && "
> -                        "ip4 && udp.src == 68 && udp.dst == 67"
> -                        " && "REGBIT_DHCP_OPTS_RESULT,
> -                        is_external ? op->od->localnet_port->json_key :
> -                            op->json_key,
> -                        op->lsp_addrs[i].ea_s);
> -
> -                    if (is_external) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      op->json_key);
> -                    }
> -
> -                    ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_DHCP_RESPONSE,
> -                                  100, ds_cstr(&match),
> -                                  ds_cstr(&response_action));
> -                    ds_destroy(&options_action);
> -                    ds_destroy(&response_action);
> -                    ds_destroy(&ipv4_addr_match);
> -                    break;
> -                }
> -            }
> -
> -            for (size_t j = 0; j < op->lsp_addrs[i].n_ipv6_addrs; j++) {
> -                struct ds options_action = DS_EMPTY_INITIALIZER;
> -                struct ds response_action = DS_EMPTY_INITIALIZER;
> -                if (build_dhcpv6_action(
> -                        op, &op->lsp_addrs[i].ipv6_addrs[j].addr,
> -                        &options_action, &response_action)) {
> -                    ds_clear(&match);
> -                    ds_put_format(
> -                        &match, "inport == %s && eth.src == %s"
> -                        " && ip6.dst == ff02::1:2 && udp.src == 546 &&"
> -                        " udp.dst == 547",
> -                        is_external ? op->od->localnet_port->json_key :
> -                            op->json_key,
> -                        op->lsp_addrs[i].ea_s);
> -
> -                    if (is_external) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      op->json_key);
> -                    }
> -
> -                    ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_DHCP_OPTIONS, 100,
> -                                  ds_cstr(&match),
> ds_cstr(&options_action));
> -
> -                    /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means
> the
> -                     * put_dhcpv6_opts action is successful */
> -                    ds_put_cstr(&match, " && "REGBIT_DHCP_OPTS_RESULT);
> -                    ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_DHCP_RESPONSE, 100,
> -                                  ds_cstr(&match),
> ds_cstr(&response_action));
> -                    ds_destroy(&options_action);
> -                    ds_destroy(&response_action);
> -                    break;
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Logical switch ingress table 14 and 15: DNS lookup and response
> -     * priority 100 flows.
> -     */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs || !ls_has_dns_records(od->nbs)) {
> -           continue;
> -        }
> -
> -        struct ds action = DS_EMPTY_INITIALIZER;
> -
> -        ds_clear(&match);
> -        ds_put_cstr(&match, "udp.dst == 53");
> -        ds_put_format(&action,
> -                      REGBIT_DNS_LOOKUP_RESULT" = dns_lookup(); next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 100,
> -                      ds_cstr(&match), ds_cstr(&action));
> -        ds_clear(&action);
> -        ds_put_cstr(&match, " && "REGBIT_DNS_LOOKUP_RESULT);
> -        ds_put_format(&action, "eth.dst <-> eth.src; ip4.src <-> ip4.dst;
> "
> -                      "udp.dst = udp.src; udp.src = 53; outport = inport;
> "
> -                      "flags.loopback = 1; output;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100,
> -                      ds_cstr(&match), ds_cstr(&action));
> -        ds_clear(&action);
> -        ds_put_format(&action, "eth.dst <-> eth.src; ip6.src <-> ip6.dst;
> "
> -                      "udp.dst = udp.src; udp.src = 53; outport = inport;
> "
> -                      "flags.loopback = 1; output;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100,
> -                      ds_cstr(&match), ds_cstr(&action));
> -        ds_destroy(&action);
> -    }
> -
> -    /* Ingress table 12 and 13: DHCP options and response, by default goto
> -     * next. (priority 0).
> -     * Ingress table 14 and 15: DNS lookup and response, by default goto
> next.
> -     * (priority 0).
> -     * Ingress table 16 - External port handling, by default goto next.
> -     * (priority 0). */
> -
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1",
> "next;");
> -    }
> -
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp || !lsp_is_external(op->nbsp) ||
> -            !op->od->localnet_port) {
> -           continue;
> -        }
> -
> -        /* Table 16: External port. Drop ARP request for router ips from
> -         * external ports  on chassis not binding those ports.
> -         * This makes the router pipeline to be run only on the chassis
> -         * binding the external ports. */
> -
> -        for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> -            for (size_t j = 0; j < op->od->n_router_ports; j++) {
> -                struct ovn_port *rp = op->od->router_ports[j];
> -                for (size_t k = 0; k < rp->n_lsp_addrs; k++) {
> -                    for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv4_addrs;
> -                         l++) {
> -                        ds_clear(&match);
> -                        ds_put_format(
> -                            &match, "inport == %s && eth.src == %s"
> -                            " && !is_chassis_resident(%s)"
> -                            " && arp.tpa == %s && arp.op == 1",
> -                            op->od->localnet_port->json_key,
> -                            op->lsp_addrs[i].ea_s, op->json_key,
> -                            rp->lsp_addrs[k].ipv4_addrs[l].addr_s);
> -                        ovn_lflow_add(lflows, op->od,
> -                                      S_SWITCH_IN_EXTERNAL_PORT, 100,
> -                                      ds_cstr(&match), "drop;");
> -                    }
> -                    for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs;
> -                         l++) {
> -                        ds_clear(&match);
> -                        ds_put_format(
> -                            &match, "inport == %s && eth.src == %s"
> -                            " && !is_chassis_resident(%s)"
> -                            " && nd_ns && ip6.dst == {%s, %s} && "
> -                            "nd.target == %s",
> -                            op->od->localnet_port->json_key,
> -                            op->lsp_addrs[i].ea_s, op->json_key,
> -                            rp->lsp_addrs[k].ipv6_addrs[l].addr_s,
> -                            rp->lsp_addrs[k].ipv6_addrs[l].sn_addr_s,
> -                            rp->lsp_addrs[k].ipv6_addrs[l].addr_s);
> -                        ovn_lflow_add(lflows, op->od,
> -                                      S_SWITCH_IN_EXTERNAL_PORT, 100,
> -                                      ds_cstr(&match), "drop;");
> -                    }
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Ingress table 17: Destination lookup, broadcast and multicast
> handling
> -     * (priority 70 - 100). */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        if (od->mcast_info.enabled) {
> -            /* Punt IGMP traffic to controller. */
> -            ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 100,
> -                          "ip4 && ip.proto == 2", "igmp;");
> -
> -            /* Flood all IP multicast traffic destined to 224.0.0.X to all
> -             * ports - RFC 4541, section 2.1.2, item 2.
> -             */
> -            ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85,
> -                          "ip4 && ip4.dst == 224.0.0.0/24",
> -                          "outport = \""MC_FLOOD"\"; output;");
> -
> -            /* Drop unregistered IP multicast if not allowed. */
> -            if (!od->mcast_info.flood_unregistered) {
> -                ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 80,
> -                              "ip4 && ip4.mcast", "drop;");
> -            }
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 70, "eth.mcast",
> -                      "outport = \""MC_FLOOD"\"; output;");
> -    }
> -
> -    /* Ingress table 17: Add IP multicast flows learnt from IGMP
> -     * (priority 90). */
> -    struct ovn_igmp_group *igmp_group, *next_igmp_group;
> -
> -    HMAP_FOR_EACH_SAFE (igmp_group, next_igmp_group, hmap_node,
> igmp_groups) {
> -        ds_clear(&match);
> -        ds_clear(&actions);
> -
> -        if (!igmp_group->datapath) {
> -            continue;
> -        }
> -
> -        struct mcast_info *mcast_info = &igmp_group->datapath->mcast_info;
> -
> -        if (mcast_info->active_flows >= mcast_info->table_size) {
> -            continue;
> -        }
> -        mcast_info->active_flows++;
> -
> -        ds_put_format(&match, "eth.mcast && ip4 && ip4.dst == %s ",
> -                      igmp_group->mcgroup.name);
> -        ds_put_format(&actions, "outport = \"%s\"; output; ",
> -                      igmp_group->mcgroup.name);
> -
> -        ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP,
> 90,
> -                      ds_cstr(&match), ds_cstr(&actions));
> -    }
> -
> -    /* Ingress table 17: Destination lookup, unicast handling (priority
> 50), */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp || lsp_is_external(op->nbsp)) {
> -            continue;
> -        }
> -
> -        for (size_t i = 0; i < op->nbsp->n_addresses; i++) {
> -            /* Addresses are owned by the logical port.
> -             * Ethernet address followed by zero or more IPv4
> -             * or IPv6 addresses (or both). */
> -            struct eth_addr mac;
> -            if (ovs_scan(op->nbsp->addresses[i],
> -                        ETH_ADDR_SCAN_FMT, ETH_ADDR_SCAN_ARGS(mac))) {
> -                ds_clear(&match);
> -                ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT,
> -                              ETH_ADDR_ARGS(mac));
> -
> -                ds_clear(&actions);
> -                ds_put_format(&actions, "outport = %s; output;",
> op->json_key);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            } else if (!strcmp(op->nbsp->addresses[i], "unknown")) {
> -                if (lsp_is_enabled(op->nbsp)) {
> -                    ovn_multicast_add(mcgroups, &mc_unknown, op);
> -                    op->od->has_unknown = true;
> -                }
> -            } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) {
> -                if (!op->nbsp->dynamic_addresses
> -                    || !ovs_scan(op->nbsp->dynamic_addresses,
> -                            ETH_ADDR_SCAN_FMT, ETH_ADDR_SCAN_ARGS(mac))) {
> -                    continue;
> -                }
> -                ds_clear(&match);
> -                ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT,
> -                              ETH_ADDR_ARGS(mac));
> -
> -                ds_clear(&actions);
> -                ds_put_format(&actions, "outport = %s; output;",
> op->json_key);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            } else if (!strcmp(op->nbsp->addresses[i], "router")) {
> -                if (!op->peer || !op->peer->nbrp
> -                    || !ovs_scan(op->peer->nbrp->mac,
> -                            ETH_ADDR_SCAN_FMT, ETH_ADDR_SCAN_ARGS(mac))) {
> -                    continue;
> -                }
> -                ds_clear(&match);
> -                ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT,
> -                              ETH_ADDR_ARGS(mac));
> -                if (op->peer->od->l3dgw_port
> -                    && op->peer->od->l3redirect_port
> -                    && op->od->localnet_port) {
> -                    bool add_chassis_resident_check = false;
> -                    if (op->peer == op->peer->od->l3dgw_port) {
> -                        /* The peer of this port represents a distributed
> -                         * gateway port. The destination lookup flow for
> the
> -                         * router's distributed gateway port MAC address
> should
> -                         * only be programmed on the "redirect-chassis".
> */
> -                        add_chassis_resident_check = true;
> -                    } else {
> -                        /* Check if the option
> 'reside-on-redirect-chassis'
> -                         * is set to true on the peer port. If set to true
> -                         * and if the logical switch has a localnet port,
> it
> -                         * means the router pipeline for the packets from
> -                         * this logical switch should be run on the
> chassis
> -                         * hosting the gateway port.
> -                         */
> -                        add_chassis_resident_check = smap_get_bool(
> -                            &op->peer->nbrp->options,
> -                            "reside-on-redirect-chassis", false);
> -                    }
> -
> -                    if (add_chassis_resident_check) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -
> op->peer->od->l3redirect_port->json_key);
> -                    }
> -                }
> -
> -                ds_clear(&actions);
> -                ds_put_format(&actions, "outport = %s; output;",
> op->json_key);
> -                ovn_lflow_add(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -
> -                /* Add ethernet addresses specified in NAT rules on
> -                 * distributed logical routers. */
> -                if (op->peer->od->l3dgw_port
> -                    && op->peer == op->peer->od->l3dgw_port) {
> -                    for (int j = 0; j < op->peer->od->nbr->n_nat; j++) {
> -                        const struct nbrec_nat *nat
> -                                                  =
> op->peer->od->nbr->nat[j];
> -                        if (!strcmp(nat->type, "dnat_and_snat")
> -                            && nat->logical_port && nat->external_mac
> -                            && eth_addr_from_string(nat->external_mac,
> &mac)) {
> -
> -                            ds_clear(&match);
> -                            ds_put_format(&match, "eth.dst ==
> "ETH_ADDR_FMT
> -                                          " &&
> is_chassis_resident(\"%s\")",
> -                                          ETH_ADDR_ARGS(mac),
> -                                          nat->logical_port);
> -
> -                            ds_clear(&actions);
> -                            ds_put_format(&actions, "outport = %s;
> output;",
> -                                          op->json_key);
> -                            ovn_lflow_add(lflows, op->od,
> S_SWITCH_IN_L2_LKUP,
> -                                          50, ds_cstr(&match),
> -                                          ds_cstr(&actions));
> -                        }
> -                    }
> -                }
> -            } else {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(1, 1);
> -
> -                VLOG_INFO_RL(&rl,
> -                             "%s: invalid syntax '%s' in addresses
> column",
> -                             op->nbsp->name, op->nbsp->addresses[i]);
> -            }
> -        }
> -    }
> -
> -    /* Ingress table 17: Destination lookup for unknown MACs (priority
> 0). */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        if (od->has_unknown) {
> -            ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1",
> -                          "outport = \""MC_UNKNOWN"\"; output;");
> -        }
> -    }
> -
> -    /* Egress tables 8: Egress port security - IP (priority 0)
> -     * Egress table 9: Egress port security L2 - multicast/broadcast
> -     *                 (priority 100). */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_PORT_SEC_IP, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_SWITCH_OUT_PORT_SEC_L2, 100,
> "eth.mcast",
> -                      "output;");
> -    }
> -
> -    /* Egress table 8: Egress port security - IP (priorities 90 and 80)
> -     * if port security enabled.
> -     *
> -     * Egress table 9: Egress port security - L2 (priorities 50 and 150).
> -     *
> -     * Priority 50 rules implement port security for enabled logical port.
> -     *
> -     * Priority 150 rules drop packets to disabled logical ports, so that
> they
> -     * don't even receive multicast or broadcast packets. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp || lsp_is_external(op->nbsp)) {
> -            continue;
> -        }
> -
> -        ds_clear(&match);
> -        ds_put_format(&match, "outport == %s", op->json_key);
> -        if (lsp_is_enabled(op->nbsp)) {
> -            build_port_security_l2("eth.dst", op->ps_addrs,
> op->n_ps_addrs,
> -                                   &match);
> -            ovn_lflow_add(lflows, op->od, S_SWITCH_OUT_PORT_SEC_L2, 50,
> -                          ds_cstr(&match), "output;");
> -        } else {
> -            ovn_lflow_add(lflows, op->od, S_SWITCH_OUT_PORT_SEC_L2, 150,
> -                          ds_cstr(&match), "drop;");
> -        }
> -
> -        if (op->nbsp->n_port_security) {
> -            build_port_security_ip(P_OUT, op, lflows);
> -        }
> -    }
> -
> -    ds_destroy(&match);
> -    ds_destroy(&actions);
> -}
> -
> -static bool
> -lrport_is_enabled(const struct nbrec_logical_router_port *lrport)
> -{
> -    return !lrport->enabled || *lrport->enabled;
> -}
> -
> -/* Returns a string of the IP address of the router port 'op' that
> - * overlaps with 'ip_s".  If one is not found, returns NULL.
> - *
> - * The caller must not free the returned string. */
> -static const char *
> -find_lrp_member_ip(const struct ovn_port *op, const char *ip_s)
> -{
> -    bool is_ipv4 = strchr(ip_s, '.') ? true : false;
> -
> -    if (is_ipv4) {
> -        ovs_be32 ip;
> -
> -        if (!ip_parse(ip_s, &ip)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad ip address %s", ip_s);
> -            return NULL;
> -        }
> -
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            const struct ipv4_netaddr *na =
> &op->lrp_networks.ipv4_addrs[i];
> -
> -            if (!((na->network ^ ip) & na->mask)) {
> -                /* There should be only 1 interface that matches the
> -                 * supplied IP.  Otherwise, it's a configuration error,
> -                 * because subnets of a router's interfaces should NOT
> -                 * overlap. */
> -                return na->addr_s;
> -            }
> -        }
> -    } else {
> -        struct in6_addr ip6;
> -
> -        if (!ipv6_parse(ip_s, &ip6)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad ipv6 address %s", ip_s);
> -            return NULL;
> -        }
> -
> -        for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -            const struct ipv6_netaddr *na =
> &op->lrp_networks.ipv6_addrs[i];
> -            struct in6_addr xor_addr = ipv6_addr_bitxor(&na->network,
> &ip6);
> -            struct in6_addr and_addr = ipv6_addr_bitand(&xor_addr,
> &na->mask);
> -
> -            if (ipv6_is_zero(&and_addr)) {
> -                /* There should be only 1 interface that matches the
> -                 * supplied IP.  Otherwise, it's a configuration error,
> -                 * because subnets of a router's interfaces should NOT
> -                 * overlap. */
> -                return na->addr_s;
> -            }
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static struct ovn_port*
> -get_outport_for_routing_policy_nexthop(struct ovn_datapath *od,
> -                                       struct hmap *ports,
> -                                       int priority, const char *nexthop)
> -{
> -    if (nexthop == NULL) {
> -        return NULL;
> -    }
> -
> -    /* Find the router port matching the next hop. */
> -    for (int i = 0; i < od->nbr->n_ports; i++) {
> -       struct nbrec_logical_router_port *lrp = od->nbr->ports[i];
> -
> -       struct ovn_port *out_port = ovn_port_find(ports, lrp->name);
> -       if (out_port && find_lrp_member_ip(out_port, nexthop)) {
> -           return out_port;
> -       }
> -    }
> -
> -    static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -    VLOG_WARN_RL(&rl, "No path for routing policy priority %d; next hop
> %s",
> -                 priority, nexthop);
> -    return NULL;
> -}
> -
> -static void
> -build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od,
> -                          struct hmap *ports,
> -                          const struct nbrec_logical_router_policy *rule)
> -{
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -
> -    if (!strcmp(rule->action, "reroute")) {
> -        struct ovn_port *out_port =
> get_outport_for_routing_policy_nexthop(
> -             od, ports, rule->priority, rule->nexthop);
> -        if (!out_port) {
> -            return;
> -        }
> -
> -        const char *lrp_addr_s = find_lrp_member_ip(out_port,
> rule->nexthop);
> -        if (!lrp_addr_s) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "lrp_addr not found for routing policy "
> -                         " priority %"PRId64" nexthop %s",
> -                         rule->priority, rule->nexthop);
> -            return;
> -        }
> -        bool is_ipv4 = strchr(rule->nexthop, '.') ? true : false;
> -        ds_put_format(&actions, "%sreg0 = %s; "
> -                      "%sreg1 = %s; "
> -                      "eth.src = %s; "
> -                      "outport = %s; "
> -                      "flags.loopback = 1; "
> -                      "next;",
> -                      is_ipv4 ? "" : "xx",
> -                      rule->nexthop,
> -                      is_ipv4 ? "" : "xx",
> -                      lrp_addr_s,
> -                      out_port->lrp_networks.ea_s,
> -                      out_port->json_key);
> -
> -    } else if (!strcmp(rule->action, "drop")) {
> -        ds_put_cstr(&actions, "drop;");
> -    } else if (!strcmp(rule->action, "allow")) {
> -        ds_put_cstr(&actions, "next;");
> -    }
> -    ds_put_format(&match, "%s", rule->match);
> -    ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, rule->priority,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -    ds_destroy(&match);
> -    ds_destroy(&actions);
> -}
> -
> -static void
> -add_distributed_nat_routes(struct hmap *lflows, const struct ovn_port *op)
> -{
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -
> -    if (!op->od->l3dgw_port) {
> -        return;
> -    }
> -
> -    if (!op->peer || !op->peer->od->nbs) {
> -        return;
> -    }
> -
> -    for (size_t i = 0; i < op->od->nbr->n_nat; i++) {
> -        const struct nbrec_nat *nat = op->od->nbr->nat[i];
> -        bool found = false;
> -
> -        if (strcmp(nat->type, "dnat_and_snat") ||
> -            !nat->external_mac  || !nat->external_ip) {
> -            continue;
> -        }
> -
> -        const struct ovn_datapath *peer_dp = op->peer->od;
> -        for (size_t j = 0; j < peer_dp->nbs->n_ports; j++) {
> -            if (!strcmp(peer_dp->nbs->ports[j]->name, nat->logical_port))
> {
> -                found = true;
> -                break;
> -            }
> -        }
> -        if (!found) {
> -            continue;
> -        }
> -
> -        ds_put_format(&match, "inport == %s && "
> -                      "ip4.src == %s && ip4.dst == %s",
> -                       op->json_key, nat->logical_ip, nat->external_ip);
> -        ds_put_format(&actions, "outport = %s; eth.dst = %s; "
> -                      REGBIT_DISTRIBUTED_NAT" = 1; "
> -                      REGBIT_NAT_REDIRECT" = 0; next;",
> -                      op->od->l3dgw_port->json_key,
> -                      nat->external_mac);
> -        ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_ROUTING, 400,
> -                      ds_cstr(&match), ds_cstr(&actions));
> -        ds_clear(&match);
> -        ds_clear(&actions);
> -
> -        for (size_t j = 0; j < op->od->nbr->n_nat; j++) {
> -            const struct nbrec_nat *nat2 = op->od->nbr->nat[j];
> -
> -            if (nat == nat2 || strcmp(nat2->type, "dnat_and_snat") ||
> -                !nat2->external_mac || !nat2->external_ip)
> -                continue;
> -
> -            ds_put_format(&match, "inport == %s && "
> -                          "ip4.src == %s && ip4.dst == %s",
> -                          op->json_key, nat->logical_ip,
> nat2->external_ip);
> -            ds_put_format(&actions, "outport = %s; "
> -                          "eth.src = %s; eth.dst = %s; "
> -                          "reg0 = ip4.dst; reg1 = %s; "
> -                          REGBIT_DISTRIBUTED_NAT" = 1; "
> -                          REGBIT_NAT_REDIRECT" = 0; next;",
> -                          op->od->l3dgw_port->json_key,
> -                          op->od->l3dgw_port->lrp_networks.ea_s,
> -                          nat2->external_mac, nat->external_ip);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_ROUTING, 400,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -            ds_clear(&match);
> -            ds_clear(&actions);
> -        }
> -    }
> -}
> -
> -static void
> -add_route(struct hmap *lflows, const struct ovn_port *op,
> -          const char *lrp_addr_s, const char *network_s, int plen,
> -          const char *gateway, const char *policy)
> -{
> -    bool is_ipv4 = strchr(network_s, '.') ? true : false;
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -    const char *dir;
> -    uint16_t priority;
> -
> -    if (policy && !strcmp(policy, "src-ip")) {
> -        dir = "src";
> -        priority = plen * 2;
> -    } else {
> -        dir = "dst";
> -        priority = (plen * 2) + 1;
> -    }
> -
> -    /* IPv6 link-local addresses must be scoped to the local router port.
> */
> -    if (!is_ipv4) {
> -        struct in6_addr network;
> -        ovs_assert(ipv6_parse(network_s, &network));
> -        if (in6_is_lla(&network)) {
> -            ds_put_format(&match, "inport == %s && ", op->json_key);
> -        }
> -    }
> -    ds_put_format(&match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : "6", dir,
> -                  network_s, plen);
> -
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&actions, "ip.ttl--; %sreg0 = ", is_ipv4 ? "" : "xx");
> -
> -    if (gateway) {
> -        ds_put_cstr(&actions, gateway);
> -    } else {
> -        ds_put_format(&actions, "ip%s.dst", is_ipv4 ? "4" : "6");
> -    }
> -    ds_put_format(&actions, "; "
> -                  "%sreg1 = %s; "
> -                  "eth.src = %s; "
> -                  "outport = %s; "
> -                  "flags.loopback = 1; "
> -                  "next;",
> -                  is_ipv4 ? "" : "xx",
> -                  lrp_addr_s,
> -                  op->lrp_networks.ea_s,
> -                  op->json_key);
> -
> -    /* The priority here is calculated to implement longest-prefix-match
> -     * routing. */
> -    ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_ROUTING, priority,
> -                  ds_cstr(&match), ds_cstr(&actions));
> -    ds_destroy(&match);
> -    ds_destroy(&actions);
> -}
> -
> -static void
> -build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od,
> -                        struct hmap *ports,
> -                        const struct nbrec_logical_router_static_route
> *route)
> -{
> -    ovs_be32 nexthop;
> -    const char *lrp_addr_s = NULL;
> -    unsigned int plen;
> -    bool is_ipv4;
> -
> -    /* Verify that the next hop is an IP address with an all-ones mask. */
> -    char *error = ip_parse_cidr(route->nexthop, &nexthop, &plen);
> -    if (!error) {
> -        if (plen != 32) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad next hop mask %s", route->nexthop);
> -            return;
> -        }
> -        is_ipv4 = true;
> -    } else {
> -        free(error);
> -
> -        struct in6_addr ip6;
> -        error = ipv6_parse_cidr(route->nexthop, &ip6, &plen);
> -        if (!error) {
> -            if (plen != 128) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(5, 1);
> -                VLOG_WARN_RL(&rl, "bad next hop mask %s", route->nexthop);
> -                return;
> -            }
> -            is_ipv4 = false;
> -        } else {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad next hop ip address %s",
> route->nexthop);
> -            free(error);
> -            return;
> -        }
> -    }
> -
> -    char *prefix_s;
> -    if (is_ipv4) {
> -        ovs_be32 prefix;
> -        /* Verify that ip prefix is a valid IPv4 address. */
> -        error = ip_parse_cidr(route->ip_prefix, &prefix, &plen);
> -        if (error) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad 'ip_prefix' in static routes %s",
> -                         route->ip_prefix);
> -            free(error);
> -            return;
> -        }
> -        prefix_s = xasprintf(IP_FMT, IP_ARGS(prefix &
> be32_prefix_mask(plen)));
> -    } else {
> -        /* Verify that ip prefix is a valid IPv6 address. */
> -        struct in6_addr prefix;
> -        error = ipv6_parse_cidr(route->ip_prefix, &prefix, &plen);
> -        if (error) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad 'ip_prefix' in static routes %s",
> -                         route->ip_prefix);
> -            free(error);
> -            return;
> -        }
> -        struct in6_addr mask = ipv6_create_mask(plen);
> -        struct in6_addr network = ipv6_addr_bitand(&prefix, &mask);
> -        prefix_s = xmalloc(INET6_ADDRSTRLEN);
> -        inet_ntop(AF_INET6, &network, prefix_s, INET6_ADDRSTRLEN);
> -    }
> -
> -    /* Find the outgoing port. */
> -    struct ovn_port *out_port = NULL;
> -    if (route->output_port) {
> -        out_port = ovn_port_find(ports, route->output_port);
> -        if (!out_port) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "Bad out port %s for static route %s",
> -                         route->output_port, route->ip_prefix);
> -            goto free_prefix_s;
> -        }
> -        lrp_addr_s = find_lrp_member_ip(out_port, route->nexthop);
> -        if (!lrp_addr_s) {
> -            /* There are no IP networks configured on the router's port
> via
> -             * which 'route->nexthop' is theoretically reachable.  But
> since
> -             * 'out_port' has been specified, we honor it by trying to
> reach
> -             * 'route->nexthop' via the first IP address of 'out_port'.
> -             * (There are cases, e.g in GCE, where each VM gets a /32 IP
> -             * address and the default gateway is still reachable from
> it.) */
> -            if (is_ipv4) {
> -                if (out_port->lrp_networks.n_ipv4_addrs) {
> -                    lrp_addr_s =
> out_port->lrp_networks.ipv4_addrs[0].addr_s;
> -                }
> -            } else {
> -                if (out_port->lrp_networks.n_ipv6_addrs) {
> -                    lrp_addr_s =
> out_port->lrp_networks.ipv6_addrs[0].addr_s;
> -                }
> -            }
> -        }
> -    } else {
> -        /* output_port is not specified, find the
> -         * router port matching the next hop. */
> -        int i;
> -        for (i = 0; i < od->nbr->n_ports; i++) {
> -            struct nbrec_logical_router_port *lrp = od->nbr->ports[i];
> -            out_port = ovn_port_find(ports, lrp->name);
> -            if (!out_port) {
> -                /* This should not happen. */
> -                continue;
> -            }
> -
> -            lrp_addr_s = find_lrp_member_ip(out_port, route->nexthop);
> -            if (lrp_addr_s) {
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!out_port || !lrp_addr_s) {
> -        /* There is no matched out port. */
> -        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -        VLOG_WARN_RL(&rl, "No path for static route %s; next hop %s",
> -                     route->ip_prefix, route->nexthop);
> -        goto free_prefix_s;
> -    }
> -
> -    char *policy = route->policy ? route->policy : "dst-ip";
> -    add_route(lflows, out_port, lrp_addr_s, prefix_s, plen,
> route->nexthop,
> -              policy);
> -
> -free_prefix_s:
> -    free(prefix_s);
> -}
> -
> -static void
> -op_put_v4_networks(struct ds *ds, const struct ovn_port *op, bool
> add_bcast)
> -{
> -    if (!add_bcast && op->lrp_networks.n_ipv4_addrs == 1) {
> -        ds_put_format(ds, "%s", op->lrp_networks.ipv4_addrs[0].addr_s);
> -        return;
> -    }
> -
> -    ds_put_cstr(ds, "{");
> -    for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -        ds_put_format(ds, "%s, ", op->lrp_networks.ipv4_addrs[i].addr_s);
> -        if (add_bcast) {
> -            ds_put_format(ds, "%s, ",
> op->lrp_networks.ipv4_addrs[i].bcast_s);
> -        }
> -    }
> -    ds_chomp(ds, ' ');
> -    ds_chomp(ds, ',');
> -    ds_put_cstr(ds, "}");
> -}
> -
> -static void
> -op_put_v6_networks(struct ds *ds, const struct ovn_port *op)
> -{
> -    if (op->lrp_networks.n_ipv6_addrs == 1) {
> -        ds_put_format(ds, "%s", op->lrp_networks.ipv6_addrs[0].addr_s);
> -        return;
> -    }
> -
> -    ds_put_cstr(ds, "{");
> -    for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -        ds_put_format(ds, "%s, ", op->lrp_networks.ipv6_addrs[i].addr_s);
> -    }
> -    ds_chomp(ds, ' ');
> -    ds_chomp(ds, ',');
> -    ds_put_cstr(ds, "}");
> -}
> -
> -static const char *
> -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, ovs_be32
> *ip)
> -{
> -    char *key = xasprintf("%s_force_snat_ip", key_type);
> -    const char *ip_address = smap_get(&od->nbr->options, key);
> -    free(key);
> -
> -    if (ip_address) {
> -        ovs_be32 mask;
> -        char *error = ip_parse_masked(ip_address, ip, &mask);
> -        if (error || mask != OVS_BE32_MAX) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"",
> -                         ip_address, UUID_ARGS(&od->key));
> -            free(error);
> -            *ip = 0;
> -            return NULL;
> -        }
> -        return ip_address;
> -    }
> -
> -    *ip = 0;
> -    return NULL;
> -}
> -
> -static void
> -add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od,
> -                   struct ds *match, struct ds *actions, int priority,
> -                   const char *lb_force_snat_ip, char *backend_ips,
> -                   bool is_udp, int addr_family)
> -{
> -    /* A match and actions for new connections. */
> -    char *new_match = xasprintf("ct.new && %s", ds_cstr(match));
> -    if (lb_force_snat_ip) {
> -        char *new_actions = xasprintf("flags.force_snat_for_lb = 1; %s",
> -                                      ds_cstr(actions));
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, priority, new_match,
> -                      new_actions);
> -        free(new_actions);
> -    } else {
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, priority, new_match,
> -                      ds_cstr(actions));
> -    }
> -
> -    /* A match and actions for established connections. */
> -    char *est_match = xasprintf("ct.est && %s", ds_cstr(match));
> -    if (lb_force_snat_ip) {
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, priority, est_match,
> -                      "flags.force_snat_for_lb = 1; ct_dnat;");
> -    } else {
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, priority, est_match,
> -                      "ct_dnat;");
> -    }
> -
> -    free(new_match);
> -    free(est_match);
> -
> -    if (!od->l3dgw_port || !od->l3redirect_port || !backend_ips) {
> -        return;
> -    }
> -
> -    /* Add logical flows to UNDNAT the load balanced reverse traffic in
> -     * the router egress pipleine stage - S_ROUTER_OUT_UNDNAT if the
> logical
> -     * router has a gateway router port associated.
> -     */
> -    struct ds undnat_match = DS_EMPTY_INITIALIZER;
> -    if (addr_family == AF_INET) {
> -        ds_put_cstr(&undnat_match, "ip4 && (");
> -    } else {
> -        ds_put_cstr(&undnat_match, "ip6 && (");
> -    }
> -    char *start, *next, *ip_str;
> -    start = next = xstrdup(backend_ips);
> -    ip_str = strsep(&next, ",");
> -    bool backend_ips_found = false;
> -    while (ip_str && ip_str[0]) {
> -        char *ip_address = NULL;
> -        uint16_t port = 0;
> -        int addr_family_;
> -        ip_address_and_port_from_lb_key(ip_str, &ip_address, &port,
> -                                        &addr_family_);
> -        if (!ip_address) {
> -            break;
> -        }
> -
> -        if (addr_family_ == AF_INET) {
> -            ds_put_format(&undnat_match, "(ip4.src == %s", ip_address);
> -        } else {
> -            ds_put_format(&undnat_match, "(ip6.src == %s", ip_address);
> -        }
> -        free(ip_address);
> -        if (port) {
> -            ds_put_format(&undnat_match, " && %s.src == %d) || ",
> -                          is_udp ? "udp" : "tcp", port);
> -        } else {
> -            ds_put_cstr(&undnat_match, ") || ");
> -        }
> -        ip_str = strsep(&next, ",");
> -        backend_ips_found = true;
> -    }
> -
> -    free(start);
> -    if (!backend_ips_found) {
> -        ds_destroy(&undnat_match);
> -        return;
> -    }
> -    ds_chomp(&undnat_match, ' ');
> -    ds_chomp(&undnat_match, '|');
> -    ds_chomp(&undnat_match, '|');
> -    ds_chomp(&undnat_match, ' ');
> -    ds_put_format(&undnat_match, ") && outport == %s && "
> -                 "is_chassis_resident(%s)", od->l3dgw_port->json_key,
> -                 od->l3redirect_port->json_key);
> -    if (lb_force_snat_ip) {
> -        ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 120,
> -                      ds_cstr(&undnat_match),
> -                      "flags.force_snat_for_lb = 1; ct_dnat;");
> -    } else {
> -        ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 120,
> -                      ds_cstr(&undnat_match), "ct_dnat;");
> -    }
> -
> -    ds_destroy(&undnat_match);
> -}
> -
> -#define ND_RA_MAX_INTERVAL_MAX 1800
> -#define ND_RA_MAX_INTERVAL_MIN 4
> -
> -#define ND_RA_MIN_INTERVAL_MAX(max) ((max) * 3 / 4)
> -#define ND_RA_MIN_INTERVAL_MIN 3
> -
> -static void
> -copy_ra_to_sb(struct ovn_port *op, const char *address_mode)
> -{
> -    struct smap options;
> -    smap_clone(&options, &op->sb->options);
> -
> -    smap_add(&options, "ipv6_ra_send_periodic", "true");
> -    smap_add(&options, "ipv6_ra_address_mode", address_mode);
> -
> -    int max_interval = smap_get_int(&op->nbrp->ipv6_ra_configs,
> -            "max_interval", ND_RA_MAX_INTERVAL_DEFAULT);
> -    if (max_interval > ND_RA_MAX_INTERVAL_MAX) {
> -        max_interval = ND_RA_MAX_INTERVAL_MAX;
> -    }
> -    if (max_interval < ND_RA_MAX_INTERVAL_MIN) {
> -        max_interval = ND_RA_MAX_INTERVAL_MIN;
> -    }
> -    smap_add_format(&options, "ipv6_ra_max_interval", "%d", max_interval);
> -
> -    int min_interval = smap_get_int(&op->nbrp->ipv6_ra_configs,
> -            "min_interval", nd_ra_min_interval_default(max_interval));
> -    if (min_interval > ND_RA_MIN_INTERVAL_MAX(max_interval)) {
> -        min_interval = ND_RA_MIN_INTERVAL_MAX(max_interval);
> -    }
> -    if (min_interval < ND_RA_MIN_INTERVAL_MIN) {
> -        min_interval = ND_RA_MIN_INTERVAL_MIN;
> -    }
> -    smap_add_format(&options, "ipv6_ra_min_interval", "%d", min_interval);
> -
> -    int mtu = smap_get_int(&op->nbrp->ipv6_ra_configs, "mtu",
> ND_MTU_DEFAULT);
> -    /* RFC 2460 requires the MTU for IPv6 to be at least 1280 */
> -    if (mtu && mtu >= 1280) {
> -        smap_add_format(&options, "ipv6_ra_mtu", "%d", mtu);
> -    }
> -
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; ++i) {
> -        struct ipv6_netaddr *addrs = &op->lrp_networks.ipv6_addrs[i];
> -        if (in6_is_lla(&addrs->network)) {
> -            smap_add(&options, "ipv6_ra_src_addr", addrs->addr_s);
> -            continue;
> -        }
> -        ds_put_format(&s, "%s/%u ", addrs->network_s, addrs->plen);
> -    }
> -    /* Remove trailing space */
> -    ds_chomp(&s, ' ');
> -    smap_add(&options, "ipv6_ra_prefixes", ds_cstr(&s));
> -    ds_destroy(&s);
> -
> -    smap_add(&options, "ipv6_ra_src_eth", op->lrp_networks.ea_s);
> -
> -    sbrec_port_binding_set_options(op->sb, &options);
> -    smap_destroy(&options);
> -}
> -
> -static void
> -build_lrouter_flows(struct hmap *datapaths, struct hmap *ports,
> -                    struct hmap *lflows)
> -{
> -    /* This flow table structure is documented in ovn-northd(8), so please
> -     * update ovn-northd.8.xml if you change anything. */
> -
> -    struct ds match = DS_EMPTY_INITIALIZER;
> -    struct ds actions = DS_EMPTY_INITIALIZER;
> -
> -    /* Logical router ingress table 0: Admission control framework. */
> -    struct ovn_datapath *od;
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        /* Logical VLANs not supported.
> -         * Broadcast/multicast source address is invalid. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 100,
> -                      "vlan.present || eth.src[40]", "drop;");
> -    }
> -
> -    /* Logical router ingress table 0: match (priority 50). */
> -    struct ovn_port *op;
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp) {
> -            continue;
> -        }
> -
> -        if (!lrport_is_enabled(op->nbrp)) {
> -            /* Drop packets from disabled logical ports (since logical
> flow
> -             * tables are default-drop). */
> -            continue;
> -        }
> -
> -        if (op->derived) {
> -            /* No ingress packets should be received on a chassisredirect
> -             * port. */
> -            continue;
> -        }
> -
> -        ds_clear(&match);
> -        ds_put_format(&match, "eth.mcast && inport == %s", op->json_key);
> -        ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ADMISSION, 50,
> -                      ds_cstr(&match), "next;");
> -
> -        ds_clear(&match);
> -        ds_put_format(&match, "eth.dst == %s && inport == %s",
> -                      op->lrp_networks.ea_s, op->json_key);
> -        if (op->od->l3dgw_port && op == op->od->l3dgw_port
> -            && op->od->l3redirect_port) {
> -            /* Traffic with eth.dst = l3dgw_port->lrp_networks.ea_s
> -             * should only be received on the "redirect-chassis". */
> -            ds_put_format(&match, " && is_chassis_resident(%s)",
> -                          op->od->l3redirect_port->json_key);
> -        }
> -        ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ADMISSION, 50,
> -                      ds_cstr(&match), "next;");
> -    }
> -
> -    /* Logical router ingress table 1: IP Input. */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        /* L3 admission control: drop multicast and broadcast source,
> localhost
> -         * source or destination, and zero network source or destination
> -         * (priority 100). */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 100,
> -                      "ip4.mcast || "
> -                      "ip4.src == 255.255.255.255 || "
> -                      "ip4.src == 127.0.0.0/8 || "
> -                      "ip4.dst == 127.0.0.0/8 || "
> -                      "ip4.src == 0.0.0.0/8 || "
> -                      "ip4.dst == 0.0.0.0/8",
> -                      "drop;");
> -
> -        /* ARP reply handling.  Use ARP replies to populate the logical
> -         * router's ARP table. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 90, "arp.op == 2",
> -                      "put_arp(inport, arp.spa, arp.sha);");
> -
> -        /* Drop Ethernet local broadcast.  By definition this traffic
> should
> -         * not be forwarded.*/
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 50,
> -                      "eth.bcast", "drop;");
> -
> -        /* TTL discard */
> -        ds_clear(&match);
> -        ds_put_cstr(&match, "ip4 && ip.ttl == {0, 1}");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 30,
> -                      ds_cstr(&match), "drop;");
> -
> -        /* ND advertisement handling.  Use advertisements to populate
> -         * the logical router's ARP/ND table. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 90, "nd_na",
> -                      "put_nd(inport, nd.target, nd.tll);");
> -
> -        /* Lean from neighbor solicitations that were not directed at
> -         * us.  (A priority-90 flow will respond to requests to us and
> -         * learn the sender's mac address. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 80, "nd_ns",
> -                      "put_nd(inport, ip6.src, nd.sll);");
> -
> -        /* Pass other traffic not already handled to the next table for
> -         * routing. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;");
> -    }
> -
> -    /* Logical router ingress table 1: IP Input for IPv4. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp) {
> -            continue;
> -        }
> -
> -        if (op->derived) {
> -            /* No ingress packets are accepted on a chassisredirect
> -             * port, so no need to program flows for that port. */
> -            continue;
> -        }
> -
> -        if (op->lrp_networks.n_ipv4_addrs) {
> -            /* L3 admission control: drop packets that originate from an
> -             * IPv4 address owned by the router or a broadcast address
> -             * known to the router (priority 100). */
> -            ds_clear(&match);
> -            ds_put_cstr(&match, "ip4.src == ");
> -            op_put_v4_networks(&match, op, true);
> -            ds_put_cstr(&match, " && "REGBIT_EGRESS_LOOPBACK" == 0");
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100,
> -                          ds_cstr(&match), "drop;");
> -
> -            /* ICMP echo reply.  These flows reply to ICMP echo requests
> -             * received for the router's IP address. Since packets only
> -             * get here as part of the logical router datapath, the inport
> -             * (i.e. the incoming locally attached net) does not matter.
> -             * The ip.ttl also does not matter (RFC1812 section 4.2.2.9)
> */
> -            ds_clear(&match);
> -            ds_put_cstr(&match, "ip4.dst == ");
> -            op_put_v4_networks(&match, op, false);
> -            ds_put_cstr(&match, " && icmp4.type == 8 && icmp4.code == 0");
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                "ip4.dst <-> ip4.src; "
> -                "ip.ttl = 255; "
> -                "icmp4.type = 0; "
> -                "flags.loopback = 1; "
> -                "next; ");
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        /* ICMP time exceeded */
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            ds_clear(&match);
> -            ds_clear(&actions);
> -
> -            ds_put_format(&match,
> -                          "inport == %s && ip4 && "
> -                          "ip.ttl == {0, 1} && !ip.later_frag",
> op->json_key);
> -            ds_put_format(&actions,
> -                          "icmp4 {"
> -                          "eth.dst <-> eth.src; "
> -                          "icmp4.type = 11; /* Time exceeded */ "
> -                          "icmp4.code = 0; /* TTL exceeded in transit */ "
> -                          "ip4.dst = ip4.src; "
> -                          "ip4.src = %s; "
> -                          "ip.ttl = 255; "
> -                          "next; };",
> -                          op->lrp_networks.ipv4_addrs[i].addr_s);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 40,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        /* ARP reply.  These flows reply to ARP requests for the router's
> own
> -         * IP address. */
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            ds_clear(&match);
> -            ds_put_format(&match,
> -                          "inport == %s && arp.spa == %s/%u && arp.tpa ==
> %s"
> -                          " && arp.op == 1",
> -                          op->json_key,
> -                          op->lrp_networks.ipv4_addrs[i].network_s,
> -                          op->lrp_networks.ipv4_addrs[i].plen,
> -                          op->lrp_networks.ipv4_addrs[i].addr_s);
> -
> -            if (op->od->l3dgw_port && op->od->l3redirect_port && op->peer
> -                && op->peer->od->localnet_port) {
> -                bool add_chassis_resident_check = false;
> -                if (op == op->od->l3dgw_port) {
> -                    /* Traffic with eth.src =
> l3dgw_port->lrp_networks.ea_s
> -                     * should only be sent from the "redirect-chassis",
> so that
> -                     * upstream MAC learning points to the
> "redirect-chassis".
> -                     * Also need to avoid generation of multiple ARP
> responses
> -                     * from different chassis. */
> -                    add_chassis_resident_check = true;
> -                } else {
> -                    /* Check if the option 'reside-on-redirect-chassis'
> -                     * is set to true on the router port. If set to true
> -                     * and if peer's logical switch has a localnet port,
> it
> -                     * means the router pipeline for the packets from
> -                     * peer's logical switch is be run on the chassis
> -                     * hosting the gateway port and it should reply to the
> -                     * ARP requests for the router port IPs.
> -                     */
> -                    add_chassis_resident_check = smap_get_bool(
> -                        &op->nbrp->options,
> -                        "reside-on-redirect-chassis", false);
> -                }
> -
> -                if (add_chassis_resident_check) {
> -                    ds_put_format(&match, " && is_chassis_resident(%s)",
> -                                  op->od->l3redirect_port->json_key);
> -                }
> -            }
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                "put_arp(inport, arp.spa, arp.sha); "
> -                "eth.dst = eth.src; "
> -                "eth.src = %s; "
> -                "arp.op = 2; /* ARP reply */ "
> -                "arp.tha = arp.sha; "
> -                "arp.sha = %s; "
> -                "arp.tpa = arp.spa; "
> -                "arp.spa = %s; "
> -                "outport = %s; "
> -                "flags.loopback = 1; "
> -                "output;",
> -                op->lrp_networks.ea_s,
> -                op->lrp_networks.ea_s,
> -                op->lrp_networks.ipv4_addrs[i].addr_s,
> -                op->json_key);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        /* Learn from ARP requests that were not directed at us. A typical
> -         * use case is GARP request handling.  (A priority-90 flow will
> -         * respond to request to us and learn the sender's mac address.)
> */
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            ds_clear(&match);
> -            ds_put_format(&match,
> -                          "inport == %s && arp.spa == %s/%u && arp.op ==
> 1",
> -                          op->json_key,
> -                          op->lrp_networks.ipv4_addrs[i].network_s,
> -                          op->lrp_networks.ipv4_addrs[i].plen);
> -            if (op->od->l3dgw_port && op == op->od->l3dgw_port
> -                && op->od->l3redirect_port) {
> -                ds_put_format(&match, " && is_chassis_resident(%s)",
> -                              op->od->l3redirect_port->json_key);
> -            }
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
> -                          ds_cstr(&match),
> -                          "put_arp(inport, arp.spa, arp.sha);");
> -
> -        }
> -
> -        /* A set to hold all load-balancer vips that need ARP responses.
> */
> -        struct sset all_ips = SSET_INITIALIZER(&all_ips);
> -        int addr_family;
> -        get_router_load_balancer_ips(op->od, &all_ips, &addr_family);
> -
> -        const char *ip_address;
> -        SSET_FOR_EACH(ip_address, &all_ips) {
> -            ds_clear(&match);
> -            if (addr_family == AF_INET) {
> -                ds_put_format(&match,
> -                              "inport == %s && arp.tpa == %s && arp.op ==
> 1",
> -                              op->json_key, ip_address);
> -            } else {
> -                ds_put_format(&match,
> -                              "inport == %s && nd_ns && nd.target == %s",
> -                              op->json_key, ip_address);
> -            }
> -
> -            ds_clear(&actions);
> -            if (addr_family == AF_INET) {
> -                ds_put_format(&actions,
> -                "eth.dst = eth.src; "
> -                "eth.src = %s; "
> -                "arp.op = 2; /* ARP reply */ "
> -                "arp.tha = arp.sha; "
> -                "arp.sha = %s; "
> -                "arp.tpa = arp.spa; "
> -                "arp.spa = %s; "
> -                "outport = %s; "
> -                "flags.loopback = 1; "
> -                "output;",
> -                op->lrp_networks.ea_s,
> -                op->lrp_networks.ea_s,
> -                ip_address,
> -                op->json_key);
> -            } else {
> -                ds_put_format(&actions,
> -                "nd_na { "
> -                "eth.src = %s; "
> -                "ip6.src = %s; "
> -                "nd.target = %s; "
> -                "nd.tll = %s; "
> -                "outport = inport; "
> -                "flags.loopback = 1; "
> -                "output; "
> -                "};",
> -                op->lrp_networks.ea_s,
> -                ip_address,
> -                ip_address,
> -                op->lrp_networks.ea_s);
> -            }
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        sset_destroy(&all_ips);
> -
> -        /* A gateway router can have 2 SNAT IP addresses to force DNATed
> and
> -         * LBed traffic respectively to be SNATed.  In addition, there
> can be
> -         * a number of SNAT rules in the NAT table. */
> -        ovs_be32 *snat_ips = xmalloc(sizeof *snat_ips *
> -                                     (op->od->nbr->n_nat + 2));
> -        size_t n_snat_ips = 0;
> -
> -        ovs_be32 snat_ip;
> -        const char *dnat_force_snat_ip = get_force_snat_ip(op->od, "dnat",
> -                                                           &snat_ip);
> -        if (dnat_force_snat_ip) {
> -            snat_ips[n_snat_ips++] = snat_ip;
> -        }
> -
> -        const char *lb_force_snat_ip = get_force_snat_ip(op->od, "lb",
> -                                                         &snat_ip);
> -        if (lb_force_snat_ip) {
> -            snat_ips[n_snat_ips++] = snat_ip;
> -        }
> -
> -        for (int i = 0; i < op->od->nbr->n_nat; i++) {
> -            const struct nbrec_nat *nat;
> -
> -            nat = op->od->nbr->nat[i];
> -
> -            ovs_be32 ip;
> -            if (!ip_parse(nat->external_ip, &ip) || !ip) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(5, 1);
> -                VLOG_WARN_RL(&rl, "bad ip address %s in nat configuration
> "
> -                             "for router %s", nat->external_ip, op->key);
> -                continue;
> -            }
> -
> -            if (!strcmp(nat->type, "snat")) {
> -                snat_ips[n_snat_ips++] = ip;
> -                continue;
> -            }
> -
> -            /* ARP handling for external IP addresses.
> -             *
> -             * DNAT IP addresses are external IP addresses that need ARP
> -             * handling. */
> -            ds_clear(&match);
> -            ds_put_format(&match,
> -                          "inport == %s && arp.tpa == "IP_FMT" && arp.op
> == 1",
> -                          op->json_key, IP_ARGS(ip));
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                "eth.dst = eth.src; "
> -                "arp.op = 2; /* ARP reply */ "
> -                "arp.tha = arp.sha; ");
> -
> -            if (op->od->l3dgw_port && op == op->od->l3dgw_port) {
> -                struct eth_addr mac;
> -                if (nat->external_mac &&
> -                    eth_addr_from_string(nat->external_mac, &mac)
> -                    && nat->logical_port) {
> -                    /* distributed NAT case, use nat->external_mac */
> -                    ds_put_format(&actions,
> -                        "eth.src = "ETH_ADDR_FMT"; "
> -                        "arp.sha = "ETH_ADDR_FMT"; ",
> -                        ETH_ADDR_ARGS(mac),
> -                        ETH_ADDR_ARGS(mac));
> -                    /* Traffic with eth.src = nat->external_mac should
> only be
> -                     * sent from the chassis where nat->logical_port is
> -                     * resident, so that upstream MAC learning points to
> the
> -                     * correct chassis.  Also need to avoid generation of
> -                     * multiple ARP responses from different chassis. */
> -                    ds_put_format(&match, " &&
> is_chassis_resident(\"%s\")",
> -                                  nat->logical_port);
> -                } else {
> -                    ds_put_format(&actions,
> -                        "eth.src = %s; "
> -                        "arp.sha = %s; ",
> -                        op->lrp_networks.ea_s,
> -                        op->lrp_networks.ea_s);
> -                    /* Traffic with eth.src =
> l3dgw_port->lrp_networks.ea_s
> -                     * should only be sent from the "redirect-chassis",
> so that
> -                     * upstream MAC learning points to the
> "redirect-chassis".
> -                     * Also need to avoid generation of multiple ARP
> responses
> -                     * from different chassis. */
> -                    if (op->od->l3redirect_port) {
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      op->od->l3redirect_port->json_key);
> -                    }
> -                }
> -            } else {
> -                ds_put_format(&actions,
> -                    "eth.src = %s; "
> -                    "arp.sha = %s; ",
> -                    op->lrp_networks.ea_s,
> -                    op->lrp_networks.ea_s);
> -            }
> -            ds_put_format(&actions,
> -                "arp.tpa = arp.spa; "
> -                "arp.spa = "IP_FMT"; "
> -                "outport = %s; "
> -                "flags.loopback = 1; "
> -                "output;",
> -                IP_ARGS(ip),
> -                op->json_key);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        if (!smap_get(&op->od->nbr->options, "chassis")
> -            && !op->od->l3dgw_port) {
> -            /* UDP/TCP port unreachable. */
> -            for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip4 && ip4.dst == %s && !ip.later_frag &&
> udp",
> -                              op->lrp_networks.ipv4_addrs[i].addr_s);
> -                const char *action = "icmp4 {"
> -                                     "eth.dst <-> eth.src; "
> -                                     "ip4.dst <-> ip4.src; "
> -                                     "ip.ttl = 255; "
> -                                     "icmp4.type = 3; "
> -                                     "icmp4.code = 3; "
> -                                     "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
> -                              ds_cstr(&match), action);
> -
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip4 && ip4.dst == %s && !ip.later_frag &&
> tcp",
> -                              op->lrp_networks.ipv4_addrs[i].addr_s);
> -                action = "tcp_reset {"
> -                         "eth.dst <-> eth.src; "
> -                         "ip4.dst <-> ip4.src; "
> -                         "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
> -                              ds_cstr(&match), action);
> -
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip4 && ip4.dst == %s && !ip.later_frag",
> -                              op->lrp_networks.ipv4_addrs[i].addr_s);
> -                action = "icmp4 {"
> -                         "eth.dst <-> eth.src; "
> -                         "ip4.dst <-> ip4.src; "
> -                         "ip.ttl = 255; "
> -                         "icmp4.type = 3; "
> -                         "icmp4.code = 2; "
> -                         "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 70,
> -                              ds_cstr(&match), action);
> -            }
> -        }
> -
> -        ds_clear(&match);
> -        ds_put_cstr(&match, "ip4.dst == {");
> -        bool has_drop_ips = false;
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            bool snat_ip_is_router_ip = false;
> -            for (int j = 0; j < n_snat_ips; j++) {
> -                /* Packets to SNAT IPs should not be dropped. */
> -                if (op->lrp_networks.ipv4_addrs[i].addr == snat_ips[j]) {
> -                    snat_ip_is_router_ip = true;
> -                    break;
> -                }
> -            }
> -            if (snat_ip_is_router_ip) {
> -                continue;
> -            }
> -            ds_put_format(&match, "%s, ",
> -                          op->lrp_networks.ipv4_addrs[i].addr_s);
> -            has_drop_ips = true;
> -        }
> -        ds_chomp(&match, ' ');
> -        ds_chomp(&match, ',');
> -        ds_put_cstr(&match, "}");
> -
> -        if (has_drop_ips) {
> -            /* Drop IP traffic to this router. */
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 60,
> -                          ds_cstr(&match), "drop;");
> -        }
> -
> -        free(snat_ips);
> -    }
> -
> -    /* Logical router ingress table 1: IP Input for IPv6. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp) {
> -            continue;
> -        }
> -
> -        if (op->derived) {
> -            /* No ingress packets are accepted on a chassisredirect
> -             * port, so no need to program flows for that port. */
> -            continue;
> -        }
> -
> -        if (op->lrp_networks.n_ipv6_addrs) {
> -            /* L3 admission control: drop packets that originate from an
> -             * IPv6 address owned by the router (priority 100). */
> -            ds_clear(&match);
> -            ds_put_cstr(&match, "ip6.src == ");
> -            op_put_v6_networks(&match, op);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100,
> -                          ds_cstr(&match), "drop;");
> -
> -            /* ICMPv6 echo reply.  These flows reply to echo requests
> -             * received for the router's IP address. */
> -            ds_clear(&match);
> -            ds_put_cstr(&match, "ip6.dst == ");
> -            op_put_v6_networks(&match, op);
> -            ds_put_cstr(&match, " && icmp6.type == 128 && icmp6.code ==
> 0");
> -
> -            ds_clear(&actions);
> -            ds_put_cstr(&actions,
> -                        "ip6.dst <-> ip6.src; "
> -                        "ip.ttl = 255; "
> -                        "icmp6.type = 129; "
> -                        "flags.loopback = 1; "
> -                        "next; ");
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -
> -            /* Drop IPv6 traffic to this router. */
> -            ds_clear(&match);
> -            ds_put_cstr(&match, "ip6.dst == ");
> -            op_put_v6_networks(&match, op);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 60,
> -                          ds_cstr(&match), "drop;");
> -        }
> -
> -        /* ND reply.  These flows reply to ND solicitations for the
> -         * router's own IP address. */
> -        for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -            ds_clear(&match);
> -            ds_put_format(&match,
> -                    "inport == %s && nd_ns && ip6.dst == {%s, %s} "
> -                    "&& nd.target == %s",
> -                    op->json_key,
> -                    op->lrp_networks.ipv6_addrs[i].addr_s,
> -                    op->lrp_networks.ipv6_addrs[i].sn_addr_s,
> -                    op->lrp_networks.ipv6_addrs[i].addr_s);
> -            if (op->od->l3dgw_port && op == op->od->l3dgw_port
> -                && op->od->l3redirect_port) {
> -                /* Traffic with eth.src = l3dgw_port->lrp_networks.ea_s
> -                 * should only be sent from the "redirect-chassis", so
> that
> -                 * upstream MAC learning points to the "redirect-chassis".
> -                 * Also need to avoid generation of multiple ND replies
> -                 * from different chassis. */
> -                ds_put_format(&match, " && is_chassis_resident(%s)",
> -                              op->od->l3redirect_port->json_key);
> -            }
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                          "put_nd(inport, ip6.src, nd.sll); "
> -                          "nd_na_router { "
> -                          "eth.src = %s; "
> -                          "ip6.src = %s; "
> -                          "nd.target = %s; "
> -                          "nd.tll = %s; "
> -                          "outport = inport; "
> -                          "flags.loopback = 1; "
> -                          "output; "
> -                          "};",
> -                          op->lrp_networks.ea_s,
> -                          op->lrp_networks.ipv6_addrs[i].addr_s,
> -                          op->lrp_networks.ipv6_addrs[i].addr_s,
> -                          op->lrp_networks.ea_s);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        /* UDP/TCP port unreachable */
> -        if (!smap_get(&op->od->nbr->options, "chassis")
> -            && !op->od->l3dgw_port) {
> -            for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip6 && ip6.dst == %s && !ip.later_frag &&
> tcp",
> -                              op->lrp_networks.ipv6_addrs[i].addr_s);
> -                const char *action = "tcp_reset {"
> -                                     "eth.dst <-> eth.src; "
> -                                     "ip6.dst <-> ip6.src; "
> -                                     "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
> -                          ds_cstr(&match), action);
> -
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip6 && ip6.dst == %s && !ip.later_frag &&
> udp",
> -                              op->lrp_networks.ipv6_addrs[i].addr_s);
> -                action = "icmp6 {"
> -                         "eth.dst <-> eth.src; "
> -                         "ip6.dst <-> ip6.src; "
> -                         "ip.ttl = 255; "
> -                         "icmp6.type = 1; "
> -                         "icmp6.code = 4; "
> -                         "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
> -                              ds_cstr(&match), action);
> -
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "ip6 && ip6.dst == %s && !ip.later_frag",
> -                              op->lrp_networks.ipv6_addrs[i].addr_s);
> -                action = "icmp6 {"
> -                         "eth.dst <-> eth.src; "
> -                         "ip6.dst <-> ip6.src; "
> -                         "ip.ttl = 255; "
> -                         "icmp6.type = 1; "
> -                         "icmp6.code = 3; "
> -                         "next; };";
> -                ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 70,
> -                              ds_cstr(&match), action);
> -            }
> -        }
> -
> -        /* ICMPv6 time exceeded */
> -        for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -            /* skip link-local address */
> -            if (in6_is_lla(&op->lrp_networks.ipv6_addrs[i].network)) {
> -                continue;
> -            }
> -
> -            ds_clear(&match);
> -            ds_clear(&actions);
> -
> -            ds_put_format(&match,
> -                          "inport == %s && ip6 && "
> -                          "ip6.src == %s/%d && "
> -                          "ip.ttl == {0, 1} && !ip.later_frag",
> -                          op->json_key,
> -                          op->lrp_networks.ipv6_addrs[i].network_s,
> -                          op->lrp_networks.ipv6_addrs[i].plen);
> -            ds_put_format(&actions,
> -                          "icmp6 {"
> -                          "eth.dst <-> eth.src; "
> -                          "ip6.dst = ip6.src; "
> -                          "ip6.src = %s; "
> -                          "ip.ttl = 255; "
> -                          "icmp6.type = 3; /* Time exceeded */ "
> -                          "icmp6.code = 0; /* TTL exceeded in transit */ "
> -                          "next; };",
> -                          op->lrp_networks.ipv6_addrs[i].addr_s);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 40,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -    }
> -
> -    /* NAT, Defrag and load balancing. */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        /* Packets are allowed by default. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;");
> -
> -        /* NAT rules are only valid on Gateway routers and routers with
> -         * l3dgw_port (router has a port with "redirect-chassis"
> -         * specified). */
> -        if (!smap_get(&od->nbr->options, "chassis") && !od->l3dgw_port) {
> -            continue;
> -        }
> -
> -        ovs_be32 snat_ip;
> -        const char *dnat_force_snat_ip = get_force_snat_ip(od, "dnat",
> -                                                           &snat_ip);
> -        const char *lb_force_snat_ip = get_force_snat_ip(od, "lb",
> -                                                         &snat_ip);
> -
> -        for (int i = 0; i < od->nbr->n_nat; i++) {
> -            const struct nbrec_nat *nat;
> -
> -            nat = od->nbr->nat[i];
> -
> -            ovs_be32 ip, mask;
> -
> -            char *error = ip_parse_masked(nat->external_ip, &ip, &mask);
> -            if (error || mask != OVS_BE32_MAX) {
> -                static struct vlog_rate_limit rl =
> VLOG_RATE_LIMIT_INIT(5, 1);
> -                VLOG_WARN_RL(&rl, "bad external ip %s for nat",
> -                             nat->external_ip);
> -                free(error);
> -                continue;
> -            }
> -
> -            /* Check the validity of nat->logical_ip. 'logical_ip' can
> -             * be a subnet when the type is "snat". */
> -            error = ip_parse_masked(nat->logical_ip, &ip, &mask);
> -            if (!strcmp(nat->type, "snat")) {
> -                if (error) {
> -                    static struct vlog_rate_limit rl =
> -                        VLOG_RATE_LIMIT_INIT(5, 1);
> -                    VLOG_WARN_RL(&rl, "bad ip network or ip %s for snat "
> -                                 "in router "UUID_FMT"",
> -                                 nat->logical_ip, UUID_ARGS(&od->key));
> -                    free(error);
> -                    continue;
> -                }
> -            } else {
> -                if (error || mask != OVS_BE32_MAX) {
> -                    static struct vlog_rate_limit rl =
> -                        VLOG_RATE_LIMIT_INIT(5, 1);
> -                    VLOG_WARN_RL(&rl, "bad ip %s for dnat in router "
> -                        ""UUID_FMT"", nat->logical_ip,
> UUID_ARGS(&od->key));
> -                    free(error);
> -                    continue;
> -                }
> -            }
> -
> -            /* For distributed router NAT, determine whether this NAT rule
> -             * satisfies the conditions for distributed NAT processing. */
> -            bool distributed = false;
> -            struct eth_addr mac;
> -            if (od->l3dgw_port && !strcmp(nat->type, "dnat_and_snat") &&
> -                nat->logical_port && nat->external_mac) {
> -                if (eth_addr_from_string(nat->external_mac, &mac)) {
> -                    distributed = true;
> -                } else {
> -                    static struct vlog_rate_limit rl =
> -                        VLOG_RATE_LIMIT_INIT(5, 1);
> -                    VLOG_WARN_RL(&rl, "bad mac %s for dnat in router "
> -                        ""UUID_FMT"", nat->external_mac,
> UUID_ARGS(&od->key));
> -                    continue;
> -                }
> -            }
> -
> -            /* Ingress UNSNAT table: It is for already established
> connections'
> -             * reverse traffic. i.e., SNAT has already been done in egress
> -             * pipeline and now the packet has entered the ingress
> pipeline as
> -             * part of a reply. We undo the SNAT here.
> -             *
> -             * Undoing SNAT has to happen before DNAT processing.  This is
> -             * because when the packet was DNATed in ingress pipeline, it
> did
> -             * not know about the possibility of eventual additional SNAT
> in
> -             * egress pipeline. */
> -            if (!strcmp(nat->type, "snat")
> -                || !strcmp(nat->type, "dnat_and_snat")) {
> -                if (!od->l3dgw_port) {
> -                    /* Gateway router. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s",
> -                                  nat->external_ip);
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 90,
> -                                  ds_cstr(&match), "ct_snat;");
> -                } else {
> -                    /* Distributed router. */
> -
> -                    /* Traffic received on l3dgw_port is subject to NAT.
> */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s"
> -                                          " && inport == %s",
> -                                  nat->external_ip,
> -                                  od->l3dgw_port->json_key);
> -                    if (!distributed && od->l3redirect_port) {
> -                        /* Flows for NAT rules that are centralized are
> only
> -                         * programmed on the "redirect-chassis". */
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      od->l3redirect_port->json_key);
> -                    }
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 100,
> -                                  ds_cstr(&match), "ct_snat;");
> -
> -                    /* Traffic received on other router ports must be
> -                     * redirected to the central instance of the
> l3dgw_port
> -                     * for NAT processing. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s",
> -                                  nat->external_ip);
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 50,
> -                                  ds_cstr(&match),
> -                                  REGBIT_NAT_REDIRECT" = 1; next;");
> -                }
> -            }
> -
> -            /* Ingress DNAT table: Packets enter the pipeline with
> destination
> -             * IP address that needs to be DNATted from a external IP
> address
> -             * to a logical IP address. */
> -            if (!strcmp(nat->type, "dnat")
> -                || !strcmp(nat->type, "dnat_and_snat")) {
> -                if (!od->l3dgw_port) {
> -                    /* Gateway router. */
> -                    /* Packet when it goes from the initiator to
> destination.
> -                     * We need to set flags.loopback because the router
> can
> -                     * send the packet back through the same interface. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s",
> -                                  nat->external_ip);
> -                    ds_clear(&actions);
> -                    if (dnat_force_snat_ip) {
> -                        /* Indicate to the future tables that a DNAT has
> taken
> -                         * place and a force SNAT needs to be done in the
> -                         * Egress SNAT table. */
> -                        ds_put_format(&actions,
> -                                      "flags.force_snat_for_dnat = 1; ");
> -                    }
> -                    ds_put_format(&actions, "flags.loopback = 1;
> ct_dnat(%s);",
> -                                  nat->logical_ip);
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 100,
> -                                  ds_cstr(&match), ds_cstr(&actions));
> -                } else {
> -                    /* Distributed router. */
> -
> -                    /* Traffic received on l3dgw_port is subject to NAT.
> */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s"
> -                                          " && inport == %s",
> -                                  nat->external_ip,
> -                                  od->l3dgw_port->json_key);
> -                    if (!distributed && od->l3redirect_port) {
> -                        /* Flows for NAT rules that are centralized are
> only
> -                         * programmed on the "redirect-chassis". */
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      od->l3redirect_port->json_key);
> -                    }
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "ct_dnat(%s);",
> -                                  nat->logical_ip);
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 100,
> -                                  ds_cstr(&match), ds_cstr(&actions));
> -
> -                    /* Traffic received on other router ports must be
> -                     * redirected to the central instance of the
> l3dgw_port
> -                     * for NAT processing. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.dst == %s",
> -                                  nat->external_ip);
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50,
> -                                  ds_cstr(&match),
> -                                  REGBIT_NAT_REDIRECT" = 1; next;");
> -                }
> -            }
> -
> -            /* Egress UNDNAT table: It is for already established
> connections'
> -             * reverse traffic. i.e., DNAT has already been done in
> ingress
> -             * pipeline and now the packet has entered the egress
> pipeline as
> -             * part of a reply. We undo the DNAT here.
> -             *
> -             * Note that this only applies for NAT on a distributed
> router.
> -             * Undo DNAT on a gateway router is done in the ingress DNAT
> -             * pipeline stage. */
> -            if (od->l3dgw_port && (!strcmp(nat->type, "dnat")
> -                || !strcmp(nat->type, "dnat_and_snat"))) {
> -                ds_clear(&match);
> -                ds_put_format(&match, "ip && ip4.src == %s"
> -                                      " && outport == %s",
> -                              nat->logical_ip,
> -                              od->l3dgw_port->json_key);
> -                if (!distributed && od->l3redirect_port) {
> -                    /* Flows for NAT rules that are centralized are only
> -                     * programmed on the "redirect-chassis". */
> -                    ds_put_format(&match, " && is_chassis_resident(%s)",
> -                                  od->l3redirect_port->json_key);
> -                }
> -                ds_clear(&actions);
> -                if (distributed) {
> -                    ds_put_format(&actions, "eth.src = "ETH_ADDR_FMT"; ",
> -                                  ETH_ADDR_ARGS(mac));
> -                }
> -                ds_put_format(&actions, "ct_dnat;");
> -                ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            }
> -
> -            /* Egress SNAT table: Packets enter the egress pipeline with
> -             * source ip address that needs to be SNATted to a external ip
> -             * address. */
> -            if (!strcmp(nat->type, "snat")
> -                || !strcmp(nat->type, "dnat_and_snat")) {
> -                if (!od->l3dgw_port) {
> -                    /* Gateway router. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.src == %s",
> -                                  nat->logical_ip);
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "ct_snat(%s);",
> nat->external_ip);
> -
> -                    /* The priority here is calculated such that the
> -                     * nat->logical_ip with the longest mask gets a higher
> -                     * priority. */
> -                    ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT,
> -                                  count_1bits(ntohl(mask)) + 1,
> -                                  ds_cstr(&match), ds_cstr(&actions));
> -                } else {
> -                    uint16_t priority = count_1bits(ntohl(mask)) + 1;
> -
> -                    /* Distributed router. */
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "ip && ip4.src == %s"
> -                                          " && outport == %s",
> -                                  nat->logical_ip,
> -                                  od->l3dgw_port->json_key);
> -                    if (!distributed && od->l3redirect_port) {
> -                        /* Flows for NAT rules that are centralized are
> only
> -                         * programmed on the "redirect-chassis". */
> -                        priority += 128;
> -                        ds_put_format(&match, " &&
> is_chassis_resident(%s)",
> -                                      od->l3redirect_port->json_key);
> -                    }
> -                    ds_clear(&actions);
> -                    if (distributed) {
> -                        ds_put_format(&actions, "eth.src =
> "ETH_ADDR_FMT"; ",
> -                                      ETH_ADDR_ARGS(mac));
> -                    }
> -                    ds_put_format(&actions, "ct_snat(%s);",
> nat->external_ip);
> -
> -                    /* The priority here is calculated such that the
> -                     * nat->logical_ip with the longest mask gets a higher
> -                     * priority. */
> -                    ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT,
> -                                  priority, ds_cstr(&match),
> -                                  ds_cstr(&actions));
> -                }
> -            }
> -
> -            /* Logical router ingress table 0:
> -             * For NAT on a distributed router, add rules allowing
> -             * ingress traffic with eth.dst matching nat->external_mac
> -             * on the l3dgw_port instance where nat->logical_port is
> -             * resident. */
> -            if (distributed) {
> -                ds_clear(&match);
> -                ds_put_format(&match,
> -                              "eth.dst == "ETH_ADDR_FMT" && inport == %s"
> -                              " && is_chassis_resident(\"%s\")",
> -                              ETH_ADDR_ARGS(mac),
> -                              od->l3dgw_port->json_key,
> -                              nat->logical_port);
> -                ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 50,
> -                              ds_cstr(&match), "next;");
> -            }
> -
> -            /* Ingress Gateway Redirect Table: For NAT on a distributed
> -             * router, add flows that are specific to a NAT rule.  These
> -             * flows indicate the presence of an applicable NAT rule that
> -             * can be applied in a distributed manner. */
> -            if (distributed) {
> -                ds_clear(&match);
> -                ds_put_format(&match, "ip4.src == %s && outport == %s",
> -                              nat->logical_ip,
> -                              od->l3dgw_port->json_key);
> -                ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100,
> -                              ds_cstr(&match), "next;");
> -            }
> -
> -            /* Egress Loopback table: For NAT on a distributed router.
> -             * If packets in the egress pipeline on the distributed
> -             * gateway port have ip.dst matching a NAT external IP, then
> -             * loop a clone of the packet back to the beginning of the
> -             * ingress pipeline with inport = outport. */
> -            if (od->l3dgw_port) {
> -                /* Distributed router. */
> -                if (!strcmp(nat->type, "dnat_and_snat") &&
> -                    nat->external_mac && nat->external_ip) {
> -                    for (int j = 0; j < od->nbr->n_nat; j++) {
> -                        const struct nbrec_nat *nat2 = od->nbr->nat[j];
> -
> -                        if (nat2 == nat ||
> -                            strcmp(nat2->type, "dnat_and_snat") ||
> -                            !nat2->external_mac || !nat2->external_ip) {
> -                            continue;
> -                        }
> -
> -                        ds_clear(&match);
> -                        ds_put_format(&match,
> "is_chassis_resident(\"%s\") && "
> -                                      "ip4.src == %s && ip4.dst == %s",
> -                                      nat->logical_port,
> nat2->external_ip,
> -                                      nat->external_ip);
> -                        ds_clear(&actions);
> -                        ds_put_format(&actions,
> -                                      "inport = outport; outport = \"\"; "
> -                                      "flags = 0; flags.loopback = 1; "
> -                                      REGBIT_EGRESS_LOOPBACK" = 1; "
> -                                      "next(pipeline=ingress, table=0);
> ");
> -                        ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP,
> 300,
> -                                      ds_cstr(&match),
> ds_cstr(&actions));
> -
> -                        ds_clear(&match);
> -                        ds_put_format(&match,
> -                                      "ip4.src == %s && ip4.dst == %s",
> -                                      nat2->external_ip,
> nat->external_ip);
> -                        ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP,
> 200,
> -                                      ds_cstr(&match), "next;");
> -                        ds_clear(&match);
> -                    }
> -                }
> -
> -                ds_clear(&match);
> -                ds_put_format(&match, "ip4.dst == %s && outport == %s",
> -                              nat->external_ip,
> -                              od->l3dgw_port->json_key);
> -                ds_clear(&actions);
> -                ds_put_format(&actions,
> -                              "clone { ct_clear; "
> -                              "inport = outport; outport = \"\"; "
> -                              "flags = 0; flags.loopback = 1; ");
> -                for (int j = 0; j < MFF_N_LOG_REGS; j++) {
> -                    ds_put_format(&actions, "reg%d = 0; ", j);
> -                }
> -                ds_put_format(&actions, REGBIT_EGRESS_LOOPBACK" = 1; "
> -                              "next(pipeline=ingress, table=0); };");
> -                ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            }
> -        }
> -
> -        /* Handle force SNAT options set in the gateway router. */
> -        if (dnat_force_snat_ip && !od->l3dgw_port) {
> -            /* If a packet with destination IP address as that of the
> -             * gateway router (as set in options:dnat_force_snat_ip) is
> seen,
> -             * UNSNAT it. */
> -            ds_clear(&match);
> -            ds_put_format(&match, "ip && ip4.dst == %s",
> dnat_force_snat_ip);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110,
> -                          ds_cstr(&match), "ct_snat;");
> -
> -            /* Higher priority rules to force SNAT with the IP addresses
> -             * configured in the Gateway router.  This only takes effect
> -             * when the packet has already been DNATed once. */
> -            ds_clear(&match);
> -            ds_put_format(&match, "flags.force_snat_for_dnat == 1 && ip");
> -            ds_clear(&actions);
> -            ds_put_format(&actions, "ct_snat(%s);", dnat_force_snat_ip);
> -            ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -        if (lb_force_snat_ip && !od->l3dgw_port) {
> -            /* If a packet with destination IP address as that of the
> -             * gateway router (as set in options:lb_force_snat_ip) is
> seen,
> -             * UNSNAT it. */
> -            ds_clear(&match);
> -            ds_put_format(&match, "ip && ip4.dst == %s",
> lb_force_snat_ip);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 100,
> -                          ds_cstr(&match), "ct_snat;");
> -
> -            /* Load balanced traffic will have flags.force_snat_for_lb
> set.
> -             * Force SNAT it. */
> -            ds_clear(&match);
> -            ds_put_format(&match, "flags.force_snat_for_lb == 1 && ip");
> -            ds_clear(&actions);
> -            ds_put_format(&actions, "ct_snat(%s);", lb_force_snat_ip);
> -            ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        if (!od->l3dgw_port) {
> -            /* For gateway router, re-circulate every packet through
> -            * the DNAT zone.  This helps with the following.
> -            *
> -            * Any packet that needs to be unDNATed in the reverse
> -            * direction gets unDNATed. Ideally this could be done in
> -            * the egress pipeline. But since the gateway router
> -            * does not have any feature that depends on the source
> -            * ip address being external IP address for IP routing,
> -            * we can do it here, saving a future re-circulation. */
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50,
> -                          "ip", "flags.loopback = 1; ct_dnat;");
> -        } else {
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 400,
> -                          REGBIT_DISTRIBUTED_NAT" == 1", "next;");
> -
> -            /* For NAT on a distributed router, add flows to Ingress
> -             * IP Routing table, Ingress ARP Resolution table, and
> -             * Ingress Gateway Redirect Table that are not specific to a
> -             * NAT rule. */
> -
> -            /* The highest priority IN_IP_ROUTING rule matches packets
> -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages),
> -             * with action "ip.ttl--; next;".  The IN_GW_REDIRECT table
> -             * will take care of setting the outport. */
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 300,
> -                          REGBIT_NAT_REDIRECT" == 1", "ip.ttl--; next;");
> -
> -            /* The highest priority IN_ARP_RESOLVE rule matches packets
> -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages),
> -             * then sets eth.dst to the distributed gateway port's
> -             * ethernet address. */
> -            ds_clear(&actions);
> -            ds_put_format(&actions, "eth.dst = %s; next;",
> -                          od->l3dgw_port->lrp_networks.ea_s);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200,
> -                          REGBIT_NAT_REDIRECT" == 1", ds_cstr(&actions));
> -
> -            /* The highest priority IN_GW_REDIRECT rule redirects packets
> -             * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages) to
> -             * the central instance of the l3dgw_port for NAT processing.
> */
> -            ds_clear(&actions);
> -            ds_put_format(&actions, "outport = %s; next;",
> -                          od->l3redirect_port->json_key);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 200,
> -                          REGBIT_NAT_REDIRECT" == 1", ds_cstr(&actions));
> -        }
> -
> -        /* Load balancing and packet defrag are only valid on
> -         * Gateway routers or router with gateway port. */
> -        if (!smap_get(&od->nbr->options, "chassis") && !od->l3dgw_port) {
> -            continue;
> -        }
> -
> -        /* A set to hold all ips that need defragmentation and tracking.
> */
> -        struct sset all_ips = SSET_INITIALIZER(&all_ips);
> -
> -        for (int i = 0; i < od->nbr->n_load_balancer; i++) {
> -            struct nbrec_load_balancer *lb = od->nbr->load_balancer[i];
> -            struct smap *vips = &lb->vips;
> -            struct smap_node *node;
> -
> -            SMAP_FOR_EACH (node, vips) {
> -                uint16_t port = 0;
> -                int addr_family;
> -
> -                /* node->key contains IP:port or just IP. */
> -                char *ip_address = NULL;
> -                ip_address_and_port_from_lb_key(node->key, &ip_address,
> &port,
> -                        &addr_family);
> -                if (!ip_address) {
> -                    continue;
> -                }
> -
> -                if (!sset_contains(&all_ips, ip_address)) {
> -                    sset_add(&all_ips, ip_address);
> -                    /* If there are any load balancing rules, we should
> send
> -                     * the packet to conntrack for defragmentation and
> -                     * tracking.  This helps with two things.
> -                     *
> -                     * 1. With tracking, we can send only new connections
> to
> -                     *    pick a DNAT ip address from a group.
> -                     * 2. If there are L4 ports in load balancing rules,
> we
> -                     *    need the defragmentation to match on L4 ports.
> */
> -                    ds_clear(&match);
> -                    if (addr_family == AF_INET) {
> -                        ds_put_format(&match, "ip && ip4.dst == %s",
> -                                      ip_address);
> -                    } else {
> -                        ds_put_format(&match, "ip && ip6.dst == %s",
> -                                      ip_address);
> -                    }
> -                    ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG,
> -                                  100, ds_cstr(&match), "ct_next;");
> -                }
> -
> -                /* Higher priority rules are added for load-balancing in
> DNAT
> -                 * table.  For every match (on a VIP[:port]), we add two
> flows
> -                 * via add_router_lb_flow().  One flow is for specific
> matching
> -                 * on ct.new with an action of "ct_lb($targets);".  The
> other
> -                 * flow is for ct.est with an action of "ct_dnat;". */
> -                ds_clear(&actions);
> -                ds_put_format(&actions, "ct_lb(%s);", node->value);
> -
> -                ds_clear(&match);
> -                if (addr_family == AF_INET) {
> -                    ds_put_format(&match, "ip && ip4.dst == %s",
> -                                ip_address);
> -                } else {
> -                    ds_put_format(&match, "ip && ip6.dst == %s",
> -                                ip_address);
> -                }
> -                free(ip_address);
> -
> -                int prio = 110;
> -                bool is_udp = lb->protocol && !strcmp(lb->protocol,
> "udp") ?
> -                    true : false;
> -                if (port) {
> -                    if (is_udp) {
> -                        ds_put_format(&match, " && udp && udp.dst == %d",
> -                                      port);
> -                    } else {
> -                        ds_put_format(&match, " && tcp && tcp.dst == %d",
> -                                      port);
> -                    }
> -                    prio = 120;
> -                }
> -
> -                if (od->l3redirect_port) {
> -                    ds_put_format(&match, " && is_chassis_resident(%s)",
> -                                  od->l3redirect_port->json_key);
> -                }
> -                add_router_lb_flow(lflows, od, &match, &actions, prio,
> -                                   lb_force_snat_ip, node->value, is_udp,
> -                                   addr_family);
> -            }
> -        }
> -        sset_destroy(&all_ips);
> -    }
> -
> -    /* Logical router ingress table 5 and 6: IPv6 Router Adv (RA) options
> and
> -     * response. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp || op->nbrp->peer || !op->peer) {
> -            continue;
> -        }
> -
> -        if (!op->lrp_networks.n_ipv6_addrs) {
> -            continue;
> -        }
> -
> -        const char *address_mode = smap_get(
> -            &op->nbrp->ipv6_ra_configs, "address_mode");
> -
> -        if (!address_mode) {
> -            continue;
> -        }
> -        if (strcmp(address_mode, "slaac") &&
> -            strcmp(address_mode, "dhcpv6_stateful") &&
> -            strcmp(address_mode, "dhcpv6_stateless")) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> -            VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined",
> -                         address_mode);
> -            continue;
> -        }
> -
> -        if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic",
> -                          false)) {
> -            copy_ra_to_sb(op, address_mode);
> -        }
> -
> -        ds_clear(&match);
> -        ds_put_format(&match, "inport == %s && ip6.dst == ff02::2 &&
> nd_rs",
> -                              op->json_key);
> -        ds_clear(&actions);
> -
> -        const char *mtu_s = smap_get(
> -            &op->nbrp->ipv6_ra_configs, "mtu");
> -
> -        /* As per RFC 2460, 1280 is minimum IPv6 MTU. */
> -        uint32_t mtu = (mtu_s && atoi(mtu_s) >= 1280) ? atoi(mtu_s) : 0;
> -
> -        ds_put_format(&actions, REGBIT_ND_RA_OPTS_RESULT" =
> put_nd_ra_opts("
> -                      "addr_mode = \"%s\", slla = %s",
> -                      address_mode, op->lrp_networks.ea_s);
> -        if (mtu > 0) {
> -            ds_put_format(&actions, ", mtu = %u", mtu);
> -        }
> -
> -        bool add_rs_response_flow = false;
> -
> -        for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -            if (in6_is_lla(&op->lrp_networks.ipv6_addrs[i].network)) {
> -                continue;
> -            }
> -
> -            ds_put_format(&actions, ", prefix = %s/%u",
> -                          op->lrp_networks.ipv6_addrs[i].network_s,
> -                          op->lrp_networks.ipv6_addrs[i].plen);
> -
> -            add_rs_response_flow = true;
> -        }
> -
> -        if (add_rs_response_flow) {
> -            ds_put_cstr(&actions, "); next;");
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ND_RA_OPTIONS, 50,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -            ds_clear(&actions);
> -            ds_clear(&match);
> -            ds_put_format(&match, "inport == %s && ip6.dst == ff02::2 && "
> -                          "nd_ra && "REGBIT_ND_RA_OPTS_RESULT,
> op->json_key);
> -
> -            char ip6_str[INET6_ADDRSTRLEN + 1];
> -            struct in6_addr lla;
> -            in6_generate_lla(op->lrp_networks.ea, &lla);
> -            memset(ip6_str, 0, sizeof(ip6_str));
> -            ipv6_string_mapped(ip6_str, &lla);
> -            ds_put_format(&actions, "eth.dst = eth.src; eth.src = %s; "
> -                          "ip6.dst = ip6.src; ip6.src = %s; "
> -                          "outport = inport; flags.loopback = 1; "
> -                          "output;",
> -                          op->lrp_networks.ea_s, ip6_str);
> -            ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ND_RA_RESPONSE, 50,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -    }
> -
> -    /* Logical router ingress table 5, 6: RS responder, by default goto
> next.
> -     * (priority 0)*/
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1",
> "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1",
> "next;");
> -    }
> -
> -    /* Logical router ingress table 7: IP Routing.
> -     *
> -     * A packet that arrives at this table is an IP packet that should be
> -     * routed to the address in 'ip[46].dst'. This table sets outport to
> -     * the correct output port, eth.src to the output port's MAC
> -     * address, and '[xx]reg0' to the next-hop IP address (leaving
> -     * 'ip[46].dst', the packet’s final destination, unchanged), and
> -     * advances to the next table for ARP/ND resolution. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp) {
> -            continue;
> -        }
> -
> -        /* create logical flows for DVR floating IPs */
> -        add_distributed_nat_routes(lflows, op);
> -
> -        for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> -            add_route(lflows, op, op->lrp_networks.ipv4_addrs[i].addr_s,
> -                      op->lrp_networks.ipv4_addrs[i].network_s,
> -                      op->lrp_networks.ipv4_addrs[i].plen, NULL, NULL);
> -        }
> -
> -        for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> -            add_route(lflows, op, op->lrp_networks.ipv6_addrs[i].addr_s,
> -                      op->lrp_networks.ipv6_addrs[i].network_s,
> -                      op->lrp_networks.ipv6_addrs[i].plen, NULL, NULL);
> -        }
> -    }
> -
> -    /* Convert the static routes to flows. */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        for (int i = 0; i < od->nbr->n_static_routes; i++) {
> -            const struct nbrec_logical_router_static_route *route;
> -
> -            route = od->nbr->static_routes[i];
> -            build_static_route_flow(lflows, od, ports, route);
> -        }
> -    }
> -
> -    /* Logical router ingress table 8: Policy.
> -     *
> -     * A packet that arrives at this table is an IP packet that should be
> -     * permitted/denied/rerouted to the address in the rule's nexthop.
> -     * This table sets outport to the correct out_port,
> -     * eth.src to the output port's MAC address,
> -     * and '[xx]reg0' to the next-hop IP address (leaving
> -     * 'ip[46].dst', the packet’s final destination, unchanged), and
> -     * advances to the next table for ARP/ND resolution. */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -        /* This is a catch-all rule. It has the lowest priority (0)
> -         * does a match-all("1") and pass-through (next) */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, 0, "1", "next;");
> -
> -        /* Convert routing policies to flows. */
> -        for (int i = 0; i < od->nbr->n_policies; i++) {
> -            const struct nbrec_logical_router_policy *rule
> -                = od->nbr->policies[i];
> -            build_routing_policy_flow(lflows, od, ports, rule);
> -        }
> -    }
> -
> -
> -    /* XXX destination unreachable */
> -
> -    /* Local router ingress table 9: ARP Resolution.
> -     *
> -     * Any packet that reaches this table is an IP packet whose next-hop
> IP
> -     * address is in reg0. (ip4.dst is the final destination.) This table
> -     * resolves the IP address in reg0 into an output port in outport and
> an
> -     * Ethernet address in eth.dst. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (op->nbsp && !lsp_is_enabled(op->nbsp)) {
> -            continue;
> -        }
> -
> -        if (op->nbrp) {
> -            /* This is a logical router port. If next-hop IP address in
> -             * '[xx]reg0' matches IP address of this router port, then
> -             * the packet is intended to eventually be sent to this
> -             * logical port. Set the destination mac address using this
> -             * port's mac address.
> -             *
> -             * The packet is still in peer's logical pipeline. So the
> match
> -             * should be on peer's outport. */
> -            if (op->peer && op->nbrp->peer) {
> -                if (op->lrp_networks.n_ipv4_addrs) {
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "outport == %s && reg0 == ",
> -                                  op->peer->json_key);
> -                    op_put_v4_networks(&match, op, false);
> -
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "eth.dst = %s; next;",
> -                                  op->lrp_networks.ea_s);
> -                    ovn_lflow_add(lflows, op->peer->od,
> S_ROUTER_IN_ARP_RESOLVE,
> -                                  100, ds_cstr(&match),
> ds_cstr(&actions));
> -                }
> -
> -                if (op->lrp_networks.n_ipv6_addrs) {
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "outport == %s && xxreg0 == ",
> -                                  op->peer->json_key);
> -                    op_put_v6_networks(&match, op);
> -
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "eth.dst = %s; next;",
> -                                  op->lrp_networks.ea_s);
> -                    ovn_lflow_add(lflows, op->peer->od,
> S_ROUTER_IN_ARP_RESOLVE,
> -                                  100, ds_cstr(&match),
> ds_cstr(&actions));
> -                }
> -            }
> -        } else if (op->od->n_router_ports && strcmp(op->nbsp->type,
> "router")) {
> -            /* This is a logical switch port that backs a VM or a
> container.
> -             * Extract its addresses. For each of the address, go through
> all
> -             * the router ports attached to the switch (to which this port
> -             * connects) and if the address in question is reachable from
> the
> -             * router port, add an ARP/ND entry in that router's
> pipeline. */
> -
> -            for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> -                const char *ea_s = op->lsp_addrs[i].ea_s;
> -                for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs;
> j++) {
> -                    const char *ip_s =
> op->lsp_addrs[i].ipv4_addrs[j].addr_s;
> -                    for (size_t k = 0; k < op->od->n_router_ports; k++) {
> -                        /* Get the Logical_Router_Port that the
> -                         * Logical_Switch_Port is connected to, as
> -                         * 'peer'. */
> -                        const char *peer_name = smap_get(
> -                            &op->od->router_ports[k]->nbsp->options,
> -                            "router-port");
> -                        if (!peer_name) {
> -                            continue;
> -                        }
> -
> -                        struct ovn_port *peer = ovn_port_find(ports,
> peer_name);
> -                        if (!peer || !peer->nbrp) {
> -                            continue;
> -                        }
> -
> -                        if (!find_lrp_member_ip(peer, ip_s)) {
> -                            continue;
> -                        }
> -
> -                        ds_clear(&match);
> -                        ds_put_format(&match, "outport == %s && reg0 ==
> %s",
> -                                      peer->json_key, ip_s);
> -
> -                        ds_clear(&actions);
> -                        ds_put_format(&actions, "eth.dst = %s; next;",
> ea_s);
> -                        ovn_lflow_add(lflows, peer->od,
> -                                      S_ROUTER_IN_ARP_RESOLVE, 100,
> -                                      ds_cstr(&match), ds_cstr(&actions));
> -                    }
> -                }
> -
> -                for (size_t j = 0; j < op->lsp_addrs[i].n_ipv6_addrs;
> j++) {
> -                    const char *ip_s =
> op->lsp_addrs[i].ipv6_addrs[j].addr_s;
> -                    for (size_t k = 0; k < op->od->n_router_ports; k++) {
> -                        /* Get the Logical_Router_Port that the
> -                         * Logical_Switch_Port is connected to, as
> -                         * 'peer'. */
> -                        const char *peer_name = smap_get(
> -                            &op->od->router_ports[k]->nbsp->options,
> -                            "router-port");
> -                        if (!peer_name) {
> -                            continue;
> -                        }
> -
> -                        struct ovn_port *peer = ovn_port_find(ports,
> peer_name);
> -                        if (!peer || !peer->nbrp) {
> -                            continue;
> -                        }
> -
> -                        if (!find_lrp_member_ip(peer, ip_s)) {
> -                            continue;
> -                        }
> -
> -                        ds_clear(&match);
> -                        ds_put_format(&match, "outport == %s && xxreg0 ==
> %s",
> -                                      peer->json_key, ip_s);
> -
> -                        ds_clear(&actions);
> -                        ds_put_format(&actions, "eth.dst = %s; next;",
> ea_s);
> -                        ovn_lflow_add(lflows, peer->od,
> -                                      S_ROUTER_IN_ARP_RESOLVE, 100,
> -                                      ds_cstr(&match), ds_cstr(&actions));
> -                    }
> -                }
> -            }
> -        } else if (!strcmp(op->nbsp->type, "router")) {
> -            /* This is a logical switch port that connects to a router. */
> -
> -            /* The peer of this switch port is the router port for which
> -             * we need to add logical flows such that it can resolve
> -             * ARP entries for all the other router ports connected to
> -             * the switch in question. */
> -
> -            const char *peer_name = smap_get(&op->nbsp->options,
> -                                             "router-port");
> -            if (!peer_name) {
> -                continue;
> -            }
> -
> -            struct ovn_port *peer = ovn_port_find(ports, peer_name);
> -            if (!peer || !peer->nbrp) {
> -                continue;
> -            }
> -
> -            for (size_t i = 0; i < op->od->n_router_ports; i++) {
> -                const char *router_port_name = smap_get(
> -
> &op->od->router_ports[i]->nbsp->options,
> -                                    "router-port");
> -                struct ovn_port *router_port = ovn_port_find(ports,
> -
>  router_port_name);
> -                if (!router_port || !router_port->nbrp) {
> -                    continue;
> -                }
> -
> -                /* Skip the router port under consideration. */
> -                if (router_port == peer) {
> -                   continue;
> -                }
> -
> -                if (router_port->lrp_networks.n_ipv4_addrs) {
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "outport == %s && reg0 == ",
> -                                  peer->json_key);
> -                    op_put_v4_networks(&match, router_port, false);
> -
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "eth.dst = %s; next;",
> -
> router_port->lrp_networks.ea_s);
> -                    ovn_lflow_add(lflows, peer->od,
> S_ROUTER_IN_ARP_RESOLVE,
> -                                  100, ds_cstr(&match),
> ds_cstr(&actions));
> -                }
> -
> -                if (router_port->lrp_networks.n_ipv6_addrs) {
> -                    ds_clear(&match);
> -                    ds_put_format(&match, "outport == %s && xxreg0 == ",
> -                                  peer->json_key);
> -                    op_put_v6_networks(&match, router_port);
> -
> -                    ds_clear(&actions);
> -                    ds_put_format(&actions, "eth.dst = %s; next;",
> -                                  router_port->lrp_networks.ea_s);
> -                    ovn_lflow_add(lflows, peer->od,
> S_ROUTER_IN_ARP_RESOLVE,
> -                                  100, ds_cstr(&match),
> ds_cstr(&actions));
> -                }
> -            }
> -        }
> -    }
> -
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 0, "ip4",
> -                      "get_arp(outport, reg0); next;");
> -
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 0, "ip6",
> -                      "get_nd(outport, xxreg0); next;");
> -    }
> -
> -    /* Local router ingress table 10: Check packet length.
> -     *
> -     * Any IPv4 packet with outport set to the distributed gateway
> -     * router port, check the packet length and store the result in the
> -     * 'REGBIT_PKT_LARGER' register bit.
> -     *
> -     * Local router ingress table 11: Handle larger packets.
> -     *
> -     * Any IPv4 packet with outport set to the distributed gateway
> -     * router port and the 'REGBIT_PKT_LARGER' register bit is set,
> -     * generate ICMPv4 packet with type 3 (Destination Unreachable) and
> -     * code 4 (Fragmentation needed).
> -     * */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        /* Packets are allowed by default. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, "1",
> -                      "next;");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, "1",
> -                      "next;");
> -
> -        if (od->l3dgw_port && od->l3redirect_port) {
> -            int gw_mtu = 0;
> -            if (od->l3dgw_port->nbrp) {
> -                 gw_mtu = smap_get_int(&od->l3dgw_port->nbrp->options,
> -                                       "gateway_mtu", 0);
> -            }
> -            /* Add the flows only if gateway_mtu is configured. */
> -            if (gw_mtu <= 0) {
> -                continue;
> -            }
> -
> -            ds_clear(&match);
> -            ds_put_format(&match, "outport == %s && ip4",
> -                          od->l3dgw_port->json_key);
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                          REGBIT_PKT_LARGER" = check_pkt_larger(%d);"
> -                          " next;", gw_mtu);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 50,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -
> -            for (size_t i = 0; i < od->nbr->n_ports; i++) {
> -                struct ovn_port *rp = ovn_port_find(ports,
> -
> od->nbr->ports[i]->name);
> -                if (!rp || rp == od->l3dgw_port) {
> -                    continue;
> -                }
> -                ds_clear(&match);
> -                ds_put_format(&match, "inport == %s && outport == %s &&
> ip4 "
> -                              "&& "REGBIT_PKT_LARGER,
> -                              rp->json_key, od->l3dgw_port->json_key);
> -
> -                ds_clear(&actions);
> -                /* Set icmp4.frag_mtu to gw_mtu - 58. 58 is the Geneve
> tunnel
> -                 * overhead. */
> -                ds_put_format(&actions,
> -                    "icmp4_error {"
> -                    REGBIT_EGRESS_LOOPBACK" = 1; "
> -                    "eth.dst = %s; "
> -                    "ip4.dst = ip4.src; "
> -                    "ip4.src = %s; "
> -                    "ip.ttl = 255; "
> -                    "icmp4.type = 3; /* Destination Unreachable. */ "
> -                    "icmp4.code = 4; /* Frag Needed and DF was Set. */ "
> -                    "icmp4.frag_mtu = %d; "
> -                    "next(pipeline=ingress, table=0); };",
> -                    rp->lrp_networks.ea_s,
> -                    rp->lrp_networks.ipv4_addrs[0].addr_s,
> -                    gw_mtu - 18);
> -                ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 50,
> -                              ds_cstr(&match), ds_cstr(&actions));
> -            }
> -        }
> -    }
> -
> -    /* Logical router ingress table 12: Gateway redirect.
> -     *
> -     * For traffic with outport equal to the l3dgw_port
> -     * on a distributed router, this table redirects a subset
> -     * of the traffic to the l3redirect_port which represents
> -     * the central instance of the l3dgw_port.
> -     */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -        if (od->l3dgw_port && od->l3redirect_port) {
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 300,
> -                          REGBIT_DISTRIBUTED_NAT" == 1", "next;");
> -
> -            /* For traffic with outport == l3dgw_port, if the
> -             * packet did not match any higher priority redirect
> -             * rule, then the traffic is redirected to the central
> -             * instance of the l3dgw_port. */
> -            ds_clear(&match);
> -            ds_put_format(&match, "outport == %s",
> -                          od->l3dgw_port->json_key);
> -            ds_clear(&actions);
> -            ds_put_format(&actions, "outport = %s; next;",
> -                          od->l3redirect_port->json_key);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -
> -            /* If the Ethernet destination has not been resolved,
> -             * redirect to the central instance of the l3dgw_port.
> -             * Such traffic will be replaced by an ARP request or ND
> -             * Neighbor Solicitation in the ARP request ingress
> -             * table, before being redirected to the central instance.
> -             */
> -            ds_put_format(&match, " && eth.dst == 00:00:00:00:00:00");
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 150,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        /* Packets are allowed by default. */
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1",
> "next;");
> -    }
> -
> -    /* Local router ingress table 13: ARP request.
> -     *
> -     * In the common case where the Ethernet destination has been
> resolved,
> -     * this table outputs the packet (priority 0).  Otherwise, it composes
> -     * and sends an ARP/IPv6 NA request (priority 100). */
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbr) {
> -            continue;
> -        }
> -
> -        for (int i = 0; i < od->nbr->n_static_routes; i++) {
> -            const struct nbrec_logical_router_static_route *route;
> -
> -            route = od->nbr->static_routes[i];
> -            struct in6_addr gw_ip6;
> -            unsigned int plen;
> -            char *error = ipv6_parse_cidr(route->nexthop, &gw_ip6, &plen);
> -            if (error || plen != 128) {
> -                free(error);
> -                continue;
> -            }
> -
> -            ds_clear(&match);
> -            ds_put_format(&match, "eth.dst == 00:00:00:00:00:00 && "
> -                          "ip6 && xxreg0 == %s", route->nexthop);
> -            struct in6_addr sn_addr;
> -            struct eth_addr eth_dst;
> -            in6_addr_solicited_node(&sn_addr, &gw_ip6);
> -            ipv6_multicast_to_ethernet(&eth_dst, &sn_addr);
> -
> -            char sn_addr_s[INET6_ADDRSTRLEN + 1];
> -            ipv6_string_mapped(sn_addr_s, &sn_addr);
> -
> -            ds_clear(&actions);
> -            ds_put_format(&actions,
> -                          "nd_ns { "
> -                          "eth.dst = "ETH_ADDR_FMT"; "
> -                          "ip6.dst = %s; "
> -                          "nd.target = %s; "
> -                          "output; "
> -                          "};", ETH_ADDR_ARGS(eth_dst), sn_addr_s,
> -                          route->nexthop);
> -            ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 200,
> -                          ds_cstr(&match), ds_cstr(&actions));
> -        }
> -
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100,
> -                      "eth.dst == 00:00:00:00:00:00",
> -                      "arp { "
> -                      "eth.dst = ff:ff:ff:ff:ff:ff; "
> -                      "arp.spa = reg1; "
> -                      "arp.tpa = reg0; "
> -                      "arp.op = 1; " /* ARP request */
> -                      "output; "
> -                      "};");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100,
> -                      "eth.dst == 00:00:00:00:00:00",
> -                      "nd_ns { "
> -                      "nd.target = xxreg0; "
> -                      "output; "
> -                      "};");
> -        ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1",
> "output;");
> -    }
> -
> -    /* Logical router egress table 1: Delivery (priority 100).
> -     *
> -     * Priority 100 rules deliver packets to enabled logical ports. */
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbrp) {
> -            continue;
> -        }
> -
> -        if (!lrport_is_enabled(op->nbrp)) {
> -            /* Drop packets to disabled logical ports (since logical flow
> -             * tables are default-drop). */
> -            continue;
> -        }
> -
> -        if (op->derived) {
> -            /* No egress packets should be processed in the context of
> -             * a chassisredirect port.  The chassisredirect port should
> -             * be replaced by the l3dgw port in the local output
> -             * pipeline stage before egress processing. */
> -            continue;
> -        }
> -
> -        ds_clear(&match);
> -        ds_put_format(&match, "outport == %s", op->json_key);
> -        ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 100,
> -                      ds_cstr(&match), "output;");
> -    }
> -
> -    ds_destroy(&match);
> -    ds_destroy(&actions);
> -}
> -
> -/* Updates the Logical_Flow and Multicast_Group tables in the OVN_SB
> database,
> - * constructing their contents based on the OVN_NB database. */
> -static void
> -build_lflows(struct northd_context *ctx, struct hmap *datapaths,
> -             struct hmap *ports, struct hmap *port_groups,
> -             struct hmap *mcgroups, struct hmap *igmp_groups)
> -{
> -    struct hmap lflows = HMAP_INITIALIZER(&lflows);
> -
> -    build_lswitch_flows(datapaths, ports, port_groups, &lflows, mcgroups,
> -                        igmp_groups);
> -    build_lrouter_flows(datapaths, ports, &lflows);
> -
> -    /* Push changes to the Logical_Flow table to database. */
> -    const struct sbrec_logical_flow *sbflow, *next_sbflow;
> -    SBREC_LOGICAL_FLOW_FOR_EACH_SAFE (sbflow, next_sbflow,
> ctx->ovnsb_idl) {
> -        struct ovn_datapath *od
> -            = ovn_datapath_from_sbrec(datapaths,
> sbflow->logical_datapath);
> -        if (!od) {
> -            sbrec_logical_flow_delete(sbflow);
> -            continue;
> -        }
> -
> -        enum ovn_datapath_type dp_type = od->nbs ? DP_SWITCH : DP_ROUTER;
> -        enum ovn_pipeline pipeline
> -            = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT;
> -        struct ovn_lflow *lflow = ovn_lflow_find(
> -            &lflows, od, ovn_stage_build(dp_type, pipeline,
> sbflow->table_id),
> -            sbflow->priority, sbflow->match, sbflow->actions,
> sbflow->hash);
> -        if (lflow) {
> -            ovn_lflow_destroy(&lflows, lflow);
> -        } else {
> -            sbrec_logical_flow_delete(sbflow);
> -        }
> -    }
> -    struct ovn_lflow *lflow, *next_lflow;
> -    HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows) {
> -        const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage);
> -        uint8_t table = ovn_stage_get_table(lflow->stage);
> -
> -        sbflow = sbrec_logical_flow_insert(ctx->ovnsb_txn);
> -        sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb);
> -        sbrec_logical_flow_set_pipeline(sbflow, pipeline);
> -        sbrec_logical_flow_set_table_id(sbflow, table);
> -        sbrec_logical_flow_set_priority(sbflow, lflow->priority);
> -        sbrec_logical_flow_set_match(sbflow, lflow->match);
> -        sbrec_logical_flow_set_actions(sbflow, lflow->actions);
> -
> -        /* Trim the source locator lflow->where, which looks something
> like
> -         * "ovn/northd/ovn-northd.c:1234", down to just the part
> following the
> -         * last slash, e.g. "ovn-northd.c:1234". */
> -        const char *slash = strrchr(lflow->where, '/');
> -#if _WIN32
> -        const char *backslash = strrchr(lflow->where, '\\');
> -        if (!slash || backslash > slash) {
> -            slash = backslash;
> -        }
> -#endif
> -        const char *where = slash ? slash + 1 : lflow->where;
> -
> -        struct smap ids = SMAP_INITIALIZER(&ids);
> -        smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage));
> -        smap_add(&ids, "source", where);
> -        if (lflow->stage_hint) {
> -            smap_add(&ids, "stage-hint", lflow->stage_hint);
> -        }
> -        sbrec_logical_flow_set_external_ids(sbflow, &ids);
> -        smap_destroy(&ids);
> -
> -        ovn_lflow_destroy(&lflows, lflow);
> -    }
> -    hmap_destroy(&lflows);
> -
> -    /* Push changes to the Multicast_Group table to database. */
> -    const struct sbrec_multicast_group *sbmc, *next_sbmc;
> -    SBREC_MULTICAST_GROUP_FOR_EACH_SAFE (sbmc, next_sbmc, ctx->ovnsb_idl)
> {
> -        struct ovn_datapath *od = ovn_datapath_from_sbrec(datapaths,
> -                                                          sbmc->datapath);
> -        if (!od) {
> -            sbrec_multicast_group_delete(sbmc);
> -            continue;
> -        }
> -
> -        struct multicast_group group = { .name = sbmc->name,
> -                                         .key = sbmc->tunnel_key };
> -        struct ovn_multicast *mc = ovn_multicast_find(mcgroups, od,
> &group);
> -        if (mc) {
> -            ovn_multicast_update_sbrec(mc, sbmc);
> -            ovn_multicast_destroy(mcgroups, mc);
> -        } else {
> -            sbrec_multicast_group_delete(sbmc);
> -        }
> -    }
> -    struct ovn_multicast *mc, *next_mc;
> -    HMAP_FOR_EACH_SAFE (mc, next_mc, hmap_node, mcgroups) {
> -        if (!mc->datapath) {
> -            ovn_multicast_destroy(mcgroups, mc);
> -            continue;
> -        }
> -        sbmc = sbrec_multicast_group_insert(ctx->ovnsb_txn);
> -        sbrec_multicast_group_set_datapath(sbmc, mc->datapath->sb);
> -        sbrec_multicast_group_set_name(sbmc, mc->group->name);
> -        sbrec_multicast_group_set_tunnel_key(sbmc, mc->group->key);
> -        ovn_multicast_update_sbrec(mc, sbmc);
> -        ovn_multicast_destroy(mcgroups, mc);
> -    }
> -}
> -
> -static void
> -sync_address_set(struct northd_context *ctx, const char *name,
> -                 const char **addrs, size_t n_addrs,
> -                 struct shash *sb_address_sets)
> -{
> -    const struct sbrec_address_set *sb_address_set;
> -    sb_address_set = shash_find_and_delete(sb_address_sets,
> -                                           name);
> -    if (!sb_address_set) {
> -        sb_address_set = sbrec_address_set_insert(ctx->ovnsb_txn);
> -        sbrec_address_set_set_name(sb_address_set, name);
> -    }
> -
> -    sbrec_address_set_set_addresses(sb_address_set,
> -                                    addrs, n_addrs);
> -}
> -
> -/* Go through 'addresses' and add found IPv4 addresses to 'ipv4_addrs'
> and IPv6
> - * addresses to 'ipv6_addrs'.
> - */
> -static void
> -split_addresses(const char *addresses, struct svec *ipv4_addrs,
> -                struct svec *ipv6_addrs)
> -{
> -    struct lport_addresses laddrs;
> -    extract_lsp_addresses(addresses, &laddrs);
> -    for (size_t k = 0; k < laddrs.n_ipv4_addrs; k++) {
> -        svec_add(ipv4_addrs, laddrs.ipv4_addrs[k].addr_s);
> -    }
> -    for (size_t k = 0; k < laddrs.n_ipv6_addrs; k++) {
> -        svec_add(ipv6_addrs, laddrs.ipv6_addrs[k].addr_s);
> -    }
> -    destroy_lport_addresses(&laddrs);
> -}
> -
> -/* OVN_Southbound Address_Set table contains same records as in north
> - * bound, plus the records generated from Port_Group table in north bound.
> - *
> - * There are 2 records generated from each port group, one for IPv4, and
> - * one for IPv6, named in the format: <port group name>_ip4 and
> - * <port group name>_ip6 respectively. MAC addresses are ignored.
> - *
> - * We always update OVN_Southbound to match the Address_Set and Port_Group
> - * in OVN_Northbound, so that the address sets used in Logical_Flows in
> - * OVN_Southbound is checked against the proper set.*/
> -static void
> -sync_address_sets(struct northd_context *ctx)
> -{
> -    struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets);
> -
> -    const struct sbrec_address_set *sb_address_set;
> -    SBREC_ADDRESS_SET_FOR_EACH (sb_address_set, ctx->ovnsb_idl) {
> -        shash_add(&sb_address_sets, sb_address_set->name, sb_address_set);
> -    }
> -
> -    /* sync port group generated address sets first */
> -    const struct nbrec_port_group *nb_port_group;
> -    NBREC_PORT_GROUP_FOR_EACH (nb_port_group, ctx->ovnnb_idl) {
> -        struct svec ipv4_addrs = SVEC_EMPTY_INITIALIZER;
> -        struct svec ipv6_addrs = SVEC_EMPTY_INITIALIZER;
> -        for (size_t i = 0; i < nb_port_group->n_ports; i++) {
> -            for (size_t j = 0; j < nb_port_group->ports[i]->n_addresses;
> j++) {
> -                const char *addrs = nb_port_group->ports[i]->addresses[j];
> -                if (!is_dynamic_lsp_address(addrs)) {
> -                    split_addresses(addrs, &ipv4_addrs, &ipv6_addrs);
> -                }
> -            }
> -            if (nb_port_group->ports[i]->dynamic_addresses) {
> -
> split_addresses(nb_port_group->ports[i]->dynamic_addresses,
> -                                &ipv4_addrs, &ipv6_addrs);
> -            }
> -        }
> -        char *ipv4_addrs_name = xasprintf("%s_ip4", nb_port_group->name);
> -        char *ipv6_addrs_name = xasprintf("%s_ip6", nb_port_group->name);
> -        sync_address_set(ctx, ipv4_addrs_name,
> -                         /* "char **" is not compatible with "const char
> **" */
> -                         (const char **)ipv4_addrs.names,
> -                         ipv4_addrs.n, &sb_address_sets);
> -        sync_address_set(ctx, ipv6_addrs_name,
> -                         /* "char **" is not compatible with "const char
> **" */
> -                         (const char **)ipv6_addrs.names,
> -                         ipv6_addrs.n, &sb_address_sets);
> -        free(ipv4_addrs_name);
> -        free(ipv6_addrs_name);
> -        svec_destroy(&ipv4_addrs);
> -        svec_destroy(&ipv6_addrs);
> -    }
> -
> -    /* sync user defined address sets, which may overwrite port group
> -     * generated address sets if same name is used */
> -    const struct nbrec_address_set *nb_address_set;
> -    NBREC_ADDRESS_SET_FOR_EACH (nb_address_set, ctx->ovnnb_idl) {
> -        sync_address_set(ctx, nb_address_set->name,
> -            /* "char **" is not compatible with "const char **" */
> -            (const char **)nb_address_set->addresses,
> -            nb_address_set->n_addresses, &sb_address_sets);
> -    }
> -
> -    struct shash_node *node, *next;
> -    SHASH_FOR_EACH_SAFE (node, next, &sb_address_sets) {
> -        sbrec_address_set_delete(node->data);
> -        shash_delete(&sb_address_sets, node);
> -    }
> -    shash_destroy(&sb_address_sets);
> -}
> -
> -/* Each port group in Port_Group table in OVN_Northbound has a
> corresponding
> - * entry in Port_Group table in OVN_Southbound. In OVN_Northbound the
> entries
> - * contains lport uuids, while in OVN_Southbound we store the lport names.
> - */
> -static void
> -sync_port_groups(struct northd_context *ctx)
> -{
> -    struct shash sb_port_groups = SHASH_INITIALIZER(&sb_port_groups);
> -
> -    const struct sbrec_port_group *sb_port_group;
> -    SBREC_PORT_GROUP_FOR_EACH (sb_port_group, ctx->ovnsb_idl) {
> -        shash_add(&sb_port_groups, sb_port_group->name, sb_port_group);
> -    }
> -
> -    const struct nbrec_port_group *nb_port_group;
> -    NBREC_PORT_GROUP_FOR_EACH (nb_port_group, ctx->ovnnb_idl) {
> -        sb_port_group = shash_find_and_delete(&sb_port_groups,
> -                                               nb_port_group->name);
> -        if (!sb_port_group) {
> -            sb_port_group = sbrec_port_group_insert(ctx->ovnsb_txn);
> -            sbrec_port_group_set_name(sb_port_group, nb_port_group->name);
> -        }
> -
> -        const char **nb_port_names = xcalloc(nb_port_group->n_ports,
> -                                             sizeof *nb_port_names);
> -        int i;
> -        for (i = 0; i < nb_port_group->n_ports; i++) {
> -            nb_port_names[i] = nb_port_group->ports[i]->name;
> -        }
> -        sbrec_port_group_set_ports(sb_port_group,
> -                                   nb_port_names,
> -                                   nb_port_group->n_ports);
> -        free(nb_port_names);
> -    }
> -
> -    struct shash_node *node, *next;
> -    SHASH_FOR_EACH_SAFE (node, next, &sb_port_groups) {
> -        sbrec_port_group_delete(node->data);
> -        shash_delete(&sb_port_groups, node);
> -    }
> -    shash_destroy(&sb_port_groups);
> -}
> -
> -struct band_entry {
> -    int64_t rate;
> -    int64_t burst_size;
> -    const char *action;
> -};
> -
> -static int
> -band_cmp(const void *band1_, const void *band2_)
> -{
> -    const struct band_entry *band1p = band1_;
> -    const struct band_entry *band2p = band2_;
> -
> -    if (band1p->rate != band2p->rate) {
> -        return band1p->rate > band2p->rate ? -1 : 1;
> -    } else if (band1p->burst_size != band2p->burst_size) {
> -        return band1p->burst_size > band2p->burst_size ? -1 : 1;
> -    } else {
> -        return strcmp(band1p->action, band2p->action);
> -    }
> -}
> -
> -static bool
> -bands_need_update(const struct nbrec_meter *nb_meter,
> -                  const struct sbrec_meter *sb_meter)
> -{
> -    if (nb_meter->n_bands != sb_meter->n_bands) {
> -        return true;
> -    }
> -
> -    /* A single band is the most common scenario, so speed up that
> -     * check. */
> -    if (nb_meter->n_bands == 1) {
> -        struct nbrec_meter_band *nb_band = nb_meter->bands[0];
> -        struct sbrec_meter_band *sb_band = sb_meter->bands[0];
> -
> -        return !(nb_band->rate == sb_band->rate
> -                 && nb_band->burst_size == sb_band->burst_size
> -                 && !strcmp(sb_band->action, nb_band->action));
> -    }
> -
> -    /* Place the Northbound entries in sorted order. */
> -    struct band_entry *nb_bands;
> -    nb_bands = xmalloc(sizeof *nb_bands * nb_meter->n_bands);
> -    for (size_t i = 0; i < nb_meter->n_bands; i++) {
> -        struct nbrec_meter_band *nb_band = nb_meter->bands[i];
> -
> -        nb_bands[i].rate = nb_band->rate;
> -        nb_bands[i].burst_size = nb_band->burst_size;
> -        nb_bands[i].action = nb_band->action;
> -    }
> -    qsort(nb_bands, nb_meter->n_bands, sizeof *nb_bands, band_cmp);
> -
> -    /* Place the Southbound entries in sorted order. */
> -    struct band_entry *sb_bands;
> -    sb_bands = xmalloc(sizeof *sb_bands * sb_meter->n_bands);
> -    for (size_t i = 0; i < sb_meter->n_bands; i++) {
> -        struct sbrec_meter_band *sb_band = sb_meter->bands[i];
> -
> -        sb_bands[i].rate = sb_band->rate;
> -        sb_bands[i].burst_size = sb_band->burst_size;
> -        sb_bands[i].action = sb_band->action;
> -    }
> -    qsort(sb_bands, sb_meter->n_bands, sizeof *sb_bands, band_cmp);
> -
> -    bool need_update = false;
> -    for (size_t i = 0; i < nb_meter->n_bands; i++) {
> -        if (nb_bands[i].rate != sb_bands[i].rate
> -            || nb_bands[i].burst_size != sb_bands[i].burst_size
> -            || strcmp(nb_bands[i].action, sb_bands[i].action)) {
> -            need_update = true;
> -            goto done;
> -        }
> -    }
> -
> -done:
> -    free(nb_bands);
> -    free(sb_bands);
> -
> -    return need_update;
> -}
> -
> -/* Each entry in the Meter and Meter_Band tables in OVN_Northbound have
> - * a corresponding entries in the Meter and Meter_Band tables in
> - * OVN_Southbound.
> - */
> -static void
> -sync_meters(struct northd_context *ctx)
> -{
> -    struct shash sb_meters = SHASH_INITIALIZER(&sb_meters);
> -
> -    const struct sbrec_meter *sb_meter;
> -    SBREC_METER_FOR_EACH (sb_meter, ctx->ovnsb_idl) {
> -        shash_add(&sb_meters, sb_meter->name, sb_meter);
> -    }
> -
> -    const struct nbrec_meter *nb_meter;
> -    NBREC_METER_FOR_EACH (nb_meter, ctx->ovnnb_idl) {
> -        bool new_sb_meter = false;
> -
> -        sb_meter = shash_find_and_delete(&sb_meters, nb_meter->name);
> -        if (!sb_meter) {
> -            sb_meter = sbrec_meter_insert(ctx->ovnsb_txn);
> -            sbrec_meter_set_name(sb_meter, nb_meter->name);
> -            new_sb_meter = true;
> -        }
> -
> -        if (new_sb_meter || bands_need_update(nb_meter, sb_meter)) {
> -            struct sbrec_meter_band **sb_bands;
> -            sb_bands = xcalloc(nb_meter->n_bands, sizeof *sb_bands);
> -            for (size_t i = 0; i < nb_meter->n_bands; i++) {
> -                const struct nbrec_meter_band *nb_band =
> nb_meter->bands[i];
> -
> -                sb_bands[i] = sbrec_meter_band_insert(ctx->ovnsb_txn);
> -
> -                sbrec_meter_band_set_action(sb_bands[i], nb_band->action);
> -                sbrec_meter_band_set_rate(sb_bands[i], nb_band->rate);
> -                sbrec_meter_band_set_burst_size(sb_bands[i],
> -                                                nb_band->burst_size);
> -            }
> -            sbrec_meter_set_bands(sb_meter, sb_bands, nb_meter->n_bands);
> -            free(sb_bands);
> -        }
> -
> -        sbrec_meter_set_unit(sb_meter, nb_meter->unit);
> -    }
> -
> -    struct shash_node *node, *next;
> -    SHASH_FOR_EACH_SAFE (node, next, &sb_meters) {
> -        sbrec_meter_delete(node->data);
> -        shash_delete(&sb_meters, node);
> -    }
> -    shash_destroy(&sb_meters);
> -}
> -
> -/*
> - * struct 'dns_info' is used to sync the DNS records between OVN
> Northbound db
> - * and Southbound db.
> - */
> -struct dns_info {
> -    struct hmap_node hmap_node;
> -    const struct nbrec_dns *nb_dns; /* DNS record in the Northbound db. */
> -    const struct sbrec_dns *sb_dns; /* DNS record in the Soutbound db. */
> -
> -    /* Datapaths to which the DNS entry is associated with it. */
> -    const struct sbrec_datapath_binding **sbs;
> -    size_t n_sbs;
> -};
> -
> -static inline struct dns_info *
> -get_dns_info_from_hmap(struct hmap *dns_map, struct uuid *uuid)
> -{
> -    struct dns_info *dns_info;
> -    size_t hash = uuid_hash(uuid);
> -    HMAP_FOR_EACH_WITH_HASH (dns_info, hmap_node, hash, dns_map) {
> -        if (uuid_equals(&dns_info->nb_dns->header_.uuid, uuid)) {
> -            return dns_info;
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static void
> -sync_dns_entries(struct northd_context *ctx, struct hmap *datapaths)
> -{
> -    struct hmap dns_map = HMAP_INITIALIZER(&dns_map);
> -    struct ovn_datapath *od;
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs || !od->nbs->n_dns_records) {
> -            continue;
> -        }
> -
> -        for (size_t i = 0; i < od->nbs->n_dns_records; i++) {
> -            struct dns_info *dns_info = get_dns_info_from_hmap(
> -                &dns_map, &od->nbs->dns_records[i]->header_.uuid);
> -            if (!dns_info) {
> -                size_t hash = uuid_hash(
> -                    &od->nbs->dns_records[i]->header_.uuid);
> -                dns_info = xzalloc(sizeof *dns_info);;
> -                dns_info->nb_dns = od->nbs->dns_records[i];
> -                hmap_insert(&dns_map, &dns_info->hmap_node, hash);
> -            }
> -
> -            dns_info->n_sbs++;
> -            dns_info->sbs = xrealloc(dns_info->sbs,
> -                                     dns_info->n_sbs * sizeof
> *dns_info->sbs);
> -            dns_info->sbs[dns_info->n_sbs - 1] = od->sb;
> -        }
> -    }
> -
> -    const struct sbrec_dns *sbrec_dns, *next;
> -    SBREC_DNS_FOR_EACH_SAFE (sbrec_dns, next, ctx->ovnsb_idl) {
> -        const char *nb_dns_uuid = smap_get(&sbrec_dns->external_ids,
> "dns_id");
> -        struct uuid dns_uuid;
> -        if (!nb_dns_uuid || !uuid_from_string(&dns_uuid, nb_dns_uuid)) {
> -            sbrec_dns_delete(sbrec_dns);
> -            continue;
> -        }
> -
> -        struct dns_info *dns_info =
> -            get_dns_info_from_hmap(&dns_map, &dns_uuid);
> -        if (dns_info) {
> -            dns_info->sb_dns = sbrec_dns;
> -        } else {
> -            sbrec_dns_delete(sbrec_dns);
> -        }
> -    }
> -
> -    struct dns_info *dns_info;
> -    HMAP_FOR_EACH_POP (dns_info, hmap_node, &dns_map) {
> -        if (!dns_info->sb_dns) {
> -            sbrec_dns = sbrec_dns_insert(ctx->ovnsb_txn);
> -            dns_info->sb_dns = sbrec_dns;
> -            char *dns_id = xasprintf(
> -                UUID_FMT, UUID_ARGS(&dns_info->nb_dns->header_.uuid));
> -            const struct smap external_ids =
> -                SMAP_CONST1(&external_ids, "dns_id", dns_id);
> -            sbrec_dns_set_external_ids(sbrec_dns, &external_ids);
> -            free(dns_id);
> -        }
> -
> -        /* Set the datapaths and records. If nothing has changed, then
> -         * this will be a no-op.
> -         */
> -        sbrec_dns_set_datapaths(
> -            dns_info->sb_dns,
> -            (struct sbrec_datapath_binding **)dns_info->sbs,
> -            dns_info->n_sbs);
> -        sbrec_dns_set_records(dns_info->sb_dns,
> &dns_info->nb_dns->records);
> -        free(dns_info->sbs);
> -        free(dns_info);
> -    }
> -    hmap_destroy(&dns_map);
> -}
> -
> -static void
> -destroy_datapaths_and_ports(struct hmap *datapaths, struct hmap *ports,
> -                            struct ovs_list *lr_list)
> -{
> -    struct ovn_datapath *router_dp;
> -    LIST_FOR_EACH_POP (router_dp, lr_list, lr_list) {
> -        if (router_dp->lr_group) {
> -            struct lrouter_group *lr_group = router_dp->lr_group;
> -
> -            for (size_t i = 0; i < lr_group->n_router_dps; i++) {
> -                lr_group->router_dps[i]->lr_group = NULL;
> -            }
> -
> -            free(lr_group->router_dps);
> -            sset_destroy(&lr_group->ha_chassis_groups);
> -            free(lr_group);
> -        }
> -    }
> -
> -    struct ovn_datapath *dp, *next_dp;
> -    HMAP_FOR_EACH_SAFE (dp, next_dp, key_node, datapaths) {
> -        ovn_datapath_destroy(datapaths, dp);
> -    }
> -    hmap_destroy(datapaths);
> -
> -    struct ovn_port *port, *next_port;
> -    HMAP_FOR_EACH_SAFE (port, next_port, key_node, ports) {
> -        ovn_port_destroy(ports, port);
> -    }
> -    hmap_destroy(ports);
> -}
> -
> -static void
> -build_ip_mcast(struct northd_context *ctx, struct hmap *datapaths)
> -{
> -    struct ovn_datapath *od;
> -
> -    HMAP_FOR_EACH (od, key_node, datapaths) {
> -        if (!od->nbs) {
> -            continue;
> -        }
> -
> -        const struct sbrec_ip_multicast *ip_mcast =
> -            ip_mcast_lookup(ctx->sbrec_ip_mcast_by_dp, od->sb);
> -
> -        if (!ip_mcast) {
> -            ip_mcast = sbrec_ip_multicast_insert(ctx->ovnsb_txn);
> -        }
> -        store_mcast_info_for_datapath(ip_mcast, od);
> -    }
> -
> -    /* Delete southbound records without northbound matches. */
> -    const struct sbrec_ip_multicast *sb, *sb_next;
> -
> -    SBREC_IP_MULTICAST_FOR_EACH_SAFE (sb, sb_next, ctx->ovnsb_idl) {
> -        if (!sb->datapath ||
> -                !ovn_datapath_from_sbrec(datapaths, sb->datapath)) {
> -            sbrec_ip_multicast_delete(sb);
> -        }
> -    }
> -}
> -
> -static void
> -build_mcast_groups(struct northd_context *ctx,
> -                   struct hmap *datapaths, struct hmap *ports,
> -                   struct hmap *mcast_groups,
> -                   struct hmap *igmp_groups)
> -{
> -    struct ovn_port *op;
> -
> -    hmap_init(mcast_groups);
> -    hmap_init(igmp_groups);
> -
> -    HMAP_FOR_EACH (op, key_node, ports) {
> -        if (!op->nbsp) {
> -            continue;
> -        }
> -
> -        if (lsp_is_enabled(op->nbsp)) {
> -            ovn_multicast_add(mcast_groups, &mc_flood, op);
> -        }
> -    }
> -
> -    const struct sbrec_igmp_group *sb_igmp, *sb_igmp_next;
> -
> -    SBREC_IGMP_GROUP_FOR_EACH_SAFE (sb_igmp, sb_igmp_next,
> ctx->ovnsb_idl) {
> -        /* If this is a stale group (e.g., controller had crashed,
> -         * purge it).
> -         */
> -        if (!sb_igmp->chassis || !sb_igmp->datapath) {
> -            sbrec_igmp_group_delete(sb_igmp);
> -            continue;
> -        }
> -
> -        /* If the datapath value is stale, purge the group. */
> -        struct ovn_datapath *od =
> -            ovn_datapath_from_sbrec(datapaths, sb_igmp->datapath);
> -        if (!od) {
> -            sbrec_igmp_group_delete(sb_igmp);
> -            continue;
> -        }
> -
> -        /* Add the IGMP group entry. Will also try to allocate an ID for
> it
> -         * if the multicast group already exists.
> -         */
> -        ovn_igmp_group_add(ctx, igmp_groups, od, sb_igmp);
> -    }
> -
> -    /* Walk the aggregated IGMP groups and allocate IDs for new entries.
> -     * Then store the ports in the associated multicast group.
> -     */
> -    struct ovn_igmp_group *igmp_group, *igmp_group_next;
> -    HMAP_FOR_EACH_SAFE (igmp_group, igmp_group_next, hmap_node,
> igmp_groups) {
> -        if (igmp_group->mcgroup.key == 0) {
> -            struct mcast_info *mcast_info =
> &igmp_group->datapath->mcast_info;
> -            igmp_group->mcgroup.key =
> ovn_mcast_group_allocate_key(mcast_info);
> -        }
> -
> -        /* If we ran out of keys just destroy the entry. */
> -        if (igmp_group->mcgroup.key == 0) {
> -            ovn_igmp_group_destroy(igmp_groups, igmp_group);
> -            continue;
> -        }
> -
> -        /* Aggregate the ports from all SB entries corresponding to this
> -         * group.
> -         */
> -        ovn_igmp_group_aggregate_ports(igmp_group, ports, mcast_groups);
> -    }
> -}
> -
> -static void
> -ovnnb_db_run(struct northd_context *ctx,
> -             struct ovsdb_idl_index *sbrec_chassis_by_name,
> -             struct ovsdb_idl_loop *sb_loop,
> -             struct hmap *datapaths, struct hmap *ports,
> -             struct ovs_list *lr_list)
> -{
> -    if (!ctx->ovnsb_txn || !ctx->ovnnb_txn) {
> -        return;
> -    }
> -    struct hmap port_groups;
> -    struct hmap mcast_groups;
> -    struct hmap igmp_groups;
> -
> -    build_datapaths(ctx, datapaths, lr_list);
> -    build_ports(ctx, sbrec_chassis_by_name, datapaths, ports);
> -    build_ipam(datapaths, ports);
> -    build_port_group_lswitches(ctx, &port_groups, ports);
> -    build_lrouter_groups(ports, lr_list);
> -    build_ip_mcast(ctx, datapaths);
> -    build_mcast_groups(ctx, datapaths, ports, &mcast_groups,
> &igmp_groups);
> -    build_lflows(ctx, datapaths, ports, &port_groups, &mcast_groups,
> -                 &igmp_groups);
> -
> -    sync_address_sets(ctx);
> -    sync_port_groups(ctx);
> -    sync_meters(ctx);
> -    sync_dns_entries(ctx, datapaths);
> -
> -    struct ovn_igmp_group *igmp_group, *next_igmp_group;
> -
> -    HMAP_FOR_EACH_SAFE (igmp_group, next_igmp_group, hmap_node,
> &igmp_groups) {
> -        ovn_igmp_group_destroy(&igmp_groups, igmp_group);
> -    }
> -
> -    struct ovn_port_group *pg, *next_pg;
> -    HMAP_FOR_EACH_SAFE (pg, next_pg, key_node, &port_groups) {
> -        ovn_port_group_destroy(&port_groups, pg);
> -    }
> -    hmap_destroy(&igmp_groups);
> -    hmap_destroy(&mcast_groups);
> -    hmap_destroy(&port_groups);
> -
> -    /* Sync ipsec configuration.
> -     * Copy nb_cfg from northbound to southbound database.
> -     * Also set up to update sb_cfg once our southbound transaction
> commits. */
> -    const struct nbrec_nb_global *nb =
> nbrec_nb_global_first(ctx->ovnnb_idl);
> -    if (!nb) {
> -        nb = nbrec_nb_global_insert(ctx->ovnnb_txn);
> -    }
> -    const struct sbrec_sb_global *sb =
> sbrec_sb_global_first(ctx->ovnsb_idl);
> -    if (!sb) {
> -        sb = sbrec_sb_global_insert(ctx->ovnsb_txn);
> -    }
> -    if (nb->ipsec != sb->ipsec) {
> -        sbrec_sb_global_set_ipsec(sb, nb->ipsec);
> -    }
> -    sbrec_sb_global_set_nb_cfg(sb, nb->nb_cfg);
> -    sbrec_sb_global_set_options(sb, &nb->options);
> -    sb_loop->next_cfg = nb->nb_cfg;
> -
> -    const char *mac_addr_prefix = smap_get(&nb->options, "mac_prefix");
> -    if (mac_addr_prefix) {
> -        struct eth_addr addr;
> -
> -        memset(&addr, 0, sizeof addr);
> -        if (ovs_scan(mac_addr_prefix, "%"SCNx8":%"SCNx8":%"SCNx8,
> -                     &addr.ea[0], &addr.ea[1], &addr.ea[2])) {
> -            mac_prefix = addr;
> -        }
> -    } else {
> -        struct smap options;
> -
> -        smap_clone(&options, &nb->options);
> -        eth_addr_random(&mac_prefix);
> -        memset(&mac_prefix.ea[3], 0, 3);
> -
> -        smap_add_format(&options, "mac_prefix",
> -                        "%02"PRIx8":%02"PRIx8":%02"PRIx8,
> -                        mac_prefix.ea[0], mac_prefix.ea[1],
> mac_prefix.ea[2]);
> -        nbrec_nb_global_verify_options(nb);
> -        nbrec_nb_global_set_options(nb, &options);
> -
> -        smap_destroy(&options);
> -    }
> -
> -    controller_event_en = smap_get_bool(&nb->options,
> -                                        "controller_event", false);
> -
> -    cleanup_macam(&macam);
> -}
> -
> -/* Stores the list of chassis which references an ha_chassis_group.
> - */
> -struct ha_ref_chassis_info {
> -    const struct sbrec_ha_chassis_group *ha_chassis_group;
> -    struct sbrec_chassis **ref_chassis;
> -    size_t n_ref_chassis;
> -    size_t free_slots;
> -};
> -
> -static void
> -add_to_ha_ref_chassis_info(struct ha_ref_chassis_info *ref_ch_info,
> -                           const struct sbrec_chassis *chassis)
> -{
> -    for (size_t j = 0; j < ref_ch_info->n_ref_chassis; j++) {
> -        if (ref_ch_info->ref_chassis[j] == chassis) {
> -           return;
> -        }
> -    }
> -
> -    /* Allocate space for 3 chassis at a time. */
> -    if (!ref_ch_info->free_slots) {
> -        ref_ch_info->ref_chassis =
> -            xrealloc(ref_ch_info->ref_chassis,
> -                     sizeof *ref_ch_info->ref_chassis *
> -                     (ref_ch_info->n_ref_chassis + 3));
> -        ref_ch_info->free_slots = 3;
> -    }
> -
> -    ref_ch_info->ref_chassis[ref_ch_info->n_ref_chassis] =
> -        CONST_CAST(struct sbrec_chassis *, chassis);
> -    ref_ch_info->n_ref_chassis++;
> -    ref_ch_info->free_slots--;
> -}
> -
> -static void
> -update_sb_ha_group_ref_chassis(struct shash *ha_ref_chassis_map)
> -{
> -    struct shash_node *node, *next;
> -    SHASH_FOR_EACH_SAFE (node, next, ha_ref_chassis_map) {
> -        struct ha_ref_chassis_info *ha_ref_info = node->data;
> -
> sbrec_ha_chassis_group_set_ref_chassis(ha_ref_info->ha_chassis_group,
> -                                               ha_ref_info->ref_chassis,
> -
>  ha_ref_info->n_ref_chassis);
> -        free(ha_ref_info->ref_chassis);
> -        free(ha_ref_info);
> -        shash_delete(ha_ref_chassis_map, node);
> -    }
> -}
> -
> -/* This function checks if the port binding 'sb' references
> - * a HA chassis group.
> - * Eg. Suppose a distributed logical router port - lr0-public
> - * uses an HA chassis group - hagrp1 and if hagrp1 has 3 ha
> - * chassis - gw1, gw2 and gw3.
> - * Or
> - * If the distributed logical router port - lr0-public has
> - * 3 gateway chassis - gw1, gw2 and gw3.
> - * ovn-northd creates ha chassis group - hagrp1 in SB DB
> - * and adds gw1, gw2 and gw3 to its ha_chassis list.
> - *
> - * If port binding 'sb' represents a logical switch port 'p1'
> - * and its logical switch is connected to the logical router
> - * 'lr0' directly or indirectly (i.e p1's logical switch is
> - *  connected to a router 'lr1' and 'lr1' has a path to lr0 via
> - *  transit logical switches) and 'sb' is claimed by chassis - 'c1' then
> - * this function adds c1 to the list of the reference chassis
> - *  - 'ref_chassis' of hagrp1.
> - */
> -static void
> -build_ha_chassis_group_ref_chassis(struct northd_context *ctx,
> -                                   const struct sbrec_port_binding *sb,
> -                                   struct ovn_port *op,
> -                                   struct shash *ha_ref_chassis_map)
> -{
> -    struct lrouter_group *lr_group = NULL;
> -    for (size_t i = 0; i < op->od->n_router_ports; i++) {
> -        if (!op->od->router_ports[i]->peer) {
> -            continue;
> -        }
> -
> -        lr_group = op->od->router_ports[i]->peer->od->lr_group;
> -        /* If a logical switch has multiple router ports, then
> -         * all the logical routers belong to the same logical
> -         * router group. */
> -        break;
> -    }
> -
> -    if (!lr_group) {
> -        return;
> -    }
> -
> -    const char *ha_group_name;
> -    SSET_FOR_EACH (ha_group_name, &lr_group->ha_chassis_groups) {
> -        const struct sbrec_ha_chassis_group *sb_ha_chassis_grp;
> -        sb_ha_chassis_grp = ha_chassis_group_lookup_by_name(
> -            ctx->sbrec_ha_chassis_grp_by_name, ha_group_name);
> -
> -        if (sb_ha_chassis_grp) {
> -            struct ha_ref_chassis_info *ref_ch_info =
> -            shash_find_data(ha_ref_chassis_map, sb_ha_chassis_grp->name);
> -            ovs_assert(ref_ch_info);
> -            add_to_ha_ref_chassis_info(ref_ch_info, sb->chassis);
> -        }
> -    }
> -}
> -
> -/* Handle changes to the 'chassis' column of the 'Port_Binding' table.
> When
> - * this column is not empty, it means we need to set the corresponding
> logical
> - * port as 'up' in the northbound DB. */
> -static void
> -handle_port_binding_changes(struct northd_context *ctx, struct hmap
> *ports,
> -                            struct shash *ha_ref_chassis_map)
> -{
> -    const struct sbrec_port_binding *sb;
> -    bool build_ha_chassis_ref = false;
> -    if (ctx->ovnsb_txn) {
> -        const struct sbrec_ha_chassis_group *ha_ch_grp;
> -        SBREC_HA_CHASSIS_GROUP_FOR_EACH (ha_ch_grp, ctx->ovnsb_idl) {
> -            struct ha_ref_chassis_info *ref_ch_info =
> -                xzalloc(sizeof *ref_ch_info);
> -            ref_ch_info->ha_chassis_group = ha_ch_grp;
> -            build_ha_chassis_ref = true;
> -            shash_add(ha_ref_chassis_map, ha_ch_grp->name, ref_ch_info);
> -        }
> -    }
> -
> -    SBREC_PORT_BINDING_FOR_EACH(sb, ctx->ovnsb_idl) {
> -        struct ovn_port *op = ovn_port_find(ports, sb->logical_port);
> -
> -        if (!op || !op->nbsp) {
> -            /* The logical port doesn't exist for this port binding.
> This can
> -             * happen under normal circumstances when ovn-northd hasn't
> gotten
> -             * around to pruning the Port_Binding yet. */
> -            continue;
> -        }
> -
> -        bool up = (sb->chassis || !strcmp(op->nbsp->type, "router"));
> -        if (!op->nbsp->up || *op->nbsp->up != up) {
> -            nbrec_logical_switch_port_set_up(op->nbsp, &up, 1);
> -        }
> -
> -        if (build_ha_chassis_ref && ctx->ovnsb_txn && sb->chassis) {
> -            /* Check and add the chassis which has claimed this 'sb'
> -             * to the ha chassis group's ref_chassis if required. */
> -            build_ha_chassis_group_ref_chassis(ctx, sb, op,
> -                                               ha_ref_chassis_map);
> -        }
> -    }
> -}
> -
> -static struct gen_opts_map supported_dhcp_opts[] = {
> -    OFFERIP,
> -    DHCP_OPT_NETMASK,
> -    DHCP_OPT_ROUTER,
> -    DHCP_OPT_DNS_SERVER,
> -    DHCP_OPT_LOG_SERVER,
> -    DHCP_OPT_LPR_SERVER,
> -    DHCP_OPT_SWAP_SERVER,
> -    DHCP_OPT_POLICY_FILTER,
> -    DHCP_OPT_ROUTER_SOLICITATION,
> -    DHCP_OPT_NIS_SERVER,
> -    DHCP_OPT_NTP_SERVER,
> -    DHCP_OPT_SERVER_ID,
> -    DHCP_OPT_TFTP_SERVER,
> -    DHCP_OPT_CLASSLESS_STATIC_ROUTE,
> -    DHCP_OPT_MS_CLASSLESS_STATIC_ROUTE,
> -    DHCP_OPT_IP_FORWARD_ENABLE,
> -    DHCP_OPT_ROUTER_DISCOVERY,
> -    DHCP_OPT_ETHERNET_ENCAP,
> -    DHCP_OPT_DEFAULT_TTL,
> -    DHCP_OPT_TCP_TTL,
> -    DHCP_OPT_MTU,
> -    DHCP_OPT_LEASE_TIME,
> -    DHCP_OPT_T1,
> -    DHCP_OPT_T2,
> -    DHCP_OPT_WPAD,
> -    DHCP_OPT_BOOTFILE,
> -    DHCP_OPT_PATH_PREFIX,
> -    DHCP_OPT_TFTP_SERVER_ADDRESS,
> -    DHCP_OPT_DOMAIN_NAME,
> -};
> -
> -static struct gen_opts_map supported_dhcpv6_opts[] = {
> -    DHCPV6_OPT_IA_ADDR,
> -    DHCPV6_OPT_SERVER_ID,
> -    DHCPV6_OPT_DOMAIN_SEARCH,
> -    DHCPV6_OPT_DNS_SERVER
> -};
> -
> -static void
> -check_and_add_supported_dhcp_opts_to_sb_db(struct northd_context *ctx)
> -{
> -    struct hmap dhcp_opts_to_add = HMAP_INITIALIZER(&dhcp_opts_to_add);
> -    for (size_t i = 0; (i < sizeof(supported_dhcp_opts) /
> -                            sizeof(supported_dhcp_opts[0])); i++) {
> -        hmap_insert(&dhcp_opts_to_add, &supported_dhcp_opts[i].hmap_node,
> -                    dhcp_opt_hash(supported_dhcp_opts[i].name));
> -    }
> -
> -    const struct sbrec_dhcp_options *opt_row, *opt_row_next;
> -    SBREC_DHCP_OPTIONS_FOR_EACH_SAFE(opt_row, opt_row_next,
> ctx->ovnsb_idl) {
> -        struct gen_opts_map *dhcp_opt =
> -            dhcp_opts_find(&dhcp_opts_to_add, opt_row->name);
> -        if (dhcp_opt) {
> -            hmap_remove(&dhcp_opts_to_add, &dhcp_opt->hmap_node);
> -        } else {
> -            sbrec_dhcp_options_delete(opt_row);
> -        }
> -    }
> -
> -    struct gen_opts_map *opt;
> -    HMAP_FOR_EACH (opt, hmap_node, &dhcp_opts_to_add) {
> -        struct sbrec_dhcp_options *sbrec_dhcp_option =
> -            sbrec_dhcp_options_insert(ctx->ovnsb_txn);
> -        sbrec_dhcp_options_set_name(sbrec_dhcp_option, opt->name);
> -        sbrec_dhcp_options_set_code(sbrec_dhcp_option, opt->code);
> -        sbrec_dhcp_options_set_type(sbrec_dhcp_option, opt->type);
> -    }
> -
> -    hmap_destroy(&dhcp_opts_to_add);
> -}
> -
> -static void
> -check_and_add_supported_dhcpv6_opts_to_sb_db(struct northd_context *ctx)
> -{
> -    struct hmap dhcpv6_opts_to_add =
> HMAP_INITIALIZER(&dhcpv6_opts_to_add);
> -    for (size_t i = 0; (i < sizeof(supported_dhcpv6_opts) /
> -                            sizeof(supported_dhcpv6_opts[0])); i++) {
> -        hmap_insert(&dhcpv6_opts_to_add,
> &supported_dhcpv6_opts[i].hmap_node,
> -                    dhcp_opt_hash(supported_dhcpv6_opts[i].name));
> -    }
> -
> -    const struct sbrec_dhcpv6_options *opt_row, *opt_row_next;
> -    SBREC_DHCPV6_OPTIONS_FOR_EACH_SAFE(opt_row, opt_row_next,
> ctx->ovnsb_idl) {
> -        struct gen_opts_map *dhcp_opt =
> -            dhcp_opts_find(&dhcpv6_opts_to_add, opt_row->name);
> -        if (dhcp_opt) {
> -            hmap_remove(&dhcpv6_opts_to_add, &dhcp_opt->hmap_node);
> -        } else {
> -            sbrec_dhcpv6_options_delete(opt_row);
> -        }
> -    }
> -
> -    struct gen_opts_map *opt;
> -    HMAP_FOR_EACH(opt, hmap_node, &dhcpv6_opts_to_add) {
> -        struct sbrec_dhcpv6_options *sbrec_dhcpv6_option =
> -            sbrec_dhcpv6_options_insert(ctx->ovnsb_txn);
> -        sbrec_dhcpv6_options_set_name(sbrec_dhcpv6_option, opt->name);
> -        sbrec_dhcpv6_options_set_code(sbrec_dhcpv6_option, opt->code);
> -        sbrec_dhcpv6_options_set_type(sbrec_dhcpv6_option, opt->type);
> -    }
> -
> -    hmap_destroy(&dhcpv6_opts_to_add);
> -}
> -
> -static const char *rbac_chassis_auth[] =
> -    {"name"};
> -static const char *rbac_chassis_update[] =
> -    {"nb_cfg", "external_ids", "encaps", "vtep_logical_switches"};
> -
> -static const char *rbac_encap_auth[] =
> -    {"chassis_name"};
> -static const char *rbac_encap_update[] =
> -    {"type", "options", "ip"};
> -
> -static const char *rbac_port_binding_auth[] =
> -    {""};
> -static const char *rbac_port_binding_update[] =
> -    {"chassis"};
> -
> -static const char *rbac_mac_binding_auth[] =
> -    {""};
> -static const char *rbac_mac_binding_update[] =
> -    {"logical_port", "ip", "mac", "datapath"};
> -
> -static struct rbac_perm_cfg {
> -    const char *table;
> -    const char **auth;
> -    int n_auth;
> -    bool insdel;
> -    const char **update;
> -    int n_update;
> -    const struct sbrec_rbac_permission *row;
> -} rbac_perm_cfg[] = {
> -    {
> -        .table = "Chassis",
> -        .auth = rbac_chassis_auth,
> -        .n_auth = ARRAY_SIZE(rbac_chassis_auth),
> -        .insdel = true,
> -        .update = rbac_chassis_update,
> -        .n_update = ARRAY_SIZE(rbac_chassis_update),
> -        .row = NULL
> -    },{
> -        .table = "Encap",
> -        .auth = rbac_encap_auth,
> -        .n_auth = ARRAY_SIZE(rbac_encap_auth),
> -        .insdel = true,
> -        .update = rbac_encap_update,
> -        .n_update = ARRAY_SIZE(rbac_encap_update),
> -        .row = NULL
> -    },{
> -        .table = "Port_Binding",
> -        .auth = rbac_port_binding_auth,
> -        .n_auth = ARRAY_SIZE(rbac_port_binding_auth),
> -        .insdel = false,
> -        .update = rbac_port_binding_update,
> -        .n_update = ARRAY_SIZE(rbac_port_binding_update),
> -        .row = NULL
> -    },{
> -        .table = "MAC_Binding",
> -        .auth = rbac_mac_binding_auth,
> -        .n_auth = ARRAY_SIZE(rbac_mac_binding_auth),
> -        .insdel = true,
> -        .update = rbac_mac_binding_update,
> -        .n_update = ARRAY_SIZE(rbac_mac_binding_update),
> -        .row = NULL
> -    },{
> -        .table = NULL,
> -        .auth = NULL,
> -        .n_auth = 0,
> -        .insdel = false,
> -        .update = NULL,
> -        .n_update = 0,
> -        .row = NULL
> -    }
> -};
> -
> -static bool
> -ovn_rbac_validate_perm(const struct sbrec_rbac_permission *perm)
> -{
> -    struct rbac_perm_cfg *pcfg;
> -    int i, j, n_found;
> -
> -    for (pcfg = rbac_perm_cfg; pcfg->table; pcfg++) {
> -        if (!strcmp(perm->table, pcfg->table)) {
> -            break;
> -        }
> -    }
> -    if (!pcfg->table) {
> -        return false;
> -    }
> -    if (perm->n_authorization != pcfg->n_auth ||
> -        perm->n_update != pcfg->n_update) {
> -        return false;
> -    }
> -    if (perm->insert_delete != pcfg->insdel) {
> -        return false;
> -    }
> -    /* verify perm->authorization vs. pcfg->auth */
> -    n_found = 0;
> -    for (i = 0; i < pcfg->n_auth; i++) {
> -        for (j = 0; j < perm->n_authorization; j++) {
> -            if (!strcmp(pcfg->auth[i], perm->authorization[j])) {
> -                n_found++;
> -                break;
> -            }
> -        }
> -    }
> -    if (n_found != pcfg->n_auth) {
> -        return false;
> -    }
> -
> -    /* verify perm->update vs. pcfg->update */
> -    n_found = 0;
> -    for (i = 0; i < pcfg->n_update; i++) {
> -        for (j = 0; j < perm->n_update; j++) {
> -            if (!strcmp(pcfg->update[i], perm->update[j])) {
> -                n_found++;
> -                break;
> -            }
> -        }
> -    }
> -    if (n_found != pcfg->n_update) {
> -        return false;
> -    }
> -
> -    /* Success, db state matches expected state */
> -    pcfg->row = perm;
> -    return true;
> -}
> -
> -static void
> -ovn_rbac_create_perm(struct rbac_perm_cfg *pcfg,
> -                     struct northd_context *ctx,
> -                     const struct sbrec_rbac_role *rbac_role)
> -{
> -    struct sbrec_rbac_permission *rbac_perm;
> -
> -    rbac_perm = sbrec_rbac_permission_insert(ctx->ovnsb_txn);
> -    sbrec_rbac_permission_set_table(rbac_perm, pcfg->table);
> -    sbrec_rbac_permission_set_authorization(rbac_perm,
> -                                            pcfg->auth,
> -                                            pcfg->n_auth);
> -    sbrec_rbac_permission_set_insert_delete(rbac_perm, pcfg->insdel);
> -    sbrec_rbac_permission_set_update(rbac_perm,
> -                                     pcfg->update,
> -                                     pcfg->n_update);
> -    sbrec_rbac_role_update_permissions_setkey(rbac_role, pcfg->table,
> -                                              rbac_perm);
> -}
> -
> -static void
> -check_and_update_rbac(struct northd_context *ctx)
> -{
> -    const struct sbrec_rbac_role *rbac_role = NULL;
> -    const struct sbrec_rbac_permission *perm_row, *perm_next;
> -    const struct sbrec_rbac_role *role_row, *role_row_next;
> -    struct rbac_perm_cfg *pcfg;
> -
> -    for (pcfg = rbac_perm_cfg; pcfg->table; pcfg++) {
> -        pcfg->row = NULL;
> -    }
> -
> -    SBREC_RBAC_PERMISSION_FOR_EACH_SAFE (perm_row, perm_next,
> ctx->ovnsb_idl) {
> -        if (!ovn_rbac_validate_perm(perm_row)) {
> -            sbrec_rbac_permission_delete(perm_row);
> -        }
> -    }
> -    SBREC_RBAC_ROLE_FOR_EACH_SAFE (role_row, role_row_next,
> ctx->ovnsb_idl) {
> -        if (strcmp(role_row->name, "ovn-controller")) {
> -            sbrec_rbac_role_delete(role_row);
> -        } else {
> -            rbac_role = role_row;
> -        }
> -    }
> -
> -    if (!rbac_role) {
> -        rbac_role = sbrec_rbac_role_insert(ctx->ovnsb_txn);
> -        sbrec_rbac_role_set_name(rbac_role, "ovn-controller");
> -    }
> -
> -    for (pcfg = rbac_perm_cfg; pcfg->table; pcfg++) {
> -        if (!pcfg->row) {
> -            ovn_rbac_create_perm(pcfg, ctx, rbac_role);
> -        }
> -    }
> -}
> -
> -/* Updates the sb_cfg and hv_cfg columns in the northbound NB_Global
> table. */
> -static void
> -update_northbound_cfg(struct northd_context *ctx,
> -                      struct ovsdb_idl_loop *sb_loop)
> -{
> -    /* Update northbound sb_cfg if appropriate. */
> -    const struct nbrec_nb_global *nbg =
> nbrec_nb_global_first(ctx->ovnnb_idl);
> -    int64_t sb_cfg = sb_loop->cur_cfg;
> -    if (nbg && sb_cfg && nbg->sb_cfg != sb_cfg) {
> -        nbrec_nb_global_set_sb_cfg(nbg, sb_cfg);
> -    }
> -
> -    /* Update northbound hv_cfg if appropriate. */
> -    if (nbg) {
> -        /* Find minimum nb_cfg among all chassis. */
> -        const struct sbrec_chassis *chassis;
> -        int64_t hv_cfg = nbg->nb_cfg;
> -        SBREC_CHASSIS_FOR_EACH (chassis, ctx->ovnsb_idl) {
> -            if (chassis->nb_cfg < hv_cfg) {
> -                hv_cfg = chassis->nb_cfg;
> -            }
> -        }
> -
> -        /* Update hv_cfg. */
> -        if (nbg->hv_cfg != hv_cfg) {
> -            nbrec_nb_global_set_hv_cfg(nbg, hv_cfg);
> -        }
> -    }
> -}
> -
> -/* Handle a fairly small set of changes in the southbound database. */
> -static void
> -ovnsb_db_run(struct northd_context *ctx, struct ovsdb_idl_loop *sb_loop,
> -             struct hmap *ports)
> -{
> -    if (!ctx->ovnnb_txn || !ovsdb_idl_has_ever_connected(ctx->ovnsb_idl))
> {
> -        return;
> -    }
> -
> -    struct shash ha_ref_chassis_map =
> SHASH_INITIALIZER(&ha_ref_chassis_map);
> -    handle_port_binding_changes(ctx, ports, &ha_ref_chassis_map);
> -    update_northbound_cfg(ctx, sb_loop);
> -    if (ctx->ovnsb_txn) {
> -        update_sb_ha_group_ref_chassis(&ha_ref_chassis_map);
> -    }
> -    shash_destroy(&ha_ref_chassis_map);
> -}
> -
> -static void
> -ovn_db_run(struct northd_context *ctx,
> -           struct ovsdb_idl_index *sbrec_chassis_by_name,
> -           struct ovsdb_idl_loop *ovnsb_idl_loop)
> -{
> -    struct hmap datapaths, ports;
> -    struct ovs_list lr_list;
> -    ovs_list_init(&lr_list);
> -    hmap_init(&datapaths);
> -    hmap_init(&ports);
> -    ovnnb_db_run(ctx, sbrec_chassis_by_name, ovnsb_idl_loop,
> -                 &datapaths, &ports, &lr_list);
> -    ovnsb_db_run(ctx, ovnsb_idl_loop, &ports);
> -    destroy_datapaths_and_ports(&datapaths, &ports, &lr_list);
> -}
> -
> -static void
> -parse_options(int argc OVS_UNUSED, char *argv[] OVS_UNUSED)
> -{
> -    enum {
> -        DAEMON_OPTION_ENUMS,
> -        VLOG_OPTION_ENUMS,
> -        SSL_OPTION_ENUMS,
> -    };
> -    static const struct option long_options[] = {
> -        {"ovnsb-db", required_argument, NULL, 'd'},
> -        {"ovnnb-db", required_argument, NULL, 'D'},
> -        {"unixctl", required_argument, NULL, 'u'},
> -        {"help", no_argument, NULL, 'h'},
> -        {"options", no_argument, NULL, 'o'},
> -        {"version", no_argument, NULL, 'V'},
> -        DAEMON_LONG_OPTIONS,
> -        VLOG_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {NULL, 0, NULL, 0},
> -    };
> -    char *short_options =
> ovs_cmdl_long_options_to_short_options(long_options);
> -
> -    for (;;) {
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, long_options, NULL);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        switch (c) {
> -        DAEMON_OPTION_HANDLERS;
> -        VLOG_OPTION_HANDLERS;
> -        STREAM_SSL_OPTION_HANDLERS;
> -
> -        case 'd':
> -            ovnsb_db = optarg;
> -            break;
> -
> -        case 'D':
> -            ovnnb_db = optarg;
> -            break;
> -
> -        case 'u':
> -            unixctl_path = optarg;
> -            break;
> -
> -        case 'h':
> -            usage();
> -            exit(EXIT_SUCCESS);
> -
> -        case 'o':
> -            ovs_cmdl_print_options(long_options);
> -            exit(EXIT_SUCCESS);
> -
> -        case 'V':
> -            ovs_print_version(0, 0);
> -            exit(EXIT_SUCCESS);
> -
> -        default:
> -            break;
> -        }
> -    }
> -
> -    if (!ovnsb_db) {
> -        ovnsb_db = default_sb_db();
> -    }
> -
> -    if (!ovnnb_db) {
> -        ovnnb_db = default_nb_db();
> -    }
> -
> -    free(short_options);
> -}
> -
> -static void
> -add_column_noalert(struct ovsdb_idl *idl,
> -                   const struct ovsdb_idl_column *column)
> -{
> -    ovsdb_idl_add_column(idl, column);
> -    ovsdb_idl_omit_alert(idl, column);
> -}
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    int res = EXIT_SUCCESS;
> -    struct unixctl_server *unixctl;
> -    int retval;
> -    bool exiting;
> -
> -    fatal_ignore_sigpipe();
> -    ovs_cmdl_proctitle_init(argc, argv);
> -    set_program_name(argv[0]);
> -    service_start(&argc, &argv);
> -    parse_options(argc, argv);
> -
> -    daemonize_start(false);
> -
> -    retval = unixctl_server_create(unixctl_path, &unixctl);
> -    if (retval) {
> -        exit(EXIT_FAILURE);
> -    }
> -    unixctl_command_register("exit", "", 0, 0, ovn_northd_exit, &exiting);
> -
> -    daemonize_complete();
> -
> -    /* We want to detect (almost) all changes to the ovn-nb db. */
> -    struct ovsdb_idl_loop ovnnb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create(ovnnb_db, &nbrec_idl_class, true, true));
> -    ovsdb_idl_omit_alert(ovnnb_idl_loop.idl, &nbrec_nb_global_col_sb_cfg);
> -    ovsdb_idl_omit_alert(ovnnb_idl_loop.idl, &nbrec_nb_global_col_hv_cfg);
> -
> -    /* We want to detect only selected changes to the ovn-sb db. */
> -    struct ovsdb_idl_loop ovnsb_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> -        ovsdb_idl_create(ovnsb_db, &sbrec_idl_class, false, true));
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_sb_global);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_sb_global_col_nb_cfg);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_sb_global_col_options);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_sb_global_col_ipsec);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_logical_flow);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_logical_flow_col_logical_datapath);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_logical_flow_col_pipeline);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_logical_flow_col_table_id);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_logical_flow_col_priority);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_logical_flow_col_match);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_logical_flow_col_actions);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_multicast_group);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_multicast_group_col_datapath);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_multicast_group_col_tunnel_key);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_multicast_group_col_name);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_multicast_group_col_ports);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl,
> &sbrec_table_datapath_binding);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_datapath_binding_col_tunnel_key);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_datapath_binding_col_external_ids);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_port_binding);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_port_binding_col_datapath);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_port_binding_col_logical_port);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_port_binding_col_tunnel_key);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_port_binding_col_parent_port);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_binding_col_tag);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_binding_col_type);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_port_binding_col_options);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_binding_col_mac);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_port_binding_col_nat_addresses);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_port_binding_col_chassis);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_port_binding_col_gateway_chassis);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_port_binding_col_ha_chassis_group);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_gateway_chassis_col_chassis);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_gateway_chassis_col_name);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_gateway_chassis_col_priority);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_gateway_chassis_col_external_ids);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> -                         &sbrec_gateway_chassis_col_options);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_port_binding_col_external_ids);
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_mac_binding);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_mac_binding_col_datapath);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_ip);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_mac_binding_col_mac);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_mac_binding_col_logical_port);
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_dhcp_options);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dhcp_options_col_code);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dhcp_options_col_type);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dhcp_options_col_name);
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_dhcpv6_options);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_dhcpv6_options_col_code);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_dhcpv6_options_col_type);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_dhcpv6_options_col_name);
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_address_set);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_address_set_col_name);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_address_set_col_addresses);
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_port_group);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_group_col_name);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_port_group_col_ports);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_dns);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dns_col_datapaths);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dns_col_records);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_dns_col_external_ids);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_rbac_role);
> -    add_column_noalert(ovnsb_idl_loop.idl, &sbrec_rbac_role_col_name);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_rbac_role_col_permissions);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_rbac_permission);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_rbac_permission_col_table);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_rbac_permission_col_authorization);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_rbac_permission_col_insert_delete);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> &sbrec_rbac_permission_col_update);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_meter);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_meter_col_name);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_meter_col_unit);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_meter_col_bands);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_meter_band);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_meter_band_col_action);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_meter_band_col_rate);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_meter_band_col_burst_size);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_chassis);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_nb_cfg);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_chassis_col_name);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_ha_chassis);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_col_chassis);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_col_priority);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_col_external_ids);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl,
> &sbrec_table_ha_chassis_group);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_group_col_name);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_group_col_ha_chassis);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_group_col_external_ids);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ha_chassis_group_col_ref_chassis);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_igmp_group);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_igmp_group_col_address);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_igmp_group_col_datapath);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl,
> &sbrec_igmp_group_col_chassis);
> -    ovsdb_idl_add_column(ovnsb_idl_loop.idl, &sbrec_igmp_group_col_ports);
> -
> -    ovsdb_idl_add_table(ovnsb_idl_loop.idl, &sbrec_table_ip_multicast);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_datapath);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_enabled);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_querier);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_eth_src);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_ip4_src);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_table_size);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_idle_timeout);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_query_interval);
> -    add_column_noalert(ovnsb_idl_loop.idl,
> -                       &sbrec_ip_multicast_col_query_max_resp);
> -
> -    struct ovsdb_idl_index *sbrec_chassis_by_name
> -        = chassis_index_create(ovnsb_idl_loop.idl);
> -
> -    struct ovsdb_idl_index *sbrec_ha_chassis_grp_by_name
> -        = ha_chassis_group_index_create(ovnsb_idl_loop.idl);
> -
> -    struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp
> -        = mcast_group_index_create(ovnsb_idl_loop.idl);
> -
> -    struct ovsdb_idl_index *sbrec_ip_mcast_by_dp
> -        = ip_mcast_index_create(ovnsb_idl_loop.idl);
> -
> -    /* Ensure that only a single ovn-northd is active in the deployment by
> -     * acquiring a lock called "ovn_northd" on the southbound database
> -     * and then only performing DB transactions if the lock is held. */
> -    ovsdb_idl_set_lock(ovnsb_idl_loop.idl, "ovn_northd");
> -    bool had_lock = false;
> -
> -    /* Main loop. */
> -    exiting = false;
> -    while (!exiting) {
> -        struct northd_context ctx = {
> -            .ovnnb_idl = ovnnb_idl_loop.idl,
> -            .ovnnb_txn = ovsdb_idl_loop_run(&ovnnb_idl_loop),
> -            .ovnsb_idl = ovnsb_idl_loop.idl,
> -            .ovnsb_txn = ovsdb_idl_loop_run(&ovnsb_idl_loop),
> -            .sbrec_ha_chassis_grp_by_name = sbrec_ha_chassis_grp_by_name,
> -            .sbrec_mcast_group_by_name_dp = sbrec_mcast_group_by_name_dp,
> -            .sbrec_ip_mcast_by_dp = sbrec_ip_mcast_by_dp,
> -        };
> -
> -        if (!had_lock && ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) {
> -            VLOG_INFO("ovn-northd lock acquired. "
> -                      "This ovn-northd instance is now active.");
> -            had_lock = true;
> -        } else if (had_lock && !ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) {
> -            VLOG_INFO("ovn-northd lock lost. "
> -                      "This ovn-northd instance is now on standby.");
> -            had_lock = false;
> -        }
> -
> -        if (ovsdb_idl_has_lock(ovnsb_idl_loop.idl)) {
> -            ovn_db_run(&ctx, sbrec_chassis_by_name, &ovnsb_idl_loop);
> -            if (ctx.ovnsb_txn) {
> -                check_and_add_supported_dhcp_opts_to_sb_db(&ctx);
> -                check_and_add_supported_dhcpv6_opts_to_sb_db(&ctx);
> -                check_and_update_rbac(&ctx);
> -            }
> -        }
> -
> -        unixctl_server_run(unixctl);
> -        unixctl_server_wait(unixctl);
> -        if (exiting) {
> -            poll_immediate_wake();
> -        }
> -        ovsdb_idl_loop_commit_and_wait(&ovnnb_idl_loop);
> -        ovsdb_idl_loop_commit_and_wait(&ovnsb_idl_loop);
> -
> -        poll_block();
> -        if (should_service_stop()) {
> -            exiting = true;
> -        }
> -    }
> -
> -    unixctl_server_destroy(unixctl);
> -    ovsdb_idl_loop_destroy(&ovnnb_idl_loop);
> -    ovsdb_idl_loop_destroy(&ovnsb_idl_loop);
> -    service_stop();
> -
> -    exit(res);
> -}
> -
> -static void
> -ovn_northd_exit(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                const char *argv[] OVS_UNUSED, void *exiting_)
> -{
> -    bool *exiting = exiting_;
> -    *exiting = true;
> -
> -    unixctl_command_reply(conn, NULL);
> -}
> diff --git a/ovn/ovn-architecture.7.xml b/ovn/ovn-architecture.7.xml
> deleted file mode 100644
> index c4099f25a..000000000
> --- a/ovn/ovn-architecture.7.xml
> +++ /dev/null
> @@ -1,2074 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-architecture" section="7" title="OVN Architecture">
> -  <h1>Name</h1>
> -  <p>ovn-architecture -- Open Virtual Network architecture</p>
> -
> -  <h1>Description</h1>
> -
> -  <p>
> -    OVN, the Open Virtual Network, is a system to support virtual network
> -    abstraction.  OVN complements the existing capabilities of OVS to add
> -    native support for virtual network abstractions, such as virtual L2
> and L3
> -    overlays and security groups.  Services such as DHCP are also
> desirable
> -    features.  Just like OVS, OVN's design goal is to have a
> production-quality
> -    implementation that can operate at significant scale.
> -  </p>
> -
> -  <p>
> -    An OVN deployment consists of several components:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      <p>
> -        A <dfn>Cloud Management System</dfn> (<dfn>CMS</dfn>), which is
> -        OVN's ultimate client (via its users and administrators).  OVN
> -        integration requires installing a CMS-specific plugin and
> -        related software (see below).  OVN initially targets OpenStack
> -        as CMS.
> -      </p>
> -
> -      <p>
> -        We generally speak of ``the'' CMS, but one can imagine scenarios
> in
> -        which multiple CMSes manage different parts of an OVN deployment.
> -      </p>
> -    </li>
> -
> -    <li>
> -      An OVN Database physical or virtual node (or, eventually, cluster)
> -      installed in a central location.
> -    </li>
> -
> -    <li>
> -      One or more (usually many) <dfn>hypervisors</dfn>.  Hypervisors
> must run
> -      Open vSwitch and implement the interface described in
> -      <code>IntegrationGuide.rst</code> in the OVS source tree.  Any
> hypervisor
> -      platform supported by Open vSwitch is acceptable.
> -    </li>
> -
> -    <li>
> -      <p>
> -        Zero or more <dfn>gateways</dfn>.  A gateway extends a
> tunnel-based
> -        logical network into a physical network by bidirectionally
> forwarding
> -        packets between tunnels and a physical Ethernet port.  This allows
> -        non-virtualized machines to participate in logical networks.  A
> gateway
> -        may be a physical host, a virtual machine, or an ASIC-based
> hardware
> -        switch that supports the <code>vtep</code>(5) schema.
> -      </p>
> -
> -      <p>
> -        Hypervisors and gateways are together called <dfn>transport
> node</dfn>
> -        or <dfn>chassis</dfn>.
> -      </p>
> -    </li>
> -  </ul>
> -
> -  <p>
> -    The diagram below shows how the major components of OVN and related
> -    software interact.  Starting at the top of the diagram, we have:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      The Cloud Management System, as defined above.
> -    </li>
> -
> -    <li>
> -      <p>
> -        The <dfn>OVN/CMS Plugin</dfn> is the component of the CMS that
> -        interfaces to OVN.  In OpenStack, this is a Neutron plugin.
> -        The plugin's main purpose is to translate the CMS's notion of
> logical
> -        network configuration, stored in the CMS's configuration database
> in a
> -        CMS-specific format, into an intermediate representation
> understood by
> -        OVN.
> -      </p>
> -
> -      <p>
> -        This component is necessarily CMS-specific, so a new plugin needs
> to be
> -        developed for each CMS that is integrated with OVN.  All of the
> -        components below this one in the diagram are CMS-independent.
> -      </p>
> -    </li>
> -
> -    <li>
> -      <p>
> -        The <dfn>OVN Northbound Database</dfn> receives the intermediate
> -        representation of logical network configuration passed down by the
> -        OVN/CMS Plugin.  The database schema is meant to be ``impedance
> -        matched'' with the concepts used in a CMS, so that it directly
> supports
> -        notions of logical switches, routers, ACLs, and so on.  See
> -        <code>ovn-nb</code>(5) for details.
> -      </p>
> -
> -      <p>
> -        The OVN Northbound Database has only two clients: the OVN/CMS
> Plugin
> -        above it and <code>ovn-northd</code> below it.
> -      </p>
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code>(8) connects to the OVN Northbound Database
> -      above it and the OVN Southbound Database below it.  It translates
> the
> -      logical network configuration in terms of conventional network
> -      concepts, taken from the OVN Northbound Database, into logical
> -      datapath flows in the OVN Southbound Database below it.
> -    </li>
> -
> -    <li>
> -      <p>
> -    The <dfn>OVN Southbound Database</dfn> is the center of the system.
> -    Its clients are <code>ovn-northd</code>(8) above it and
> -    <code>ovn-controller</code>(8) on every transport node below it.
> -      </p>
> -
> -      <p>
> -        The OVN Southbound Database contains three kinds of data:
> <dfn>Physical
> -        Network</dfn> (PN) tables that specify how to reach hypervisor and
> -        other nodes, <dfn>Logical Network</dfn> (LN) tables that describe
> the
> -        logical network in terms of ``logical datapath flows,'' and
> -        <dfn>Binding</dfn> tables that link logical network components'
> -        locations to the physical network.  The hypervisors populate the
> PN and
> -        Port_Binding tables, whereas <code>ovn-northd</code>(8) populates
> the
> -        LN tables.
> -      </p>
> -
> -      <p>
> -    OVN Southbound Database performance must scale with the number of
> -    transport nodes.  This will likely require some work on
> -    <code>ovsdb-server</code>(1) as we encounter bottlenecks.
> -    Clustering for availability may be needed.
> -      </p>
> -    </li>
> -  </ul>
> -
> -  <p>
> -    The remaining components are replicated onto each hypervisor:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      <code>ovn-controller</code>(8) is OVN's agent on each hypervisor and
> -      software gateway.  Northbound, it connects to the OVN Southbound
> -      Database to learn about OVN configuration and status and to
> -      populate the PN table and the <code>Chassis</code> column in
> -      <code>Binding</code> table with the hypervisor's status.
> -      Southbound, it connects to <code>ovs-vswitchd</code>(8) as an
> -      OpenFlow controller, for control over network traffic, and to the
> -      local <code>ovsdb-server</code>(1) to allow it to monitor and
> -      control Open vSwitch configuration.
> -    </li>
> -
> -    <li>
> -      <code>ovs-vswitchd</code>(8) and <code>ovsdb-server</code>(1) are
> -      conventional components of Open vSwitch.
> -    </li>
> -  </ul>
> -
> -  <pre fixed="yes">
> -                                  CMS
> -                                   |
> -                                   |
> -                       +-----------|-----------+
> -                       |           |           |
> -                       |     OVN/CMS Plugin    |
> -                       |           |           |
> -                       |           |           |
> -                       |   OVN Northbound DB   |
> -                       |           |           |
> -                       |           |           |
> -                       |       ovn-northd      |
> -                       |           |           |
> -                       +-----------|-----------+
> -                                   |
> -                                   |
> -                         +-------------------+
> -                         | OVN Southbound DB |
> -                         +-------------------+
> -                                   |
> -                                   |
> -                +------------------+------------------+
> -                |                  |                  |
> -  HV 1          |                  |    HV n          |
> -+---------------|---------------+  .  +---------------|---------------+
> -|               |               |  .  |               |               |
> -|        ovn-controller         |  .  |        ovn-controller         |
> -|         |          |          |  .  |         |          |          |
> -|         |          |          |     |         |          |          |
> -|  ovs-vswitchd   ovsdb-server  |     |  ovs-vswitchd   ovsdb-server  |
> -|                               |     |                               |
> -+-------------------------------+     +-------------------------------+
> -  </pre>
> -
> -  <h2>Information Flow in OVN</h2>
> -
> -  <p>
> -    Configuration data in OVN flows from north to south.  The CMS,
> through its
> -    OVN/CMS plugin, passes the logical network configuration to
> -    <code>ovn-northd</code> via the northbound database.  In turn,
> -    <code>ovn-northd</code> compiles the configuration into a lower-level
> form
> -    and passes it to all of the chassis via the southbound database.
> -  </p>
> -
> -  <p>
> -    Status information in OVN flows from south to north.  OVN currently
> -    provides only a few forms of status information.  First,
> -    <code>ovn-northd</code> populates the <code>up</code> column in the
> -    northbound <code>Logical_Switch_Port</code> table: if a logical port's
> -    <code>chassis</code> column in the southbound
> <code>Port_Binding</code>
> -    table is nonempty, it sets <code>up</code> to <code>true</code>,
> otherwise
> -    to <code>false</code>.  This allows the CMS to detect when a VM's
> -    networking has come up.
> -  </p>
> -
> -  <p>
> -    Second, OVN provides feedback to the CMS on the realization of its
> -    configuration, that is, whether the configuration provided by the CMS
> has
> -    taken effect.  This feature requires the CMS to participate in a
> sequence
> -    number protocol, which works the following way:
> -  </p>
> -
> -  <ol>
> -    <li>
> -      When the CMS updates the configuration in the northbound database,
> as
> -      part of the same transaction, it increments the value of the
> -      <code>nb_cfg</code> column in the <code>NB_Global</code> table.
> (This is
> -      only necessary if the CMS wants to know when the configuration has
> been
> -      realized.)
> -    </li>
> -
> -    <li>
> -      When <code>ovn-northd</code> updates the southbound database based
> on a
> -      given snapshot of the northbound database, it copies
> <code>nb_cfg</code>
> -      from northbound <code>NB_Global</code> into the southbound database
> -      <code>SB_Global</code> table, as part of the same transaction.
> (Thus, an
> -      observer monitoring both databases can determine when the southbound
> -      database is caught up with the northbound.)
> -    </li>
> -
> -    <li>
> -      After <code>ovn-northd</code> receives confirmation from the
> southbound
> -      database server that its changes have committed, it updates
> -      <code>sb_cfg</code> in the northbound <code>NB_Global</code> table
> to the
> -      <code>nb_cfg</code> version that was pushed down.  (Thus, the CMS or
> -      another observer can determine when the southbound database is
> caught up
> -      without a connection to the southbound database.)
> -    </li>
> -
> -    <li>
> -      The <code>ovn-controller</code> process on each chassis receives the
> -      updated southbound database, with the updated <code>nb_cfg</code>.
> This
> -      process in turn updates the physical flows installed in the
> chassis's
> -      Open vSwitch instances.  When it receives confirmation from Open
> vSwitch
> -      that the physical flows have been updated, it updates
> <code>nb_cfg</code>
> -      in its own <code>Chassis</code> record in the southbound database.
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code> monitors the <code>nb_cfg</code> column in
> all of
> -      the <code>Chassis</code> records in the southbound database.  It
> keeps
> -      track of the minimum value among all the records and copies it into
> the
> -      <code>hv_cfg</code> column in the northbound <code>NB_Global</code>
> -      table.  (Thus, the CMS or another observer can determine when all
> of the
> -      hypervisors have caught up to the northbound configuration.)
> -    </li>
> -  </ol>
> -
> -  <h2>Chassis Setup</h2>
> -
> -  <p>
> -    Each chassis in an OVN deployment must be configured with an Open
> vSwitch
> -    bridge dedicated for OVN's use, called the <dfn>integration
> bridge</dfn>.
> -    System startup scripts may create this bridge prior to starting
> -    <code>ovn-controller</code> if desired.  If this bridge does not
> exist when
> -    ovn-controller starts, it will be created automatically with the
> default
> -    configuration suggested below.  The ports on the integration bridge
> include:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      On any chassis, tunnel ports that OVN uses to maintain logical
> network
> -      connectivity.  <code>ovn-controller</code> adds, updates, and
> removes
> -      these tunnel ports.
> -    </li>
> -
> -    <li>
> -      On a hypervisor, any VIFs that are to be attached to logical
> networks.
> -      The hypervisor itself, or the integration between Open vSwitch and
> the
> -      hypervisor (described in <code>IntegrationGuide.rst</code>) takes
> care of
> -      this.  (This is not part of OVN or new to OVN; this is pre-existing
> -      integration work that has already been done on hypervisors that
> support
> -      OVS.)
> -    </li>
> -
> -    <li>
> -      On a gateway, the physical port used for logical network
> connectivity.
> -      System startup scripts add this port to the bridge prior to starting
> -      <code>ovn-controller</code>.  This can be a patch port to another
> bridge,
> -      instead of a physical port, in more sophisticated setups.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    Other ports should not be attached to the integration bridge.  In
> -    particular, physical ports attached to the underlay network (as
> opposed to
> -    gateway ports, which are physical ports attached to logical networks)
> must
> -    not be attached to the integration bridge.  Underlay physical ports
> should
> -    instead be attached to a separate Open vSwitch bridge (they need not
> be
> -    attached to any bridge at all, in fact).
> -  </p>
> -
> -  <p>
> -    The integration bridge should be configured as described below.
> -    The effect of each of these settings is documented in
> -    <code>ovs-vswitchd.conf.db</code>(5):
> -  </p>
> -
> -  <!-- Keep the following in sync with create_br_int() in
> -       ovn/controller/ovn-controller.c. -->
> -  <dl>
> -    <dt><code>fail-mode=secure</code></dt>
> -    <dd>
> -      Avoids switching packets between isolated logical networks before
> -      <code>ovn-controller</code> starts up.  See <code>Controller Failure
> -      Settings</code> in <code>ovs-vsctl</code>(8) for more information.
> -    </dd>
> -
> -    <dt><code>other-config:disable-in-band=true</code></dt>
> -    <dd>
> -      Suppresses in-band control flows for the integration bridge.  It
> would be
> -      unusual for such flows to show up anyway, because OVN uses a local
> -      controller (over a Unix domain socket) instead of a remote
> controller.
> -      It's possible, however, for some other bridge in the same system to
> have
> -      an in-band remote controller, and in that case this suppresses the
> flows
> -      that in-band control would ordinarily set up.  Refer to the
> documentation
> -      for more information.
> -    </dd>
> -  </dl>
> -
> -  <p>
> -    The customary name for the integration bridge is <code>br-int</code>,
> but
> -    another name may be used.
> -  </p>
> -
> -  <h2>Logical Networks</h2>
> -
> -  <p>
> -    A <dfn>logical network</dfn> implements the same concepts as physical
> -    networks, but they are insulated from the physical network with
> tunnels or
> -    other encapsulations.  This allows logical networks to have separate
> IP and
> -    other address spaces that overlap, without conflicting, with those
> used for
> -    physical networks.  Logical network topologies can be arranged without
> -    regard for the topologies of the physical networks on which they run.
> -  </p>
> -
> -  <p>
> -    Logical network concepts in OVN include:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      <dfn>Logical switches</dfn>, the logical version of Ethernet
> switches.
> -    </li>
> -
> -    <li>
> -      <dfn>Logical routers</dfn>, the logical version of IP routers.
> Logical
> -      switches and routers can be connected into sophisticated topologies.
> -    </li>
> -
> -    <li>
> -      <dfn>Logical datapaths</dfn> are the logical version of an OpenFlow
> -      switch.  Logical switches and routers are both implemented as
> logical
> -      datapaths.
> -    </li>
> -
> -    <li>
> -      <p>
> -        <dfn>Logical ports</dfn> represent the points of connectivity in
> and
> -        out of logical switches and logical routers.  Some common types of
> -        logical ports are:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          Logical ports representing VIFs.
> -        </li>
> -
> -        <li>
> -          <dfn>Localnet ports</dfn> represent the points of connectivity
> -          between logical switches and the physical network.  They are
> -          implemented as OVS patch ports between the integration bridge
> -          and the separate Open vSwitch bridge that underlay physical
> -          ports attach to.
> -        </li>
> -
> -        <li>
> -          <dfn>Logical patch ports</dfn> represent the points of
> -          connectivity between logical switches and logical routers, and
> -          in some cases between peer logical routers.  There is a pair of
> -          logical patch ports at each such point of connectivity, one on
> -          each side.
> -        </li>
> -        <li>
> -          <dfn>Localport ports</dfn> represent the points of local
> -          connectivity between logical switches and VIFs. These ports are
> -          present in every chassis (not bound to any particular one) and
> -          traffic from them will never go through a tunnel. A
> -          <code>localport</code> is expected to only generate traffic
> destined
> -          for a local destination, typically in response to a request it
> -          received.
> -          One use case is how OpenStack Neutron uses a
> <code>localport</code>
> -          port for serving metadata to VM's residing on every hypervisor.
> A
> -          metadata proxy process is attached to this port on every host
> and all
> -          VM's within the same network will reach it at the same IP/MAC
> address
> -          without any traffic being sent over a tunnel. Further details
> can be
> -          seen at
> https://docs.openstack.org/developer/networking-ovn/design/metadata_api.html
> .
> -        </li>
> -      </ul>
> -    </li>
> -  </ul>
> -
> -  <h2>Life Cycle of a VIF</h2>
> -
> -  <p>
> -    Tables and their schemas presented in isolation are difficult to
> -    understand.  Here's an example.
> -  </p>
> -
> -  <p>
> -    A VIF on a hypervisor is a virtual network interface attached either
> -    to a VM or a container running directly on that hypervisor (This is
> -    different from the interface of a container running inside a VM).
> -  </p>
> -
> -  <p>
> -    The steps in this example refer often to details of the OVN and OVN
> -    Northbound database schemas.  Please see <code>ovn-sb</code>(5) and
> -    <code>ovn-nb</code>(5), respectively, for the full story on these
> -    databases.
> -  </p>
> -
> -  <ol>
> -    <li>
> -      A VIF's life cycle begins when a CMS administrator creates a new VIF
> -      using the CMS user interface or API and adds it to a switch (one
> -      implemented by OVN as a logical switch).  The CMS updates its own
> -      configuration.  This includes associating unique, persistent
> identifier
> -      <var>vif-id</var> and Ethernet address <var>mac</var> with the VIF.
> -    </li>
> -
> -    <li>
> -      The CMS plugin updates the OVN Northbound database to include the
> new
> -      VIF, by adding a row to the <code>Logical_Switch_Port</code>
> -      table.  In the new row, <code>name</code> is <var>vif-id</var>,
> -      <code>mac</code> is <var>mac</var>, <code>switch</code> points to
> -      the OVN logical switch's Logical_Switch record, and other columns
> -      are initialized appropriately.
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code> receives the OVN Northbound database
> update.  In
> -      turn, it makes the corresponding updates to the OVN Southbound
> database,
> -      by adding rows to the OVN Southbound database
> <code>Logical_Flow</code>
> -      table to reflect the new port, e.g. add a flow to recognize that
> packets
> -      destined to the new port's MAC address should be delivered to it,
> and
> -      update the flow that delivers broadcast and multicast packets to
> include
> -      the new port.  It also creates a record in the <code>Binding</code>
> table
> -      and populates all its columns except the column that identifies the
> -      <code>chassis</code>.
> -    </li>
> -
> -    <li>
> -      On every hypervisor, <code>ovn-controller</code> receives the
> -      <code>Logical_Flow</code> table updates that
> <code>ovn-northd</code> made
> -      in the previous step.  As long as the VM that owns the VIF is
> powered
> -      off, <code>ovn-controller</code> cannot do much; it cannot, for
> example,
> -      arrange to send packets to or receive packets from the VIF, because
> the
> -      VIF does not actually exist anywhere.
> -    </li>
> -
> -    <li>
> -      Eventually, a user powers on the VM that owns the VIF.  On the
> hypervisor
> -      where the VM is powered on, the integration between the hypervisor
> and
> -      Open vSwitch (described in <code>IntegrationGuide.rst</code>) adds
> the VIF
> -      to the OVN integration bridge and stores <var>vif-id</var> in
> -      <code>external_ids</code>:<code>iface-id</code> to indicate that the
> -      interface is an instantiation of the new VIF.  (None of this code
> is new
> -      in OVN; this is pre-existing integration work that has already been
> done
> -      on hypervisors that support OVS.)
> -    </li>
> -
> -    <li>
> -      On the hypervisor where the VM is powered on,
> <code>ovn-controller</code>
> -      notices <code>external_ids</code>:<code>iface-id</code> in the new
> -      Interface. In response, in the OVN Southbound DB, it updates the
> -      <code>Binding</code> table's <code>chassis</code> column for the
> -      row that links the logical port from
> <code>external_ids</code>:<code>
> -      iface-id</code> to the hypervisor. Afterward,
> <code>ovn-controller</code>
> -      updates the local hypervisor's OpenFlow tables so that packets to
> and from
> -      the VIF are properly handled.
> -    </li>
> -
> -    <li>
> -      Some CMS systems, including OpenStack, fully start a VM only when
> its
> -      networking is ready.  To support this, <code>ovn-northd</code>
> notices
> -      the <code>chassis</code> column updated for the row in
> -      <code>Binding</code> table and pushes this upward by updating the
> -      <ref column="up" table="Logical_Switch_Port" db="OVN_NB"/> column
> -      in the OVN Northbound database's <ref table="Logical_Switch_Port"
> -      db="OVN_NB"/> table to indicate that the VIF is now up.  The CMS,
> -      if it uses this feature, can then react by allowing the VM's
> -      execution to proceed.
> -    </li>
> -
> -    <li>
> -      On every hypervisor but the one where the VIF resides,
> -      <code>ovn-controller</code> notices the completely populated row in
> the
> -      <code>Binding</code> table.  This provides
> <code>ovn-controller</code>
> -      the physical location of the logical port, so each instance updates
> the
> -      OpenFlow tables of its switch (based on logical datapath flows in
> the OVN
> -      DB <code>Logical_Flow</code> table) so that packets to and from the
> VIF
> -      can be properly handled via tunnels.
> -    </li>
> -
> -    <li>
> -      Eventually, a user powers off the VM that owns the VIF.  On the
> -      hypervisor where the VM was powered off, the VIF is deleted from
> the OVN
> -      integration bridge.
> -    </li>
> -
> -    <li>
> -      On the hypervisor where the VM was powered off,
> -      <code>ovn-controller</code> notices that the VIF was deleted.  In
> -      response, it removes the <code>Chassis</code> column content in the
> -      <code>Binding</code> table for the logical port.
> -    </li>
> -
> -    <li>
> -      On every hypervisor, <code>ovn-controller</code> notices the empty
> -      <code>Chassis</code> column in the <code>Binding</code> table's row
> -      for the logical port.  This means that <code>ovn-controller</code>
> no
> -      longer knows the physical location of the logical port, so each
> instance
> -      updates its OpenFlow table to reflect that.
> -    </li>
> -
> -    <li>
> -      Eventually, when the VIF (or its entire VM) is no longer needed by
> -      anyone, an administrator deletes the VIF using the CMS user
> interface or
> -      API.  The CMS updates its own configuration.
> -    </li>
> -
> -    <li>
> -      The CMS plugin removes the VIF from the OVN Northbound database,
> -      by deleting its row in the <code>Logical_Switch_Port</code> table.
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code> receives the OVN Northbound update and in
> turn
> -      updates the OVN Southbound database accordingly, by removing or
> updating
> -      the rows from the OVN Southbound database <code>Logical_Flow</code>
> table
> -      and <code>Binding</code> table that were related to the
> now-destroyed
> -      VIF.
> -    </li>
> -
> -    <li>
> -      On every hypervisor, <code>ovn-controller</code> receives the
> -      <code>Logical_Flow</code> table updates that
> <code>ovn-northd</code> made
> -      in the previous step.  <code>ovn-controller</code> updates OpenFlow
> -      tables to reflect the update, although there may not be much to do,
> since
> -      the VIF had already become unreachable when it was removed from the
> -      <code>Binding</code> table in a previous step.
> -    </li>
> -  </ol>
> -
> -  <h2>Life Cycle of a Container Interface Inside a VM</h2>
> -
> -  <p>
> -    OVN provides virtual network abstractions by converting information
> -    written in OVN_NB database to OpenFlow flows in each hypervisor.
> Secure
> -    virtual networking for multi-tenants can only be provided if OVN
> controller
> -    is the only entity that can modify flows in Open vSwitch.  When the
> -    Open vSwitch integration bridge resides in the hypervisor, it is a
> -    fair assumption to make that tenant workloads running inside VMs
> cannot
> -    make any changes to Open vSwitch flows.
> -  </p>
> -
> -  <p>
> -    If the infrastructure provider trusts the applications inside the
> -    containers not to break out and modify the Open vSwitch flows, then
> -    containers can be run in hypervisors.  This is also the case when
> -    containers are run inside the VMs and Open vSwitch integration bridge
> -    with flows added by OVN controller resides in the same VM.  For both
> -    the above cases, the workflow is the same as explained with an example
> -    in the previous section ("Life Cycle of a VIF").
> -  </p>
> -
> -  <p>
> -    This section talks about the life cycle of a container interface (CIF)
> -    when containers are created in the VMs and the Open vSwitch
> integration
> -    bridge resides inside the hypervisor.  In this case, even if a
> container
> -    application breaks out, other tenants are not affected because the
> -    containers running inside the VMs cannot modify the flows in the
> -    Open vSwitch integration bridge.
> -  </p>
> -
> -  <p>
> -    When multiple containers are created inside a VM, there are multiple
> -    CIFs associated with them.  The network traffic associated with these
> -    CIFs need to reach the Open vSwitch integration bridge running in the
> -    hypervisor for OVN to support virtual network abstractions.  OVN
> should
> -    also be able to distinguish network traffic coming from different
> CIFs.
> -    There are two ways to distinguish network traffic of CIFs.
> -  </p>
> -
> -  <p>
> -    One way is to provide one VIF for every CIF (1:1 model).  This means
> that
> -    there could be a lot of network devices in the hypervisor.  This
> would slow
> -    down OVS because of all the additional CPU cycles needed for the
> management
> -    of all the VIFs.  It would also mean that the entity creating the
> -    containers in a VM should also be able to create the corresponding
> VIFs in
> -    the hypervisor.
> -  </p>
> -
> -  <p>
> -    The second way is to provide a single VIF for all the CIFs (1:many
> model).
> -    OVN could then distinguish network traffic coming from different CIFs
> via
> -    a tag written in every packet.  OVN uses this mechanism and uses VLAN
> as
> -    the tagging mechanism.
> -  </p>
> -
> -  <ol>
> -    <li>
> -      A CIF's life cycle begins when a container is spawned inside a VM by
> -      the either the same CMS that created the VM or a tenant that owns
> that VM
> -      or even a container Orchestration System that is different than the
> CMS
> -      that initially created the VM.  Whoever the entity is, it will need
> to
> -      know the <var>vif-id</var> that is associated with the network
> interface
> -      of the VM through which the container interface's network traffic is
> -      expected to go through.  The entity that creates the container
> interface
> -      will also need to choose an unused VLAN inside that VM.
> -    </li>
> -
> -    <li>
> -      The container spawning entity (either directly or through the CMS
> that
> -      manages the underlying infrastructure) updates the OVN Northbound
> -      database to include the new CIF, by adding a row to the
> -      <code>Logical_Switch_Port</code> table.  In the new row,
> -      <code>name</code> is any unique identifier,
> -      <code>parent_name</code> is the <var>vif-id</var> of the VM
> -      through which the CIF's network traffic is expected to go through
> -      and the <code>tag</code> is the VLAN tag that identifies the
> -      network traffic of that CIF.
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code> receives the OVN Northbound database
> update.  In
> -      turn, it makes the corresponding updates to the OVN Southbound
> database,
> -      by adding rows to the OVN Southbound database's
> <code>Logical_Flow</code>
> -      table to reflect the new port and also by creating a new row in the
> -      <code>Binding</code> table and populating all its columns except the
> -      column that identifies the <code>chassis</code>.
> -    </li>
> -
> -    <li>
> -      On every hypervisor, <code>ovn-controller</code> subscribes to the
> -      changes in the <code>Binding</code> table.  When a new row is
> created
> -      by <code>ovn-northd</code> that includes a value in
> -      <code>parent_port</code> column of <code>Binding</code> table, the
> -      <code>ovn-controller</code> in the hypervisor whose OVN integration
> bridge
> -      has that same value in <var>vif-id</var> in
> -      <code>external_ids</code>:<code>iface-id</code>
> -      updates the local hypervisor's OpenFlow tables so that packets to
> and
> -      from the VIF with the particular VLAN <code>tag</code> are properly
> -      handled.  Afterward it updates the <code>chassis</code> column of
> -      the <code>Binding</code> to reflect the physical location.
> -    </li>
> -
> -    <li>
> -      One can only start the application inside the container after the
> -      underlying network is ready.  To support this,
> <code>ovn-northd</code>
> -      notices the updated <code>chassis</code> column in
> <code>Binding</code>
> -      table and updates the <ref column="up" table="Logical_Switch_Port"
> -      db="OVN_NB"/> column in the OVN Northbound database's
> -      <ref table="Logical_Switch_Port" db="OVN_NB"/> table to indicate
> that the
> -      CIF is now up.  The entity responsible to start the container
> application
> -      queries this value and starts the application.
> -    </li>
> -
> -    <li>
> -      Eventually the entity that created and started the container, stops
> it.
> -      The entity, through the CMS (or directly) deletes its row in the
> -      <code>Logical_Switch_Port</code> table.
> -    </li>
> -
> -    <li>
> -      <code>ovn-northd</code> receives the OVN Northbound update and in
> turn
> -      updates the OVN Southbound database accordingly, by removing or
> updating
> -      the rows from the OVN Southbound database <code>Logical_Flow</code>
> table
> -      that were related to the now-destroyed CIF.  It also deletes the
> row in
> -      the <code>Binding</code> table for that CIF.
> -    </li>
> -
> -    <li>
> -      On every hypervisor, <code>ovn-controller</code> receives the
> -      <code>Logical_Flow</code> table updates that
> <code>ovn-northd</code> made
> -      in the previous step.  <code>ovn-controller</code> updates OpenFlow
> -      tables to reflect the update.
> -    </li>
> -  </ol>
> -
> -  <h2>Architectural Physical Life Cycle of a Packet</h2>
> -
> -  <p>
> -    This section describes how a packet travels from one virtual machine
> or
> -    container to another through OVN.  This description focuses on the
> physical
> -    treatment of a packet; for a description of the logical life cycle of
> a
> -    packet, please refer to the <code>Logical_Flow</code> table in
> -    <code>ovn-sb</code>(5).
> -  </p>
> -
> -  <p>
> -    This section mentions several data and metadata fields, for clarity
> -    summarized here:
> -  </p>
> -
> -  <dl>
> -    <dt>tunnel key</dt>
> -    <dd>
> -      When OVN encapsulates a packet in Geneve or another tunnel, it
> attaches
> -      extra data to it to allow the receiving OVN instance to process it
> -      correctly.  This takes different forms depending on the particular
> -      encapsulation, but in each case we refer to it here as the ``tunnel
> -      key.''  See <code>Tunnel Encapsulations</code>, below, for details.
> -    </dd>
> -
> -    <dt>logical datapath field</dt>
> -    <dd>
> -      A field that denotes the logical datapath through which a packet is
> being
> -      processed.
> -      <!-- Keep the following in sync with MFF_LOG_DATAPATH in
> -           ovn/lib/logical-fields.h. -->
> -      OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
> -      ``metadata'' to store the logical datapath.  (This field is passed
> across
> -      tunnels as part of the tunnel key.)
> -    </dd>
> -
> -    <dt>logical input port field</dt>
> -    <dd>
> -      <p>
> -        A field that denotes the logical port from which the packet
> -        entered the logical datapath.
> -        <!-- Keep the following in sync with MFF_LOG_INPORT in
> -             ovn/lib/logical-fields.h. -->
> -        OVN stores this in Open vSwitch extension register number 14.
> -      </p>
> -
> -      <p>
> -        Geneve and STT tunnels pass this field as part of the tunnel key.
> -        Although VXLAN tunnels do not explicitly carry a logical input
> port,
> -        OVN only uses VXLAN to communicate with gateways that from OVN's
> -        perspective consist of only a single logical port, so that OVN
> can set
> -        the logical input port field to this one on ingress to the OVN
> logical
> -        pipeline.
> -      </p>
> -    </dd>
> -
> -    <dt>logical output port field</dt>
> -    <dd>
> -      <p>
> -        A field that denotes the logical port from which the packet will
> -        leave the logical datapath.  This is initialized to 0 at the
> -        beginning of the logical ingress pipeline.
> -        <!-- Keep the following in sync with MFF_LOG_OUTPORT in
> -             ovn/lib/logical-fields.h. -->
> -        OVN stores this in Open vSwitch extension register number 15.
> -      </p>
> -
> -      <p>
> -        Geneve and STT tunnels pass this field as part of the tunnel key.
> -        VXLAN tunnels do not transmit the logical output port field.
> -        Since VXLAN tunnels do not carry a logical output port field in
> -        the tunnel key, when a packet is received from VXLAN tunnel by
> -        an OVN hypervisor, the packet is resubmitted to table 8 to
> -        determine the output port(s);  when the packet reaches table 32,
> -        these packets are resubmitted to table 33 for local delivery by
> -        checking a MLF_RCV_FROM_VXLAN flag, which is set when the packet
> -        arrives from a VXLAN tunnel.
> -      </p>
> -    </dd>
> -
> -    <dt>conntrack zone field for logical ports</dt>
> -    <dd>
> -      A field that denotes the connection tracking zone for logical ports.
> -      The value only has local significance and is not meaningful between
> -      chassis.  This is initialized to 0 at the beginning of the logical
> -        <!-- Keep the following in sync with MFF_LOG_CT_ZONE in
> -             ovn/lib/logical-fields.h. -->
> -      ingress pipeline.  OVN stores this in Open vSwitch extension
> register
> -      number 13.
> -    </dd>
> -
> -    <dt>conntrack zone fields for routers</dt>
> -    <dd>
> -      Fields that denote the connection tracking zones for routers.  These
> -      values only have local significance and are not meaningful between
> -      chassis.  OVN stores the zone information for DNATting in Open
> vSwitch
> -        <!-- Keep the following in sync with MFF_LOG_DNAT_ZONE and
> -        MFF_LOG_SNAT_ZONE in ovn/lib/logical-fields.h. -->
> -      extension register number 11 and zone information for SNATing in
> -      Open vSwitch extension register number 12.
> -    </dd>
> -
> -    <dt>logical flow flags</dt>
> -    <dd>
> -      The logical flags are intended to handle keeping context between
> -      tables in order to decide which rules in subsequent tables are
> -      matched.  These values only have local significance and are not
> -      meaningful between chassis.  OVN stores the logical flags in
> -        <!-- Keep the following in sync with MFF_LOG_FLAGS in
> -             ovn/lib/logical-fields.h. -->
> -      Open vSwitch extension register number 10.
> -    </dd>
> -
> -    <dt>VLAN ID</dt>
> -    <dd>
> -      The VLAN ID is used as an interface between OVN and containers
> nested
> -      inside a VM (see <code>Life Cycle of a container interface inside a
> -      VM</code>, above, for more information).
> -    </dd>
> -  </dl>
> -
> -  <p>
> -    Initially, a VM or container on the ingress hypervisor sends a packet
> on a
> -    port attached to the OVN integration bridge.  Then:
> -  </p>
> -
> -  <ol>
> -    <li>
> -      <p>
> -        OpenFlow table 0 performs physical-to-logical translation.  It
> matches
> -        the packet's ingress port.  Its actions annotate the packet with
> -        logical metadata, by setting the logical datapath field to
> identify the
> -        logical datapath that the packet is traversing and the logical
> input
> -        port field to identify the ingress port.  Then it resubmits to
> table 8
> -        to enter the logical ingress pipeline.
> -      </p>
> -
> -      <p>
> -        Packets that originate from a container nested within a VM are
> treated
> -        in a slightly different way.  The originating container can be
> -        distinguished based on the VIF-specific VLAN ID, so the
> -        physical-to-logical translation flows additionally match on VLAN
> ID and
> -        the actions strip the VLAN header.  Following this step, OVN
> treats
> -        packets from containers just like any other packets.
> -      </p>
> -
> -      <p>
> -        Table 0 also processes packets that arrive from other chassis.  It
> -        distinguishes them from other packets by ingress port, which is a
> -        tunnel.  As with packets just entering the OVN pipeline, the
> actions
> -        annotate these packets with logical datapath and logical ingress
> port
> -        metadata.  In addition, the actions set the logical output port
> field,
> -        which is available because in OVN tunneling occurs after the
> logical
> -        output port is known.  These three pieces of information are
> obtained
> -        from the tunnel encapsulation metadata (see <code>Tunnel
> -        Encapsulations</code> for encoding details).  Then the actions
> resubmit
> -        to table 33 to enter the logical egress pipeline.
> -      </p>
> -    </li>
> -
> -    <li>
> -      <p>
> -        OpenFlow tables 8 through 31 execute the logical ingress pipeline
> from
> -        the <code>Logical_Flow</code> table in the OVN Southbound
> database.
> -        These tables are expressed entirely in terms of logical concepts
> like
> -        logical ports and logical datapaths.  A big part of
> -        <code>ovn-controller</code>'s job is to translate them into
> equivalent
> -        OpenFlow (in particular it translates the table numbers:
> -        <code>Logical_Flow</code> tables 0 through 23 become OpenFlow
> tables 8
> -        through 31).
> -      </p>
> -
> -      <p>
> -        Each logical flow maps to one or more OpenFlow flows.  An actual
> packet
> -        ordinarily matches only one of these, although in some cases it
> can
> -        match more than one of these flows (which is not a problem
> because all
> -        of them have the same actions).  <code>ovn-controller</code> uses
> the
> -        first 32 bits of the logical flow's UUID as the cookie for its
> OpenFlow
> -        flow or flows.  (This is not necessarily unique, since the first
> 32
> -        bits of a logical flow's UUID is not necessarily unique.)
> -      </p>
> -
> -      <p>
> -        Some logical flows can map to the Open vSwitch ``conjunctive
> match''
> -        extension (see <code>ovs-fields</code>(7)).  Flows with a
> -        <code>conjunction</code> action use an OpenFlow cookie of 0,
> because
> -        they can correspond to multiple logical flows.  The OpenFlow flow
> for a
> -        conjunctive match includes a match on <code>conj_id</code>.
> -      </p>
> -
> -      <p>
> -        Some logical flows may not be represented in the OpenFlow tables
> on a
> -        given hypervisor, if they could not be used on that hypervisor.
> For
> -        example, if no VIF in a logical switch resides on a given
> hypervisor,
> -        and the logical switch is not otherwise reachable on that
> hypervisor
> -        (e.g. over a series of hops through logical switches and routers
> -        starting from a VIF on the hypervisor), then the logical flow may
> not
> -        be represented there.
> -      </p>
> -
> -      <p>
> -        Most OVN actions have fairly obvious implementations in OpenFlow
> (with
> -        OVS extensions), e.g. <code>next;</code> is implemented as
> -        <code>resubmit</code>, <code><var>field</var> =
> -        <var>constant</var>;</code> as <code>set_field</code>.  A few are
> worth
> -        describing in more detail:
> -      </p>
> -
> -      <dl>
> -        <dt><code>output:</code></dt>
> -        <dd>
> -          Implemented by resubmitting the packet to table 32.  If the
> pipeline
> -          executes more than one <code>output</code> action, then each
> one is
> -          separately resubmitted to table 32.  This can be used to send
> -          multiple copies of the packet to multiple ports.  (If the
> packet was
> -          not modified between the <code>output</code> actions, and some
> of the
> -          copies are destined to the same hypervisor, then using a logical
> -          multicast output port would save bandwidth between hypervisors.)
> -        </dd>
> -
> -        <dt><code>get_arp(<var>P</var>, <var>A</var>);</code></dt>
> -        <dt><code>get_nd(<var>P</var>, <var>A</var>);</code></dt>
> -        <dd>
> -          <p>
> -            Implemented by storing arguments into OpenFlow fields, then
> -            resubmitting to table 66, which <code>ovn-controller</code>
> -            populates with flows generated from the
> <code>MAC_Binding</code>
> -            table in the OVN Southbound database.  If there is a match in
> table
> -            66, then its actions store the bound MAC in the Ethernet
> -            destination address field.
> -          </p>
> -
> -          <p>
> -            (The OpenFlow actions save and restore the OpenFlow fields
> used for
> -            the arguments, so that the OVN actions do not have to be
> aware of
> -            this temporary use.)
> -          </p>
> -        </dd>
> -
> -        <dt><code>put_arp(<var>P</var>, <var>A</var>,
> <var>E</var>);</code></dt>
> -        <dt><code>put_nd(<var>P</var>, <var>A</var>,
> <var>E</var>);</code></dt>
> -        <dd>
> -          <p>
> -            Implemented by storing the arguments into OpenFlow fields,
> then
> -            outputting a packet to <code>ovn-controller</code>, which
> updates
> -            the <code>MAC_Binding</code> table.
> -          </p>
> -
> -          <p>
> -            (The OpenFlow actions save and restore the OpenFlow fields
> used for
> -            the arguments, so that the OVN actions do not have to be
> aware of
> -            this temporary use.)
> -          </p>
> -        </dd>
> -      </dl>
> -    </li>
> -
> -    <li>
> -      <p>
> -        OpenFlow tables 32 through 47 implement the <code>output</code>
> action
> -        in the logical ingress pipeline.  Specifically, table 32 handles
> -        packets to remote hypervisors, table 33 handles packets to the
> local
> -        hypervisor, and table 34 checks whether packets whose logical
> ingress
> -        and egress port are the same should be discarded.
> -      </p>
> -
> -      <p>
> -        Logical patch ports are a special case.  Logical patch ports do
> not
> -        have a physical location and effectively reside on every
> hypervisor.
> -        Thus, flow table 33, for output to ports on the local hypervisor,
> -        naturally implements output to unicast logical patch ports too.
> -        However, applying the same logic to a logical patch port that is
> part
> -        of a logical multicast group yields packet duplication, because
> each
> -        hypervisor that contains a logical port in the multicast group
> will
> -        also output the packet to the logical patch port.  Thus, multicast
> -        groups implement output to logical patch ports in table 32.
> -      </p>
> -
> -      <p>
> -        Each flow in table 32 matches on a logical output port for
> unicast or
> -        multicast logical ports that include a logical port on a remote
> -        hypervisor.  Each flow's actions implement sending a packet to
> the port
> -        it matches.  For unicast logical output ports on remote
> hypervisors,
> -        the actions set the tunnel key to the correct value, then send the
> -        packet on the tunnel port to the correct hypervisor.  (When the
> remote
> -        hypervisor receives the packet, table 0 there will recognize it
> as a
> -        tunneled packet and pass it along to table 33.)  For multicast
> logical
> -        output ports, the actions send one copy of the packet to each
> remote
> -        hypervisor, in the same way as for unicast destinations.  If a
> -        multicast group includes a logical port or ports on the local
> -        hypervisor, then its actions also resubmit to table 33.  Table 32
> also
> -        includes:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          A higher-priority rule to match packets received from VXLAN
> tunnels,
> -          based on flag MLF_RCV_FROM_VXLAN, and resubmit these packets to
> table
> -          33 for local delivery.  Packets received from VXLAN tunnels
> reach
> -          here because of a lack of logical output port field in the
> tunnel key
> -          and thus these packets needed to be submitted to table 8 to
> -          determine the output port.
> -        </li>
> -        <li>
> -          A higher-priority rule to match packets received from ports of
> type
> -          <code>localport</code>, based on the logical input port, and
> resubmit
> -          these packets to table 33 for local delivery.  Ports of type
> -          <code>localport</code> exist on every hypervisor and by
> definition
> -          their traffic should never go out through a tunnel.
> -        </li>
> -        <li>
> -          A higher-priority rule to match packets that have the
> MLF_LOCAL_ONLY
> -          logical flow flag set, and whose destination is a multicast
> address.
> -          This flag indicates that the packet should not be delivered to
> remote
> -          hypervisors, even if the multicast destination includes ports on
> -          remote hypervisors. This flag is used when
> -          <code>ovn-controller</code> is the originator of the multicast
> packet.
> -          Since each <code>ovn-controller</code> instance is originating
> these
> -          packets, the packets only need to be delivered to local ports.
> -        </li>
> -        <li>
> -          A fallback flow that resubmits to table 33 if there is no other
> -          match.
> -        </li>
> -      </ul>
> -
> -      <p>
> -        Flows in table 33 resemble those in table 32 but for logical
> ports that
> -        reside locally rather than remotely.  For unicast logical output
> ports
> -        on the local hypervisor, the actions just resubmit to table 34.
> For
> -        multicast output ports that include one or more logical ports on
> the
> -        local hypervisor, for each such logical port <var>P</var>, the
> actions
> -        change the logical output port to <var>P</var>, then resubmit to
> table
> -        34.
> -      </p>
> -
> -      <p>
> -        A special case is that when a localnet port exists on the
> datapath,
> -        remote port is connected by switching to the localnet port. In
> this
> -        case, instead of adding a flow in table 32 to reach the remote
> port, a
> -        flow is added in table 33 to switch the logical outport to the
> localnet
> -        port, and resubmit to table 33 as if it were unicasted to a
> logical
> -        port on the local hypervisor.
> -      </p>
> -
> -      <p>
> -        Table 34 matches and drops packets for which the logical input and
> -        output ports are the same and the MLF_ALLOW_LOOPBACK flag is not
> -        set.  It resubmits other packets to table 40.
> -      </p>
> -    </li>
> -
> -    <li>
> -      <p>
> -        OpenFlow tables 40 through 63 execute the logical egress pipeline
> from
> -        the <code>Logical_Flow</code> table in the OVN Southbound
> database.
> -        The egress pipeline can perform a final stage of validation before
> -        packet delivery.  Eventually, it may execute an
> <code>output</code>
> -        action, which <code>ovn-controller</code> implements by
> resubmitting to
> -        table 64.  A packet for which the pipeline never executes
> -        <code>output</code> is effectively dropped (although it may have
> been
> -        transmitted through a tunnel across a physical network).
> -      </p>
> -
> -      <p>
> -        The egress pipeline cannot change the logical output port or cause
> -        further tunneling.
> -      </p>
> -    </li>
> -
> -    <li>
> -     <p>
> -       Table 64 bypasses OpenFlow loopback when MLF_ALLOW_LOOPBACK is set.
> -       Logical loopback was handled in table 34, but OpenFlow by default
> also
> -       prevents loopback to the OpenFlow ingress port.  Thus, when
> -       MLF_ALLOW_LOOPBACK is set, OpenFlow table 64 saves the OpenFlow
> ingress
> -       port, sets it to zero, resubmits to table 65 for
> logical-to-physical
> -       transformation, and then restores the OpenFlow ingress port,
> -       effectively disabling OpenFlow loopback prevents.  When
> -       MLF_ALLOW_LOOPBACK is unset, table 64 flow simply resubmits to
> table
> -       65.
> -     </p>
> -    </li>
> -
> -    <li>
> -      <p>
> -        OpenFlow table 65 performs logical-to-physical translation, the
> -        opposite of table 0.  It matches the packet's logical egress
> port.  Its
> -        actions output the packet to the port attached to the OVN
> integration
> -        bridge that represents that logical port.  If the logical egress
> port
> -        is a container nested with a VM, then before sending the packet
> the
> -        actions push on a VLAN header with an appropriate VLAN ID.
> -      </p>
> -    </li>
> -  </ol>
> -
> -  <h2>Logical Routers and Logical Patch Ports</h2>
> -
> -  <p>
> -    Typically logical routers and logical patch ports do not have a
> -    physical location and effectively reside on every hypervisor.  This is
> -    the case for logical patch ports between logical routers and logical
> -    switches behind those logical routers, to which VMs (and VIFs) attach.
> -  </p>
> -
> -  <p>
> -    Consider a packet sent from one virtual machine or container to
> another
> -    VM or container that resides on a different subnet.  The packet will
> -    traverse tables 0 to 65 as described in the previous section
> -    <code>Architectural Physical Life Cycle of a Packet</code>, using the
> -    logical datapath representing the logical switch that the sender is
> -    attached to.  At table 32, the packet will use the fallback flow that
> -    resubmits locally to table 33 on the same hypervisor.  In this case,
> -    all of the processing from table 0 to table 65 occurs on the
> hypervisor
> -    where the sender resides.
> -  </p>
> -
> -  <p>
> -    When the packet reaches table 65, the logical egress port is a logical
> -    patch port.  The implementation in table 65 differs depending on the
> OVS
> -    version, although the observed behavior is meant to be the same:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      In OVS versions 2.6 and earlier, table 65 outputs to an OVS patch
> -      port that represents the logical patch port.  The packet re-enters
> -      the OpenFlow flow table from the OVS patch port's peer in table 0,
> -      which identifies the logical datapath and logical input port based
> -      on the OVS patch port's OpenFlow port number.
> -    </li>
> -
> -    <li>
> -      In OVS versions 2.7 and later, the packet is cloned and resubmitted
> -      directly to the first OpenFlow flow table in the ingress pipeline,
> -      setting the logical ingress port to the peer logical patch port, and
> -      using the peer logical patch port's logical datapath (that
> -      represents the logical router).
> -    </li>
> -  </ul>
> -
> -  <p>
> -    The packet re-enters the ingress pipeline in order to traverse tables
> -    8 to 65 again, this time using the logical datapath representing the
> -    logical router.  The processing continues as described in the previous
> -    section <code>Architectural Physical Life Cycle of a Packet</code>.
> -    When the packet reachs table 65, the logical egress port will once
> -    again be a logical patch port.  In the same manner as described above,
> -    this logical patch port will cause the packet to be resubmitted to
> -    OpenFlow tables 8 to 65, this time using the logical datapath
> -    representing the logical switch that the destination VM or container
> -    is attached to.
> -  </p>
> -
> -  <p>
> -    The packet traverses tables 8 to 65 a third and final time.  If the
> -    destination VM or container resides on a remote hypervisor, then table
> -    32 will send the packet on a tunnel port from the sender's hypervisor
> -    to the remote hypervisor.  Finally table 65 will output the packet
> -    directly to the destination VM or container.
> -  </p>
> -
> -  <p>
> -    The following sections describe two exceptions, where logical routers
> -    and/or logical patch ports are associated with a physical location.
> -  </p>
> -
> -  <h3>Gateway Routers</h3>
> -
> -  <p>
> -    A <dfn>gateway router</dfn> is a logical router that is bound to a
> -    physical location.  This includes all of the logical patch ports of
> -    the logical router, as well as all of the peer logical patch ports on
> -    logical switches.  In the OVN Southbound database, the
> -    <code>Port_Binding</code> entries for these logical patch ports use
> -    the type <code>l3gateway</code> rather than <code>patch</code>, in
> -    order to distinguish that these logical patch ports are bound to a
> -    chassis.
> -  </p>
> -
> -  <p>
> -    When a hypervisor processes a packet on a logical datapath
> -    representing a logical switch, and the logical egress port is a
> -    <code>l3gateway</code> port representing connectivity to a gateway
> -    router, the packet will match a flow in table 32 that sends the
> -    packet on a tunnel port to the chassis where the gateway router
> -    resides.  This processing in table 32 is done in the same manner as
> -    for VIFs.
> -  </p>
> -
> -  <p>
> -    Gateway routers are typically used in between distributed logical
> -    routers and physical networks.  The distributed logical router and
> -    the logical switches behind it, to which VMs and containers attach,
> -    effectively reside on each hypervisor.  The distributed router and
> -    the gateway router are connected by another logical switch, sometimes
> -    referred to as a <code>join</code> logical switch.  On the other
> -    side, the gateway router connects to another logical switch that has
> -    a localnet port connecting to the physical network.
> -  </p>
> -
> -  <p>
> -    When using gateway routers, DNAT and SNAT rules are associated with
> -    the gateway router, which provides a central location that can handle
> -    one-to-many SNAT (aka IP masquerading).
> -  </p>
> -
> -  <h3>Distributed Gateway Ports</h3>
> -
> -  <p>
> -    <dfn>Distributed gateway ports</dfn> are logical router patch ports
> -    that directly connect distributed logical routers to logical
> -    switches with localnet ports.
> -  </p>
> -
> -  <p>
> -    The primary design goal of distributed gateway ports is to allow as
> -    much traffic as possible to be handled locally on the hypervisor
> -    where a VM or container resides.  Whenever possible, packets from
> -    the VM or container to the outside world should be processed
> -    completely on that VM's or container's hypervisor, eventually
> -    traversing a localnet port instance on that hypervisor to the
> -    physical network.  Whenever possible, packets from the outside
> -    world to a VM or container should be directed through the physical
> -    network directly to the VM's or container's hypervisor, where the
> -    packet will enter the integration bridge through a localnet port.
> -  </p>
> -
> -  <p>
> -    In order to allow for the distributed processing of packets
> -    described in the paragraph above, distributed gateway ports need to
> -    be logical patch ports that effectively reside on every hypervisor,
> -    rather than <code>l3gateway</code> ports that are bound to a
> -    particular chassis.  However, the flows associated with distributed
> -    gateway ports often need to be associated with physical locations,
> -    for the following reasons:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      <p>
> -        The physical network that the localnet port is attached to
> -        typically uses L2 learning.  Any Ethernet address used over the
> -        distributed gateway port must be restricted to a single physical
> -        location so that upstream L2 learning is not confused.  Traffic
> -        sent out the distributed gateway port towards the localnet port
> -        with a specific Ethernet address must be sent out one specific
> -        instance of the distributed gateway port on one specific
> -        chassis.  Traffic received from the localnet port (or from a VIF
> -        on the same logical switch as the localnet port) with a specific
> -        Ethernet address must be directed to the logical switch's patch
> -        port instance on that specific chassis.
> -      </p>
> -
> -      <p>
> -        Due to the implications of L2 learning, the Ethernet address and
> -        IP address of the distributed gateway port need to be restricted
> -        to a single physical location.  For this reason, the user must
> -        specify one chassis associated with the distributed gateway
> -        port.  Note that traffic traversing the distributed gateway port
> -        using other Ethernet addresses and IP addresses (e.g. one-to-one
> -        NAT) is not restricted to this chassis.
> -      </p>
> -
> -      <p>
> -        Replies to ARP and ND requests must be restricted to a single
> -        physical location, where the Ethernet address in the reply
> -        resides.  This includes ARP and ND replies for the IP address
> -        of the distributed gateway port, which are restricted to the
> -        chassis that the user associated with the distributed gateway
> -        port.
> -      </p>
> -    </li>
> -
> -    <li>
> -      In order to support one-to-many SNAT (aka IP masquerading), where
> -      multiple logical IP addresses spread across multiple chassis are
> -      mapped to a single external IP address, it will be necessary to
> -      handle some of the logical router processing on a specific chassis
> -      in a centralized manner.  Since the SNAT external IP address is
> -      typically the distributed gateway port IP address, and for
> -      simplicity, the same chassis associated with the distributed
> -      gateway port is used.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    The details of flow restrictions to specific chassis are described
> -    in the <code>ovn-northd</code> documentation.
> -  </p>
> -
> -  <p>
> -    While most of the physical location dependent aspects of distributed
> -    gateway ports can be handled by restricting some flows to specific
> -    chassis, one additional mechanism is required.  When a packet
> -    leaves the ingress pipeline and the logical egress port is the
> -    distributed gateway port, one of two different sets of actions is
> -    required at table 32:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      If the packet can be handled locally on the sender's hypervisor
> -      (e.g. one-to-one NAT traffic), then the packet should just be
> -      resubmitted locally to table 33, in the normal manner for
> -      distributed logical patch ports.
> -    </li>
> -
> -    <li>
> -      However, if the packet needs to be handled on the chassis
> -      associated with the distributed gateway port (e.g. one-to-many
> -      SNAT traffic or non-NAT traffic), then table 32 must send the
> -      packet on a tunnel port to that chassis.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    In order to trigger the second set of actions, the
> -    <code>chassisredirect</code> type of southbound
> -    <code>Port_Binding</code> has been added.  Setting the logical
> -    egress port to the type <code>chassisredirect</code> logical port is
> -    simply a way to indicate that although the packet is destined for
> -    the distributed gateway port, it needs to be redirected to a
> -    different chassis.  At table 32, packets with this logical egress
> -    port are sent to a specific chassis, in the same way that table 32
> -    directs packets whose logical egress port is a VIF or a type
> -    <code>l3gateway</code> port to different chassis.  Once the packet
> -    arrives at that chassis, table 33 resets the logical egress port to
> -    the value representing the distributed gateway port.  For each
> -    distributed gateway port, there is one type
> -    <code>chassisredirect</code> port, in addition to the distributed
> -    logical patch port representing the distributed gateway port.
> -  </p>
> -
> -  <h3>High Availability for Distributed Gateway Ports</h3>
> -
> -  <p>
> -    OVN allows you to specify a prioritized list of chassis for a
> distributed
> -    gateway port.  This is done by associating multiple
> -    <code>Gateway_Chassis</code> rows with a
> <code>Logical_Router_Port</code>
> -    in the <code>OVN_Northbound</code> database.
> -  </p>
> -
> -  <p>
> -    When multiple chassis have been specified for a gateway, all chassis
> that
> -    may send packets to that gateway will enable BFD on tunnels to all
> -    configured gateway chassis.  The current master chassis for the
> gateway
> -    is the highest priority gateway chassis that is currently viewed as
> -    active based on BFD status.
> -  </p>
> -
> -  <p>
> -    For more information on L3 gateway high availability, please refer to
> -    http://docs.openvswitch.org/en/latest/topics/high-availability.
> -  </p>
> -
> -  <h2>Multiple localnet logical switches connected to a Logical
> Router</h2>
> -
> -  <p>
> -    It is possible to have multiple logical switches each with a localnet
> port
> -    (representing physical networks) connected to a logical router, in
> which
> -    one localnet logical switch may provide the external connectivity via
> a
> -    distributed gateway port and rest of the localnet logical switches use
> -    VLAN tagging in the physical network. It is expected that
> -    <code>ovn-bridge-mappings</code> is configured appropriately on the
> -    chassis for all these localnet networks.
> -  </p>
> -
> -  <h3>East West routing</h3>
> -  <p>
> -    East-West routing between these localnet VLAN tagged logical switches
> -    work almost the same way as normal logical switches. When the VM sends
> -    such a packet, then:
> -  </p>
> -  <ol>
> -    <li>
> -      It first enters the ingress pipeline, and then egress pipeline of
> the
> -      source localnet logical switch datapath. It then enters the ingress
> -      pipeline of the logical router datapath via the logical router port
> in
> -      the source chassis.
> -    </li>
> -
> -    <li>
> -      Routing decision is taken.
> -    </li>
> -
> -    <li>
> -      <p>
> -        From the router datapath, packet enters the ingress pipeline and
> then
> -        egress pipeline of the destination localnet logical switch
> datapath
> -        and goes out of the integration bridge to the provider bridge (
> -        belonging to the destination logical switch) via the localnet
> port.
> -        While sending the packet to provider bridge, we also replace
> router
> -        port MAC as source MAC with a chassis unique MAC.
> -      </p>
> -
> -      <p>
> -        This chassis unique MAC is configured as global ovs config on each
> -        chassis (eg. via "<code>ovs-vsctl set open . external-ids:
> -        ovn-chassis-mac-mappings="phys:aa:bb:cc:dd:ee:$i$i"</code>").
> For more
> -        details, see <code>ovn-controller</code>(8).
> -      </p>
> -
> -      <p>
> -        If the above is not configured, then source MAC would be the
> router
> -        port MAC.  This could create problem if we have more than one
> chassis.
> -        This is because, since the router port is distributed, the same
> -        (MAC,VLAN) tuple will seen by physical network from other chassis
> as
> -        well, which could cause these issues:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          Continuous MAC moves in top-of-rack switch (ToR).
> -        </li>
> -        <li>
> -          ToR dropping the traffic, which is causing continuous MAC moves.
> -        </li>
> -        <li>
> -          ToR blocking the ports from which MAC moves are happening.
> -        </li>
> -      </ul>
> -    </li>
> -
> -    <li>
> -      The destination chassis receives the packet via the localnet port
> and
> -      sends it to the integration bridge. The packet enters the
> -      ingress pipeline and then egress pipeline of the destination
> localnet
> -      logical switch and finally gets delivered to the destination VM
> port.
> -    </li>
> -  </ol>
> -
> -  <h3>External traffic</h3>
> -
> -  <p>
> -    The following happens when a VM sends an external traffic (which
> requires
> -    NATting) and the chassis hosting the VM doesn't have a distributed
> gateway
> -    port.
> -  </p>
> -
> -  <ol>
> -    <li>
> -      The packet first enters the ingress pipeline, and then egress
> pipeline of
> -      the source localnet logical switch datapath. It then enters the
> ingress
> -      pipeline of the logical router datapath via the logical router port
> in
> -      the source chassis.
> -    </li>
> -
> -    <li>
> -      Routing decision is taken. Since the gateway router or the
> distributed
> -      gateway port doesn't reside in the source chassis, the traffic is
> -      redirected to the gateway chassis via the tunnel port.
> -    </li>
> -
> -    <li>
> -      The gateway chassis receives the packet via the tunnel port and the
> -      packet enters the egress pipeline of the logical router datapath.
> NAT
> -      rules are applied here. The packet then enters the ingress pipeline
> and
> -      then egress pipeline of the localnet logical switch datapath which
> -      provides external connectivity and finally goes out via the localnet
> -      port of the logical switch which provides external connectivity.
> -    </li>
> -  </ol>
> -
> -  <p>
> -    Although this works, the VM traffic is tunnelled when sent from the
> compute
> -    chassis to the gateway chassis. In order for it to work properly, the
> MTU
> -    of the localnet logical switches must be lowered to account for the
> tunnel
> -    encapsulation.
> -  </p>
> -
> -  <h2>
> -    Centralized routing for localnet VLAN tagged logical switches
> connected
> -    to a Logical Router
> -  </h2>
> -
> -  <p>
> -    To overcome the tunnel encapsulation problem described in the previous
> -    section, <code>OVN</code> supports the option of enabling centralized
> -    routing for localnet VLAN tagged logical switches. CMS can configure
> the
> -    option <ref column="options:reside-on-redirect-chassis"
> -    table="Logical_Router_Port" db="OVN_NB"/> to <code>true</code> for
> each
> -    <ref table="Logical_Router_Port" db="OVN_NB"/> which connects to the
> -    localnet VLAN tagged logical switches. This causes the gateway
> -    chassis (hosting the distributed gateway port) to handle all the
> -    routing for these networks, making it centralized. It will reply to
> -    the ARP requests for the logical router port IPs.
> -  </p>
> -
> -  <p>
> -    If the logical router doesn't have a distributed gateway port
> connecting
> -    to the localnet logical switch which provides external connectivity,
> -    then this option is ignored by <code>OVN</code>.
> -  </p>
> -
> -  <p>
> -    The following happens when a VM sends an east-west traffic which
> needs to
> -    be routed:
> -  </p>
> -
> -  <ol>
> -    <li>
> -      The packet first enters the ingress pipeline, and then egress
> pipeline of
> -      the source localnet logical switch datapath and is sent out via the
> -      localnet port of the source localnet logical switch (instead of
> sending
> -      it to router pipeline).
> -    </li>
> -
> -    <li>
> -      The gateway chassis receives the packet via the localnet port of the
> -      source localnet logical switch and sends it to the integration
> bridge.
> -      The packet then enters the ingress pipeline, and then egress
> pipeline of
> -      the source localnet logical switch datapath and enters the ingress
> -      pipeline of the logical router datapath.
> -    </li>
> -
> -    <li>
> -      Routing decision is taken.
> -    </li>
> -
> -    <li>
> -      From the router datapath, packet enters the ingress pipeline and
> then
> -      egress pipeline of the destination localnet logical switch datapath.
> -      It then goes out of the integration bridge to the provider bridge (
> -      belonging to the destination logical switch) via the localnet port.
> -    </li>
> -
> -    <li>
> -      The destination chassis receives the packet via the localnet port
> and
> -      sends it to the integration bridge. The packet enters the
> -      ingress pipeline and then egress pipeline of the destination
> localnet
> -      logical switch and finally delivered to the destination VM port.
> -    </li>
> -  </ol>
> -
> -  <p>
> -    The following happens when a VM sends an external traffic which
> requires
> -    NATting:
> -  </p>
> -
> -  <ol>
> -    <li>
> -      The packet first enters the ingress pipeline, and then egress
> pipeline of
> -      the source localnet logical switch datapath and is sent out via the
> -      localnet port of the source localnet logical switch (instead of
> sending
> -      it to router pipeline).
> -    </li>
> -
> -    <li>
> -      The gateway chassis receives the packet via the localnet port of the
> -      source localnet logical switch and sends it to the integration
> bridge.
> -      The packet then enters the ingress pipeline, and then egress
> pipeline of
> -      the source localnet logical switch datapath and enters the ingress
> -      pipeline of the logical router datapath.
> -    </li>
> -
> -    <li>
> -      Routing decision is taken and NAT rules are applied.
> -    </li>
> -
> -    <li>
> -      From the router datapath, packet enters the ingress pipeline and
> then
> -      egress pipeline of the localnet logical switch datapath which
> provides
> -      external connectivity. It then goes out of the integration bridge
> to the
> -      provider bridge (belonging to the logical switch which provides
> external
> -      connectivity) via the localnet port.
> -    </li>
> -  </ol>
> -
> -  <p>
> -    The following happens for the reverse external traffic.
> -  </p>
> -
> -  <ol>
> -    <li>
> -      The gateway chassis receives the packet from the localnet port of
> -      the logical switch which provides external connectivity. The packet
> then
> -      enters the ingress pipeline and then egress pipeline of the localnet
> -      logical switch (which provides external connectivity). The packet
> then
> -      enters the ingress pipeline of the logical router datapath.
> -    </li>
> -
> -    <li>
> -      The ingress pipeline of the logical router datapath applies the
> unNATting
> -      rules. The packet then enters the ingress pipeline and then egress
> -      pipeline of the source localnet logical switch. Since the source VM
> -      doesn't reside in the gateway chassis, the packet is sent out via
> the
> -      localnet port of the source logical switch.
> -    </li>
> -
> -    <li>
> -      The source chassis receives the packet via the localnet port and
> -      sends it to the integration bridge. The packet enters the
> -      ingress pipeline and then egress pipeline of the source localnet
> -      logical switch and finally gets delivered to the source VM port.
> -    </li>
> -  </ol>
> -
> -  <h2>Life Cycle of a VTEP gateway</h2>
> -
> -  <p>
> -    A gateway is a chassis that forwards traffic between the OVN-managed
> -    part of a logical network and a physical VLAN,  extending a
> -    tunnel-based logical network into a physical network.
> -  </p>
> -
> -  <p>
> -    The steps below refer often to details of the OVN and VTEP database
> -    schemas.  Please see <code>ovn-sb</code>(5), <code>ovn-nb</code>(5)
> -    and <code>vtep</code>(5), respectively, for the full story on these
> -    databases.
> -  </p>
> -
> -  <ol>
> -    <li>
> -      A VTEP gateway's life cycle begins with the administrator
> registering
> -      the VTEP gateway as a <code>Physical_Switch</code> table entry in
> the
> -      <code>VTEP</code> database.  The <code>ovn-controller-vtep</code>
> -      connected to this VTEP database, will recognize the new VTEP gateway
> -      and create a new <code>Chassis</code> table entry for it in the
> -      <code>OVN_Southbound</code> database.
> -    </li>
> -
> -    <li>
> -      The administrator can then create a new <code>Logical_Switch</code>
> -      table entry, and bind a particular vlan on a VTEP gateway's port to
> -      any VTEP logical switch.  Once a VTEP logical switch is bound to
> -      a VTEP gateway, the <code>ovn-controller-vtep</code> will detect
> -      it and add its name to the <var>vtep_logical_switches</var>
> -      column of the <code>Chassis</code> table in the <code>
> -      OVN_Southbound</code> database.  Note, the <var>tunnel_key</var>
> -      column of VTEP logical switch is not filled at creation.  The
> -      <code>ovn-controller-vtep</code> will set the column when the
> -      correponding vtep logical switch is bound to an OVN logical network.
> -    </li>
> -
> -    <li>
> -      Now, the administrator can use the CMS to add a VTEP logical switch
> -      to the OVN logical network.  To do that, the CMS must first create a
> -      new <code>Logical_Switch_Port</code> table entry in the <code>
> -      OVN_Northbound</code> database.  Then, the <var>type</var> column
> -      of this entry must be set to "vtep".  Next, the <var>
> -      vtep-logical-switch</var> and <var>vtep-physical-switch</var> keys
> -      in the <var>options</var> column must also be specified, since
> -      multiple VTEP gateways can attach to the same VTEP logical switch.
> -    </li>
> -
> -    <li>
> -      The newly created logical port in the <code>OVN_Northbound</code>
> -      database and its configuration will be passed down to the <code>
> -      OVN_Southbound</code> database as a new <code>Port_Binding</code>
> -      table entry.  The <code>ovn-controller-vtep</code> will recognize
> the
> -      change and bind the logical port to the corresponding VTEP gateway
> -      chassis.  Configuration of binding the same VTEP logical switch to
> -      a different OVN logical networks is not allowed and a warning will
> be
> -      generated in the log.
> -    </li>
> -
> -    <li>
> -      Beside binding to the VTEP gateway chassis, the <code>
> -      ovn-controller-vtep</code> will update the <var>tunnel_key</var>
> -      column of the VTEP logical switch to the corresponding <code>
> -      Datapath_Binding</code> table entry's <var>tunnel_key</var> for the
> -      bound OVN logical network.
> -    </li>
> -
> -    <li>
> -      Next, the <code>ovn-controller-vtep</code> will keep reacting to the
> -      configuration change in the <code>Port_Binding</code> in the
> -      <code>OVN_Northbound</code> database, and updating the
> -      <code>Ucast_Macs_Remote</code> table in the <code>VTEP</code>
> database.
> -      This allows the VTEP gateway to understand where to forward the
> unicast
> -      traffic coming from the extended external network.
> -    </li>
> -
> -    <li>
> -      Eventually, the VTEP gateway's life cycle ends when the
> administrator
> -      unregisters the VTEP gateway from the <code>VTEP</code> database.
> -      The <code>ovn-controller-vtep</code> will recognize the event and
> -      remove all related configurations (<code>Chassis</code> table entry
> -      and port bindings) in the <code>OVN_Southbound</code> database.
> -    </li>
> -
> -    <li>
> -      When the <code>ovn-controller-vtep</code> is terminated, all related
> -      configurations in the <code>OVN_Southbound</code> database and
> -      the <code>VTEP</code> database will be cleaned, including
> -      <code>Chassis</code> table entries for all registered VTEP gateways
> -      and their port bindings, and all <code>Ucast_Macs_Remote</code>
> table
> -      entries and the <code>Logical_Switch</code> tunnel keys.
> -    </li>
> -  </ol>
> -
> -  <h2>Native OVN services for external logical ports</h2>
> -
> -  <p>
> -    To support OVN native services (like DHCP/IPv6 RA/DNS lookup) to the
> -    cloud resources which are external, OVN supports <code>external</code>
> -    logical ports.
> -  </p>
> -
> -  <p>
> -    Below are some of the use cases where <code>external</code> ports can
> be
> -    used.
> -  </p>
> -
> -  <ul>
> -    <li>
> -      VMs connected to SR-IOV nics - Traffic from these VMs by passes the
> -      kernel stack and local <code>ovn-controller</code> do not bind these
> -      ports and cannot serve the native services.
> -    </li>
> -    <li>
> -      When CMS supports provisioning baremetal servers.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    OVN will provide the native services if CMS has done the below
> -    configuration in the <dfn>OVN Northbound Database</dfn>.
> -  </p>
> -
> -  <ul>
> -    <li>
> -      A row is created in <code>Logical_Switch_Port</code>, configuring
> the
> -      <ref column="addresses" table="Logical_Switch_Port" db="OVN_NB"/>
> column
> -      and setting the <ref column="type" table="Logical_Switch_Port"
> -      db="OVN_NB"/> to <code>external</code>.
> -    </li>
> -
> -    <li>
> -      <ref column="ha_chassis_group" table="Logical_Switch_Port"
> -      db="OVN_NB"/> column is configured.
> -    </li>
> -
> -    <li>
> -      The HA chassis which belongs to the HA chassis group has the
> -      <code>ovn-bridge-mappings</code> configured and has proper L2
> -      connectivity so that it can receive the DHCP and other related
> request
> -      packets from these external resources.
> -    </li>
> -
> -    <li>
> -      The Logical_Switch of this port has a <code>localnet</code> port.
> -    </li>
> -
> -    <li>
> -      Native OVN services are enabled by configuring the DHCP and other
> -      options like the way it is done for the normal logical ports.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    It is recommended to use the same HA chassis group for all the
> external
> -    ports of a logical switch. Otherwise, the physical switch might see
> MAC
> -    flap issue when different chassis provide the native services. For
> -    example when supporting native DHCPv4 service, DHCPv4 server mac
> -    (configured in <ref column="options:server_mac" table="DHCP_Options"
> -    db="OVN_NB"/> column in table <ref table="DHCP_Options"/>) originating
> -    from different ports can cause MAC flap issue.
> -    The MAC of the logical router IP(s) can also flap if the same HA
> chassis
> -    group is not set for all the external ports of a logical switch.
> -  </p>
> -
> -  <h1>Security</h1>
> -
> -  <h2>Role-Based Access Controls for the Soutbound DB</h2>
> -  <p>
> -    In order to provide additional security against the possibility of an
> OVN
> -    chassis becoming compromised in such a way as to allow rogue software
> to
> -    make arbitrary modifications to the southbound database state and thus
> -    disrupt the OVN network, role-based access controls (see
> -    <code>ovsdb-server(1)</code> for additional details) are provided for
> the
> -    southbound database.
> -  </p>
> -
> -  <p>
> -    The implementation of role-based access controls (RBAC) requires the
> -    addition of two tables to an OVSDB schema: the <code>RBAC_Role</code>
> -    table, which is indexed by role name and maps the the names of the
> various
> -    tables that may be modifiable for a given role to individual rows in a
> -    permissions table containing detailed permission information for that
> role,
> -    and the permission table itself which consists of rows containing the
> -    following information:
> -  </p>
> -  <dl>
> -    <dt><code>Table Name</code></dt>
> -    <dd>
> -      The name of the associated table. This column exists primarily as an
> -      aid for humans reading the contents of this table.
> -    </dd>
> -
> -    <dt><code>Auth Criteria</code></dt>
> -    <dd>
> -      A set of strings containing the names of columns (or column:key
> pairs
> -      for columns containing string:string maps).  The contents of at
> least
> -      one of the columns or column:key values in a row to be modified,
> -      inserted, or deleted must be equal to the ID of the client
> attempting
> -      to act on the row in order for the authorization check to pass. If
> the
> -      authorization criteria is empty, authorization checking is disabled
> and
> -      all clients for the role will be treated as authorized.
> -    </dd>
> -
> -    <dt><code>Insert/Delete</code></dt>
> -    <dd>
> -       Row insertion/deletion permission; boolean value indicating whether
> -       insertion and deletion of rows is allowed for the associated table.
> -       If true, insertion and deletion of rows is allowed for authorized
> -       clients.
> -    </dd>
> -
> -    <dt><code>Updatable Columns</code></dt>
> -    <dd>
> -      A set of strings containing the names of columns or column:key pairs
> -      that may be updated or mutated by authorized clients. Modifications
> to
> -      columns within a row are only permitted when the authorization check
> -      for the client passes and all columns to be modified are included in
> -      this set of modifiable columns.
> -    </dd>
> -  </dl>
> -
> -  <p>
> -    RBAC configuration for the OVN southbound database is maintained by
> -    ovn-northd. With RBAC enabled, modifications are only permitted for
> the
> -    <code>Chassis</code>, <code>Encap</code>, <code>Port_Binding</code>,
> and
> -    <code>MAC_Binding</code> tables, and are resstricted as follows:
> -  </p>
> -  <dl>
> -    <dt><code>Chassis</code></dt>
> -    <dd>
> -      <p>
> -       <code>Authorization</code>: client ID must match the chassis name.
> -      </p>
> -      <p>
> -        <code>Insert/Delete</code>: authorized row insertion and deletion
> -        are permitted.
> -      </p>
> -      <p>
> -        <code>Update</code>: The columns <code>nb_cfg</code>,
> -        <code>external_ids</code>, <code>encaps</code>, and
> -        <code>vtep_logical_switches</code> may be modified when
> authorized.
> -      </p>
> -    </dd>
> -
> -    <dt><code>Encap</code></dt>
> -    <dd>
> -      <p>
> -        <code>Authorization</code>: client ID must match the chassis name.
> -      </p>
> -      <p>
> -        <code>Insert/Delete</code>: row insertion and row deletion
> -        are permitted.
> -      </p>
> -      <p>
> -        <code>Update</code>: The columns <code>type</code>,
> -        <code>options</code>, and <code>ip</code> can be modified.
> -      </p>
> -    </dd>
> -
> -    <dt><code>Port_Binding</code></dt>
> -    <dd>
> -      <p>
> -        <code>Authorization</code>: disabled (all clients are considered
> -        authorized. A future enhancement may add columns (or keys to
> -        <code>external_ids</code>) in order to control which chassis are
> -        allowed to bind each port.
> -      </p>
> -      <p>
> -        <code>Insert/Delete</code>: row insertion/deletion are not
> permitted
> -        (ovn-northd maintains rows in this table.
> -      </p>
> -      <p>
> -        <code>Update</code>: Only modifications to the
> <code>chassis</code>
> -        column are permitted.
> -      </p>
> -    </dd>
> -
> -    <dt><code>MAC_Binding</code></dt>
> -    <dd>
> -      <p>
> -        <code>Authorization</code>: disabled (all clients are considered
> -        to be authorized).
> -      </p>
> -      <p>
> -        <code>Insert/Delete</code>: row insertion/deletion are permitted.
> -      </p>
> -      <p>
> -        <code>Update</code>: The columns <code>logical_port</code>,
> -        <code>ip</code>, <code>mac</code>, and <code>datapath</code> may
> be
> -        modified by ovn-controller.
> -      </p>
> -    </dd>
> -  </dl>
> -
> -  <p>
> -    Enabling RBAC for ovn-controller connections to the southbound
> database
> -    requires the following steps:
> -  </p>
> -
> -  <ol>
> -    <li>
> -      Creating SSL certificates for each chassis with the certificate CN
> field
> -      set to the chassis name (e.g. for a chassis with
> -      <code>external-ids:system-id=chassis-1</code>, via the command
> -      "<code>ovs-pki -u req+sign chassis-1 switch</code>").
> -    </li>
> -    <li>
> -      Configuring each ovn-controller to use SSL when connecting to the
> -      southbound database (e.g. via "<code>ovs-vsctl set open .
> -      external-ids:ovn-remote=ssl:x.x.x.x:6642</code>").
> -    </li>
> -    <li>
> -      Configuring a southbound database SSL remote with "ovn-controller"
> role
> -      (e.g. via "<code>ovn-sbctl set-connection role=ovn-controller
> -      pssl:6642</code>").
> -    </li>
> -  </ol>
> -
> -  <h2>Encrypt Tunnel Traffic with IPsec</h2>
> -  <p>
> -    OVN tunnel traffic goes through physical routers and switches. These
> -    physical devices could be untrusted (devices in public network) or
> might be
> -    compromised. Enabling encryption to the tunnel traffic can prevent the
> -    traffic data from being monitored and manipulated.
> -  </p>
> -  <p>
> -    The tunnel traffic is encrypted with IPsec. The CMS sets the
> -    <code>ipsec</code> column in the northbound <code>NB_Global</code>
> table to
> -    enable or disable IPsec encrytion. If <code>ipsec</code> is true, all
> OVN
> -    tunnels will be encrypted. If <code>ipsec</code> is false, no OVN
> tunnels
> -    will be encrypted.
> -  </p>
> -  <p>
> -    When CMS updates the <code>ipsec</code> column in the northbound
> -    <code>NB_Global</code> table, <code>ovn-northd</code> copies the
> value to
> -    the <code>ipsec</code> column in the southbound <code>SB_Global</code>
> -    table. <code>ovn-controller</code> in each chassis monitors the
> southbound
> -    database and sets the options of the OVS tunnel interface
> accordingly. OVS
> -    tunnel interface options are monitored by the
> -    <code>ovs-monitor-ipsec</code> daemon which configures IKE daemon to
> set up
> -    IPsec connections.
> -  </p>
> -  <p>
> -    Chassis authenticates each other by using certificate. The
> authentication
> -    succeeds if the other end in tunnel presents a certificate signed by a
> -    trusted CA and the common name (CN) matches the expected chassis
> name.  The
> -    SSL certificates used in role-based access controls (RBAC) can be
> used in
> -    IPsec. Or use <code>ovs-pki</code> to create different certificates.
> The
> -    certificate is required to be x.509 version 3, and with CN field and
> -    subjectAltName field being set to the chassis name.
> -  </p>
> -  <p>
> -    The CA certificate, chassis certificate and private key are required
> to be
> -    installed in each chassis before enabling IPsec. Please see
> -    <code>ovs-vswitchd.conf.db</code>(5) for setting up CA based IPsec
> -    authentication.
> -  </p>
> -  <h1>Design Decisions</h1>
> -
> -  <h2>Tunnel Encapsulations</h2>
> -
> -  <p>
> -    OVN annotates logical network packets that it sends from one
> hypervisor to
> -    another with the following three pieces of metadata, which are
> encoded in
> -    an encapsulation-specific fashion:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      24-bit logical datapath identifier, from the <code>tunnel_key</code>
> -      column in the OVN Southbound <code>Datapath_Binding</code> table.
> -    </li>
> -
> -    <li>
> -      15-bit logical ingress port identifier.  ID 0 is reserved for
> internal
> -      use within OVN.  IDs 1 through 32767, inclusive, may be assigned to
> -      logical ports (see the <code>tunnel_key</code> column in the OVN
> -      Southbound <code>Port_Binding</code> table).
> -    </li>
> -
> -    <li>
> -      16-bit logical egress port identifier.  IDs 0 through 32767 have
> the same
> -      meaning as for logical ingress ports.  IDs 32768 through 65535,
> -      inclusive, may be assigned to logical multicast groups (see the
> -      <code>tunnel_key</code> column in the OVN Southbound
> -      <code>Multicast_Group</code> table).
> -    </li>
> -  </ul>
> -
> -  <p>
> -    For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
> -    encapsulations, for the following reasons:
> -  </p>
> -
> -  <ul>
> -    <li>
> -      Only STT and Geneve support the large amounts of metadata (over 32
> bits
> -      per packet) that OVN uses (as described above).
> -    </li>
> -
> -    <li>
> -      STT and Geneve use randomized UDP or TCP source ports that allows
> -      efficient distribution among multiple paths in environments that
> use ECMP
> -      in their underlay.
> -    </li>
> -
> -    <li>
> -      NICs are available to offload STT and Geneve encapsulation and
> -      decapsulation.
> -    </li>
> -  </ul>
> -
> -  <p>
> -    Due to its flexibility, the preferred encapsulation between
> hypervisors is
> -    Geneve.  For Geneve encapsulation, OVN transmits the logical datapath
> -    identifier in the Geneve VNI.
> -
> -    <!-- Keep the following in sync with ovn/controller/physical.h. -->
> -    OVN transmits the logical ingress and logical egress ports in a TLV
> with
> -    class 0x0102, type 0x80, and a 32-bit value encoded as follows, from
> MSB to
> -    LSB:
> -  </p>
> -
> -  <diagram>
> -    <header name="">
> -      <bits name="rsv" above="1" below="0" width=".25"/>
> -      <bits name="ingress port" above="15" width=".75"/>
> -      <bits name="egress port" above="16" width=".75"/>
> -    </header>
> -  </diagram>
> -
> -  <p>
> -    Environments whose NICs lack Geneve offload may prefer STT
> encapsulation
> -    for performance reasons.  For STT encapsulation, OVN encodes all three
> -    pieces of logical metadata in the STT 64-bit tunnel ID as follows,
> from MSB
> -    to LSB:
> -  </p>
> -
> -  <diagram>
> -    <header name="">
> -      <bits name="reserved" above="9" below="0" width=".5"/>
> -      <bits name="ingress port" above="15" width=".75"/>
> -      <bits name="egress port" above="16" width=".75"/>
> -      <bits name="datapath" above="24" width="1.25"/>
> -    </header>
> -  </diagram>
> -
> -  <p>
> -    For connecting to gateways, in addition to Geneve and STT, OVN
> supports
> -    VXLAN, because only VXLAN support is common on top-of-rack (ToR)
> switches.
> -    Currently, gateways have a feature set that matches the capabilities
> as
> -    defined by the VTEP schema, so fewer bits of metadata are necessary.
> In
> -    the future, gateways that do not support encapsulations with large
> amounts
> -    of metadata may continue to have a reduced feature set.
> -  </p>
> -</manpage>
> diff --git a/ovn/ovn-nb.ovsschema b/ovn/ovn-nb.ovsschema
> deleted file mode 100644
> index 2c87cbba7..000000000
> --- a/ovn/ovn-nb.ovsschema
> +++ /dev/null
> @@ -1,449 +0,0 @@
> -{
> -    "name": "OVN_Northbound",
> -    "version": "5.16.0",
> -    "cksum": "923459061 23095",
> -    "tables": {
> -        "NB_Global": {
> -            "columns": {
> -                "nb_cfg": {"type": {"key": "integer"}},
> -                "sb_cfg": {"type": {"key": "integer"}},
> -                "hv_cfg": {"type": {"key": "integer"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "connections": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "Connection"},
> -                                     "min": 0,
> -                                     "max": "unlimited"}},
> -                "ssl": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "SSL"},
> -                                     "min": 0, "max": 1}},
> -                "options": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "ipsec": {"type": "boolean"}},
> -            "maxRows": 1,
> -            "isRoot": true},
> -        "Logical_Switch": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ports": {"type": {"key": {"type": "uuid",
> -                                           "refTable":
> "Logical_Switch_Port",
> -                                           "refType": "strong"},
> -                                   "min": 0,
> -                                   "max": "unlimited"}},
> -                "acls": {"type": {"key": {"type": "uuid",
> -                                          "refTable": "ACL",
> -                                          "refType": "strong"},
> -                                  "min": 0,
> -                                  "max": "unlimited"}},
> -                "qos_rules": {"type": {"key": {"type": "uuid",
> -                                          "refTable": "QoS",
> -                                          "refType": "strong"},
> -                                  "min": 0,
> -                                  "max": "unlimited"}},
> -                "load_balancer": {"type": {"key": {"type": "uuid",
> -                                                  "refTable":
> "Load_Balancer",
> -                                                  "refType": "weak"},
> -                                           "min": 0,
> -                                           "max": "unlimited"}},
> -                "dns_records": {"type": {"key": {"type": "uuid",
> -                                         "refTable": "DNS",
> -                                         "refType": "weak"},
> -                                  "min": 0,
> -                                  "max": "unlimited"}},
> -                "other_config": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "Logical_Switch_Port": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "type": {"type": "string"},
> -                "options": {
> -                     "type": {"key": "string",
> -                              "value": "string",
> -                              "min": 0,
> -                              "max": "unlimited"}},
> -                "parent_name": {"type": {"key": "string", "min": 0,
> "max": 1}},
> -                "tag_request": {
> -                     "type": {"key": {"type": "integer",
> -                                      "minInteger": 0,
> -                                      "maxInteger": 4095},
> -                              "min": 0, "max": 1}},
> -                "tag": {
> -                     "type": {"key": {"type": "integer",
> -                                      "minInteger": 1,
> -                                      "maxInteger": 4095},
> -                              "min": 0, "max": 1}},
> -                "addresses": {"type": {"key": "string",
> -                                       "min": 0,
> -                                       "max": "unlimited"}},
> -                "dynamic_addresses": {"type": {"key": "string",
> -                                       "min": 0,
> -                                       "max": 1}},
> -                "port_security": {"type": {"key": "string",
> -                                           "min": 0,
> -                                           "max": "unlimited"}},
> -                "up": {"type": {"key": "boolean", "min": 0, "max": 1}},
> -                "enabled": {"type": {"key": "boolean", "min": 0, "max":
> 1}},
> -                "dhcpv4_options": {"type": {"key": {"type": "uuid",
> -                                            "refTable": "DHCP_Options",
> -                                            "refType": "weak"},
> -                                 "min": 0,
> -                                 "max": 1}},
> -                "dhcpv6_options": {"type": {"key": {"type": "uuid",
> -                                            "refTable": "DHCP_Options",
> -                                            "refType": "weak"},
> -                                 "min": 0,
> -                                 "max": 1}},
> -                "ha_chassis_group": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "HA_Chassis_Group",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": 1}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": false},
> -        "Address_Set": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "addresses": {"type": {"key": "string",
> -                                       "min": 0,
> -                                       "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Port_Group": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ports": {"type": {"key": {"type": "uuid",
> -                                           "refTable":
> "Logical_Switch_Port",
> -                                           "refType": "weak"},
> -                                   "min": 0,
> -                                   "max": "unlimited"}},
> -                "acls": {"type": {"key": {"type": "uuid",
> -                                          "refTable": "ACL",
> -                                          "refType": "strong"},
> -                                  "min": 0,
> -                                  "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Load_Balancer": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "vips": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "protocol": {
> -                    "type": {"key": {"type": "string",
> -                             "enum": ["set", ["tcp", "udp"]]},
> -                             "min": 0, "max": 1}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "ACL": {
> -            "columns": {
> -                "name": {"type": {"key": {"type": "string",
> -                                          "maxLength": 63},
> -                                          "min": 0, "max": 1}},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "direction": {"type": {"key": {"type": "string",
> -                                            "enum": ["set",
> ["from-lport", "to-lport"]]}}},
> -                "match": {"type": "string"},
> -                "action": {"type": {"key": {"type": "string",
> -                                            "enum": ["set", ["allow",
> "allow-related", "drop", "reject"]]}}},
> -                "log": {"type": "boolean"},
> -                "severity": {"type": {"key": {"type": "string",
> -                                              "enum": ["set",
> -                                                       ["alert",
> "warning",
> -                                                        "notice", "info",
> -                                                        "debug"]]},
> -                                      "min": 0, "max": 1}},
> -                "meter": {"type": {"key": "string", "min": 0, "max": 1}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "QoS": {
> -            "columns": {
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "direction": {"type": {"key": {"type": "string",
> -                                            "enum": ["set",
> ["from-lport", "to-lport"]]}}},
> -                "match": {"type": "string"},
> -                "action": {"type": {"key": {"type": "string",
> -                                            "enum": ["set", ["dscp"]]},
> -                                    "value": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 63},
> -                                    "min": 0, "max": "unlimited"}},
> -                "bandwidth": {"type": {"key": {"type": "string",
> -                                               "enum": ["set", ["rate",
> -
> "burst"]]},
> -                                       "value": {"type": "integer",
> -                                                 "minInteger": 1,
> -                                                 "maxInteger":
> 4294967295},
> -                                       "min": 0, "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "Meter": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "unit": {"type": {"key": {"type": "string",
> -                                          "enum": ["set", ["kbps",
> "pktps"]]}}},
> -                "bands": {"type": {"key": {"type": "uuid",
> -                                           "refTable": "Meter_Band",
> -                                           "refType": "strong"},
> -                                   "min": 1,
> -                                   "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Meter_Band": {
> -            "columns": {
> -                "action": {"type": {"key": {"type": "string",
> -                                            "enum": ["set", ["drop"]]}}},
> -                "rate": {"type": {"key": {"type": "integer",
> -                                          "minInteger": 1,
> -                                          "maxInteger": 4294967295}}},
> -                "burst_size": {"type": {"key": {"type": "integer",
> -                                                "minInteger": 0,
> -                                                "maxInteger":
> 4294967295}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "Logical_Router": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ports": {"type": {"key": {"type": "uuid",
> -                                           "refTable":
> "Logical_Router_Port",
> -                                           "refType": "strong"},
> -                                   "min": 0,
> -                                   "max": "unlimited"}},
> -                "static_routes": {"type": {"key": {"type": "uuid",
> -                                            "refTable":
> "Logical_Router_Static_Route",
> -                                            "refType": "strong"},
> -                                   "min": 0,
> -                                   "max": "unlimited"}},
> -                "policies": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "Logical_Router_Policy",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "enabled": {"type": {"key": "boolean", "min": 0, "max":
> 1}},
> -                "nat": {"type": {"key": {"type": "uuid",
> -                                         "refTable": "NAT",
> -                                         "refType": "strong"},
> -                                 "min": 0,
> -                                 "max": "unlimited"}},
> -                "load_balancer": {"type": {"key": {"type": "uuid",
> -                                                  "refTable":
> "Load_Balancer",
> -                                                  "refType": "weak"},
> -                                           "min": 0,
> -                                           "max": "unlimited"}},
> -                "options": {
> -                     "type": {"key": "string",
> -                              "value": "string",
> -                              "min": 0,
> -                              "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "Logical_Router_Port": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "gateway_chassis": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "Gateway_Chassis",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "ha_chassis_group": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "HA_Chassis_Group",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": 1}},
> -                "options": {
> -                    "type": {"key": "string",
> -                             "value": "string",
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "networks": {"type": {"key": "string",
> -                                      "min": 1,
> -                                      "max": "unlimited"}},
> -                "mac": {"type": "string"},
> -                "peer": {"type": {"key": "string", "min": 0, "max": 1}},
> -                "enabled": {"type": {"key": "boolean", "min": 0, "max":
> 1}},
> -                "ipv6_ra_configs": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": false},
> -        "Logical_Router_Static_Route": {
> -            "columns": {
> -                "ip_prefix": {"type": "string"},
> -                "policy": {"type": {"key": {"type": "string",
> -                                            "enum": ["set", ["src-ip",
> -                                                             "dst-ip"]]},
> -                                    "min": 0, "max": 1}},
> -                "nexthop": {"type": "string"},
> -                "output_port": {"type": {"key": "string", "min": 0,
> "max": 1}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "Logical_Router_Policy": {
> -            "columns": {
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "match": {"type": "string"},
> -                "action": {"type": {
> -                    "key": {"type": "string",
> -                            "enum": ["set", ["allow", "drop",
> "reroute"]]}}},
> -                "nexthop": {"type": {"key": "string", "min": 0, "max":
> 1}}},
> -            "isRoot": false},
> -        "NAT": {
> -            "columns": {
> -                "external_ip": {"type": "string"},
> -                "external_mac": {"type": {"key": "string",
> -                                          "min": 0, "max": 1}},
> -                "logical_ip": {"type": "string"},
> -                "logical_port": {"type": {"key": "string",
> -                                          "min": 0, "max": 1}},
> -                "type": {"type": {"key": {"type": "string",
> -                                           "enum": ["set", ["dnat",
> -                                                             "snat",
> -
>  "dnat_and_snat"
> -                                                               ]]}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "DHCP_Options": {
> -            "columns": {
> -                "cidr": {"type": "string"},
> -                "options": {"type": {"key": "string", "value": "string",
> -                                     "min": 0, "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "Connection": {
> -            "columns": {
> -                "target": {"type": "string"},
> -                "max_backoff": {"type": {"key": {"type": "integer",
> -                                         "minInteger": 1000},
> -                                         "min": 0,
> -                                         "max": 1}},
> -                "inactivity_probe": {"type": {"key": "integer",
> -                                              "min": 0,
> -                                              "max": 1}},
> -                "other_config": {"type": {"key": "string",
> -                                          "value": "string",
> -                                          "min": 0,
> -                                          "max": "unlimited"}},
> -                "external_ids": {"type": {"key": "string",
> -                                 "value": "string",
> -                                 "min": 0,
> -                                 "max": "unlimited"}},
> -                "is_connected": {"type": "boolean", "ephemeral": true},
> -                "status": {"type": {"key": "string",
> -                                    "value": "string",
> -                                    "min": 0,
> -                                    "max": "unlimited"},
> -                                    "ephemeral": true}},
> -            "indexes": [["target"]]},
> -        "DNS": {
> -            "columns": {
> -                "records": {"type": {"key": "string",
> -                                     "value": "string",
> -                                     "min": 0,
> -                                     "max": "unlimited"}},
> -                "external_ids": {"type": {"key": "string",
> -                                 "value": "string",
> -                                 "min": 0,
> -                                 "max": "unlimited"}}},
> -            "isRoot": true},
> -        "SSL": {
> -            "columns": {
> -                "private_key": {"type": "string"},
> -                "certificate": {"type": "string"},
> -                "ca_cert": {"type": "string"},
> -                "bootstrap_ca_cert": {"type": "boolean"},
> -                "ssl_protocols": {"type": "string"},
> -                "ssl_ciphers": {"type": "string"},
> -                "external_ids": {"type": {"key": "string",
> -                                          "value": "string",
> -                                          "min": 0,
> -                                          "max": "unlimited"}}},
> -            "maxRows": 1},
> -        "Gateway_Chassis": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "chassis_name": {"type": "string"},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "options": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": false},
> -        "HA_Chassis": {
> -            "columns": {
> -                "chassis_name": {"type": "string"},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "HA_Chassis_Group": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ha_chassis": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "HA_Chassis",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true}}
> -    }
> diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml
> deleted file mode 100644
> index 57b6edbf8..000000000
> --- a/ovn/ovn-nb.xml
> +++ /dev/null
> @@ -1,2917 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<database name="ovn-nb" title="OVN Northbound Database">
> -  <p>
> -    This database is the interface between OVN and the cloud management
> system
> -    (CMS), such as OpenStack, running above it.  The CMS produces almost
> all of
> -    the contents of the database.  The <code>ovn-northd</code> program
> -    monitors the database contents, transforms it, and stores it into the
> <ref
> -    db="OVN_Southbound"/> database.
> -  </p>
> -
> -  <p>
> -    We generally speak of ``the'' CMS, but one can imagine scenarios in
> -    which multiple CMSes manage different parts of an OVN deployment.
> -  </p>
> -
> -  <h2>External IDs</h2>
> -
> -  <p>
> -    Each of the tables in this database contains a special column, named
> -    <code>external_ids</code>.  This column has the same form and purpose
> each
> -    place it appears.
> -  </p>
> -
> -  <dl>
> -    <dt><code>external_ids</code>: map of string-string pairs</dt>
> -    <dd>
> -      Key-value pairs for use by the CMS.  The CMS might use certain
> pairs, for
> -      example, to identify entities in its own configuration that
> correspond to
> -      those in this database.
> -    </dd>
> -  </dl>
> -
> -  <table name="NB_Global" title="Northbound configuration">
> -    <p>
> -      Northbound configuration for an OVN system.  This table must have
> exactly
> -      one row.
> -    </p>
> -
> -    <group title="Status">
> -      These columns allow a client to track the overall configuration
> state of
> -      the system.
> -
> -      <column name="nb_cfg">
> -        Sequence number for client to increment.  When a client modifies
> any
> -        part of the northbound database configuration and wishes to wait
> for
> -        <code>ovn-northd</code> and possibly all of the hypervisors to
> finish
> -        applying the changes, it may increment this sequence number.
> -      </column>
> -
> -      <column name="sb_cfg">
> -        Sequence number that <code>ovn-northd</code> sets to the value of
> <ref
> -        column="nb_cfg"/> after it finishes applying the corresponding
> -        configuration changes to the <ref db="OVN_Southbound"/> database.
> -      </column>
> -
> -      <column name="hv_cfg">
> -        Sequence number that <code>ovn-northd</code> sets to the smallest
> -        sequence number of all the chassis in the system, as reported in
> the
> -        <code>Chassis</code> table in the southbound database.  Thus, <ref
> -        column="hv_cfg"/> equals <ref column="nb_cfg"/> if all chassis are
> -        caught up with the northbound configuration (which may never
> happen, if
> -        any chassis is down).  This value can regress, if a chassis was
> removed
> -        from the system and rejoins before catching up.
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -
> -    <group title="Common options">
> -      <column name="options">
> -        This column provides general key/value settings. The supported
> -        options are described individually below.
> -      </column>
> -
> -      <group title="Options for configuring BFD">
> -        <p>
> -          These options apply when <code>ovn-controller</code> configures
> -          BFD on tunnels interfaces.
> -        </p>
> -
> -        <column name="options" key="bfd-min-rx">
> -          BFD option <code>min-rx</code> value to use when configuring
> BFD on
> -          tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-decay-min-rx">
> -          BFD option <code>decay-min-rx</code> value to use when
> configuring
> -          BFD on tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-min-tx">
> -          BFD option <code>min-tx</code> value to use when configuring
> BFD on
> -          tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-mult">
> -          BFD option <code>mult</code> value to use when configuring BFD
> on
> -          tunnel interfaces.
> -        </column>
> -      </group>
> -
> -      <column name="options" key="mac_prefix">
> -        Configure a given OUI to be used as prefix when L2 address is
> -        dynamically assigned, e.g. <code>00:11:22</code>
> -      </column>
> -
> -      <column name="options" key="controller_event" type='{"type":
> "boolean"}'>
> -        Value set by the CMS to enable/disable ovn-controller event
> reporting.
> -        Traffic into OVS can raise a 'controller' event that results in a
> -        Controller_Event being written to the <ref
> table="Controller_Event"/>
> -        table in SBDB. When the CMS has seen the event and taken
> appropriate
> -        action, it can remove the correponding row in
> -        <ref table="Controller_Event"/> table.
> -        The intention is for a CMS to see the events and take some sort of
> -        action. Please see the <ref table="Controller_Event"/> table in
> SBDB.
> -      </column>
> -    </group>
> -
> -    <group title="Connection Options">
> -      <column name="connections">
> -        Database clients to which the Open vSwitch database server should
> -        connect or on which it should listen, along with options for how
> these
> -        connections should be configured.  See the <ref
> table="Connection"/>
> -        table for more information.
> -      </column>
> -      <column name="ssl">
> -        Global SSL configuration.
> -      </column>
> -    </group>
> -    <group title="Security Configurations">
> -      <column name="ipsec">
> -        Tunnel encryption configuration. If this column is set to be
> true, all
> -        OVN tunnels will be encrypted with IPsec.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Logical_Switch" title="L2 logical switch">
> -    <p>
> -      Each row represents one L2 logical switch.
> -    </p>
> -
> -    <p>
> -      There are two kinds of logical switches, that is, ones that fully
> -      virtualize the network (overlay logical switches) and ones that
> provide
> -      simple connectivity to a physical network (bridged logical
> switches).
> -      They work in the same way when providing connectivity between
> logical
> -      ports on same chasis, but differently when connecting remote logical
> -      ports.  Overlay logical switches connect remote logical ports by
> tunnels,
> -      while bridged logical switches provide connectivity to remote ports
> by
> -      bridging the packets to directly connected physical L2 segment with
> the
> -      help of <code>localnet</code> ports.  Each bridged logical switch
> has
> -      one and only one <code>localnet</code> port, which has only one
> special
> -      address <code>unknown</code>.
> -    </p>
> -
> -    <column name="ports">
> -      <p>
> -        The logical ports connected to the logical switch.
> -      </p>
> -
> -      <p>
> -        It is an error for multiple logical switches to include the same
> -        logical port.
> -      </p>
> -    </column>
> -
> -    <column name="load_balancer">
> -      Load balance a virtual ip address to a set of logical port endpoint
> -      ip addresses.
> -    </column>
> -
> -    <column name="acls">
> -      Access control rules that apply to packets within the logical
> switch.
> -    </column>
> -
> -    <column name="qos_rules">
> -      QoS marking and metering rules that apply to packets within the
> -      logical switch.
> -    </column>
> -
> -    <column name="dns_records">
> -      This column defines the DNS records to be used for resolving
> internal
> -      DNS queries within the logical switch by the native DNS resolver.
> -      Please see the <ref table="DNS"/> table.
> -    </column>
> -
> -    <group title="Naming">
> -      <p>
> -        These columns provide names for the logical switch.  From OVN's
> -        perspective, these names have no special meaning or purpose other
> than
> -        to provide convenience for human interaction with the  database.
> -        There is no requirement for the name to be unique.  (For a unique
> -        identifier for a logical switch, use its row UUID.)
> -      </p>
> -
> -      <p>
> -        (Originally, <ref column="name"/> was intended to serve the
> purpose of
> -        a human-friendly name, but the Neutron integration used it to
> uniquely
> -        identify its own switch object, in the format
> -        <code>neutron-<var>uuid</var></code>.  Later on, Neutron started
> -        propagating the friendly name of a switch as <ref
> column="external_ids"
> -        key="neutron:network_name"/>.  Perhaps this can be cleaned up
> someday.)
> -      </p>
> -
> -      <column name="name">
> -        A name for the logical switch.
> -      </column>
> -
> -      <column name="external_ids" key="neutron:network_name">
> -        Another name for the logical switch.
> -      </column>
> -    </group>
> -
> -    <group title="IP Address Assignment">
> -      <p>
> -        These options control automatic IP address management (IPAM) for
> ports
> -        attached to the logical switch.  To enable IPAM for IPv4, set <ref
> -        column="other_config" key="subnet"/> and optionally <ref
> -        column="other_config:exclude_ips"/>.  To enable IPAM for IPv6, set
> -        <ref column="other_config" key="ipv6_prefix"/>.  IPv4 and IPv6 may
> -        be enabled together or separately.
> -      </p>
> -
> -      <p>
> -        To request dynamic address assignment for a particular port, use
> the
> -        <code>dynamic</code> keyword in the <ref
> table="Logical_Switch_Port"
> -        column="addresses"/> column of the port's <ref
> -        table="Logical_Switch_Port"/> row.  This requests both an IPv4
> and an
> -        IPv6 address, if IPAM for IPv4 and IPv6 are both enabled.
> -      </p>
> -
> -      <column name="other_config" key="subnet">
> -        Set this to an IPv4 subnet, e.g. <code>192.168.0.0/24</code>, to
> enable
> -        <code>ovn-northd</code> to automatically assign IP addresses
> within
> -        that subnet.
> -      </column>
> -
> -      <column name="other_config" key="exclude_ips">
> -        <p>
> -          To exclude some addresses from automatic IP address management,
> set
> -          this to a list of the IPv4 addresses or
> <code>..</code>-delimited
> -          ranges to exclude.  The addresses or ranges should be a subset
> of
> -          those in <ref column="other_config" key="subnet"/>.
> -        </p>
> -        <p>
> -          Whether listed or not, <code>ovn-northd</code> will never
> allocate
> -          the first or last address in a subnet, such as 192.168.0.0 or
> -          192.168.0.255 in 192.168.0.0/24.
> -        </p>
> -        <p>
> -          Examples:
> -        </p>
> -        <ul>
> -          <li><code>192.168.0.2 192.168.0.10</code></li>
> -          <li><code>192.168.0.4 192.168.0.30..192.168.0.60
> 192.168.0.110..192.168.0.120</code></li>
> -          <li><code>192.168.0.110..192.168.0.120
> 192.168.0.25..192.168.0.30 192.168.0.144</code></li>
> -        </ul>
> -      </column>
> -
> -      <column name="other_config" key="ipv6_prefix">
> -        Set this to an IPv6 prefix to enable <code>ovn-northd</code> to
> -        automatically assign IPv6 addresses using this prefix.  The
> assigned
> -        IPv6 address will be generated using the IPv6 prefix and the MAC
> -        address (converted to an IEEE EUI64 identifier) of the port.  The
> IPv6
> -        prefix defined here should be a valid IPv6 address ending with
> -        <code>::</code>.
> -        <p>
> -          Examples:
> -        </p>
> -        <ul>
> -          <li><code>aef0::</code></li>
> -          <li><code>bef0:1234:a890:5678::</code></li>
> -          <li><code>8230:5678::</code></li>
> -        </ul>
> -      </column>
> -
> -      <column name="other_config" key="mac_only" type='{"type":
> "boolean"}'>
> -        Value used to request to assign L2 address only if neither subnet
> -        nor ipv6_prefix are specified
> -      </column>
> -    </group>
> -
> -    <group title="IP Multicast Snooping Options">
> -      <p>
> -        These options control IP Multicast Snooping configuration of the
> -        logical switch. To enable IP Multicast Snooping set
> -        <ref column="other_config" key="mcast_snoop"/> to true. To enable
> IP
> -        Multicast Querier set <ref column="other_config"
> key="mcast_snoop"/>
> -        to true. If IP Multicast Querier is enabled
> -        <ref column="other_config" key="mcast_eth_src"/> and
> -        <ref column="other_config" key="mcast_ip4_src"/> must be set.
> -      </p>
> -      <column name="other_config" key="mcast_snoop"
> -          type='{"type": "boolean"}'>
> -        Enables/disables IP Multicast Snooping on the logical switch.
> -      </column>
> -      <column name="other_config" key="mcast_querier"
> -          type='{"type": "boolean"}'>
> -        Enables/disables IP Multicast Querier on the logical switch.
> -      </column>
> -      <column name="other_config" key="mcast_flood_unregistered"
> -          type='{"type": "boolean"}'>
> -        Determines whether unregistered multicast traffic should be
> flooded
> -        or not. Only applicable if
> -        <ref column="other_config" key="mcast_snoop"/> is enabled.
> -      </column>
> -      <column name="other_config" key="mcast_table_size"
> -          type='{"type": "integer", "minInteger": 1, "maxInteger":
> 32766}'>
> -        Number of multicast groups to be stored. Default: 2048.
> -      </column>
> -      <column name="other_config" key="mcast_idle_timeout"
> -          type='{"type": "integer", "minInteger": 15, "maxInteger":
> 3600}'>
> -        Configures the IP Multicast Snooping group idle timeout (in
> seconds).
> -        Default: 300 seconds.
> -      </column>
> -      <column name="other_config" key="mcast_query_interval"
> -          type='{"type": "integer", "minInteger": 1, "maxInteger": 3600}'>
> -        Configures the IP Multicast Querier interval between queries (in
> -        seconds). Default:
> -        <ref column="other_config" key="mcast_idle_timeout"/> / 2.
> -      </column>
> -      <column name="other_config" key="mcast_query_max_response"
> -          type='{"type": "integer", "minInteger": 1, "maxInteger": 10}'>
> -        Configures the value of the "max-response" field in the multicast
> -        queries originated by the logical switch. Default: 1 second.
> -      </column>
> -      <column name="other_config" key="mcast_eth_src">
> -        Configures the source Ethernet address for queries originated by
> the
> -        logical switch.
> -      </column>
> -      <column name="other_config" key="mcast_ip4_src">
> -        Configures the source IPv4 address for queries originated by the
> -        logical switch.
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Logical_Switch_Port" title="L2 logical switch port">
> -    <p>
> -      A port within an L2 logical switch.
> -    </p>
> -
> -    <group title="Core Features">
> -      <column name="name">
> -        <p>
> -          The logical port name.
> -        </p>
> -
> -        <p>
> -          For entities (VMs or containers) that are spawned in the
> hypervisor,
> -          the name used here must match those used in the <ref
> key="iface-id"
> -          table="Interface" column="external_ids" db="Open_vSwitch"/> in
> the
> -          <ref db="Open_vSwitch"/> database's <ref table="Interface"
> -          db="Open_vSwitch"/> table, because hypervisors use <ref
> key="iface-id"
> -          table="Interface" column="external_ids" db="Open_vSwitch"/> as
> a lookup
> -          key to identify the network interface of that entity.
> -        </p>
> -
> -        <p>
> -          For containers that share a VIF within a VM, the name can be any
> -          unique identifier.  See <code>Containers</code>, below, for more
> -          information.
> -        </p>
> -      </column>
> -
> -      <column name="type">
> -        <p>
> -          Specify a type for this logical port.  Logical ports can be
> used to
> -          model other types of connectivity into an OVN logical switch.
> The
> -          following types are defined:
> -        </p>
> -
> -        <dl>
> -          <dt>(empty string)</dt>
> -          <dd>
> -            A VM (or VIF) interface.
> -          </dd>
> -
> -          <dt><code>router</code></dt>
> -          <dd>
> -            A connection to a logical router.
> -          </dd>
> -
> -          <dt><code>localnet</code></dt>
> -          <dd>
> -            A connection to a locally accessible network from each
> -            <code>ovn-controller</code> instance.  A logical switch can
> only
> -            have a single <code>localnet</code> port attached.  This is
> used
> -            to model direct connectivity to an existing network.
> -          </dd>
> -
> -          <dt><code>localport</code></dt>
> -          <dd>
> -            A connection to a local VIF. Traffic that arrives on a
> -            <code>localport</code> is never forwarded over a tunnel to
> another
> -            chassis. These ports are present on every chassis and have
> the same
> -            address in all of them. This is used to model connectivity to
> local
> -            services that run on every hypervisor.
> -          </dd>
> -
> -          <dt><code>l2gateway</code></dt>
> -          <dd>
> -            A connection to a physical network.
> -          </dd>
> -
> -          <dt><code>vtep</code></dt>
> -          <dd>
> -            A port to a logical switch on a VTEP gateway.
> -          </dd>
> -
> -          <dt><code>external</code></dt>
> -          <dd>
> -            <p>
> -              Represents a logical port which is external and not having
> -              an OVS port in the integration bridge.
> -              <code>OVN</code> will never receive any traffic from this
> port or
> -              send any traffic to this port. <code>OVN</code> can support
> -              native services like DHCPv4/DHCPv6/DNS for this port.
> -              If <ref column="ha_chassis_group"/> is defined,
> -              <code>ovn-controller</code> running in the master chassis of
> -              the HA chassis group will bind this port to provide these
> native
> -              services. It is expected that this port belong to a bridged
> -              logical switch (with a <code>localnet</code> port).
> -            </p>
> -
> -            <p>
> -              It is recommended to use the same HA chassis group for all
> the
> -              external ports of a logical switch. Otherwise, the physical
> -              switch might see MAC flap issue when different chassis
> provide
> -              the native services. For example when supporting native
> DHCPv4
> -              service, DHCPv4 server mac (configured in
> -              <ref column="options:server_mac" table="DHCP_Options"
> -              db="OVN_NB"/> column in table <ref table="DHCP_Options"/>)
> -              originating from different ports can cause MAC flap issue.
> -              The MAC of the logical router IP(s) can also flap if the
> -              same HA chassis group is not set for all the external ports
> -              of a logical switch.
> -            </p>
> -
> -            <p>
> -              Below are some of the use cases where <code>external</code>
> -              ports can be used.
> -            </p>
> -
> -            <ul>
> -              <li>
> -                VMs connected to SR-IOV nics - Traffic from these VMs by
> passes
> -                the kernel stack and local <code>ovn-controller</code> do
> not
> -                bind these ports and cannot serve the native services.
> -              </li>
> -
> -              <li>
> -                When CMS supports provisioning baremetal servers.
> -              </li>
> -            </ul>
> -          </dd>
> -        </dl>
> -      </column>
> -    </group>
> -
> -    <group title="Options">
> -      <column name="options">
> -        This column provides key/value settings specific to the logical
> port
> -        <ref column="type"/>.  The type-specific options are described
> -        individually below.
> -      </column>
> -
> -      <group title="Options for router ports">
> -        <p>
> -          These options apply when <ref column="type"/> is
> <code>router</code>.
> -        </p>
> -
> -        <column name="options" key="router-port">
> -          Required.  The <ref column="name"/> of the <ref
> -          table="Logical_Router_Port"/> to which this logical switch port
> is
> -          connected.
> -        </column>
> -
> -        <column name="options" key="nat-addresses">
> -          <p>
> -            This is used to send gratuitous ARPs for SNAT and DNAT IP
> -            addresses via the <code>localnet</code> port that is attached
> -            to the same logical switch as this type <code>router</code>
> -            port.  This option is specified on a logical switch port that
> is
> -            connected to a gateway router, or a logical switch port that
> is
> -            connected to a distributed gateway port on a logical router.
> -          </p>
> -
> -          <p>
> -            This must take one of the following forms:
> -          </p>
> -
> -          <dl>
> -            <dt><code>router</code></dt>
> -            <dd>
> -              <p>
> -                Gratuitous ARPs will be sent for all SNAT and DNAT
> external IP
> -                addresses and for all load balancer IP addresses defined
> on the
> -                <ref column="options" key="router-port"/>'s logical
> router,
> -                using the <ref column="options" key="router-port"/>'s MAC
> -                address.
> -              </p>
> -
> -              <p>
> -                This form of <ref column="options" key="nat-addresses"/>
> is
> -                valid for logical switch ports where <ref column="options"
> -                key="router-port"/> is the name of a port on a gateway
> router,
> -                or the name of a distributed gateway port.
> -              </p>
> -
> -              <p>
> -                Supported only in OVN 2.8 and later.  Earlier versions
> required
> -                NAT addresses to be manually synchronized.
> -              </p>
> -            </dd>
> -
> -            <dt><code>Ethernet address followed by one or more IPv4
> addresses</code></dt>
> -            <dd>
> -              <p>
> -                Example: <code>80:fa:5b:06:72:b7 158.36.44.22
> -                158.36.44.24</code>. This would result in generation of
> -                gratuitous ARPs for IP addresses 158.36.44.22 and
> 158.36.44.24
> -                with a MAC address of 80:fa:5b:06:72:b7.
> -              </p>
> -
> -              <p>
> -                This form of <ref column="options" key="nat-addresses"/>
> is
> -                only valid for logical switch ports where <ref
> column="options"
> -                key="router-port"/> is the name of a port on a gateway
> router.
> -              </p>
> -            </dd>
> -          </dl>
> -        </column>
> -      </group>
> -
> -      <group title="Options for localnet ports">
> -        <p>
> -          These options apply when <ref column="type"/> is
> -          <code>localnet</code>.
> -        </p>
> -
> -        <column name="options" key="network_name">
> -          Required.  The name of the network to which the
> <code>localnet</code>
> -          port is connected.  Each hypervisor, via
> <code>ovn-controller</code>,
> -          uses its local configuration to determine exactly how to
> connect to
> -          this locally accessible network.
> -        </column>
> -      </group>
> -
> -      <group title="Options for l2gateway ports">
> -        <p>
> -          These options apply when <ref column="type"/> is
> -          <code>l2gateway</code>.
> -        </p>
> -
> -        <column name="options" key="network_name">
> -          Required.  The name of the network to which the
> <code>l2gateway</code>
> -          port is connected.  The L2 gateway, via
> <code>ovn-controller</code>,
> -          uses its local configuration to determine exactly how to
> connect to
> -          this network.
> -        </column>
> -
> -        <column name="options" key="l2gateway-chassis">
> -          Required. The chassis on which the <code>l2gateway</code>
> logical
> -          port should be bound to. <code>ovn-controller</code> running on
> the
> -          defined chassis will connect this logical port to the physical
> network.
> -        </column>
> -
> -      </group>
> -
> -      <group title="Options for vtep ports">
> -        <p>
> -          These options apply when <ref column="type"/> is
> <code>vtep</code>.
> -        </p>
> -
> -        <column name="options" key="vtep-physical-switch">
> -          Required.  The name of the VTEP gateway.
> -        </column>
> -
> -        <column name="options" key="vtep-logical-switch">
> -          Required.  A logical switch name connected by the VTEP gateway.
> -        </column>
> -      </group>
> -
> -      <group title="VMI (or VIF) Options">
> -        <p>
> -          These options apply to logical ports with <ref column="type"/>
> having
> -          (empty string)
> -        </p>
> -
> -        <column name="options" key="requested-chassis">
> -          If set, identifies a specific chassis (by name or hostname) that
> -          is allowed to bind this port. Using this option will prevent
> -          thrashing between two chassis trying to bind the same port
> during
> -          a live migration. It can also prevent similar thrashing due to a
> -          mis-configuration, if a port is accidentally created on more
> than
> -          one chassis.
> -        </column>
> -
> -        <column name="options" key="qos_max_rate">
> -          If set, indicates the maximum rate for data sent from this
> interface,
> -          in bit/s. The traffic will be shaped according to this limit.
> -        </column>
> -
> -        <column name="options" key="qos_burst">
> -          If set, indicates the maximum burst size for data sent from this
> -          interface, in bits.
> -        </column>
> -      </group>
> -    </group>
> -
> -    <group title="Containers">
> -      <p>
> -        When a large number of containers are nested within a VM, it may
> be too
> -        expensive to dedicate a VIF to each container.  OVN can use VLAN
> tags
> -        to support such cases.  Each container is assigned a VLAN ID and
> each
> -        packet that passes between the hypervisor and the VM is tagged
> with the
> -        appropriate ID for the container.  Such VLAN IDs never appear on a
> -        physical wire, even inside a tunnel, so they need not be unique
> except
> -        relative to a single VM on a hypervisor.
> -      </p>
> -
> -      <p>
> -        These columns are used for VIFs that represent nested containers
> using
> -        shared VIFs.  For VMs and for containers that have dedicated
> VIFs, they
> -        are empty.
> -      </p>
> -
> -      <column name="parent_name">
> -        The VM interface through which the nested container sends its
> network
> -        traffic.  This must match the <ref column="name"/> column for some
> -        other <ref table="Logical_Switch_Port"/>.
> -      </column>
> -
> -      <column name="tag_request">
> -        <p>
> -          The VLAN tag in the network traffic associated with a
> container's
> -          network interface.  The client can request
> <code>ovn-northd</code>
> -          to allocate a tag that is unique within the scope of a specific
> -          parent (specified in <ref column="parent_name"/>) by setting a
> value
> -          of <code>0</code> in this column.  The allocated value is
> written
> -          by <code>ovn-northd</code> in the <ref column="tag"/> column.
> -          (Note that these tags are allocated and managed locally in
> -          <code>ovn-northd</code>, so they cannot be reconstructed in the
> event
> -          that the database is lost.)  The client can also request a
> specific
> -          non-zero tag and <code>ovn-northd</code> will honor it and copy
> that
> -          value to the <ref column="tag"/> column.
> -        </p>
> -
> -        <p>
> -          When <ref column="type"/> is set to <code>localnet</code> or
> -          <code>l2gateway</code>, this can
> -          be set to indicate that the port represents a connection to a
> -          specific VLAN on a locally accessible network. The VLAN ID is
> used
> -          to match incoming traffic and is also added to outgoing traffic.
> -        </p>
> -      </column>
> -
> -      <column name="tag">
> -        <p>
> -          The VLAN tag allocated by <code>ovn-northd</code> based on the
> -          contents of the <ref column="tag_request"/> column.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Port State">
> -      <column name="up">
> -        <p>
> -          This column is populated by <code>ovn-northd</code>, rather
> -          than by the CMS plugin as is most of this database.  When a
> -          logical port is bound to a physical location in the OVN
> -          Southbound database <ref db="OVN_Southbound"
> -          table="Binding"/> table, <code>ovn-northd</code> sets this
> -          column to <code>true</code>; otherwise, or if the port
> -          becomes unbound later, it sets it to <code>false</code>.
> -          This allows the CMS to wait for a VM's (or container's)
> -          networking to become active before it allows the VM (or
> -          container) to start.
> -        </p>
> -
> -        <p>
> -          Logical ports of router type are an exception to this rule.
> -          They are considered to be always up, that is this column is
> -          always set to <code>true</code>.
> -        </p>
> -      </column>
> -
> -      <column name="enabled">
> -        This column is used to administratively set port state.  If this
> column
> -        is empty or is set to <code>true</code>, the port is enabled.  If
> this
> -        column is set to <code>false</code>, the port is disabled.  A
> disabled
> -        port has all ingress and egress traffic dropped.
> -      </column>
> -
> -    </group>
> -
> -    <group title="Addressing">
> -      <column name="addresses">
> -        <p>
> -          Addresses owned by the logical port.
> -        </p>
> -
> -        <p>
> -          Each element in the set must take one of the following forms:
> -        </p>
> -
> -        <dl>
> -          <dt><code>Ethernet address followed by zero or more IPv4 or
> IPv6 addresses (or both)</code></dt>
> -          <dd>
> -            <p>
> -              An Ethernet address defined is owned by the logical port.
> -              Like a physical Ethernet NIC, a logical port ordinarily has
> -              a single fixed Ethernet address.
> -            </p>
> -
> -            <p>
> -              When a OVN logical switch processes a unicast Ethernet frame
> -              whose destination MAC address is in a logical port's <ref
> -              column="addresses"/> column, it delivers it only to that
> port, as
> -              if a MAC learning process had learned that MAC address on
> the
> -              port.
> -            </p>
> -
> -            <p>
> -              If IPv4 or IPv6 address(es) (or both) are defined, it
> indicates
> -              that the logical port owns the given IP addresses.
> -            </p>
> -
> -            <p>
> -              If IPv4 address(es) are defined, the OVN logical switch
> uses this
> -              information to synthesize responses to ARP requests without
> -              traversing the physical network. The OVN logical router
> connected
> -              to the logical switch, if any, uses this information to
> avoid
> -              issuing ARP requests for logical switch ports.
> -            </p>
> -
> -            <p>
> -              Note that the order here is important. The Ethernet address
> must
> -              be listed before the IP address(es) if defined.
> -            </p>
> -
> -            <p>
> -              Examples:
> -            </p>
> -
> -            <dl>
> -              <dt><code>80:fa:5b:06:72:b7</code></dt>
> -              <dd>
> -                This indicates that the logical port owns the above mac
> address.
> -              </dd>
> -
> -              <dt><code>80:fa:5b:06:72:b7 10.0.0.4 20.0.0.4</code></dt>
> -              <dd>
> -                This indicates that the logical port owns the mac address
> and two
> -                IPv4 addresses.
> -              </dd>
> -
> -              <dt><code>80:fa:5b:06:72:b7
> fdaa:15f2:72cf:0:f816:3eff:fe20:3f41</code></dt>
> -              <dd>
> -                This indicates that the logical port owns the mac address
> and
> -                1 IPv6 address.
> -              </dd>
> -
> -              <dt><code>80:fa:5b:06:72:b7 10.0.0.4
> fdaa:15f2:72cf:0:f816:3eff:fe20:3f41</code></dt>
> -              <dd>
> -                This indicates that the logical port owns the mac address
> and
> -                1 IPv4 address and 1 IPv6 address.
> -              </dd>
> -            </dl>
> -          </dd>
> -
> -          <dt><code>unknown</code></dt>
> -          <dd>
> -            This indicates that the logical port has an unknown set of
> Ethernet
> -            addresses.  When an OVN logical switch processes a unicast
> Ethernet
> -            frame whose destination MAC address is not in any logical
> port's
> -            <ref column="addresses"/> column, it delivers it to the port
> (or
> -            ports) whose <ref column="addresses"/> columns include
> -            <code>unknown</code>.
> -          </dd>
> -
> -          <dt><code>dynamic</code></dt>
> -          <dd>
> -            Use this keyword to make <code>ovn-northd</code> generate a
> -            globally unique MAC address and choose an unused IPv4 address
> with
> -            the logical port's subnet and store them in the port's <ref
> -            column="dynamic_addresses"/> column.  <code>ovn-northd</code>
> will
> -            use the subnet specified in <ref table="Logical_Switch"
> -            column="other_config" key="subnet"/> in the port's <ref
> -            table="Logical_Switch"/>.
> -          </dd>
> -
> -          <dt><code>Ethernet address followed by keyword
> "dynamic"</code></dt>
> -          <dd>
> -
> -            <p>
> -              The keyword <code>dynamic</code> after the MAC address
> indicates
> -              that <code>ovn-northd</code> should choose an unused IPv4
> address
> -              from the logical port's subnet and store it with the
> specified
> -              MAC in the port's <ref column="dynamic_addresses"/> column.
> -              <code>ovn-northd</code> will use the subnet specified in
> <ref
> -              table="Logical_Switch" column="other_config" key="subnet"/>
> in
> -              the port's <ref table="Logical_Switch"/> table.
> -            </p>
> -
> -            <p>
> -              Examples:
> -            </p>
> -
> -            <dl>
> -              <dt><code>80:fa:5b:06:72:b7 dynamic</code></dt>
> -              <dd>
> -                This indicates that the logical port owns the specified
> -                MAC address and <code>ovn-northd</code> should allocate an
> -                unused IPv4 address for the logical port from the
> corresponding
> -                logical switch subnet.
> -              </dd>
> -            </dl>
> -          </dd>
> -
> -          <dt><code>Keyword "dynamic" followed by an IPv4/IPv6
> address</code></dt>
> -          <dd>
> -
> -            <p>
> -              The keyword <code>dynamic</code> followed by an IPv4/IPv6
> -              address indicates that <code>ovn-northd</code> should choose
> -              a dynamic ethernet address and use the provided IPv4/IPv6
> address
> -              as network address.
> -            </p>
> -
> -            <p>
> -              Examples:
> -            </p>
> -
> -            <dl>
> -              <dt><code>dynamic 192.168.0.1 2001::1</code></dt>
> -              <dd>
> -                This indicates that <code>ovn-northd</code> should
> allocate
> -                a unique MAC address and use the provided IPv4/IPv6
> address
> -                for the related port
> -              </dd>
> -            </dl>
> -          </dd>
> -
> -          <dt><code>router</code></dt>
> -          <dd>
> -            <p>
> -              Accepted only when <ref column="type"/> is
> <code>router</code>.
> -              This indicates that the Ethernet, IPv4, and IPv6 addresses
> for
> -              this logical switch port should be obtained from the
> connected
> -              logical router port, as specified by
> <code>router-port</code> in
> -              <ref column="options"/>.
> -            </p>
> -
> -            <p>
> -              The resulting addresses are used to populate the logical
> -              switch's destination lookup, and also for the logical switch
> -              to generate ARP and ND replies.
> -            </p>
> -
> -            <p>
> -              If the connected logical router port has a
> -              <code>redirect-chassis</code> specified and the logical
> router
> -              has rules specified in <ref column="nat"
> table="Logical_Router"/>
> -              with <ref column="external_mac" table="NAT"/>, then those
> -              addresses are also used to populate the switch's destination
> -              lookup.
> -            </p>
> -
> -            <p>
> -              Supported only in OVN 2.7 and later.  Earlier versions
> required
> -              router addresses to be manually synchronized.
> -            </p>
> -          </dd>
> -
> -        </dl>
> -      </column>
> -
> -      <column name="dynamic_addresses">
> -        <p>
> -          Addresses assigned to the logical port by
> <code>ovn-northd</code>, if
> -          <code>dynamic</code> is specified in <ref column="addresses"/>.
> -          Addresses will be of the same format as those that populate the
> <ref
> -          column="addresses"/> column.  Note that dynamically assigned
> -          addresses are constructed and managed locally in ovn-northd, so
> they
> -          cannot be reconstructed in the event that the database is lost.
> -        </p>
> -      </column>
> -
> -      <column name="port_security">
> -        <p>
> -          This column controls the addresses from which the host attached
> to the
> -          logical port (``the host'') is allowed to send packets and to
> which it
> -          is allowed to receive packets.  If this column is empty, all
> addresses
> -          are permitted.
> -        </p>
> -
> -        <p>
> -          Each element in the set must begin with one Ethernet address.
> -          This would restrict the host to sending packets from and
> receiving
> -          packets to the ethernet addresses defined in the logical port's
> -          <ref column="port_security"/> column. It also restricts the
> inner
> -          source MAC addresses that the host may send in ARP and IPv6
> -          Neighbor Discovery packets. The host is always allowed to
> receive packets
> -          to multicast and broadcast Ethernet addresses.
> -        </p>
> -
> -        <p>
> -          Each element in the set may additionally contain one or more
> IPv4 or
> -          IPv6 addresses (or both), with optional masks.  If a mask is
> given, it
> -          must be a CIDR mask.  In addition to the restrictions described
> for
> -          Ethernet addresses above, such an element restricts the IPv4 or
> IPv6
> -          addresses from which the host may send and to which it may
> receive
> -          packets to the specified addresses.  A masked address, if the
> host part
> -          is zero, indicates that the host is allowed to use any address
> in the
> -          subnet; if the host part is nonzero, the mask simply indicates
> the size
> -          of the subnet. In addition:
> -        </p>
> -
> -        <ul>
> -          <li>
> -            <p>
> -              If any IPv4 address is given, the host is also allowed to
> receive
> -              packets to the IPv4 local broadcast address 255.255.255.255
> and to
> -              IPv4 multicast addresses (224.0.0.0/4).  If an IPv4
> address with a
> -              mask is given, the host is also allowed to receive packets
> to the
> -              broadcast address in that specified subnet.
> -            </p>
> -
> -            <p>
> -              If any IPv4 address is given, the host is additionally
> restricted
> -              to sending ARP packets with the specified source IPv4
> address.
> -              (RARP is not restricted.)
> -            </p>
> -          </li>
> -
> -          <li>
> -            <p>
> -              If any IPv6 address is given, the host is also allowed to
> receive
> -              packets to IPv6 multicast addresses (ff00::/8).
> -            </p>
> -
> -            <p>
> -              If any IPv6 address is given, the host is additionally
> restricted
> -              to sending IPv6 Neighbor Discovery Solicitation or
> Advertisement
> -              packets with the specified source address or, for
> solicitations,
> -              the unspecified address.
> -            </p>
> -          </li>
> -        </ul>
> -
> -        <p>
> -          If an element includes an IPv4 address, but no IPv6 addresses,
> then
> -          IPv6 traffic is not allowed.  If an element includes an IPv6
> address,
> -          but no IPv4 address, then IPv4 and ARP traffic is not allowed.
> -        </p>
> -
> -        <p>
> -          This column uses the same lexical syntax as the <ref
> column="match"
> -          table="Pipeline" db="OVN_Southbound"/> column in the OVN
> Southbound
> -          database's <ref table="Pipeline" db="OVN_Southbound"/> table.
> Multiple
> -          addresses within an element may be space or comma separated.
> -        </p>
> -
> -        <p>
> -          This column is provided as a convenience to cloud management
> systems,
> -          but all of the features that it implements can be implemented
> as ACLs
> -          using the <ref table="ACL"/> table.
> -        </p>
> -
> -        <p>
> -          Examples:
> -        </p>
> -
> -        <dl>
> -          <dt><code>80:fa:5b:06:72:b7</code></dt>
> -          <dd>
> -            The host may send traffic from and receive traffic to the
> specified
> -            MAC address, and to receive traffic to Ethernet multicast and
> -            broadcast addresses, but not otherwise.  The host may not
> send ARP or
> -            IPv6 Neighbor Discovery packets with inner source Ethernet
> addresses
> -            other than the one specified.
> -          </dd>
> -
> -          <dt><code>80:fa:5b:06:72:b7 192.168.1.10/24</code></dt>
> -          <dd>
> -            This adds further restrictions to the first example.  The
> host may
> -            send IPv4 packets from or receive IPv4 packets to only
> 192.168.1.10,
> -            except that it may also receive IPv4 packets to 192.168.1.255
> (based
> -            on the subnet mask), 255.255.255.255, and any address in
> 224.0.0.0/4.
> -            The host may not send ARPs with a source Ethernet address
> other than
> -            80:fa:5b:06:72:b7 or source IPv4 address other than
> 192.168.1.10.
> -            The host may not send or receive any IPv6 (including IPv6
> Neighbor
> -            Discovery) traffic.
> -          </dd>
> -
> -          <dt><code>"80:fa:5b:12:42:ba", "80:fa:5b:06:72:b7
> 192.168.1.10/24"</code></dt>
> -          <dd>
> -            The host may send traffic from and receive traffic to the
> -            specified MAC addresses, and
> -            to receive traffic to Ethernet multicast and broadcast
> addresses,
> -            but not otherwise.   With MAC 80:fa:5b:12:42:ba, the host may
> -            send traffic from and receive traffic to any L3 address.
> -            With MAC 80:fa:5b:06:72:b7, the host may send IPv4 packets
> from or
> -            receive IPv4 packets to only 192.168.1.10, except that it may
> also
> -            receive IPv4 packets to 192.168.1.255 (based on the subnet
> mask),
> -            255.255.255.255, and any address in 224.0.0.0/4.  The host
> may not
> -            send or receive any IPv6 (including IPv6 Neighbor Discovery)
> traffic.
> -          </dd>
> -        </dl>
> -      </column>
> -    </group>
> -
> -    <group title="DHCP">
> -      <column name="dhcpv4_options">
> -        This column defines the DHCPv4 Options to be included by the
> -        <code>ovn-controller</code> when it replies to the DHCPv4
> requests.
> -        Please see the <ref table="DHCP_Options"/> table.
> -      </column>
> -
> -      <column name="dhcpv6_options">
> -        This column defines the DHCPv6 Options to be included by the
> -        <code>ovn-controller</code> when it replies to the DHCPv6
> requests.
> -        Please see the <ref table="DHCP_Options"/> table.
> -      </column>
> -    </group>
> -
> -    <column name="ha_chassis_group">
> -      References a row in the OVN Northbound database's
> -      <ref table="HA_Chassis_Group" db="OVN_Northbound"/> table.
> -      It indicates the HA chassis group to use if the
> -      <ref column="type"/> is set to <code>external</code>.
> -      If <ref column="type"/> is not <code>external</code>, this
> -      column is ignored.
> -    </column>
> -
> -    <group title="Naming">
> -      <column name="external_ids" key="neutron:port_name">
> -        <p>
> -          This column gives an optional human-friendly name for the
> port.  This
> -          name has no special meaning or purpose other than to provide
> -          convenience for human interaction with the northbound database.
> -        </p>
> -
> -        <p>
> -          Neutron copies this from its own port object's name.  (Neutron
> ports
> -          do are not assigned human-friendly names by default, so it will
> often
> -          be empty.)
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        <p>
> -          See <em>External IDs</em> at the beginning of this document.
> -        </p>
> -
> -        <p>
> -          The <code>ovn-northd</code> program copies all these pairs into
> the
> -          <ref column="external_ids"/> column of the
> -          <ref table="Port_Binding"/> table in <ref db="OVN_Southbound"/>
> -          database.
> -        </p>
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Address_Set" title="Address Sets">
> -    <p>
> -      Each row in this table represents a named set of addresses.
> -      An address set may contain Ethernet, IPv4, or IPv6 addresses
> -      with optional bitwise or CIDR masks.
> -      Address set may ultimately be used in ACLs to compare against
> -      fields such as <code>ip4.src</code> or <code>ip6.src</code>.
> -      A single address set must contain addresses of the
> -      same type. As an example, the following would create an address set
> -      with three IP addresses:
> -    </p>
> -
> -    <pre>
> -      ovn-nbctl create Address_Set name=set1 addresses='10.0.0.1 10.0.0.2
> 10.0.0.3'
> -    </pre>
> -
> -    <p>
> -      Address sets may be used in the <ref column="match" table="ACL"/>
> column
> -      of the <ref table="ACL"/> table.  For syntax information, see the
> details
> -      of the expression language used for the <ref column="match"
> -      table="Logical_Flow" db="OVN_Southbound"/> column in the <ref
> -      table="Logical_Flow" db="OVN_Southbound"/> table of the <ref
> -      db="OVN_Southbound"/> database.
> -    </p>
> -
> -    <column name="name">
> -      A name for the address set.  Names are ASCII and must match
> -      <code>[a-zA-Z_.][a-zA-Z_.0-9]*</code>.
> -    </column>
> -
> -    <column name="addresses">
> -      The set of addresses in string form.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Port_Group" title="Port Groups">
> -    <p>
> -      Each row in this table represents a named group of logical switch
> ports.
> -    </p>
> -
> -    <p>
> -      Port groups may be used in the <ref column="match" table="ACL"/>
> column
> -      of the <ref table="ACL"/> table.  For syntax information, see the
> details
> -      of the expression language used for the <ref column="match"
> -      table="Logical_Flow" db="OVN_Southbound"/> column in the <ref
> -      table="Logical_Flow" db="OVN_Southbound"/> table of the <ref
> -      db="OVN_Southbound"/> database.
> -    </p>
> -
> -    <p>
> -      For each port group, there are two address sets generated to the
> -      <ref table="Address_Set" db="OVN_Southbound"/> table of the
> -      <ref db="OVN_Southbound"/> database, containing the IP addresses
> -      of the group of ports, one for IPv4, and the other for IPv6, with
> -      <ref column="name" table="Address_Set" db="OVN_Southbound"/> being
> -      the <ref column="name" table="Port_Group" db="OVN_Northbound"/>
> -      of the <ref table="Port_Group" db="OVN_Northbound"/> followed by
> -      a suffix <code>_ip4</code> for IPv4 and <code>_ip6</code> for IPv6.
> -      The generated address sets can be used in the same way as regular
> -      address sets in the <ref column="match" table="ACL"/> column
> -      of the <ref table="ACL"/> table. For syntax information, see the
> details
> -      of the expression language used for the <ref column="match"
> -      table="Logical_Flow" db="OVN_Southbound"/> column in the <ref
> -      table="Logical_Flow" db="OVN_Southbound"/> table of the <ref
> -      db="OVN_Southbound"/> database.
> -    </p>
> -
> -    <column name="name">
> -      A name for the port group.  Names are ASCII and must match
> -      <code>[a-zA-Z_.][a-zA-Z_.0-9]*</code>.
> -    </column>
> -
> -    <column name="ports">
> -      The logical switch ports belonging to the group in uuids.
> -    </column>
> -
> -    <column name="acls">
> -      Access control rules that apply to the port group. Applying an ACL
> -      to a port group has the same effect as applying the ACL to all
> logical
> -      lswitches that the ports of the port group belong to.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Load_Balancer" title="load balancer">
> -    <p>
> -      Each row represents one load balancer.
> -    </p>
> -
> -    <column name="name">
> -      A name for the load balancer.  This name has no special meaning or
> -      purpose other than to provide convenience for human interaction with
> -      the ovn-nb database.
> -    </column>
> -
> -    <column name="vips">
> -      <p>
> -        A map of virtual IP addresses (and an optional port number with
> -        <code>:</code> as a separator) associated with this load balancer
> and
> -        their corresponding endpoint IP addresses (and optional port
> numbers
> -        with <code>:</code> as separators) separated by commas.  If
> -        the destination IP address (and port number) of a packet leaving a
> -        container or a VM matches the virtual IP address (and port number)
> -        provided here as a key, then OVN will statefully replace the
> -        destination IP address by one of the provided IP address (and port
> -        number) in this map as a value.  IPv4 and IPv6 addresses are
> supported
> -        for load balancing; however a VIP of one address family may not be
> -        mapped to a destination IP address of a different family.  If
> -        specifying an IPv6 address with a port, the address portion must
> be
> -        enclosed in square brackets.  Examples for keys are "192.168.1.4"
> and
> -        "[fd0f::1]:8800".  Examples for value are "10.0.0.1, 10.0.0.2" and
> -        "20.0.0.10:8800, 20.0.0.11:8800".
> -      </p>
> -      <p>
> -        When the <code>Load_Balancer</code> is added to the
> -        <code>logical_switch</code>, the VIP has to be in a different
> subnet
> -        than the one used for the <code>logical_switch</code>.  Since VIP
> is
> -        in a different subnet, you should connect your logical switch to
> -        either a OVN logical router or a real router (this is because the
> -        client can now send a packet with VIP as the destination IP
> address
> -        and router's mac address as the destination MAC address).
> -      </p>
> -    </column>
> -
> -    <column name="protocol">
> -      <p>
> -        Valid protocols are <code>tcp</code> or <code>udp</code>.  This
> column
> -        is useful when a port number is provided as part of the
> -        <code>vips</code> column.  If this column is empty and a port
> number
> -        is provided as part of <code>vips</code> column, OVN assumes the
> -        protocol to be <code>tcp</code>.
> -      </p>
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="ACL" title="Access Control List (ACL) rule">
> -    <p>
> -      Each row in this table represents one ACL rule for a logical switch
> -      or a port group that points to it through its <ref column="acls"/>
> -      column.  The <ref column="action"/> column for the
> -      highest-<ref column="priority"/> matching row in this table
> determines a
> -      packet's treatment.  If no row matches, packets are allowed by
> default.
> -      (Default-deny treatment is possible: add a rule with
> -      <ref column="priority"/> 0, <code>1</code> as <ref column="match"/>,
> -      and <code>deny</code> as <ref column="action"/>.)
> -    </p>
> -
> -    <column name="priority">
> -      <p>
> -        The ACL rule's priority.  Rules with numerically higher priority
> -        take precedence over those with lower.  If two ACL rules with
> -        the same priority both match, then the one actually applied to a
> -        packet is undefined.
> -      </p>
> -
> -      <p>
> -        Return traffic from an <code>allow-related</code> flow is always
> -        allowed and cannot be changed through an ACL.
> -      </p>
> -    </column>
> -
> -    <column name="direction">
> -      <p>Direction of the traffic to which this rule should apply:</p>
> -      <ul>
> -        <li>
> -          <code>from-lport</code>: Used to implement filters on traffic
> -          arriving from a logical port.  These rules are applied to the
> -          logical switch's ingress pipeline.
> -        </li>
> -        <li>
> -          <code>to-lport</code>: Used to implement filters on traffic
> -          forwarded to a logical port.  These rules are applied to the
> -          logical switch's egress pipeline.
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <column name="match">
> -      <p>
> -        The packets that the ACL should match, in the same expression
> -        language used for the <ref column="match" table="Logical_Flow"
> -        db="OVN_Southbound"/> column in the OVN Southbound database's
> -        <ref table="Logical_Flow" db="OVN_Southbound"/> table.  The
> -        <code>outport</code> logical port is only available in the
> -        <code>to-lport</code> direction (the <code>inport</code> is
> -        available in both directions).
> -      </p>
> -
> -      <p>
> -        By default all traffic is allowed.  When writing a more
> -        restrictive policy, it is important to remember to allow flows
> -        such as ARP and IPv6 neighbor discovery packets.
> -      </p>
> -
> -      <p>
> -        Note that you can not create an ACL matching on a port with
> -        type=router or type=localnet.
> -      </p>
> -    </column>
> -
> -    <column name="action">
> -      <p>The action to take when the ACL rule matches:</p>
> -      <ul>
> -        <li>
> -          <code>allow</code>: Forward the packet.
> -        </li>
> -
> -        <li>
> -          <code>allow-related</code>: Forward the packet and related
> traffic
> -          (e.g. inbound replies to an outbound connection).
> -        </li>
> -
> -        <li>
> -          <code>drop</code>: Silently drop the packet.
> -        </li>
> -
> -        <li>
> -          <code>reject</code>: Drop the packet, replying with a RST for
> TCP or
> -          ICMPv4/ICMPv6 unreachable message for other IPv4/IPv6-based
> -          protocols.
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <group title="Logging">
> -      <p>
> -        These columns control whether and how OVN logs packets that match
> an
> -        ACL.
> -      </p>
> -
> -      <column name="log">
> -        <p>
> -          If set to <code>true</code>, packets that match the ACL will
> trigger
> -          a log message on the transport node or nodes that perform ACL
> -          processing.  Logging may be combined with any <ref
> column="action"/>.
> -        </p>
> -
> -        <p>
> -          If set to <code>false</code>, the remaining columns in this
> group
> -          have no significance.
> -        </p>
> -      </column>
> -
> -      <column name="name">
> -        <p>
> -          This name, if it is provided, is included in log records.  It
> -          provides the administrator and the cloud management system a
> way to
> -          associate a log record with a particular ACL.
> -        </p>
> -      </column>
> -
> -      <column name="severity">
> -        <p>
> -          The severity of the ACL.  The severity levels match those of
> syslog,
> -          in decreasing level of severity: <code>alert</code>,
> -          <code>warning</code>, <code>notice</code>, <code>info</code>, or
> -          <code>debug</code>.  When the column is empty, the default is
> -          <code>info</code>.
> -        </p>
> -      </column>
> -
> -      <column name="meter">
> -        <p>
> -            The name of a meter to rate-limit log messages for the ACL.
> -            The string must match the <ref column="name" table="meter"/>
> -            column of a row in the <ref table="Meter"/> table.  By
> -            default, log messages are not rate-limited.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Logical_Router" title="L3 logical router">
> -    <p>
> -      Each row represents one L3 logical router.
> -    </p>
> -
> -    <column name="ports">
> -      The router's ports.
> -    </column>
> -
> -    <column name="static_routes">
> -      Zero or more static routes for the router.
> -    </column>
> -
> -    <column name="policies">
> -      Zero or more routing policies for the router.
> -    </column>
> -
> -    <column name="enabled">
> -      This column is used to administratively set router state.  If this
> column
> -      is empty or is set to <code>true</code>, the router is enabled.  If
> this
> -      column is set to <code>false</code>, the router is disabled.  A
> disabled
> -      router has all ingress and egress traffic dropped.
> -    </column>
> -
> -    <column name="nat">
> -      One or more NAT rules for the router.  NAT rules only work on
> -      Gateway routers, and on distributed routers with one logical router
> -      port with a <code>redirect-chassis</code> specified.
> -    </column>
> -
> -    <column name="load_balancer">
> -      Load balance a virtual ip address to a set of logical port ip
> -      addresses.  Load balancer rules only work on the Gateway routers.
> -    </column>
> -
> -    <group title="Naming">
> -      <p>
> -        These columns provide names for the logical router.  From OVN's
> -        perspective, these names have no special meaning or purpose other
> than
> -        to provide convenience for human interaction with the northbound
> -        database.  There is no requirement for the name to be unique.
> (For a
> -        unique identifier for a logical router, use its row UUID.)
> -      </p>
> -
> -      <p>
> -        (Originally, <ref column="name"/> was intended to serve the
> purpose of
> -        a human-friendly name, but the Neutron integration used it to
> uniquely
> -        identify its own router object, in the format
> -        <code>neutron-<var>uuid</var></code>.  Later on, Neutron started
> -        propagating the friendly name of a router as <ref
> column="external_ids"
> -        key="neutron:router_name"/>.  Perhaps this can be cleaned up
> someday.)
> -      </p>
> -
> -      <column name="name">
> -        A name for the logical router.
> -      </column>
> -
> -      <column name="external_ids" key="neutron:router_name">
> -        Another name for the logical router.
> -      </column>
> -    </group>
> -
> -    <group title="Options">
> -      <p>
> -        Additional options for the logical router.
> -      </p>
> -
> -      <column name="options" key="chassis">
> -        <p>
> -          If set, indicates that the logical router in question is a
> Gateway
> -          router (which is centralized) and resides in the set chassis.
> The
> -          same value is also used by <code>ovn-controller</code> to
> -          uniquely identify the chassis in the OVN deployment and
> -          comes from <code>external_ids:system-id</code> in the
> -          <code>Open_vSwitch</code> table of Open_vSwitch database.
> -        </p>
> -
> -        <p>
> -          The Gateway router can only be connected to a distributed router
> -          via a switch if SNAT and DNAT are to be configured in the
> Gateway
> -          router.
> -        </p>
> -      </column>
> -      <column name="options" key="dnat_force_snat_ip">
> -        <p>
> -          If set, indicates the IP address to use to force SNAT a packet
> -          that has already been DNATed in the gateway router.  When
> multiple
> -          gateway routers are configured, a packet can potentially enter
> any
> -          of the gateway router, get DNATted and eventually reach the
> logical
> -          switch port.  For the return traffic to go back to the same
> gateway
> -          router (for unDNATing), the packet needs a SNAT in the first
> place.
> -          This can be achieved by setting the above option with a gateway
> -          specific IP address.
> -        </p>
> -      </column>
> -      <column name="options" key="lb_force_snat_ip">
> -        <p>
> -          If set, indicates the IP address to use to force SNAT a packet
> -          that has already been load-balanced in the gateway router.  When
> -          multiple gateway routers are configured, a packet can
> potentially
> -          enter any of the gateway routers, get DNATted as part of the
> load-
> -          balancing and eventually reach the logical switch port.
> -          For the return traffic to go back to the same gateway router
> (for
> -          unDNATing), the packet needs a SNAT in the first place.  This
> can be
> -          achieved by setting the above option with a gateway specific IP
> -          address.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="QoS" title="QoS rule">
> -    <p>
> -      Each row in this table represents one QoS rule for a logical switch
> -      that points to it through its <ref column="qos_rules"/> column.
> -      Two types of QoS are supported: DSCP marking and metering.  A
> -      <ref column="match"/> with the highest-<ref column="priority"/>
> -      will have QoS applied to it.  If the <ref column="action"/> column
> is
> -      specified, then matching packets will have DSCP marking applied.
> -      If the <ref column="bandwdith"/> column is specified, then matching
> -      packets will have metering applied.  <ref column="action"/> and
> -      <ref column="bandwdith"/> are not exclusive, so both marking and
> -      metering by defined for the same QoS entry. If no row matches,
> -      packets will not have any QoS applied.
> -    </p>
> -
> -    <column name="priority">
> -      <p>
> -        The QoS rule's priority.  Rules with numerically higher priority
> -        take precedence over those with lower.  If two QoS rules with
> -        the same priority both match, then the one actually applied to a
> -        packet is undefined.
> -      </p>
> -    </column>
> -
> -    <column name="direction">
> -      <p>
> -        The value of this field is similar to <ref colun="direction"
> -        table="ACL" db="OVN_Northbound"/> column in the OVN Northbound
> -        database's <ref table="ACL" db="OVN_Northbound"/> table.
> -      </p>
> -    </column>
> -
> -    <column name="match">
> -      <p>
> -        The packets that the QoS rules should match, in the same
> expression
> -        language used for the <ref column="match" table="Logical_Flow"
> -        db="OVN_Southbound"/> column in the OVN Southbound database's
> -        <ref table="Logical_Flow" db="OVN_Southbound"/> table.  The
> -        <code>outport</code> logical port is only available in the
> -        <code>to-lport</code> direction (the <code>inport</code> is
> -        available in both directions).
> -      </p>
> -    </column>
> -
> -    <column name="action">
> -      <p>When specified, matching flows will have DSCP marking
> applied.</p>
> -      <ul>
> -        <li>
> -          <code>dscp</code>: The value of this action should be in the
> -          range of 0 to 63 (inclusive).
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <column name="bandwidth">
> -      <p>
> -         When specified, matching packets will have bandwidth metering
> -         applied.  Traffic over the limit will be dropped.
> -      </p>
> -      <ul>
> -        <li>
> -          <code>rate</code>: The value of rate limit in kbps.
> -        </li>
> -        <li>
> -          <code>burst</code>: The value of burst rate limit in kilobits.
> -          This is optional and needs to specify the <code>rate</code>.
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <column name="external_ids">
> -      See <em>External IDs</em> at the beginning of this document.
> -    </column>
> -  </table>
> -
> -  <table name="Meter" title="Meter entry">
> -    <p>
> -      Each row in this table represents a meter that can be used for QoS
> or
> -      rate-limiting.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        A name for this meter.
> -      </p>
> -
> -      <p>
> -        Names that begin with "__" (two underscores) are reserved for
> -        OVN internal use and should not be added manually.
> -      </p>
> -    </column>
> -
> -    <column name="unit">
> -      <p>
> -        The unit for <ref column="rate" table="Meter_Band"/> and
> -        <ref column="burst_rate" table="Meter_Band"/> parameters in
> -        the <ref column="bands"/> entry.  <code>kbps</code> specifies
> -        kilobits per second, and <code>pktps</code> specifies packets
> -        per second.
> -      </p>
> -    </column>
> -
> -    <column name="bands">
> -      <p>
> -        The bands associated with this meter.  Each band specifies a
> -        rate above which the band is to take the action
> -        <code>action</code>.  If multiple bands' rates are exceeded,
> -        then the band with the highest rate among the exceeded bands is
> -        selected.
> -      </p>
> -    </column>
> -
> -    <column name="external_ids">
> -      See <em>External IDs</em> at the beginning of this document.
> -    </column>
> -  </table>
> -
> -  <table name="Meter_Band" title="Band for meter entries">
> -    <p>
> -      Each row in this table represents a meter band which specifies the
> -      rate above which the configured action should be applied.  These
> bands
> -      are referenced by the <ref column="bands" table="Meter"/> column in
> -      the <ref table="Meter"/> table.
> -    </p>
> -
> -    <column name="action">
> -      <p>
> -        The action to execute when this band matches.  The only supported
> -        action is <code>drop</code>.
> -      </p>
> -    </column>
> -
> -    <column name="rate">
> -      <p>
> -        The rate limit for this band, in kilobits per second or bits per
> -        second, depending on whether the parent <ref table="Meter"/>
> -        entry's <ref column="unit" table="Meter"/> column specified
> -        <code>kbps</code> or <code>pktps</code>.
> -      </p>
> -    </column>
> -
> -    <column name="burst_size">
> -      <p>
> -        The maximum burst allowed for the band in kilobits or packets,
> -        depending on whether <code>kbps</code> or <code>pktps</code> was
> -        selected in the parent <ref table="Meter"/> entry's
> -        <ref column="unit" table="Meter"/> column.  If the size is zero,
> -        the switch is free to select some reasonable value depending on
> -        its configuration.
> -      </p>
> -    </column>
> -
> -    <column name="external_ids">
> -      See <em>External IDs</em> at the beginning of this document.
> -    </column>
> -  </table>
> -
> -  <table name="Logical_Router_Port" title="L3 logical router port">
> -    <p>
> -      A port within an L3 logical router.
> -    </p>
> -
> -    <p>
> -      Exactly one <ref table="Logical_Router"/> row must reference a given
> -      logical router port.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        A name for the logical router port.
> -      </p>
> -
> -      <p>
> -        In addition to provide convenience for human interaction with the
> -        northbound database, this column is used as reference by its
> patch port
> -        in <ref table="Logical_Switch_Port"/> or another logical router
> port in
> -        <ref table="Logical_Router_Port"/>.
> -      </p>
> -    </column>
> -
> -    <column name="gateway_chassis">
> -      <p>
> -        This column is ignored if the column
> -        <ref column="ha_chassis_group" table="Logical_Router_Port"/>.
> -        is set.
> -      </p>
> -
> -      <p>
> -        If set, this indicates that this logical router port represents
> -        a distributed gateway port that connects this router to a logical
> -        switch with a localnet port.  There may be at most one such
> -        logical router port on each logical router.
> -      </p>
> -
> -      <p>
> -        Several <ref table="Gateway_Chassis"/> can be referenced for a
> given
> -        logical router port.  A single <ref table="Gateway_Chassis"/> is
> -        functionally equivalent to setting
> -        <ref column="options" key="redirect-chassis"/>.  Refer to the
> -        description of <ref column="options" key="redirect-chassis"/>
> -        for additional details on gateway handling.
> -      </p>
> -
> -      <p>
> -        Defining more than one <ref table="Gateway_Chassis"/> will enable
> -        gateway high availability.  Only one gateway will be active at a
> -        time.  OVN chassis will use BFD to monitor connectivity to a
> -        gateway.  If connectivity to the active gateway is interrupted,
> -        another gateway will become active.
> -        The <ref column="priority" table="Gateway_Chassis"/> column
> -        specifies the order that gateways will be chosen by OVN.
> -      </p>
> -    </column>
> -
> -    <column name="ha_chassis_group">
> -      <p>
> -        If set, this indicates that this logical router port represents
> -        a distributed gateway port that connects this router to a logical
> -        switch with a localnet port.  There may be at most one such
> -        logical router port on each logical router. The HA chassis which
> -        are part of the HA chassis group will provide the gateway high
> -        availability. Please see the <ref table="HA_Chassis_Group"/> for
> -        more details.
> -      </p>
> -
> -      <p>
> -        When this column is set, the column
> -        <ref column="gateway_chassis" table="Logical_Router_Port"/> will
> -        be ignored.
> -      </p>
> -    </column>
> -
> -    <column name="networks">
> -      <p>
> -        The IP addresses and netmasks of the router.  For example,
> -        <code>192.168.0.1/24</code> indicates that the router's IP
> -        address is 192.168.0.1 and that packets destined to
> -        192.168.0.<var>x</var> should be routed to this port.
> -      </p>
> -
> -      <p>
> -        A logical router port always adds a link-local IPv6 address
> -        (fe80::/64) automatically generated from the interface's MAC
> -        address using the modified EUI-64 format.
> -      </p>
> -    </column>
> -
> -    <column name="mac">
> -      The Ethernet address that belongs to this router port.
> -    </column>
> -
> -    <column name="enabled">
> -      This column is used to administratively set port state.  If this
> column
> -      is empty or is set to <code>true</code>, the port is enabled.  If
> this
> -      column is set to <code>false</code>, the port is disabled.  A
> disabled
> -      port has all ingress and egress traffic dropped.
> -    </column>
> -
> -    <group title="ipv6_ra_configs">
> -      <p>
> -        This column defines the IPv6 ND RA address mode and ND MTU Option
> to be
> -        included by <code>ovn-controller</code> when it replies to the
> IPv6
> -        Router solicitation requests.
> -      </p>
> -
> -      <column name="ipv6_ra_configs" key="address_mode">
> -        The address mode to be used for IPv6 address configuration.
> -        The supported values are:
> -        <ul>
> -          <li>
> -            <code>slaac</code>: Address configuration using Router
> -            Advertisement (RA) packet. The IPv6 prefixes defined in the
> -            <ref table="Logical_Router_Port"/> table's
> -            <ref table="Logical_Router_Port" column="networks"/> column
> will
> -            be included in the RA's ICMPv6 option - Prefix information.
> -          </li>
> -
> -          <li>
> -            <code>dhcpv6_stateful</code>: Address configuration using
> DHCPv6.
> -          </li>
> -
> -          <li>
> -            <code>dhcpv6_stateless</code>: Address configuration using
> Router
> -            Advertisement (RA) packet. Other IPv6 options are provided by
> -            DHCPv6.
> -          </li>
> -        </ul>
> -      </column>
> -
> -      <column name="ipv6_ra_configs" key="mtu">
> -        The recommended MTU for the link. Default is 0, which means no MTU
> -        Option will be included in RA packet replied by ovn-controller.
> -        Per RFC 2460, the mtu value is recommended no less than 1280, so
> -        any mtu value less than 1280 will be considered as no MTU Option.
> -      </column>
> -
> -      <column name="ipv6_ra_configs" key="send_periodic">
> -        If set to true, then this router interface will send router
> -        advertisements periodically.  The default is false.
> -      </column>
> -
> -      <column name="ipv6_ra_configs" key="max_interval">
> -        The maximum number of seconds to wait between sending periodic
> router
> -        advertisements.  This option has no effect if <ref
> -        column="ipv6_ra_configs" key="send_periodic"/> is false.  The
> default
> -        is 600.
> -      </column>
> -
> -      <column name="ipv6_ra_configs" key="min_interval">
> -        The minimum number of seconds to wait between sending periodic
> router
> -        advertisements.  This option has no effect if <ref
> -        column="ipv6_ra_configs" key="send_periodic"/> is false.  The
> default
> -        is one-third of <ref column="ipv6_ra_configs"
> key="max_interval"/>,
> -        i.e. 200 seconds if that key is unset.
> -      </column>
> -    </group>
> -
> -    <group title="Options">
> -      <p>
> -        Additional options for the logical router port.
> -      </p>
> -
> -      <column name="options" key="redirect-chassis">
> -        <p>
> -          If set, this indicates that this logical router port represents
> -          a distributed gateway port that connects this router to a
> logical
> -          switch with a localnet port.  There may be at most one such
> -          logical router port on each logical router.
> -        </p>
> -
> -        <p>
> -          Even when a <code>redirect-chassis</code> is specified, the
> -          logical router port still effectively resides on each chassis.
> -          However, due to the implications of the use of L2 learning in
> the
> -          physical network, as well as the need to support advanced
> features
> -          such as one-to-many NAT (aka IP masquerading), a subset of the
> -          logical router processing is handled in a centralized manner on
> -          the specified <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          When this option is specified, the peer logical switch port's
> -          <ref column="addresses" table="Logical_Switch_Port"/> must be
> -          set to <code>router</code>.  With this setting, the <ref
> -          column="external_mac" table="NAT"/>s specified in NAT rules are
> -          automatically programmed in the peer logical switch's
> -          destination lookup on the chassis where the <ref
> -          column="logical_port" table="NAT"/> resides.  In addition, the
> -          logical router's MAC address is automatically programmed in the
> -          peer logical switch's destination lookup flow on the
> -          <code>redirect-chassis</code>.
> -        </p>
> -
> -        <p>
> -          When this option is specified and it is desired to generate
> -          gratuitous ARPs for NAT addresses, then the peer logical switch
> -          port's <ref column="options" key="nat-addresses"
> -          table="Logical_Switch_Port"/> should be set to
> -          <code>router</code>.
> -        </p>
> -
> -        <p>
> -          While <ref column="options" key="redirect-chassis"/> is still
> -          supported for backwards compatibility, it is now preferred to
> -          specify one or more <ref column="gateway_chassis"/> instead.
> -          It is functionally equivalent, but allows you to specify
> multiple
> -          chassis to enable high availability.
> -        </p>
> -      </column>
> -
> -      <column name="options" key="reside-on-redirect-chassis">
> -        <p>
> -          Generally routing is distributed in <code>OVN</code>. The packet
> -          from a logical port which needs to be routed hits the router
> pipeline
> -          in the source chassis. For the East-West traffic, the packet is
> -          sent directly to the destination chassis. For the outside
> traffic
> -          the packet is sent to the gateway chassis.
> -        </p>
> -
> -        <p>
> -          When this option is set, <code>OVN</code> considers this only if
> -        </p>
> -
> -        <ul>
> -          <li>
> -            The logical router to which this logical router port belongs
> to
> -            has a distributed gateway port.
> -          </li>
> -
> -          <li>
> -            The peer's logical switch has a localnet port (representing
> -            a VLAN tagged network)
> -          </li>
> -        </ul>
> -
> -        <p>
> -          When this option is set to <code>true</code>, then the packet
> -          which needs to be routed hits the router pipeline in the chassis
> -          hosting the distributed gateway router port. The source chassis
> -          pushes out this traffic via the localnet port. With this the
> -          East-West traffic is no more distributed and will always go
> through
> -          the gateway chassis.
> -        </p>
> -
> -        <p>
> -          Without this option set, for any traffic destined to outside
> from a
> -          logical port which belongs to a logical switch with localnet
> port,
> -          the source chassis will send the traffic to the gateway chassis
> via
> -          the tunnel port instead of the localnet port and this could
> cause MTU
> -          issues.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Attachment">
> -      <p>
> -        A given router port serves one of two purposes:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          To attach a logical switch to a logical router.  A logical
> router
> -          port of this type is referenced by exactly one <ref
> -          table="Logical_Switch_Port"/> of type <code>router</code>.
> -          The value of <ref column="name"/> is set as
> -          <code>router-port</code> in column <ref column="options"/> of
> -          <ref table="Logical_Switch_Port"/>.  In this case <ref
> -          column="peer"/> column is empty.
> -        </li>
> -
> -        <li>
> -          To connect one logical router to another.  This requires a pair
> of
> -          logical router ports, each connected to a different router.
> Each
> -          router port in the pair specifies the other in its <ref
> -          column="peer"/> column.  No <ref table="Logical_Switch"/>
> refers to
> -          the router port.
> -        </li>
> -      </ul>
> -
> -      <column name="peer">
> -        <p>
> -          For a router port used to connect two logical routers, this
> -          identifies the other router port in the pair by <ref
> column="name"/>.
> -        </p>
> -
> -        <p>
> -          For a router port attached to a logical switch, this column is
> empty.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Logical_Router_Static_Route" title="Logical router static
> routes">
> -    <p>
> -      Each record represents a static route.
> -    </p>
> -
> -    <p>
> -      When multiple routes match a packet, the longest-prefix match is
> chosen.
> -      For a given prefix length, a <code>dst-ip</code> route is preferred
> over
> -      a <code>src-ip</code> route.
> -    </p>
> -
> -    <column name="ip_prefix">
> -      <p>
> -        IP prefix of this route (e.g. 192.168.100.0/24).
> -      </p>
> -    </column>
> -
> -    <column name="policy">
> -      <p>
> -        If it is specified, this setting describes the policy used to make
> -        routing decisions.  This setting must be one of the following
> strings:
> -      </p>
> -      <ul>
> -        <li>
> -          <code>src-ip</code>: This policy sends the packet to the
> -          <ref column="nexthop"/> when the packet's source IP address
> matches
> -          <ref column="ip_prefix"/>.
> -       </li>
> -        <li>
> -          <code>dst-ip</code>: This policy sends the packet to the
> -          <ref column="nexthop"/> when the packet's destination IP address
> -          matches <ref column="ip_prefix"/>.
> -        </li>
> -      </ul>
> -      <p>
> -        If not specified, the default is <code>dst-ip</code>.
> -     </p>
> -    </column>
> -
> -    <column name="nexthop">
> -      <p>
> -        Nexthop IP address for this route.  Nexthop IP address should be
> the IP
> -        address of a connected router port or the IP address of a logical
> port.
> -      </p>
> -    </column>
> -
> -    <column name="output_port">
> -      <p>
> -        The name of the <ref table="Logical_Router_Port"/> via which the
> packet
> -        needs to be sent out.  This is optional and when not specified,
> -        OVN will automatically figure this out based on the
> -        <ref column="nexthop"/>.  When this is specified and there are
> -        multiple IP addresses on the router port and none of them are in
> the
> -        same subnet of <ref column="nexthop"/>, OVN chooses the first IP
> -        address as the one via which the <ref column="nexthop"/> is
> reachable.
> -      </p>
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -
> -  </table>
> -
> -  <table name="Logical_Router_Policy" title="Logical router policies">
> -    <p>
> -      Each row in this table represents one routing policy for a logical
> router
> -      that points to it through its <ref column="policies"/> column.  The
> <ref
> -      column="action"/> column for the highest-<ref column="priority"/>
> -      matching row in this table determines a packet's treatment.  If no
> row
> -      matches, packets are allowed by default. (Default-deny treatment is
> -      possible: add a rule with <ref column="priority"/> 0,
> <code>1</code> as
> -      <ref column="match"/>, and <code>drop</code> as <ref
> column="action"/>.)
> -    </p>
> -
> -    <column name="priority">
> -      <p>
> -        The routing policy's priority.  Rules with numerically higher
> priority
> -        take precedence over those with lower. A rule is uniquely
> identified
> -        by the priority and match string.
> -      </p>
> -    </column>
> -
> -    <column name="match">
> -      <p>
> -        The packets that the routing policy should match,
> -        in the same expression language used for the
> -        <ref column="match" table="Logical_Flow" db="OVN_Southbound"/>
> -        column in the OVN Southbound database's
> -        <ref table="Logical_Flow" db="OVN_Southbound"/> table.
> -      </p>
> -
> -      <p>
> -        By default all traffic is allowed.  When writing a more
> -        restrictive policy, it is important to remember to allow flows
> -        such as ARP and IPv6 neighbor discovery packets.
> -      </p>
> -    </column>
> -
> -    <column name="action">
> -      <p>The action to take when the routing policy matches:</p>
> -      <ul>
> -        <li>
> -          <code>allow</code>: Forward the packet.
> -        </li>
> -
> -        <li>
> -          <code>drop</code>: Silently drop the packet.
> -        </li>
> -
> -        <li>
> -          <code>reroute</code>: Reroute packet to <ref column="nexthop"/>.
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <column name="nexthop">
> -      <p>
> -        Next-hop IP address for this route, which should be the IP
> -        address of a connected router port or the IP address of a logical
> port.
> -      </p>
> -    </column>
> -  </table>
> -
> -  <table name="NAT" title="NAT rules">
> -    <p>
> -      Each record represents a NAT rule.
> -    </p>
> -
> -    <column name="type">
> -      <p>Type of the NAT rule.</p>
> -      <ul>
> -        <li>
> -          When <ref column="type"/> is <code>dnat</code>, the externally
> -          visible IP address <ref column="external_ip"/> is DNATted to
> the IP
> -          address <ref column="logical_ip"/> in the logical space.
> -        </li>
> -        <li>
> -          When <ref column="type"/> is <code>snat</code>, IP packets
> -          with their source IP address that either matches the IP address
> -          in <ref column="logical_ip"/> or is in the network provided by
> -          <ref column="logical_ip"/> is SNATed into the IP address in
> -          <ref column="external_ip"/>.
> -        </li>
> -        <li>
> -          When <ref column="type"/> is <code>dnat_and_snat</code>, the
> -          externally visible IP address <ref column="external_ip"/> is
> -          DNATted to the IP address <ref column="logical_ip"/> in the
> -          logical space. In addition, IP packets with the source IP
> -          address that matches <ref column="logical_ip"/> is SNATed into
> -          the IP address in <ref column="external_ip"/>.
> -        </li>
> -      </ul>
> -    </column>
> -
> -    <column name="external_ip">
> -      An IPv4 address.
> -    </column>
> -
> -    <column name="external_mac">
> -      <p>
> -        A MAC address.
> -      </p>
> -
> -      <p>
> -        This is only used on the gateway port on distributed routers.
> -        This must be specified in order for the NAT rule to be
> -        processed in a distributed manner on all chassis.  If this is
> -        not specified for a NAT rule on a distributed router, then
> -        this NAT rule will be processed in a centralized manner on
> -        the gateway port instance on the <code>redirect-chassis</code>.
> -      </p>
> -
> -      <p>
> -        This MAC address must be unique on the logical switch that the
> -        gateway port is attached to.  If the MAC address used on the
> -        <ref column="logical_port"/> is globally unique, then that MAC
> -        address can be specified as this <ref column="external_mac"/>.
> -      </p>
> -    </column>
> -
> -    <column name="logical_ip">
> -      An IPv4 network (e.g 192.168.1.0/24) or an IPv4 address.
> -    </column>
> -
> -    <column name="logical_port">
> -      <p>
> -        The name of the logical port where the <ref column="logical_ip"/>
> -        resides.
> -      </p>
> -
> -      <p>
> -        This is only used on distributed routers.  This must be
> -        specified in order for the NAT rule to be processed in a
> -        distributed manner on all chassis.  If this is not specified
> -        for a NAT rule on a distributed router, then this NAT rule
> -        will be processed in a centralized manner on the gateway
> -        port instance on the <code>redirect-chassis</code>.
> -      </p>
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -
> -  </table>
> -
> -  <table name="DHCP_Options" title="DHCP options">
> -    <p>
> -      OVN implements native DHCPv4 support which caters to the common
> -      use case of providing an IPv4 address to a booting instance by
> -      providing stateless replies to DHCPv4 requests based on statically
> -      configured address mappings. To do this it allows a short list of
> -      DHCPv4 options to be configured and applied at each compute host
> -      running <code>ovn-controller</code>.
> -    </p>
> -
> -    <p>
> -      OVN also implements native DHCPv6 support which provides stateless
> -      replies to DHCPv6 requests.
> -    </p>
> -
> -    <column name="cidr">
> -      <p>
> -        The DHCPv4/DHCPv6 options will be included if the logical port
> has its
> -        IP address in this <ref column="cidr"/>.
> -      </p>
> -    </column>
> -
> -    <group title="DHCPv4 options">
> -      <p>
> -        The CMS should define the set of DHCPv4 options as key/value pairs
> -        in the <ref column="options"/> column of this table. For
> -        <code>ovn-controller</code> to include these DHCPv4 options, the
> -        <ref column="dhcpv4_options"/> of <ref
> table="Logical_Switch_Port"/>
> -        should refer to an entry in this table.
> -      </p>
> -
> -      <group title="Mandatory DHCPv4 options">
> -        <p>
> -          The following options must be defined.
> -        </p>
> -
> -        <column name="options" key="server_id">
> -          The IP address for the DHCP server to use.  This should be in
> the
> -          subnet of the offered IP.  This is also included in the DHCP
> offer as
> -          option 54, ``server identifier.''
> -        </column>
> -
> -        <column name="options" key="server_mac">
> -          The Ethernet address for the DHCP server to use.
> -        </column>
> -
> -        <column name="options" key="lease_time"
> -                type='{"type": "integer", "minInteger": 0, "maxInteger":
> 4294967295}'>
> -          <p>
> -            The offered lease time in seconds,
> -          </p>
> -
> -          <p>
> -            The DHCPv4 option code for this option is 51.
> -          </p>
> -        </column>
> -      </group>
> -
> -      <group title="IPv4 DHCP Options">
> -        <p>
> -          Below are the supported DHCPv4 options whose values are an IPv4
> -          address, e.g. <code>192.168.1.1</code>.  Some options accept
> multiple
> -          IPv4 addresses enclosed within curly braces, e.g.
> <code>{192.168.1.2,
> -          192.168.1.3}</code>. Please refer to RFC 2132 for more details
> on
> -          DHCPv4 options and their codes.
> -        </p>
> -
> -        <column name="options" key="router">
> -          <p>
> -            The IP address of a gateway for the client to use.  This
> should be
> -            in the subnet of the offered IP.  The DHCPv4 option code for
> this
> -            option is 3.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="netmask">
> -          <p>
> -            The DHCPv4 option code for this option is 1.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="dns_server">
> -          <p>
> -            The DHCPv4 option code for this option is 6.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="log_server">
> -          <p>
> -            The DHCPv4 option code for this option is 7.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="lpr_server">
> -          <p>
> -            The DHCPv4 option code for this option is 9.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="swap_server">
> -          <p>
> -            The DHCPv4 option code for this option is 16.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="policy_filter">
> -          <p>
> -            The DHCPv4 option code for this option is 21.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="router_solicitation">
> -          <p>
> -            The DHCPv4 option code for this option is 32.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="nis_server">
> -          <p>
> -            The DHCPv4 option code for this option is 41.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="ntp_server">
> -          <p>
> -            The DHCPv4 option code for this option is 42.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="tftp_server">
> -          <p>
> -            The DHCPv4 option code for this option is 66.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="classless_static_route">
> -          <p>
> -            The DHCPv4 option code for this option is 121.
> -          </p>
> -
> -          <p>
> -             This option can contain one or more static routes, each of
> which
> -             consists of a destination descriptor and the IP address of
> the
> -             router that should be used to reach that destination. Please
> see
> -             RFC 3442 for more details.
> -          </p>
> -
> -          <p>
> -            Example: <code>{30.0.0.0/24,10.0.0.10, 0.0.0.0/0,10.0.0.1
> }</code>
> -          </p>
> -        </column>
> -
> -        <column name="options" key="ms_classless_static_route">
> -          <p>
> -            The DHCPv4 option code for this option is 249. This option is
> -            similar to <code>classless_static_route</code> supported by
> -            Microsoft Windows DHCPv4 clients.
> -          </p>
> -        </column>
> -
> -      </group>
> -
> -      <group title="Boolean DHCP Options">
> -        <p>
> -          These options accept a Boolean value, expressed as
> <code>0</code> for
> -          false or <code>1</code> for true.
> -        </p>
> -
> -        <column name="options" key="ip_forward_enable"
> -                type='{"type": "string", "enum": ["set", ["0", "1"]]}'>
> -          <p>
> -            The DHCPv4 option code for this option is 19.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="router_discovery"
> -                type='{"type": "string", "enum": ["set", ["0", "1"]]}'>
> -          <p>
> -            The DHCPv4 option code for this option is 31.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="ethernet_encap"
> -                type='{"type": "string", "enum": ["set", ["0", "1"]]}'>
> -          <p>
> -            The DHCPv4 option code for this option is 36.
> -          </p>
> -        </column>
> -      </group>
> -
> -      <group title="Integer DHCP Options">
> -        <p>
> -          These options accept a nonnegative integer value.
> -        </p>
> -
> -        <column name="options" key="default_ttl"
> -                type='{"type": "integer", "minInteger": 0, "maxInteger":
> 255}'>
> -          The DHCPv4 option code for this option is 23.
> -        </column>
> -
> -        <column name="options" key="tcp_ttl"
> -                type='{"type": "integer", "minInteger": 0, "maxInteger":
> 255}'>
> -          The DHCPv4 option code for this option is 37.
> -        </column>
> -
> -        <column name="options" key="mtu"
> -                type='{"type": "integer", "minInteger": 68, "maxInteger":
> 65535}'>
> -          The DHCPv4 option code for this option is 26.
> -        </column>
> -
> -        <column name="options" key="T1"
> -                type='{"type": "integer", "minInteger": 68, "maxInteger":
> 4294967295}'>
> -          This specifies the time interval from address assignment until
> the
> -          client begins trying to renew its address.  The DHCPv4 option
> code
> -          for this option is 58.
> -        </column>
> -
> -        <column name="options" key="T2"
> -                type='{"type": "integer", "minInteger": 68, "maxInteger":
> 4294967295}'>
> -          This specifies the time interval from address assignment until
> the
> -          client begins trying to rebind its address.  The DHCPv4 option
> code
> -          for this option is 59.
> -        </column>
> -      </group>
> -
> -      <group title="String DHCP Options">
> -        <p>
> -          These options accept a string value.
> -        </p>
> -
> -        <column name="options" key="wpad">
> -          <p>
> -            The DHCPv4 option code for this option is 252. This option is
> used
> -            as part of web proxy auto discovery to provide a URL for a web
> -            proxy.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="bootfile_name">
> -          <p>
> -            The DHCPv4 option code for this option is 67. This option is
> used
> -            to identify a bootfile.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="path_prefix">
> -          <p>
> -            The DHCPv4 option code for this option is 210. In PXELINUX'
> -            case this option is used to set a common path prefix,
> -            instead of deriving it from the bootfile name.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="tftp_server_address">
> -          <p>
> -            The DHCPv4 option code for this option is 150. The option
> -            contains one or more IPv4 addresses that the client MAY
> -            use. This option is Cisco proprietary, the IEEE standard
> -            that matches with this requirement is option 66 (tftp_server).
> -          </p>
> -        </column>
> -
> -        <column name="options" key="domain_name">
> -          <p>
> -            The DHCPv4 option code for this option is 15. This option
> -            specifies the domain name that client should use when
> -            resolving hostnames via the Domain Name System.
> -          </p>
> -        </column>
> -      </group>
> -    </group>
> -
> -    <group title="DHCPv6 options">
> -      <p>
> -        OVN also implements native DHCPv6 support. The CMS should define
> -        the set of DHCPv6 options as key/value pairs. The define DHCPv6
> -        options will be included in the DHCPv6 response to the DHCPv6
> -        Solicit/Request/Confirm packet from the logical ports having the
> -        IPv6 addresses in the <ref column="cidr"/>.
> -      </p>
> -
> -      <group title="Mandatory DHCPv6 options">
> -        <p>
> -          The following options must be defined.
> -        </p>
> -
> -        <column name="options" key="server_id">
> -          <p>
> -            The Ethernet address for the DHCP server to use. This is also
> -            included in the DHCPv6 reply as option 2, ``Server
> Identifier''
> -            to carry a DUID identifying a server between a client and a
> server.
> -            <code>ovn-controller</code> defines DUID based on
> -            Link-layer Address [DUID-LL].
> -          </p>
> -        </column>
> -      </group>
> -
> -      <group title="IPv6 DHCPv6 options">
> -        <p>
> -          Below are the supported DHCPv6 options whose values are an IPv6
> -          address, e.g. <code>aef0::4</code>.  Some options accept
> multiple
> -          IPv6 addresses enclosed within curly braces, e.g.
> <code>{aef0::4,
> -          aef0::5}</code>. Please refer to RFC 3315 for more details on
> -          DHCPv6 options and their codes.
> -        </p>
> -
> -        <column name="options" key="dns_server">
> -          <p>
> -            The DHCPv6 option code for this option is 23. This option
> specifies
> -            the DNS servers that the VM should use.
> -          </p>
> -        </column>
> -      </group>
> -
> -      <group title="String DHCPv6 options">
> -        <p>
> -          These options accept string values.
> -        </p>
> -
> -        <column name="options" key="domain_search">
> -          <p>
> -            The DHCPv6 option code for this option is 24. This option
> specifies
> -            the domain search list the client should use to resolve
> hostnames
> -            with DNS.
> -          </p>
> -
> -          <p>
> -            Example: <code>"ovn.org"</code>.
> -          </p>
> -        </column>
> -
> -        <column name="options" key="dhcpv6_stateless">
> -          <p>
> -            This option specifies the OVN native DHCPv6 will work in
> stateless
> -            mode, which means OVN native DHCPv6 will not offer IPv6
> addresses
> -            for VM/VIF ports, but only reply other configurations, such as
> -            DNS and domain search list. When setting this option with
> string
> -            value "true", VM/VIF will configure IPv6 addresses by
> stateless
> -            way. Default value for this option is false.
> -          </p>
> -        </column>
> -      </group>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Connection" title="OVSDB client connections.">
> -    <p>
> -      Configuration for a database connection to an Open vSwitch database
> -      (OVSDB) client.
> -    </p>
> -
> -    <p>
> -      This table primarily configures the Open vSwitch database server
> -      (<code>ovsdb-server</code>).
> -    </p>
> -
> -    <p>
> -      The Open vSwitch database server can initiate and maintain active
> -      connections to remote clients.  It can also listen for database
> -      connections.
> -    </p>
> -
> -    <group title="Core Features">
> -      <column name="target">
> -        <p>Connection methods for clients.</p>
> -        <p>
> -          The following connection methods are currently supported:
> -        </p>
> -        <dl>
> -
> <dt><code>ssl:<var>host</var></code>[<code>:<var>port</var></code>]</dt>
> -          <dd>
> -            <p>
> -              The specified SSL <var>port</var> on the host at the given
> -              <var>host</var>, which can either be a DNS name (if built
> with
> -              unbound library) or an IP address. A valid SSL
> configuration must
> -              be provided when this form is used, this configuration can
> be
> -              specified via command-line options or the <ref
> table="SSL"/> table.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -            <p>
> -              SSL support is an optional feature that is not always
> -              built as part of Open vSwitch.
> -            </p>
> -          </dd>
> -
> -
> <dt><code>tcp:<var>host</var></code>[<code>:<var>port</var></code>]</dt>
> -          <dd>
> -            <p>
> -              The specified TCP <var>port</var> on the host at the given
> -              <var>host</var>, which can either be a DNS name (if built
> with
> -              unbound library) or an IP address.  If <var>host</var> is
> an IPv6
> -              address, wrap it in square brackets, e.g.
> <code>tcp:[::1]:6640</code>.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -          </dd>
> -
> <dt><code>pssl:</code>[<var>port</var>][<code>:<var>host</var></code>]</dt>
> -          <dd>
> -            <p>
> -              Listens for SSL connections on the specified TCP
> <var>port</var>.
> -              Specify 0 for <var>port</var> to have the kernel
> automatically
> -              choose an available port.  If <var>host</var>, which can
> either
> -              be a DNS name (if built with unbound library) or an IP
> address,
> -              is specified, then connections are restricted to the
> resolved or
> -              specified local IPaddress (either IPv4 or IPv6 address).  If
> -              <var>host</var> is an IPv6 address, wrap in square brackets,
> -              e.g. <code>pssl:6640:[::1]</code>.  If <var>host</var> is
> not
> -              specified then it listens only on IPv4 (but not IPv6)
> addresses.
> -              A valid SSL configuration must be provided when this form
> is used,
> -             this can be specified either via command-line options or the
> -             <ref table="SSL"/> table.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -            <p>
> -              SSL support is an optional feature that is not always built
> as
> -              part of Open vSwitch.
> -            </p>
> -          </dd>
> -
> <dt><code>ptcp:</code>[<var>port</var>][<code>:<var>host</var></code>]</dt>
> -          <dd>
> -            <p>
> -              Listens for connections on the specified TCP
> <var>port</var>.
> -              Specify 0 for <var>port</var> to have the kernel
> automatically
> -              choose an available port.  If <var>host</var>, which can
> either
> -              be a DNS name (if built with unbound library) or an IP
> address,
> -              is specified, then connections are restricted to the
> resolved or
> -              specified local IP address (either IPv4 or IPv6 address).
> If
> -              <var>host</var> is an IPv6 address, wrap it in square
> brackets,
> -              e.g. <code>ptcp:6640:[::1]</code>.  If <var>host</var> is
> not
> -              specified then it listens only on IPv4 addresses.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -          </dd>
> -        </dl>
> -        <p>When multiple clients are configured, the <ref
> column="target"/>
> -        values must be unique.  Duplicate <ref column="target"/> values
> yield
> -        unspecified results.</p>
> -      </column>
> -    </group>
> -
> -    <group title="Client Failure Detection and Handling">
> -      <column name="max_backoff">
> -        Maximum number of milliseconds to wait between connection
> attempts.
> -        Default is implementation-specific.
> -      </column>
> -
> -      <column name="inactivity_probe">
> -        Maximum number of milliseconds of idle time on connection to the
> client
> -        before sending an inactivity probe message.  If Open vSwitch does
> not
> -        communicate with the client for the specified number of seconds,
> it
> -        will send a probe.  If a response is not received for the same
> -        additional amount of time, Open vSwitch assumes the connection
> has been
> -        broken and attempts to reconnect.  Default is
> implementation-specific.
> -        A value of 0 disables inactivity probes.
> -      </column>
> -    </group>
> -
> -    <group title="Status">
> -      <p>
> -        Key-value pair of <ref column="is_connected"/> is always updated.
> -        Other key-value pairs in the status columns may be updated depends
> -        on the <ref column="target"/> type.
> -      </p>
> -
> -      <p>
> -        When <ref column="target"/> specifies a connection method that
> -        listens for inbound connections (e.g. <code>ptcp:</code> or
> -        <code>punix:</code>), both <ref column="n_connections"/> and
> -        <ref column="is_connected"/> may also be updated while the
> -        remaining key-value pairs are omitted.
> -      </p>
> -
> -      <p>
> -        On the other hand, when <ref column="target"/> specifies an
> -        outbound connection, all key-value pairs may be updated, except
> -        the above-mentioned two key-value pairs associated with inbound
> -        connection targets. They are omitted.
> -      </p>
> -
> -    <column name="is_connected">
> -        <code>true</code> if currently connected to this client,
> -        <code>false</code> otherwise.
> -      </column>
> -
> -      <column name="status" key="last_error">
> -        A human-readable description of the last error on the connection
> -        to the manager; i.e. <code>strerror(errno)</code>.  This key
> -        will exist only if an error has occurred.
> -      </column>
> -
> -      <column name="status" key="state"
> -              type='{"type": "string", "enum": ["set", ["VOID",
> "BACKOFF", "CONNECTING", "ACTIVE", "IDLE"]]}'>
> -        <p>
> -          The state of the connection to the manager:
> -        </p>
> -        <dl>
> -          <dt><code>VOID</code></dt>
> -          <dd>Connection is disabled.</dd>
> -
> -          <dt><code>BACKOFF</code></dt>
> -          <dd>Attempting to reconnect at an increasing period.</dd>
> -
> -          <dt><code>CONNECTING</code></dt>
> -          <dd>Attempting to connect.</dd>
> -
> -          <dt><code>ACTIVE</code></dt>
> -          <dd>Connected, remote host responsive.</dd>
> -
> -          <dt><code>IDLE</code></dt>
> -          <dd>Connection is idle.  Waiting for response to
> keep-alive.</dd>
> -        </dl>
> -        <p>
> -          These values may change in the future.  They are provided only
> for
> -          human consumption.
> -        </p>
> -      </column>
> -
> -      <column name="status" key="sec_since_connect"
> -              type='{"type": "integer", "minInteger": 0}'>
> -        The amount of time since this client last successfully connected
> -        to the database (in seconds). Value is empty if client has never
> -        successfully been connected.
> -      </column>
> -
> -      <column name="status" key="sec_since_disconnect"
> -              type='{"type": "integer", "minInteger": 0}'>
> -        The amount of time since this client last disconnected from the
> -        database (in seconds). Value is empty if client has never
> -        disconnected.
> -      </column>
> -
> -      <column name="status" key="locks_held">
> -        Space-separated list of the names of OVSDB locks that the
> connection
> -        holds.  Omitted if the connection does not hold any locks.
> -      </column>
> -
> -      <column name="status" key="locks_waiting">
> -        Space-separated list of the names of OVSDB locks that the
> connection is
> -        currently waiting to acquire.  Omitted if the connection is not
> waiting
> -        for any locks.
> -      </column>
> -
> -      <column name="status" key="locks_lost">
> -        Space-separated list of the names of OVSDB locks that the
> connection
> -        has had stolen by another OVSDB client.  Omitted if no locks have
> been
> -        stolen from this connection.
> -      </column>
> -
> -      <column name="status" key="n_connections"
> -              type='{"type": "integer", "minInteger": 2}'>
> -        When <ref column="target"/> specifies a connection method that
> -        listens for inbound connections (e.g. <code>ptcp:</code> or
> -        <code>pssl:</code>) and more than one connection is actually
> active,
> -        the value is the number of active connections.  Otherwise, this
> -        key-value pair is omitted.
> -      </column>
> -
> -      <column name="status" key="bound_port" type='{"type": "integer"}'>
> -        When <ref column="target"/> is <code>ptcp:</code> or
> -        <code>pssl:</code>, this is the TCP port on which the OVSDB
> server is
> -        listening.  (This is particularly useful when <ref
> -        column="target"/> specifies a port of 0, allowing the kernel to
> -        choose any available port.)
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -      <column name="other_config"/>
> -    </group>
> -  </table>
> -  <table name="DNS" title="Native DNS resolution">
> -    <p>
> -      Each row in this table stores the DNS records. The
> -      <ref table="Logical_Switch"/> table's <ref table="Logical_Switch"
> -      column="dns_records"/> references these records.
> -    </p>
> -
> -    <column name="records">
> -      Key-value pair of DNS records with <code>DNS query name</code> as
> the key
> -      and value as a string of IP address(es) separated by comma or space.
> -
> -      <p><b>Example: </b> "vm1.ovn.org" = "10.0.0.4 aef0::4"</p>
> -    </column>
> -
> -    <column name="external_ids">
> -      See <em>External IDs</em> at the beginning of this document.
> -    </column>
> -  </table>
> -  <table name="SSL">
> -    SSL configuration for ovn-nb database access.
> -
> -    <column name="private_key">
> -      Name of a PEM file containing the private key used as the switch's
> -      identity for SSL connections to the controller.
> -    </column>
> -
> -    <column name="certificate">
> -      Name of a PEM file containing a certificate, signed by the
> -      certificate authority (CA) used by the controller and manager,
> -      that certifies the switch's private key, identifying a trustworthy
> -      switch.
> -    </column>
> -
> -    <column name="ca_cert">
> -      Name of a PEM file containing the CA certificate used to verify
> -      that the switch is connected to a trustworthy controller.
> -    </column>
> -
> -    <column name="bootstrap_ca_cert">
> -      If set to <code>true</code>, then Open vSwitch will attempt to
> -      obtain the CA certificate from the controller on its first SSL
> -      connection and save it to the named PEM file. If it is successful,
> -      it will immediately drop the connection and reconnect, and from then
> -      on all SSL connections must be authenticated by a certificate signed
> -      by the CA certificate thus obtained.  <em>This option exposes the
> -      SSL connection to a man-in-the-middle attack obtaining the initial
> -      CA certificate.</em>  It may still be useful for bootstrapping.
> -    </column>
> -
> -    <column name="ssl_protocols">
> -      List of SSL protocols to be enabled for SSL connections. The default
> -      when this option is omitted is <code>TLSv1,TLSv1.1,TLSv1.2</code>.
> -    </column>
> -
> -    <column name="ssl_ciphers">
> -      List of ciphers (in OpenSSL cipher string format) to be supported
> -      for SSL connections. The default when this option is omitted is
> -      <code>HIGH:!aNULL:!MD5</code>.
> -    </column>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -  </table>
> -  <table name="Gateway_Chassis">
> -    <p>
> -      Association of one or more chassis to a logical router port. The
> traffic
> -      going out through an specific router port will be redirected to a
> -      chassis, or a set of them in high availability configurations.
> -      A single <ref table="Gateway_Chassis"/> is equivalent to setting
> -      <ref column="options" key="redirect-chassis"/>.  Using
> -      <ref table="Gateway_Chassis"/> allows associating multiple
> prioritized
> -      chassis with a single logical router port.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        Name of the <ref table="Gateway_Chassis"/>.
> -      </p>
> -      <p>
> -        A suggested, but not required naming convention is
> -        <code>${port_name}_${chassis_name}</code>.
> -      </p>
> -    </column>
> -
> -    <column name="chassis_name">
> -      <p>
> -        Name of the chassis that we want to redirect traffic through for
> the
> -        associated logical router port.  The value must match the
> -        <ref db="OVN_Southbound" table="Chassis" column="name"/> column
> -        of the <ref db="OVN_Southbound" table="Chassis"/> table in the
> -        <ref db="OVN_Southbound"/> database.
> -      </p>
> -    </column>
> -
> -    <column name="priority">
> -      <p>
> -        This is the priority of a chassis among all
> -        <ref table="Gateway_Chassis"/> belonging to the same logical
> router
> -        port.
> -      </p>
> -    </column>
> -
> -    <column name="options">
> -      Reserved for future use.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="HA_Chassis_Group">
> -    <p>
> -      Table representing a group of chassis which can provide High
> availability
> -      services. Each chassis in the group is represented by the table
> -      <ref table="HA_Chassis"/>. The HA chassis with highest priority will
> -      be the master of this group. If the master chassis failover is
> detected,
> -      the HA chassis with the next higher priority takes over the
> -      responsibility of providing the HA. If a distributed gateway router
> port
> -      references a row in this table, then the master HA chassis in this
> group
> -      provides the gateway functionality.
> -    </p>
> -
> -    <column name="name">
> -      Name of the <ref table="HA_Chassis_Group"/>. Name should be unique.
> -    </column>
> -
> -    <column name="ha_chassis">
> -      A list of HA chassis which belongs to this group.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="HA_Chassis">
> -    <column name="chassis_name">
> -      <p>
> -        Name of the chassis which is part of the HA chassis group.
> -        The value must match the
> -        <ref db="OVN_Southbound" table="Chassis" column="name"/> column
> -        of the <ref db="OVN_Southbound" table="Chassis"/> table in the
> -        <ref db="OVN_Southbound"/> database.
> -      </p>
> -    </column>
> -
> -    <column name="priority">
> -      <p>
> -        Priority of the chassis. Chassis with highest priority will be
> -        the master.
> -      </p>
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -</database>
> diff --git a/ovn/ovn-sb.ovsschema b/ovn/ovn-sb.ovsschema
> deleted file mode 100644
> index 2b7bc57a7..000000000
> --- a/ovn/ovn-sb.ovsschema
> +++ /dev/null
> @@ -1,404 +0,0 @@
> -{
> -    "name": "OVN_Southbound",
> -    "version": "2.4.0",
> -    "cksum": "3059284885 20260",
> -    "tables": {
> -        "SB_Global": {
> -            "columns": {
> -                "nb_cfg": {"type": {"key": "integer"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "connections": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "Connection"},
> -                                     "min": 0,
> -                                     "max": "unlimited"}},
> -                "ssl": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "SSL"},
> -                                     "min": 0, "max": 1}},
> -                "options": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "ipsec": {"type": "boolean"}},
> -            "maxRows": 1,
> -            "isRoot": true},
> -        "Chassis": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "hostname": {"type": "string"},
> -                "encaps": {"type": {"key": {"type": "uuid",
> -                                            "refTable": "Encap"},
> -                                    "min": 1, "max": "unlimited"}},
> -                "vtep_logical_switches" : {"type": {"key": "string",
> -                                                    "min": 0,
> -                                                    "max": "unlimited"}},
> -                "nb_cfg": {"type": {"key": "integer"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "transport_zones" : {"type": {"key": "string",
> -                                              "min": 0,
> -                                              "max": "unlimited"}}},
> -            "isRoot": true,
> -            "indexes": [["name"]]},
> -        "Encap": {
> -            "columns": {
> -                "type": {"type": {"key": {
> -                           "type": "string",
> -                           "enum": ["set", ["geneve", "stt", "vxlan"]]}}},
> -                "options": {"type": {"key": "string",
> -                                     "value": "string",
> -                                     "min": 0,
> -                                     "max": "unlimited"}},
> -                "ip": {"type": "string"},
> -                "chassis_name": {"type": "string"}},
> -            "indexes": [["type", "ip"]]},
> -        "Address_Set": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "addresses": {"type": {"key": "string",
> -                                       "min": 0,
> -                                       "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Port_Group": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ports": {"type": {"key": "string",
> -                                   "min": 0,
> -                                   "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Logical_Flow": {
> -            "columns": {
> -                "logical_datapath": {"type": {"key": {"type": "uuid",
> -                                                      "refTable":
> "Datapath_Binding"}}},
> -                "pipeline": {"type": {"key": {"type": "string",
> -                                      "enum": ["set", ["ingress",
> -                                                       "egress"]]}}},
> -                "table_id": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 23}}},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 65535}}},
> -                "match": {"type": "string"},
> -                "actions": {"type": "string"},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "Multicast_Group": {
> -            "columns": {
> -                "datapath": {"type": {"key": {"type": "uuid",
> -                                              "refTable":
> "Datapath_Binding"}}},
> -                "name": {"type": "string"},
> -                "tunnel_key": {
> -                    "type": {"key": {"type": "integer",
> -                                     "minInteger": 32768,
> -                                     "maxInteger": 65535}}},
> -                "ports": {"type": {"key": {"type": "uuid",
> -                                           "refTable": "Port_Binding",
> -                                           "refType": "weak"},
> -                                   "min": 1, "max": "unlimited"}}},
> -            "indexes": [["datapath", "tunnel_key"],
> -                        ["datapath", "name"]],
> -            "isRoot": true},
> -        "Meter": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "unit": {"type": {"key": {"type": "string",
> -                                          "enum": ["set", ["kbps",
> "pktps"]]}}},
> -                "bands": {"type": {"key": {"type": "uuid",
> -                                           "refTable": "Meter_Band",
> -                                           "refType": "strong"},
> -                                   "min": 1,
> -                                   "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Meter_Band": {
> -            "columns": {
> -                "action": {"type": {"key": {"type": "string",
> -                                            "enum": ["set", ["drop"]]}}},
> -                "rate": {"type": {"key": {"type": "integer",
> -                                          "minInteger": 1,
> -                                          "maxInteger": 4294967295}}},
> -                "burst_size": {"type": {"key": {"type": "integer",
> -                                                "minInteger": 0,
> -                                                "maxInteger":
> 4294967295}}}},
> -            "isRoot": false},
> -        "Datapath_Binding": {
> -            "columns": {
> -                "tunnel_key": {
> -                     "type": {"key": {"type": "integer",
> -                                      "minInteger": 1,
> -                                      "maxInteger": 16777215}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["tunnel_key"]],
> -            "isRoot": true},
> -        "Port_Binding": {
> -            "columns": {
> -                "logical_port": {"type": "string"},
> -                "type": {"type": "string"},
> -                "gateway_chassis": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "Gateway_Chassis",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "ha_chassis_group": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "HA_Chassis_Group",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": 1}},
> -                "options": {
> -                     "type": {"key": "string",
> -                              "value": "string",
> -                              "min": 0,
> -                              "max": "unlimited"}},
> -                "datapath": {"type": {"key": {"type": "uuid",
> -                                              "refTable":
> "Datapath_Binding"}}},
> -                "tunnel_key": {
> -                     "type": {"key": {"type": "integer",
> -                                      "minInteger": 1,
> -                                      "maxInteger": 32767}}},
> -                "parent_port": {"type": {"key": "string", "min": 0,
> "max": 1}},
> -                "tag": {
> -                     "type": {"key": {"type": "integer",
> -                                      "minInteger": 1,
> -                                      "maxInteger": 4095},
> -                              "min": 0, "max": 1}},
> -                "chassis": {"type": {"key": {"type": "uuid",
> -                                             "refTable": "Chassis",
> -                                             "refType": "weak"},
> -                                     "min": 0, "max": 1}},
> -                "encap": {"type": {"key": {"type": "uuid",
> -                                            "refTable": "Encap",
> -                                             "refType": "weak"},
> -                                    "min": 0, "max": 1}},
> -                "mac": {"type": {"key": "string",
> -                                 "min": 0,
> -                                 "max": "unlimited"}},
> -                "nat_addresses": {"type": {"key": "string",
> -                                           "min": 0,
> -                                           "max": "unlimited"}},
> -                "external_ids": {"type": {"key": "string",
> -                                 "value": "string",
> -                                 "min": 0,
> -                                 "max": "unlimited"}}},
> -            "indexes": [["datapath", "tunnel_key"], ["logical_port"]],
> -            "isRoot": true},
> -        "MAC_Binding": {
> -            "columns": {
> -                "logical_port": {"type": "string"},
> -                "ip": {"type": "string"},
> -                "mac": {"type": "string"},
> -                "datapath": {"type": {"key": {"type": "uuid",
> -                                              "refTable":
> "Datapath_Binding"}}}},
> -            "indexes": [["logical_port", "ip"]],
> -            "isRoot": true},
> -        "DHCP_Options": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "code": {
> -                    "type": {"key": {"type": "integer",
> -                                     "minInteger": 0, "maxInteger":
> 254}}},
> -                "type": {
> -                    "type": {"key": {
> -                        "type": "string",
> -                        "enum": ["set", ["bool", "uint8", "uint16",
> "uint32",
> -                                         "ipv4", "static_routes",
> "str"]]}}}},
> -            "isRoot": true},
> -        "DHCPv6_Options": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "code": {
> -                    "type": {"key": {"type": "integer",
> -                                     "minInteger": 0, "maxInteger":
> 254}}},
> -                "type": {
> -                    "type": {"key": {
> -                        "type": "string",
> -                        "enum": ["set", ["ipv6", "str", "mac"]]}}}},
> -            "isRoot": true},
> -        "Connection": {
> -            "columns": {
> -                "target": {"type": "string"},
> -                "max_backoff": {"type": {"key": {"type": "integer",
> -                                         "minInteger": 1000},
> -                                         "min": 0,
> -                                         "max": 1}},
> -                "inactivity_probe": {"type": {"key": "integer",
> -                                              "min": 0,
> -                                              "max": 1}},
> -                "read_only": {"type": "boolean"},
> -                "role": {"type": "string"},
> -                "other_config": {"type": {"key": "string",
> -                                          "value": "string",
> -                                          "min": 0,
> -                                          "max": "unlimited"}},
> -                "external_ids": {"type": {"key": "string",
> -                                 "value": "string",
> -                                 "min": 0,
> -                                 "max": "unlimited"}},
> -                "is_connected": {"type": "boolean", "ephemeral": true},
> -                "status": {"type": {"key": "string",
> -                                    "value": "string",
> -                                    "min": 0,
> -                                    "max": "unlimited"},
> -                                    "ephemeral": true}},
> -            "indexes": [["target"]]},
> -        "SSL": {
> -            "columns": {
> -                "private_key": {"type": "string"},
> -                "certificate": {"type": "string"},
> -                "ca_cert": {"type": "string"},
> -                "bootstrap_ca_cert": {"type": "boolean"},
> -                "ssl_protocols": {"type": "string"},
> -                "ssl_ciphers": {"type": "string"},
> -                "external_ids": {"type": {"key": "string",
> -                                          "value": "string",
> -                                          "min": 0,
> -                                          "max": "unlimited"}}},
> -            "maxRows": 1},
> -        "DNS": {
> -            "columns": {
> -                "records": {"type": {"key": "string",
> -                                            "value": "string",
> -                                            "min": 0,
> -                                            "max": "unlimited"}},
> -                "datapaths": {"type": {"key": {"type": "uuid",
> -                                               "refTable":
> "Datapath_Binding"},
> -                                       "min": 1,
> -                                       "max": "unlimited"}},
> -                "external_ids": {"type": {"key": "string",
> -                                          "value": "string",
> -                                          "min": 0,
> -                                          "max": "unlimited"}}},
> -            "isRoot": true},
> -        "RBAC_Role": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "permissions": {
> -                    "type": {"key": {"type": "string"},
> -                             "value": {"type": "uuid",
> -                                       "refTable": "RBAC_Permission",
> -                                       "refType": "weak"},
> -                                     "min": 0, "max": "unlimited"}}},
> -            "isRoot": true},
> -        "RBAC_Permission": {
> -            "columns": {
> -                "table": {"type": "string"},
> -                "authorization": {"type": {"key": "string",
> -                                           "min": 0,
> -                                           "max": "unlimited"}},
> -                "insert_delete": {"type": "boolean"},
> -                "update" : {"type": {"key": "string",
> -                                     "min": 0,
> -                                     "max": "unlimited"}}},
> -            "isRoot": true},
> -        "Gateway_Chassis": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "chassis": {"type": {"key": {"type": "uuid",
> -                                             "refTable": "Chassis",
> -                                             "refType": "weak"},
> -                                     "min": 0, "max": 1}},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}},
> -                "options": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": false},
> -        "HA_Chassis": {
> -            "columns": {
> -                "chassis": {"type": {"key": {"type": "uuid",
> -                                             "refTable": "Chassis",
> -                                             "refType": "weak"},
> -                                     "min": 0, "max": 1}},
> -                "priority": {"type": {"key": {"type": "integer",
> -                                              "minInteger": 0,
> -                                              "maxInteger": 32767}}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "isRoot": false},
> -        "HA_Chassis_Group": {
> -            "columns": {
> -                "name": {"type": "string"},
> -                "ha_chassis": {
> -                    "type": {"key": {"type": "uuid",
> -                                     "refTable": "HA_Chassis",
> -                                     "refType": "strong"},
> -                             "min": 0,
> -                             "max": "unlimited"}},
> -                "ref_chassis": {"type": {"key": {"type": "uuid",
> -                                                 "refTable": "Chassis",
> -                                                 "refType": "weak"},
> -                                         "min": 0, "max": "unlimited"}},
> -                "external_ids": {
> -                    "type": {"key": "string", "value": "string",
> -                             "min": 0, "max": "unlimited"}}},
> -            "indexes": [["name"]],
> -            "isRoot": true},
> -        "Controller_Event": {
> -            "columns": {
> -                "event_type": {"type": {"key": {"type": "string",
> -                               "enum": ["set", ["empty_lb_backends"]]}}},
> -                "event_info": {"type": {"key": "string", "value":
> "string",
> -                               "min": 0, "max": "unlimited"}},
> -                "chassis": {"type": {"key": {"type": "uuid",
> -                                             "refTable": "Chassis",
> -                                             "refType": "weak"},
> -                                     "min": 0, "max": 1}},
> -                "seq_num": {"type": {"key": "integer"}}
> -            },
> -            "isRoot": true},
> -        "IP_Multicast": {
> -            "columns": {
> -                "datapath": {"type": {"key": {"type": "uuid",
> -                                              "refTable":
> "Datapath_Binding",
> -                                              "refType": "weak"}}},
> -                "enabled": {"type": {"key": "boolean", "min": 0, "max":
> 1}},
> -                "querier": {"type": {"key": "boolean", "min": 0, "max":
> 1}},
> -                "eth_src": {"type": "string"},
> -                "ip4_src": {"type": "string"},
> -                "table_size": {"type": {"key": "integer",
> -                                        "min": 0, "max": 1}},
> -                "idle_timeout": {"type": {"key": "integer",
> -                                          "min": 0, "max": 1}},
> -                "query_interval": {"type": {"key": "integer",
> -                                            "min": 0, "max": 1}},
> -                "query_max_resp": {"type": {"key": "integer",
> -                                            "min": 0, "max": 1}},
> -                "seq_no": {"type": "integer"}},
> -            "indexes": [["datapath"]],
> -            "isRoot": true},
> -        "IGMP_Group": {
> -            "columns": {
> -                "address": {"type": "string"},
> -                "datapath": {"type": {"key": {"type": "uuid",
> -                                              "refTable":
> "Datapath_Binding",
> -                                              "refType": "weak"},
> -                                      "min": 0,
> -                                      "max": 1}},
> -                "chassis": {"type": {"key": {"type": "uuid",
> -                                             "refTable": "Chassis",
> -                                             "refType": "weak"},
> -                                     "min": 0,
> -                                     "max": 1}},
> -                "ports": {"type": {"key": {"type": "uuid",
> -                                           "refTable": "Port_Binding",
> -                                           "refType": "weak"},
> -                                   "min": 0, "max": "unlimited"}}},
> -            "indexes": [["address", "datapath", "chassis"]],
> -            "isRoot": true}}}
> diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml
> deleted file mode 100644
> index 544a071fa..000000000
> --- a/ovn/ovn-sb.xml
> +++ /dev/null
> @@ -1,3638 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<database name="ovn-sb" title="OVN Southbound Database">
> -  <p>
> -    This database holds logical and physical configuration and state for
> the
> -    Open Virtual Network (OVN) system to support virtual network
> abstraction.
> -    For an introduction to OVN, please see
> <code>ovn-architecture</code>(7).
> -  </p>
> -
> -  <p>
> -    The OVN Southbound database sits at the center of the OVN
> -    architecture.  It is the one component that speaks both southbound
> -    directly to all the hypervisors and gateways, via
> -    <code>ovn-controller</code>/<code>ovn-controller-vtep</code>, and
> -    northbound to the Cloud Management System, via
> <code>ovn-northd</code>:
> -  </p>
> -
> -  <h2>Database Structure</h2>
> -
> -  <p>
> -    The OVN Southbound database contains classes of data with
> -    different properties, as described in the sections below.
> -  </p>
> -
> -  <h3>Physical network</h3>
> -
> -  <p>
> -    Physical network tables contain information about the chassis nodes
> in the
> -    system.  This contains all the information necessary to wire the
> overlay,
> -    such as IP addresses, supported tunnel types, and security keys.
> -  </p>
> -
> -  <p>
> -    The amount of physical network data is small (O(n) in the number of
> -    chassis) and it changes infrequently, so it can be replicated to every
> -    chassis.
> -  </p>
> -
> -  <p>
> -    The <ref table="Chassis"/> and <ref table="Encap"/> tables are the
> physical
> -    network tables.
> -  </p>
> -
> -  <h3>Logical Network</h3>
> -
> -  <p>
> -    Logical network tables contain the topology of logical switches and
> -    routers, ACLs, firewall rules, and everything needed to describe how
> -    packets traverse a logical network, represented as logical datapath
> flows
> -    (see Logical Datapath Flows, below).
> -  </p>
> -
> -  <p>
> -    Logical network data may be large (O(n) in the number of logical
> ports, ACL
> -    rules, etc.).  Thus, to improve scaling, each chassis should receive
> only
> -    data related to logical networks in which that chassis participates.
> -  </p>
> -
> -  <p>
> -    The logical network data is ultimately controlled by the cloud
> management
> -    system (CMS) running northbound of OVN.  That CMS determines the
> entire OVN
> -    logical configuration and therefore the logical network data at any
> given
> -    time is a deterministic function of the CMS's configuration, although
> that
> -    happens indirectly via the <ref db="OVN_Northbound"/> database and
> -    <code>ovn-northd</code>.
> -  </p>
> -
> -  <p>
> -    Logical network data is likely to change more quickly than physical
> network
> -    data.  This is especially true in a container environment where
> containers
> -    are created and destroyed (and therefore added to and deleted from
> logical
> -    switches) quickly.
> -  </p>
> -
> -  <p>
> -    The <ref table="Logical_Flow"/>, <ref table="Multicast_Group"/>, <ref
> -    table="Address_Group"/>, <ref table="DHCP_Options"/>, <ref
> -    table="DHCPv6_Options"/>, and <ref table="DNS"/> tables contain
> logical
> -    network data.
> -  </p>
> -
> -  <h3>Logical-physical bindings</h3>
> -
> -  <p>
> -    These tables link logical and physical components.  They show the
> current
> -    placement of logical components (such as VMs and VIFs) onto chassis,
> and
> -    map logical entities to the values that represent them in tunnel
> -    encapsulations.
> -  </p>
> -
> -  <p>
> -    These tables change frequently, at least every time a VM powers up or
> down
> -    or migrates, and especially quickly in a container environment.  The
> -    amount of data per VM (or VIF) is small.
> -  </p>
> -
> -  <p>
> -    Each chassis is authoritative about the VMs and VIFs that it hosts at
> any
> -    given time and can efficiently flood that state to a central
> location, so
> -    the consistency needs are minimal.
> -  </p>
> -
> -  <p>
> -    The <ref table="Port_Binding"/> and <ref table="Datapath_Binding"/>
> tables
> -    contain binding data.
> -  </p>
> -
> -  <h3>MAC bindings</h3>
> -
> -  <p>
> -    The <ref table="MAC_Binding"/> table tracks the bindings from IP
> addresses
> -    to Ethernet addresses that are dynamically discovered using ARP (for
> IPv4)
> -    and neighbor discovery (for IPv6).  Usually, IP-to-MAC bindings for
> virtual
> -    machines are statically populated into the <ref table="Port_Binding"/>
> -    table, so <ref table="MAC_Binding"/> is primarily used to discover
> bindings
> -    on physical networks.
> -  </p>
> -
> -  <h2>Common Columns</h2>
> -
> -  <p>
> -    Some tables contain a special column named
> <code>external_ids</code>.  This
> -    column has the same form and purpose each place that it appears, so we
> -    describe it here to save space later.
> -  </p>
> -
> -  <dl>
> -    <dt><code>external_ids</code>: map of string-string pairs</dt>
> -    <dd>
> -      Key-value pairs for use by the software that manages the OVN
> Southbound
> -      database rather than by
> -      <code>ovn-controller</code>/<code>ovn-controller-vtep</code>.  In
> -      particular, <code>ovn-northd</code> can use key-value pairs in this
> -      column to relate entities in the southbound database to higher-level
> -      entities (such as entities in the OVN Northbound database).
> Individual
> -      key-value pairs in this column may be documented in some cases to
> aid
> -      in understanding and troubleshooting, but the reader should not
> mistake
> -      such documentation as comprehensive.
> -    </dd>
> -  </dl>
> -
> -  <table name="SB_Global" title="Southbound configuration">
> -    <p>
> -      Southbound configuration for an OVN system.  This table must have
> exactly
> -      one row.
> -    </p>
> -
> -    <group title="Status">
> -      This column allow a client to track the overall configuration state
> of
> -      the system.
> -
> -      <column name="nb_cfg">
> -        Sequence number for the configuration.  When a CMS or
> -        <code>ovn-nbctl</code> updates the northbound database, it
> increments
> -        the <code>nb_cfg</code> column in the <code>NB_Global</code>
> table in
> -        the northbound database.  In turn, when <code>ovn-northd</code>
> updates
> -        the southbound database to bring it up to date with these
> changes, it
> -        updates this column to the same value.
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -
> -      <column name="options">
> -      </column>
> -    </group>
> -
> -    <group title="Common options">
> -      <column name="options">
> -        This column provides general key/value settings. The supported
> -        options are described individually below.
> -      </column>
> -
> -      <group title="Options for configuring BFD">
> -        <p>
> -          These options apply when <code>ovn-controller</code> configures
> -          BFD on tunnels interfaces.
> -        </p>
> -
> -        <column name="options" key="bfd-min-rx">
> -          BFD option <code>min-rx</code> value to use when configuring
> BFD on
> -          tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-decay-min-rx">
> -          BFD option <code>decay-min-rx</code> value to use when
> configuring
> -          BFD on tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-min-tx">
> -          BFD option <code>min-tx</code> value to use when configuring
> BFD on
> -          tunnel interfaces.
> -        </column>
> -
> -        <column name="options" key="bfd-mult">
> -          BFD option <code>mult</code> value to use when configuring BFD
> on
> -          tunnel interfaces.
> -        </column>
> -      </group>
> -    </group>
> -
> -    <group title="Connection Options">
> -      <column name="connections">
> -        Database clients to which the Open vSwitch database server should
> -        connect or on which it should listen, along with options for how
> these
> -        connections should be configured.  See the <ref
> table="Connection"/>
> -        table for more information.
> -      </column>
> -      <column name="ssl">
> -        Global SSL configuration.
> -      </column>
> -    </group>
> -    <group title="Security Configurations">
> -      <column name="ipsec">
> -        Tunnel encryption configuration. If this column is set to be
> true, all
> -        OVN tunnels will be encrypted with IPsec.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Chassis" title="Physical Network Hypervisor and Gateway
> Information">
> -    <p>
> -      Each row in this table represents a hypervisor or gateway (a
> chassis) in
> -      the physical network.  Each chassis, via
> -      <code>ovn-controller</code>/<code>ovn-controller-vtep</code>, adds
> -      and updates its own row, and keeps a copy of the remaining rows to
> -      determine how to reach other hypervisors.
> -    </p>
> -
> -    <p>
> -      When a chassis shuts down gracefully, it should remove its own row.
> -      (This is not critical because resources hosted on the chassis are
> equally
> -      unreachable regardless of whether the row is present.)  If a chassis
> -      shuts down permanently without removing its row, some kind of
> manual or
> -      automatic cleanup is eventually needed; we can devise a process for
> that
> -      as necessary.
> -    </p>
> -
> -    <column name="name">
> -      OVN does not prescribe a particular format for chassis names.
> -      ovn-controller populates this column using <ref key="system-id"
> -      table="Open_vSwitch" column="external_ids" db="Open_vSwitch"/>
> -      in the Open_vSwitch database's <ref table="Open_vSwitch"
> -      db="Open_vSwitch"/> table.  ovn-controller-vtep populates this
> -      column with <ref table="Physical_Switch" column="name"
> -      db="hardware_vtep"/> in the hardware_vtep database's
> -      <ref table="Physical_Switch" db="hardware_vtep"/> table.
> -    </column>
> -
> -    <column name="hostname">
> -      The hostname of the chassis, if applicable.  ovn-controller will
> populate
> -      this column with the hostname of the host it is running on.
> -      ovn-controller-vtep will leave this column empty.
> -    </column>
> -
> -    <column name="nb_cfg">
> -      Sequence number for the configuration.  When
> <code>ovn-controller</code>
> -      updates the configuration of a chassis from the contents of the
> -      southbound database, it copies <ref table="SB_Global"
> column="nb_cfg"/>
> -      from the <ref table="SB_Global"/> table into this column.
> -    </column>
> -
> -    <column name="external_ids" key="ovn-bridge-mappings">
> -      <code>ovn-controller</code> populates this key with the set of
> bridge
> -      mappings it has been configured to use.  Other applications should
> treat
> -      this key as read-only.  See <code>ovn-controller</code>(8) for more
> -      information.
> -    </column>
> -
> -    <column name="external_ids" key="datapath-type">
> -      <code>ovn-controller</code> populates this key with the datapath
> type
> -      configured in the <ref table="Bridge" column="datapath_type"/>
> column of
> -      the Open_vSwitch database's <ref table="Bridge" db="Open_vSwitch"/>
> -      table.  Other applications should treat this key as read-only. See
> -      <code>ovn-controller</code>(8) for more information.
> -    </column>
> -
> -    <column name="external_ids" key="iface-types">
> -      <code>ovn-controller</code> populates this key with the interface
> types
> -      configured in the <ref table="Open_vSwitch" column="iface_types"/>
> column
> -      of the Open_vSwitch database's <ref table="Open_vSwitch"
> -      db="Open_vSwitch"/> table.  Other applications should treat this
> key as
> -      read-only. See <code>ovn-controller</code>(8) for more information.
> -    </column>
> -
> -    <column name="external_ids" key="ovn-cms-options">
> -      <code>ovn-controller</code> populates this key with the set of
> options
> -      configured in the <ref table="Open_vSwitch"
> -      column="external_ids:ovn-cms-options"/> column of the Open_vSwitch
> -      database's <ref table="Open_vSwitch" db="Open_vSwitch"/> table.
> -      See <code>ovn-controller</code>(8) for more information.
> -    </column>
> -
> -    <column name="transport_zones">
> -      <code>ovn-controller</code> populates this key with the transport
> -      zones configured in the <ref table="Open_vSwitch"
> -      column="external_ids:ovn-transport-zones"/> column of the
> Open_vSwitch
> -      database's <ref table="Open_vSwitch" db="Open_vSwitch"/> table.
> -      See <code>ovn-controller</code>(8) for more information.
> -    </column>
> -
> -    <column name="external_ids" key="ovn-chassis-mac-mappings">
> -      <code>ovn-controller</code> populates this key with the set of
> options
> -      configured in the <ref table="Open_vSwitch"
> -      column="external_ids:ovn-chassis-mac-mappings"/> column of the
> -      Open_vSwitch database's <ref table="Open_vSwitch"
> db="Open_vSwitch"/>
> -      table. See <code>ovn-controller</code>(8) for more information.
> -    </column>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -
> -    <group title="Encapsulation Configuration">
> -      <p>
> -        OVN uses encapsulation to transmit logical dataplane packets
> -        between chassis.
> -      </p>
> -
> -      <column name="encaps">
> -        Points to supported encapsulation configurations to transmit
> -        logical dataplane packets to this chassis.  Each entry is a <ref
> -        table="Encap"/> record that describes the configuration.
> -      </column>
> -    </group>
> -
> -    <group title="Gateway Configuration">
> -      <p>
> -        A <dfn>gateway</dfn> is a chassis that forwards traffic between
> the
> -        OVN-managed part of a logical network and a physical VLAN,
> extending a
> -        tunnel-based logical network into a physical network.  Gateways
> are
> -        typically dedicated nodes that do not host VMs and will be
> controlled
> -        by <code>ovn-controller-vtep</code>.
> -      </p>
> -
> -      <column name="vtep_logical_switches">
> -        Stores all VTEP logical switch names connected by this gateway
> -        chassis.  The <ref table="Port_Binding"/> table entry with
> -        <ref column="options"
> table="Port_Binding"/>:<code>vtep-physical-switch</code>
> -        equal <ref table="Chassis"/> <ref column="name"
> table="Chassis"/>, and
> -        <ref column="options"
> table="Port_Binding"/>:<code>vtep-logical-switch</code>
> -        value in <ref table="Chassis"/>
> -        <ref column="vtep_logical_switches" table="Chassis"/>, will be
> -        associated with this <ref table="Chassis"/>.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="Encap" title="Encapsulation Types">
> -    <p>
> -      The <ref column="encaps" table="Chassis"/> column in the <ref
> -      table="Chassis"/> table refers to rows in this table to identify
> -      how OVN may transmit logical dataplane packets to this chassis.
> -      Each chassis, via <code>ovn-controller</code>(8) or
> -      <code>ovn-controller-vtep</code>(8), adds and updates its own rows
> -      and keeps a copy of the remaining rows to determine how to reach
> -      other chassis.
> -    </p>
> -
> -    <column name="type">
> -      The encapsulation to use to transmit packets to this chassis.
> -      Hypervisors must use either <code>geneve</code> or
> -      <code>stt</code>.  Gateways may use <code>vxlan</code>,
> -      <code>geneve</code>, or <code>stt</code>.
> -    </column>
> -
> -    <column name="options">
> -      Options for configuring the encapsulation, which may be <ref
> column="type"/> specific.
> -    </column>
> -
> -    <column name="options" key="csum" type='{"type": "boolean"}'>
> -      <p>
> -        <code>csum</code> indicates whether this chassis can transmit and
> -        receive packets that include checksums with reasonable
> performance.  It
> -        hints
> -        to senders transmitting data to this chassis that they should use
> -        checksums to protect OVN metadata. <code>ovn-controller</code>
> -        populates this key with the value defined in
> -        <ref table="Open_vSwitch" column="external_ids:ovn-encap-csum"/>
> column
> -        of the Open_vSwitch database's <ref table="Open_vSwitch"
> -        db="Open_vSwitch"/> table.  Other applications should treat this
> key as
> -        read-only. See <code>ovn-controller</code>(8) for more
> information.
> -      </p>
> -
> -      <p>
> -        In terms of performance, checksumming actually significantly
> increases
> -        throughput in most common cases when running on Linux based hosts
> -        without NICs supporting encapsulation hardware offload (around
> 60% for
> -        bulk traffic). The reason is that generally all NICs are capable
> of
> -        offloading transmitted and received TCP/UDP checksums (viewed as
> -        ordinary data packets and not as tunnels). The benefit comes on
> the
> -        receive side where the validated outer checksum can be used to
> -        additionally validate an inner checksum (such as TCP), which in
> turn
> -        allows aggregation of packets to be more efficiently handled by
> the
> -        rest of the stack.
> -      </p>
> -
> -      <p>
> -        Not all devices see such a benefit. The most notable exception is
> -        hardware VTEPs. These devices are designed to not buffer entire
> -        packets in their switching engines and are therefore unable to
> -        efficiently compute or validate full packet checksums. In addition
> -        certain versions of the Linux kernel are not able to fully take
> -        advantage of encapsulation NIC offloads in the presence of
> checksums.
> -        (This is actually a pretty narrow corner case though: earlier
> -        versions of Linux don't support encapsulation offloads at all and
> -        later versions support both offloads and checksums well.)
> -      </p>
> -
> -      <p>
> -        <code>csum</code> defaults to <code>false</code> for hardware
> VTEPs and
> -        <code>true</code> for all other cases.
> -      </p>
> -
> -      <p>
> -        This option applies to <code>geneve</code> and <code>vxlan</code>
> -        encapsulations.
> -      </p>
> -    </column>
> -
> -    <column name="options" key="dst_port" type='{"type": "integer"}'>
> -      <p>
> -        If set, overrides the UDP (for <code>geneve</code> and
> -        <code>vxlan</code>) or TCP (for <code>stt</code>) destination
> port.
> -      </p>
> -    </column>
> -
> -    <column name="ip">
> -      The IPv4 address of the encapsulation tunnel endpoint.
> -    </column>
> -    <column name="chassis_name">
> -      The name of the chassis that created this encap.
> -    </column>
> -  </table>
> -
> -  <table name="Address_Set" title="Address Sets">
> -    <p>
> -      This table contains address sets synced from the <ref
> table="Address_Set"
> -      db="OVN_Northbound"/> table in the <ref db="OVN_Northbound"/>
> database
> -      and address sets generated from the <ref table="Port_Group"
> -      db="OVN_Northbound"/> table in the <ref db="OVN_Northbound"/>
> database.
> -    </p>
> -
> -    <p>
> -      See the documentation for the <ref table="Address_Set"
> -      db="OVN_Northbound"/> table and <ref table="Port_Group"
> -      db="OVN_Northbound"/> table in the <ref db="OVN_Northbound"/>
> -      database for details.
> -    </p>
> -
> -    <column name="name"/>
> -    <column name="addresses"/>
> -  </table>
> -
> -  <table name="Port_Group" title="Port Groups">
> -    <p>
> -      This table contains names for the logical switch ports in the
> -      <ref db="OVN_Northbound"/> database that belongs to the same group
> -      that is defined in <ref table="Port_Group" db="OVN_Northbound"/>
> -      in the <ref db="OVN_Northbound"/> database.
> -    </p>
> -
> -    <column name="name"/>
> -    <column name="ports"/>
> -  </table>
> -
> -  <table name="Logical_Flow" title="Logical Network Flows">
> -    <p>
> -      Each row in this table represents one logical flow.
> -      <code>ovn-northd</code> populates this table with logical flows
> -      that implement the L2 and L3 topologies specified in the
> -      <ref db="OVN_Northbound"/> database.  Each hypervisor, via
> -      <code>ovn-controller</code>, translates the logical flows into
> -      OpenFlow flows specific to its hypervisor and installs them into
> -      Open vSwitch.
> -    </p>
> -
> -    <p>
> -      Logical flows are expressed in an OVN-specific format, described
> here.  A
> -      logical datapath flow is much like an OpenFlow flow, except that the
> -      flows are written in terms of logical ports and logical datapaths
> instead
> -      of physical ports and physical datapaths.  Translation between
> logical
> -      and physical flows helps to ensure isolation between logical
> datapaths.
> -      (The logical flow abstraction also allows the OVN centralized
> -      components to do less work, since they do not have to separately
> -      compute and push out physical flows to each chassis.)
> -    </p>
> -
> -    <p>
> -      The default action when no flow matches is to drop packets.
> -    </p>
> -
> -    <p><em>Architectural Logical Life Cycle of a Packet</em></p>
> -
> -    <p>
> -      This following description focuses on the life cycle of a packet
> through
> -      a logical datapath, ignoring physical details of the implementation.
> -      Please refer to <em>Architectural Physical Life Cycle of a
> Packet</em> in
> -      <code>ovn-architecture</code>(7) for the physical information.
> -    </p>
> -
> -    <p>
> -      The description here is written as if OVN itself executes these
> steps,
> -      but in fact OVN (that is, <code>ovn-controller</code>) programs Open
> -      vSwitch, via OpenFlow and OVSDB, to execute them on its behalf.
> -    </p>
> -
> -    <p>
> -      At a high level, OVN passes each packet through the logical
> datapath's
> -      logical ingress pipeline, which may output the packet to one or more
> -      logical port or logical multicast groups.  For each such logical
> output
> -      port, OVN passes the packet through the datapath's logical egress
> -      pipeline, which may either drop the packet or deliver it to the
> -      destination.  Between the two pipelines, outputs to logical
> multicast
> -      groups are expanded into logical ports, so that the egress pipeline
> only
> -      processes a single logical output port at a time.  Between the two
> -      pipelines is also where, when necessary, OVN encapsulates a packet
> in a
> -      tunnel (or tunnels) to transmit to remote hypervisors.
> -    </p>
> -
> -    <p>
> -      In more detail, to start, OVN searches the <ref
> table="Logical_Flow"/>
> -      table for a row with correct <ref column="logical_datapath"/>, a
> <ref
> -      column="pipeline"/> of <code>ingress</code>, a <ref
> column="table_id"/>
> -      of 0, and a <ref column="match"/> that is true for the packet.  If
> none
> -      is found, OVN drops the packet.  If OVN finds more than one, it
> chooses
> -      the match with the highest <ref column="priority"/>.  Then OVN
> executes
> -      each of the actions specified in the row's <ref table="actions"/>
> column,
> -      in the order specified.  Some actions, such as those to modify
> packet
> -      headers, require no further details.  The <code>next</code> and
> -      <code>output</code> actions are special.
> -    </p>
> -
> -    <p>
> -      The <code>next</code> action causes the above process to be repeated
> -      recursively, except that OVN searches for <ref column="table_id"/>
> of 1
> -      instead of 0.  Similarly, any <code>next</code> action in a row
> found in
> -      that table would cause a further search for a <ref
> column="table_id"/> of
> -      2, and so on.  When recursive processing completes, flow control
> returns
> -      to the action following <code>next</code>.
> -    </p>
> -
> -    <p>
> -      The <code>output</code> action also introduces recursion.  Its
> effect
> -      depends on the current value of the <code>outport</code> field.
> Suppose
> -      <code>outport</code> designates a logical port.  First, OVN compares
> -      <code>inport</code> to <code>outport</code>; if they are equal, it
> treats
> -      the <code>output</code> as a no-op by default.  In the common
> -      case, where they are different, the packet enters the egress
> -      pipeline.  This transition to the egress pipeline discards
> -      register data, e.g. <code>reg0</code> ...  <code>reg9</code> and
> -      connection tracking state, to achieve uniform behavior regardless
> -      of whether the egress pipeline is on a different hypervisor
> -      (because registers aren't preserve across tunnel encapsulation).
> -    </p>
> -
> -    <p>
> -      To execute the egress pipeline, OVN again searches the <ref
> -      table="Logical_Flow"/> table for a row with correct <ref
> -      column="logical_datapath"/>, a <ref column="table_id"/> of 0, a <ref
> -      column="match"/> that is true for the packet, but now looking for a
> <ref
> -      column="pipeline"/> of <code>egress</code>.  If no matching row is
> found,
> -      the output becomes a no-op.  Otherwise, OVN executes the actions
> for the
> -      matching flow (which is chosen from multiple, if necessary, as
> already
> -      described).
> -    </p>
> -
> -    <p>
> -      In the <code>egress</code> pipeline, the <code>next</code> action
> acts as
> -      already described, except that it, of course, searches for
> -      <code>egress</code> flows.  The <code>output</code> action,
> however, now
> -      directly outputs the packet to the output port (which is now fixed,
> -      because <code>outport</code> is read-only within the egress
> pipeline).
> -    </p>
> -
> -    <p>
> -      The description earlier assumed that <code>outport</code> referred
> to a
> -      logical port.  If it instead designates a logical multicast group,
> then
> -      the description above still applies, with the addition of fan-out
> from
> -      the logical multicast group to each logical port in the group.  For
> each
> -      member of the group, OVN executes the logical pipeline as
> described, with
> -      the logical output port replaced by the group member.
> -    </p>
> -
> -    <p><em>Pipeline Stages</em></p>
> -
> -    <p>
> -      <code>ovn-northd</code> populates the <ref table="Logical_Flow"/>
> table
> -      with the logical flows described in detail in
> <code>ovn-northd</code>(8).
> -    </p>
> -
> -    <column name="logical_datapath">
> -      The logical datapath to which the logical flow belongs.
> -    </column>
> -
> -    <column name="pipeline">
> -      <p>
> -        The primary flows used for deciding on a packet's destination are
> the
> -        <code>ingress</code> flows.  The <code>egress</code> flows
> implement
> -        ACLs.  See <em>Logical Life Cycle of a Packet</em>, above, for
> details.
> -      </p>
> -    </column>
> -
> -    <column name="table_id">
> -      The stage in the logical pipeline, analogous to an OpenFlow table
> number.
> -    </column>
> -
> -    <column name="priority">
> -      The flow's priority.  Flows with numerically higher priority take
> -      precedence over those with lower.  If two logical datapath flows
> with the
> -      same priority both match, then the one actually applied to the
> packet is
> -      undefined.
> -    </column>
> -
> -    <column name="match">
> -      <p>
> -        A matching expression.  OVN provides a superset of OpenFlow
> matching
> -        capabilities, using a syntax similar to Boolean expressions in a
> -        programming language.
> -      </p>
> -
> -      <p>
> -        The most important components of match expression are
> -        <dfn>comparisons</dfn> between <dfn>symbols</dfn> and
> -        <dfn>constants</dfn>, e.g. <code>ip4.dst == 192.168.0.1</code>,
> -        <code>ip.proto == 6</code>, <code>arp.op == 1</code>,
> <code>eth.type ==
> -        0x800</code>.  The logical AND operator <code>&amp;&amp;</code>
> and
> -        logical OR operator <code>||</code> can combine comparisons into a
> -        larger expression.
> -      </p>
> -
> -      <p>
> -        Matching expressions also support parentheses for grouping, the
> logical
> -        NOT prefix operator <code>!</code>, and literals <code>0</code>
> and
> -        <code>1</code> to express ``false'' or ``true,'' respectively.
> The
> -        latter is useful by itself as a catch-all expression that matches
> every
> -        packet.
> -      </p>
> -
> -      <p>
> -        Match expressions also support a kind of function syntax.  The
> -        following functions are supported:
> -      </p>
> -
> -      <dl>
> -        <dt><code>is_chassis_resident(<var>lport</var>)</code></dt>
> -        <dd>
> -          Evaluates to true on a chassis on which logical port
> <var>lport</var>
> -          (a quoted string) resides, and to false elsewhere.  This
> function was
> -          introduced in OVN 2.7.
> -        </dd>
> -      </dl>
> -
> -      <p><em>Symbols</em></p>
> -
> -      <p>
> -        <em>Type</em>.  Symbols have <dfn>integer</dfn> or
> <dfn>string</dfn>
> -        type.  Integer symbols have a <dfn>width</dfn> in bits.
> -      </p>
> -
> -      <p>
> -        <em>Kinds</em>.  There are three kinds of symbols:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          <p>
> -            <dfn>Fields</dfn>.  A field symbol represents a packet header
> or
> -            metadata field.  For example, a field
> -            named <code>vlan.tci</code> might represent the VLAN TCI
> field in a
> -            packet.
> -          </p>
> -
> -          <p>
> -            A field symbol can have integer or string type.  Integer
> fields can
> -            be nominal or ordinal (see <em>Level of Measurement</em>,
> -            below).
> -          </p>
> -        </li>
> -
> -        <li>
> -          <p>
> -            <dfn>Subfields</dfn>.  A subfield represents a subset of bits
> from
> -            a larger field.  For example, a field <code>vlan.vid</code>
> might
> -            be defined as an alias for <code>vlan.tci[0..11]</code>.
> Subfields
> -            are provided for syntactic convenience, because it is always
> -            possible to instead refer to a subset of bits from a field
> -            directly.
> -          </p>
> -
> -          <p>
> -            Only ordinal fields (see <em>Level of Measurement</em>,
> -            below) may have subfields.  Subfields are always ordinal.
> -          </p>
> -        </li>
> -
> -        <li>
> -          <p>
> -            <dfn>Predicates</dfn>.  A predicate is shorthand for a Boolean
> -            expression.  Predicates may be used much like 1-bit fields.
> For
> -            example, <code>ip4</code> might expand to <code>eth.type ==
> -            0x800</code>.  Predicates are provided for syntactic
> convenience,
> -            because it is always possible to instead specify the
> underlying
> -            expression directly.
> -          </p>
> -
> -          <p>
> -            A predicate whose expansion refers to any nominal field or
> -            predicate (see <em>Level of Measurement</em>, below) is
> nominal;
> -            other predicates have Boolean level of measurement.
> -          </p>
> -        </li>
> -      </ul>
> -
> -      <p>
> -        <em>Level of Measurement</em>.  See
> -        http://en.wikipedia.org/wiki/Level_of_measurement for the
> statistical
> -        concept on which this classification is based.  There are three
> -        levels:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          <p>
> -            <dfn>Ordinal</dfn>.  In statistics, ordinal values can be
> ordered
> -            on a scale.  OVN considers a field (or subfield) to be
> ordinal if
> -            its bits can be examined individually.  This is true for the
> -            OpenFlow fields that OpenFlow or Open vSwitch makes
> ``maskable.''
> -          </p>
> -
> -          <p>
> -            Any use of a ordinal field may specify a single bit or a
> range of
> -            bits, e.g. <code>vlan.tci[13..15]</code> refers to the PCP
> field
> -            within the VLAN TCI, and <code>eth.dst[40]</code> refers to
> the
> -            multicast bit in the Ethernet destination address.
> -          </p>
> -
> -          <p>
> -            OVN supports all the usual arithmetic relations
> (<code>==</code>,
> -            <code>!=</code>, <code>&lt;</code>, <code>&lt;=</code>,
> -            <code>&gt;</code>, and <code>&gt;=</code>) on ordinal fields
> and
> -            their subfields, because OVN can implement these in OpenFlow
> and
> -            Open vSwitch as collections of bitwise tests.
> -          </p>
> -        </li>
> -
> -        <li>
> -          <p>
> -            <dfn>Nominal</dfn>.  In statistics, nominal values cannot be
> -            usefully compared except for equality.  This is true of
> OpenFlow
> -            port numbers, Ethernet types, and IP protocols are examples:
> all of
> -            these are just identifiers assigned arbitrarily with no deeper
> -            meaning.  In OpenFlow and Open vSwitch, bits in these fields
> -            generally aren't individually addressable.
> -          </p>
> -
> -          <p>
> -            OVN only supports arithmetic tests for equality on nominal
> fields,
> -            because OpenFlow and Open vSwitch provide no way for a flow to
> -            efficiently implement other comparisons on them.  (A test for
> -            inequality can be sort of built out of two flows with
> different
> -            priorities, but OVN matching expressions always generate
> flows with
> -            a single priority.)
> -          </p>
> -
> -          <p>
> -            String fields are always nominal.
> -          </p>
> -        </li>
> -
> -        <li>
> -          <p>
> -            <dfn>Boolean</dfn>.  A nominal field that has only two
> values, 0
> -            and 1, is somewhat exceptional, since it is easy to support
> both
> -            equality and inequality tests on such a field: either one can
> be
> -            implemented as a test for 0 or 1.
> -          </p>
> -
> -          <p>
> -            Only predicates (see above) have a Boolean level of
> measurement.
> -          </p>
> -
> -          <p>
> -            This isn't a standard level of measurement.
> -          </p>
> -        </li>
> -      </ul>
> -
> -      <p>
> -        <em>Prerequisites</em>.  Any symbol can have prerequisites, which
> are
> -        additional condition implied by the use of the symbol.  For
> example,
> -        For example, <code>icmp4.type</code> symbol might have
> prerequisite
> -        <code>icmp4</code>, which would cause an expression
> <code>icmp4.type ==
> -        0</code> to be interpreted as <code>icmp4.type == 0 &amp;&amp;
> -        icmp4</code>, which would in turn expand to <code>icmp4.type == 0
> -        &amp;&amp; eth.type == 0x800 &amp;&amp; ip4.proto == 1</code>
> (assuming
> -        <code>icmp4</code> is a predicate defined as suggested under
> -        <em>Types</em> above).
> -      </p>
> -
> -      <p><em>Relational operators</em></p>
> -
> -      <p>
> -        All of the standard relational operators <code>==</code>,
> -        <code>!=</code>, <code>&lt;</code>, <code>&lt;=</code>,
> -        <code>&gt;</code>, and <code>&gt;=</code> are supported.  Nominal
> -        fields support only <code>==</code> and <code>!=</code>, and only
> in a
> -        positive sense when outer <code>!</code> are taken into account,
> -        e.g. given string field <code>inport</code>, <code>inport ==
> -        "eth0"</code> and <code>!(inport != "eth0")</code> are
> acceptable, but
> -        not <code>inport != "eth0"</code>.
> -      </p>
> -
> -      <p>
> -        The implementation of <code>==</code> (or <code>!=</code> when it
> is
> -        negated), is more efficient than that of the other relational
> -        operators.
> -      </p>
> -
> -      <p><em>Constants</em></p>
> -
> -      <p>
> -        Integer constants may be expressed in decimal, hexadecimal
> prefixed by
> -        <code>0x</code>, or as dotted-quad IPv4 addresses, IPv6 addresses
> in
> -        their standard forms, or Ethernet addresses as colon-separated hex
> -        digits.  A constant in any of these forms may be followed by a
> slash
> -        and a second constant (the mask) in the same form, to form a
> masked
> -        constant.  IPv4 and IPv6 masks may be given as integers, to
> express
> -        CIDR prefixes.
> -      </p>
> -
> -      <p>
> -        String constants have the same syntax as quoted strings in JSON
> (thus,
> -        they are Unicode strings).
> -      </p>
> -
> -      <p>
> -        Some operators support sets of constants written inside curly
> braces
> -        <code>{</code> ... <code>}</code>.  Commas between elements of a
> set,
> -        and after the last elements, are optional.  With <code>==</code>,
> -        ``<code><var>field</var> == { <var>constant1</var>,
> -        <var>constant2</var>,</code> ... <code>}</code>'' is syntactic
> sugar
> -        for ``<code><var>field</var> == <var>constant1</var> ||
> -        <var>field</var> == <var>constant2</var> ||
> </code>...<code></code>.
> -        Similarly, ``<code><var>field</var> != { <var>constant1</var>,
> -        <var>constant2</var>, </code>...<code> }</code>'' is equivalent to
> -        ``<code><var>field</var> != <var>constant1</var> &amp;&amp;
> -        <var>field</var> != <var>constant2</var> &amp;&amp;
> -        </code>...<code></code>''.
> -      </p>
> -
> -      <p>
> -        You may refer to a set of IPv4, IPv6, or MAC addresses stored in
> the
> -        <ref table="Address_Set"/> table by its <ref column="name"
> -        table="Address_Set"/>.  An <ref table="Address_Set"/> with a name
> -        of <code>set1</code> can be referred to as
> -        <code>$set1</code>.
> -      </p>
> -
> -      <p>
> -        You may refer to a group of logical switch ports stored in the
> -        <ref table="Port_Group"/> table by its <ref column="name"
> -        table="Port_Group"/>.  An <ref table="Port_Group"/> with a name
> -        of <code>port_group1</code> can be referred to as
> -        <code>@port_group1</code>.
> -      </p>
> -
> -      <p>
> -        Additionally, you may refer to the set of addresses belonging to a
> -        group of logical switch ports stored in the <ref
> table="Port_Group"/>
> -        table by its <ref column="name" table="Port_Group"/> followed by
> -        a suffix '_ip4'/'_ip6'.  The IPv4 address set of a
> -        <ref table="Port_Group"/> with a name of <code>port_group1</code>
> -        can be referred to as <code>$port_group1_ip4</code>, and the IPv6
> -        address set of the same <ref table="Port_Group"/> can be referred
> to
> -        as <code>$port_group1_ip6</code>
> -      </p>
> -
> -      <p><em>Miscellaneous</em></p>
> -
> -      <p>
> -        Comparisons may name the symbol or the constant first,
> -        e.g. <code>tcp.src == 80</code> and <code>80 == tcp.src</code>
> are both
> -        acceptable.
> -      </p>
> -
> -      <p>
> -        Tests for a range may be expressed using a syntax like <code>1024
> &lt;=
> -        tcp.src &lt;= 49151</code>, which is equivalent to <code>1024
> &lt;=
> -        tcp.src &amp;&amp; tcp.src &lt;= 49151</code>.
> -      </p>
> -
> -      <p>
> -        For a one-bit field or predicate, a mention of its name is
> equivalent
> -        to <code><var>symobl</var> == 1</code>, e.g.
> <code>vlan.present</code>
> -        is equivalent to <code>vlan.present == 1</code>.  The same is
> true for
> -        one-bit subfields, e.g. <code>vlan.tci[12]</code>.  There is no
> -        technical limitation to implementing the same for ordinal fields
> of all
> -        widths, but the implementation is expensive enough that the syntax
> -        parser requires writing an explicit comparison against zero to
> make
> -        mistakes less likely, e.g. in <code>tcp.src != 0</code> the
> comparison
> -        against 0 is required.
> -      </p>
> -
> -      <p>
> -        <em>Operator precedence</em> is as shown below, from highest to
> lowest.
> -        There are two exceptions where parentheses are required even
> though the
> -        table would suggest that they are not: <code>&amp;&amp;</code> and
> -        <code>||</code> require parentheses when used together, and
> -        <code>!</code> requires parentheses when applied to a relational
> -        expression.  Thus, in <code>(eth.type == 0x800 || eth.type ==
> 0x86dd)
> -        &amp;&amp; ip.proto == 6</code> or <code>!(arp.op == 1)</code>,
> the
> -        parentheses are mandatory.
> -      </p>
> -
> -      <ul>
> -        <li><code>()</code></li>
> -        <li><code>==   !=   &lt;   &lt;=   &gt;   &gt;=</code></li>
> -        <li><code>!</code></li>
> -        <li><code>&amp;&amp;   ||</code></li>
> -      </ul>
> -
> -      <p>
> -        <em>Comments</em> may be introduced by <code>//</code>, which
> extends
> -        to the next new-line.  Comments within a line may be bracketed by
> -        <code>/*</code> and <code>*/</code>.  Multiline comments are not
> -        supported.
> -      </p>
> -
> -      <p><em>Symbols</em></p>
> -
> -      <p>
> -        Most of the symbols below have integer type.  Only
> <code>inport</code>
> -        and <code>outport</code> have string type.  <code>inport</code>
> names a
> -        logical port.  Thus, its value is a <ref column="logical_port"/>
> name
> -        from the <ref table="Port_Binding"/> table.  <code>outport</code>
> may
> -        name a logical port, as <code>inport</code>, or a logical
> multicast
> -        group defined in the <ref table="Multicast_Group"/> table.  For
> both
> -        symbols, only names within the flow's logical datapath may be
> used.
> -      </p>
> -
> -      <p>
> -        The <code>reg</code><var>X</var> symbols are 32-bit integers.
> -        The <code>xxreg</code><var>X</var> symbols are 128-bit integers,
> -        which overlay four of the 32-bit registers: <code>xxreg0</code>
> -        overlays <code>reg0</code> through <code>reg3</code>, with
> -        <code>reg0</code> supplying the most-significant bits of
> -        <code>xxreg0</code> and <code>reg3</code> the least-signficant.
> -        <code>xxreg1</code> similarly overlays <code>reg4</code> through
> -        <code>reg7</code>.
> -      </p>
> -
> -      <ul>
> -        <li><code>reg0</code>...<code>reg9</code></li>
> -        <li><code>xxreg0</code> <code>xxreg1</code></li>
> -        <li><code>inport</code> <code>outport</code></li>
> -        <li><code>flags.loopback</code></li>
> -        <li><code>eth.src</code> <code>eth.dst</code>
> <code>eth.type</code></li>
> -        <li><code>vlan.tci</code> <code>vlan.vid</code>
> <code>vlan.pcp</code> <code>vlan.present</code></li>
> -        <li><code>ip.proto</code> <code>ip.dscp</code>
> <code>ip.ecn</code> <code>ip.ttl</code> <code>ip.frag</code></li>
> -        <li><code>ip4.src</code> <code>ip4.dst</code></li>
> -        <li><code>ip6.src</code> <code>ip6.dst</code>
> <code>ip6.label</code></li>
> -        <li><code>arp.op</code> <code>arp.spa</code> <code>arp.tpa</code>
> <code>arp.sha</code> <code>arp.tha</code></li>
> -        <li><code>tcp.src</code> <code>tcp.dst</code>
> <code>tcp.flags</code></li>
> -        <li><code>udp.src</code> <code>udp.dst</code></li>
> -        <li><code>sctp.src</code> <code>sctp.dst</code></li>
> -        <li><code>icmp4.type</code> <code>icmp4.code</code></li>
> -        <li><code>icmp6.type</code> <code>icmp6.code</code></li>
> -        <li><code>nd.target</code> <code>nd.sll</code>
> <code>nd.tll</code></li>
> -        <li><code>ct_mark</code> <code>ct_label</code></li>
> -        <li>
> -          <p>
> -            <code>ct_state</code>, which has several Boolean subfields.
> The
> -            <code>ct_next</code> action initializes the following
> subfields:
> -          </p>
> -          <ul>
> -            <li>
> -              <code>ct.trk</code>: Always set to true by
> <code>ct_next</code>
> -              to indicate that connection tracking has taken place.  All
> other
> -              <code>ct</code> subfields have <code>ct.trk</code> as a
> -              prerequisite.
> -            </li>
> -            <li><code>ct.new</code>: True for a new flow</li>
> -            <li><code>ct.est</code>: True for an established flow</li>
> -            <li><code>ct.rel</code>: True for a related flow</li>
> -            <li><code>ct.rpl</code>: True for a reply flow</li>
> -            <li><code>ct.inv</code>: True for a connection entry in a bad
> state</li>
> -          </ul>
> -          <p>
> -            The <code>ct_dnat</code>, <code>ct_snat</code>, and
> -            <code>ct_lb</code> actions initialize the following subfields:
> -          </p>
> -          <ul>
> -            <li>
> -              <code>ct.dnat</code>: True for a packet whose destination IP
> -              address has been changed.
> -            </li>
> -            <li>
> -              <code>ct.snat</code>: True for a packet whose source IP
> -              address has been changed.
> -            </li>
> -          </ul>
> -        </li>
> -      </ul>
> -
> -      <p>
> -        The following predicates are supported:
> -      </p>
> -
> -      <ul>
> -        <li><code>eth.bcast</code> expands to <code>eth.dst ==
> ff:ff:ff:ff:ff:ff</code></li>
> -        <li><code>eth.mcast</code> expands to
> <code>eth.dst[40]</code></li>
> -        <li><code>vlan.present</code> expands to
> <code>vlan.tci[12]</code></li>
> -        <li><code>ip4</code> expands to <code>eth.type ==
> 0x800</code></li>
> -        <li><code>ip4.mcast</code> expands to <code>ip4.dst[28..31] ==
> 0xe</code></li>
> -        <li><code>ip6</code> expands to <code>eth.type ==
> 0x86dd</code></li>
> -        <li><code>ip</code> expands to <code>ip4 || ip6</code></li>
> -        <li><code>icmp4</code> expands to <code>ip4 &amp;&amp; ip.proto
> == 1</code></li>
> -        <li><code>icmp6</code> expands to <code>ip6 &amp;&amp; ip.proto
> == 58</code></li>
> -        <li><code>icmp</code> expands to <code>icmp4 || icmp6</code></li>
> -        <li><code>ip.is_frag</code> expands to
> <code>ip.frag[0]</code></li>
> -        <li><code>ip.later_frag</code> expands to
> <code>ip.frag[1]</code></li>
> -        <li><code>ip.first_frag</code> expands to <code>ip.is_frag
> &amp;&amp; !ip.later_frag</code></li>
> -        <li><code>arp</code> expands to <code>eth.type ==
> 0x806</code></li>
> -        <li><code>nd</code> expands to <code>icmp6.type == {135, 136}
> &amp;&amp; icmp6.code == 0 &amp;&amp; ip.ttl == 255</code></li>
> -        <li><code>nd_ns</code> expands to <code>icmp6.type == 135
> &amp;&amp; icmp6.code == 0 &amp;&amp; ip.ttl == 255</code></li>
> -        <li><code>nd_na</code> expands to <code>icmp6.type == 136
> &amp;&amp; icmp6.code == 0 &amp;&amp; ip.ttl == 255</code></li>
> -        <li><code>nd_rs</code> expands to <code>icmp6.type == 133
> &amp;&amp;
> -        icmp6.code == 0 &amp;&amp; ip.ttl == 255</code></li>
> -        <li><code>nd_ra</code> expands to <code>icmp6.type == 134
> &amp;&amp;
> -        icmp6.code == 0 &amp;&amp; ip.ttl == 255</code></li>
> -        <li><code>tcp</code> expands to <code>ip.proto == 6</code></li>
> -        <li><code>udp</code> expands to <code>ip.proto == 17</code></li>
> -        <li><code>sctp</code> expands to <code>ip.proto == 132</code></li>
> -      </ul>
> -    </column>
> -
> -    <column name="actions">
> -      <p>
> -        Logical datapath actions, to be executed when the logical flow
> -        represented by this row is the highest-priority match.
> -      </p>
> -
> -      <p>
> -        Actions share lexical syntax with the <ref column="match"/>
> column.  An
> -        empty set of actions (or one that contains just white space or
> -        comments), or a set of actions that consists of just
> -        <code>drop;</code>, causes the matched packets to be dropped.
> -        Otherwise, the column should contain a sequence of actions, each
> -        terminated by a semicolon.
> -      </p>
> -
> -      <p>
> -        The following actions are defined:
> -      </p>
> -
> -      <dl>
> -        <dt><code>output;</code></dt>
> -        <dd>
> -          <p>
> -            In the ingress pipeline, this action executes the
> -            <code>egress</code> pipeline as a subroutine.  If
> -            <code>outport</code> names a logical port, the egress pipeline
> -            executes once; if it is a multicast group, the egress
> pipeline runs
> -            once for each logical port in the group.
> -          </p>
> -
> -          <p>
> -            In the egress pipeline, this action performs the actual
> -            output to the <code>outport</code> logical port.  (In the
> egress
> -            pipeline, <code>outport</code> never names a multicast group.)
> -          </p>
> -
> -          <p>
> -            By default, output to the input port is implicitly dropped,
> -            that is, <code>output</code> becomes a no-op if
> -            <code>outport</code> == <code>inport</code>.  Occasionally
> -            it may be useful to override this behavior, e.g. to send an
> -            ARP reply to an ARP request; to do so, use
> -            <code>flags.loopback = 1</code> to allow the packet to
> -            "hair-pin" back to the input port.
> -          </p>
> -        </dd>
> -
> -        <dt><code>next;</code></dt>
> -        <dt><code>next(<var>table</var>);</code></dt>
> -        <dt><code>next(pipeline=<var>pipeline</var>,
> table=<var>table</var>);</code></dt>
> -        <dd>
> -          Executes the given logical datapath <var>table</var> in
> -          <var>pipeline</var> as a subroutine.  The default
> <var>table</var> is
> -          just after the current one.  If <var>pipeline</var> is
> specified, it
> -          may be <code>ingress</code> or <code>egress</code>; the default
> -          <var>pipeline</var> is the one currently executing.  Actions in
> the
> -          ingress pipeline may not use <code>next</code> to jump into the
> -          egress pipeline (use the <code>output</code> instead), but
> -          transitions in the opposite direction are allowed.
> -        </dd>
> -
> -        <dt><code><var>field</var> = <var>constant</var>;</code></dt>
> -        <dd>
> -          <p>
> -            Sets data or metadata field <var>field</var> to constant value
> -            <var>constant</var>, e.g. <code>outport = "vif0";</code> to
> set the
> -            logical output port.  To set only a subset of bits in a field,
> -            specify a subfield for <var>field</var> or a masked
> -            <var>constant</var>, e.g. one may use <code>vlan.pcp[2] =
> 1;</code>
> -            or <code>vlan.pcp = 4/4;</code> to set the most sigificant
> bit of
> -            the VLAN PCP.
> -          </p>
> -
> -          <p>
> -            Assigning to a field with prerequisites implicitly adds those
> -            prerequisites to <ref column="match"/>; thus, for example, a
> flow
> -            that sets <code>tcp.dst</code> applies only to TCP flows,
> -            regardless of whether its <ref column="match"/> mentions any
> TCP
> -            field.
> -          </p>
> -
> -          <p>
> -            Not all fields are modifiable (e.g. <code>eth.type</code> and
> -            <code>ip.proto</code> are read-only), and not all modifiable
> fields
> -            may be partially modified (e.g. <code>ip.ttl</code> must
> assigned
> -            as a whole).  The <code>outport</code> field is modifiable in
> the
> -            <code>ingress</code> pipeline but not in the
> <code>egress</code>
> -            pipeline.
> -          </p>
> -        </dd>
> -
> -        <dt><code><var>ovn_field</var> = <var>constant</var>;</code></dt>
> -        <dd>
> -          <p>
> -            Sets OVN field <var>ovn_field</var> to constant value
> -            <var>constant</var>.
> -          </p>
> -
> -          <p>
> -            <code>OVN</code> supports setting the values of certain fields
> -            which are not yet supported in OpenFlow to set or modify them.
> -          </p>
> -
> -          <p>
> -            Below are the supported <code>OVN fields</code>:
> -          </p>
> -
> -          <ul>
> -            <li>
> -              <code>icmp4.frag_mtu</code>
> -              <p>
> -                This field sets the low-order 16 bits of the ICMP4 header
> field
> -                that is labelled "unused" in the ICMP specification as
> defined
> -                in the RFC 1191 with the value specified in
> -                <var>constant</var>.
> -              </p>
> -
> -              <p>
> -                Eg. icmp4.frag_mtu = 1500;
> -              </p>
> -            </li>
> -          </ul>
> -        </dd>
> -
> -        <dt><code><var>field1</var> = <var>field2</var>;</code></dt>
> -        <dd>
> -          <p>
> -            Sets data or metadata field <var>field1</var> to the value of
> data
> -            or metadata field <var>field2</var>, e.g. <code>reg0 =
> -            ip4.src;</code> copies <code>ip4.src</code> into
> <code>reg0</code>.
> -            To modify only a subset of a field's bits, specify a subfield
> for
> -            <var>field1</var> or <var>field2</var> or both, e.g.
> <code>vlan.pcp
> -            = reg0[0..2];</code> copies the least-significant bits of
> -            <code>reg0</code> into the VLAN PCP.
> -          </p>
> -
> -          <p>
> -            <var>field1</var> and <var>field2</var> must be the same type,
> -            either both string or both integer fields.  If they are both
> -            integer fields, they must have the same width.
> -          </p>
> -
> -          <p>
> -            If <var>field1</var> or <var>field2</var> has prerequisites,
> they
> -            are added implicitly to <ref column="match"/>.  It is
> possible to
> -            write an assignment with contradictory prerequisites, such as
> -            <code>ip4.src = ip6.src[0..31];</code>, but the contradiction
> means
> -            that a logical flow with such an assignment will never be
> matched.
> -          </p>
> -        </dd>
> -
> -        <dt><code><var>field1</var> &lt;-&gt;
> <var>field2</var>;</code></dt>
> -        <dd>
> -          <p>
> -            Similar to <code><var>field1</var> = <var>field2</var>;</code>
> -            except that the two values are exchanged instead of copied.
> Both
> -            <var>field1</var> and <var>field2</var> must modifiable.
> -          </p>
> -        </dd>
> -
> -        <dt><code>ip.ttl--;</code></dt>
> -        <dd>
> -          <p>
> -            Decrements the IPv4 or IPv6 TTL.  If this would make the TTL
> zero
> -            or negative, then processing of the packet halts; no further
> -            actions are processed.  (To properly handle such cases, a
> -            higher-priority flow should match on
> -            <code>ip.ttl == {0, 1};</code>.)
> -          </p>
> -
> -          <p><b>Prerequisite:</b> <code>ip</code></p>
> -        </dd>
> -
> -        <dt><code>ct_next;</code></dt>
> -        <dd>
> -          <p>
> -            Apply connection tracking to the flow, initializing
> -            <code>ct_state</code> for matching in later tables.
> -            Automatically moves on to the next table, as if followed by
> -            <code>next</code>.
> -          </p>
> -
> -          <p>
> -            As a side effect, IP fragments will be reassembled for
> matching.
> -            If a fragmented packet is output, then it will be sent with
> any
> -            overlapping fragments squashed.  The connection tracking
> state is
> -            scoped by the logical port when the action is used in a flow
> for
> -            a logical switch, so overlapping addresses may be used.  To
> allow
> -            traffic related to the matched flow, execute <code>ct_commit
> -            </code>.  Connection tracking state is scoped by the logical
> -            topology when the action is used in a flow for a router.
> -          </p>
> -
> -          <p>
> -            It is possible to have actions follow <code>ct_next</code>,
> -            but they will not have access to any of its side-effects and
> -            is not generally useful.
> -          </p>
> -        </dd>
> -
> -        <dt><code>ct_commit;</code></dt>
> -        <dt><code>ct_commit(ct_mark=<var>value[/mask]</var>);</code></dt>
> -        <dt><code>ct_commit(ct_label=<var>value[/mask]</var>);</code></dt>
> -        <dt><code>ct_commit(ct_mark=<var>value[/mask]</var>,
> ct_label=<var>value[/mask]</var>);</code></dt>
> -        <dd>
> -          <p>
> -            Commit the flow to the connection tracking entry associated
> with it
> -            by a previous call to <code>ct_next</code>.  When
> -            <code>ct_mark=<var>value[/mask]</var></code> and/or
> -            <code>ct_label=<var>value[/mask]</var></code> are supplied,
> -            <code>ct_mark</code> and/or <code>ct_label</code> will be set
> to the
> -            values indicated by <var>value[/mask]</var> on the connection
> -            tracking entry. <code>ct_mark</code> is a 32-bit field.
> -            <code>ct_label</code> is a 128-bit field. The
> <var>value[/mask]</var>
> -            should be specified in hex string if more than 64bits are to
> be used.
> -          </p>
> -
> -          <p>
> -            Note that if you want processing to continue in the next
> table,
> -            you must execute the <code>next</code> action after
> -            <code>ct_commit</code>.  You may also leave out
> <code>next</code>
> -            which will commit connection tracking state, and then drop the
> -            packet.  This could be useful for setting <code>ct_mark</code>
> -            on a connection tracking entry before dropping a packet,
> -            for example.
> -          </p>
> -        </dd>
> -
> -        <dt><code>ct_dnat;</code></dt>
> -        <dt><code>ct_dnat(<var>IP</var>);</code></dt>
> -        <dd>
> -          <p>
> -            <code>ct_dnat</code> sends the packet through the DNAT zone in
> -            connection tracking table to unDNAT any packet that was
> DNATed in
> -            the opposite direction.  The packet is then automatically
> sent to
> -            to the next tables as if followed by <code>next;</code>
> action.
> -            The next tables will see the changes in the packet caused by
> -            the connection tracker.
> -          </p>
> -          <p>
> -            <code>ct_dnat(<var>IP</var>)</code> sends the packet through
> the
> -            DNAT zone to change the destination IP address of the packet
> to
> -            the one provided inside the parentheses and commits the
> connection.
> -            The packet is then automatically sent to the next tables as if
> -            followed by <code>next;</code> action.  The next tables will
> see
> -            the changes in the packet caused by the connection tracker.
> -          </p>
> -        </dd>
> -
> -        <dt><code>ct_snat;</code></dt>
> -        <dt><code>ct_snat(<var>IP</var>);</code></dt>
> -        <dd>
> -          <p>
> -            <code>ct_snat</code> sends the packet through the SNAT zone to
> -            unSNAT any packet that was SNATed in the opposite direction.
> The
> -            packet is automatically sent to the next tables as if
> followed by
> -            the <code>next;</code> action.   The next tables will see the
> -            changes in the packet caused by the connection tracker.
> -          </p>
> -          <p>
> -            <code>ct_snat(<var>IP</var>)</code> sends the packet through
> the
> -            SNAT zone to change the source IP address of the packet to
> -            the one provided inside the parenthesis and commits the
> connection.
> -            The packet is then automatically sent to the next tables as if
> -            followed by <code>next;</code> action.  The next tables will
> see the
> -            changes in the packet caused by the connection tracker.
> -          </p>
> -        </dd>
> -
> -        <dt><code>ct_clear;</code></dt>
> -        <dd>
> -          Clears connection tracking state.
> -        </dd>
> -
> -        <dt><code>clone { <var>action</var>; </code>...<code>
> };</code></dt>
> -        <dd>
> -          Makes a copy of the packet being processed and executes each
> -          <code>action</code> on the copy.  Actions following the
> -          <var>clone</var> action, if any, apply to the original,
> unmodified
> -          packet.  This can be used as a way to ``save and restore'' the
> packet
> -          around a set of actions that may modify it and should not
> persist.
> -        </dd>
> -
> -        <dt><code>arp { <var>action</var>; </code>...<code> };</code></dt>
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv4 packet being processed by an ARP
> -            packet and executes each nested <var>action</var> on the ARP
> -            packet.  Actions following the <var>arp</var> action, if any,
> apply
> -            to the original, unmodified packet.
> -          </p>
> -
> -          <p>
> -            The ARP packet that this action operates on is initialized
> based on
> -            the IPv4 packet being processed, as follows.  These are
> default
> -            values that the nested actions will probably want to change:
> -          </p>
> -
> -          <ul>
> -            <li><code>eth.src</code> unchanged</li>
> -            <li><code>eth.dst</code> unchanged</li>
> -            <li><code>eth.type = 0x0806</code></li>
> -            <li><code>arp.op = 1</code> (ARP request)</li>
> -            <li><code>arp.sha</code> copied from <code>eth.src</code></li>
> -            <li><code>arp.spa</code> copied from <code>ip4.src</code></li>
> -            <li><code>arp.tha = 00:00:00:00:00:00</code></li>
> -            <li><code>arp.tpa</code> copied from <code>ip4.dst</code></li>
> -          </ul>
> -
> -          <p>
> -            The ARP packet has the same VLAN header, if any, as the IP
> packet
> -            it replaces.
> -          </p>
> -
> -          <p><b>Prerequisite:</b> <code>ip4</code></p>
> -        </dd>
> -
> -        <dt><code>get_arp(<var>P</var>, <var>A</var>);</code></dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: logical port string field <var>P</var>,
> 32-bit
> -            IP address field <var>A</var>.
> -          </p>
> -
> -          <p>
> -            Looks up <var>A</var> in <var>P</var>'s mac binding table.
> -            If an entry is found, stores its Ethernet address in
> -            <code>eth.dst</code>, otherwise stores
> -            <code>00:00:00:00:00:00</code> in <code>eth.dst</code>.
> -          </p>
> -
> -          <p><b>Example:</b> <code>get_arp(outport, ip4.dst);</code></p>
> -        </dd>
> -
> -        <dt>
> -          <code>put_arp(<var>P</var>, <var>A</var>, <var>E</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: logical port string field <var>P</var>,
> 32-bit
> -            IP address field <var>A</var>, 48-bit Ethernet address field
> -            <var>E</var>.
> -          </p>
> -
> -          <p>
> -            Adds or updates the entry for IP address <var>A</var> in
> -            logical port <var>P</var>'s mac binding table, setting its
> -            Ethernet address to <var>E</var>.
> -          </p>
> -
> -          <p><b>Example:</b> <code>put_arp(inport, arp.spa,
> arp.sha);</code></p>
> -        </dd>
> -
> -        <dt><code>nd_ns { <var>action</var>; </code>...<code>
> };</code></dt>
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv6 packet being processed by an
> IPv6
> -            Neighbor Solicitation packet and executes each nested
> -            <var>action</var> on the IPv6 NS packet.  Actions following
> the
> -            <var>nd_ns</var> action, if any, apply to the original,
> unmodified
> -            packet.
> -          </p>
> -
> -          <p>
> -            The IPv6 NS packet that this action operates on is initialized
> -            based on the IPv6 packet being processed, as follows.  These
> are
> -            default values that the nested actions will probably want to
> -            change:
> -          </p>
> -
> -          <ul>
> -            <li><code>eth.src</code> unchanged</li>
> -            <li><code>eth.dst</code> set to IPv6 multicast MAC
> address</li>
> -            <li><code>eth.type = 0x86dd</code></li>
> -            <li><code>ip6.src</code> copied from <code>ip6.src</code></li>
> -            <li>
> -              <code>ip6.dst</code> set to IPv6 Solicited-Node multicast
> address
> -            </li>
> -            <li><code>icmp6.type = 135</code> (Neighbor Solicitation)</li>
> -            <li><code>nd.target</code> copied from
> <code>ip6.dst</code></li>
> -          </ul>
> -
> -          <p>
> -            The IPv6 NS packet has the same VLAN header, if any, as the IP
> -            packet it replaces.
> -          </p>
> -
> -          <p><b>Prerequisite:</b> <code>ip6</code></p>
> -        </dd>
> -
> -        <dt>
> -          <code>nd_na { <var>action</var>; </code>...<code> };</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv6 neighbor solicitation packet
> -            being processed by an IPv6 neighbor advertisement (NA)
> -            packet and executes each nested <var>action</var> on the NA
> -            packet.  Actions following the <code>nd_na</code> action,
> -            if any, apply to the original, unmodified packet.
> -          </p>
> -
> -          <p>
> -            The NA packet that this action operates on is initialized
> based on
> -            the IPv6 packet being processed, as follows. These are default
> -            values that the nested actions will probably want to change:
> -          </p>
> -
> -          <ul>
> -            <li><code>eth.dst</code> exchanged with
> <code>eth.src</code></li>
> -            <li><code>eth.type = 0x86dd</code></li>
> -            <li><code>ip6.dst</code> copied from <code>ip6.src</code></li>
> -            <li><code>ip6.src</code> copied from
> <code>nd.target</code></li>
> -            <li><code>icmp6.type = 136</code> (Neighbor
> Advertisement)</li>
> -            <li><code>nd.target</code> unchanged</li>
> -            <li><code>nd.sll = 00:00:00:00:00:00</code></li>
> -            <li><code>nd.tll</code> copied from <code>eth.dst</code></li>
> -          </ul>
> -
> -          <p>
> -            The ND packet has the same VLAN header, if any, as the IPv6
> packet
> -            it replaces.
> -          </p>
> -
> -          <p>
> -            <b>Prerequisite:</b> <code>nd_ns</code>
> -          </p>
> -        </dd>
> -
> -        <dt>
> -          <code>nd_na_router { <var>action</var>; </code>...<code>
> };</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv6 neighbor solicitation packet
> -            being processed by an IPv6 neighbor advertisement (NA)
> -            packet, sets ND_NSO_ROUTER in the RSO flags and executes each
> -            nested <var>action</var> on the NA packet.  Actions following
> -            the <code>nd_na_router</code> action, if any, apply to the
> -            original, unmodified packet.
> -          </p>
> -
> -          <p>
> -            The NA packet that this action operates on is initialized
> based on
> -            the IPv6 packet being processed, as follows. These are default
> -            values that the nested actions will probably want to change:
> -          </p>
> -
> -          <ul>
> -            <li><code>eth.dst</code> exchanged with
> <code>eth.src</code></li>
> -            <li><code>eth.type = 0x86dd</code></li>
> -            <li><code>ip6.dst</code> copied from <code>ip6.src</code></li>
> -            <li><code>ip6.src</code> copied from
> <code>nd.target</code></li>
> -            <li><code>icmp6.type = 136</code> (Neighbor
> Advertisement)</li>
> -            <li><code>nd.target</code> unchanged</li>
> -            <li><code>nd.sll = 00:00:00:00:00:00</code></li>
> -            <li><code>nd.tll</code> copied from <code>eth.dst</code></li>
> -          </ul>
> -
> -          <p>
> -            The ND packet has the same VLAN header, if any, as the IPv6
> packet
> -            it replaces.
> -          </p>
> -
> -          <p>
> -            <b>Prerequisite:</b> <code>nd_ns</code>
> -          </p>
> -        </dd>
> -
> -        <dt><code>get_nd(<var>P</var>, <var>A</var>);</code></dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: logical port string field <var>P</var>,
> 128-bit
> -            IPv6 address field <var>A</var>.
> -          </p>
> -
> -          <p>
> -            Looks up <var>A</var> in <var>P</var>'s mac binding table.
> -            If an entry is found, stores its Ethernet address in
> -            <code>eth.dst</code>, otherwise stores
> -            <code>00:00:00:00:00:00</code> in <code>eth.dst</code>.
> -          </p>
> -
> -          <p><b>Example:</b> <code>get_nd(outport, ip6.dst);</code></p>
> -        </dd>
> -
> -        <dt>
> -          <code>put_nd(<var>P</var>, <var>A</var>, <var>E</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: logical port string field <var>P</var>,
> -            128-bit IPv6 address field <var>A</var>, 48-bit Ethernet
> -            address field <var>E</var>.
> -          </p>
> -
> -          <p>
> -            Adds or updates the entry for IPv6 address <var>A</var> in
> -            logical port <var>P</var>'s mac binding table, setting its
> -            Ethernet address to <var>E</var>.
> -          </p>
> -
> -          <p><b>Example:</b> <code>put_nd(inport, nd.target,
> nd.tll);</code></p>
> -        </dd>
> -
> -        <dt>
> -          <code><var>R</var> = put_dhcp_opts(<var>D1</var> =
> <var>V1</var>, <var>D2</var> = <var>V2</var>, ..., <var>Dn</var> =
> <var>Vn</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: one or more DHCP option/value pairs, which
> must
> -            include an <code>offerip</code> option (with code 0).
> -          </p>
> -
> -          <p>
> -            <b>Result</b>: stored to a 1-bit subfield <var>R</var>.
> -          </p>
> -
> -          <p>
> -            Valid only in the ingress pipeline.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a DHCP request packet
> (DHCPDISCOVER
> -            or DHCPREQUEST), it changes the packet into a DHCP reply
> (DHCPOFFER
> -            or DHCPACK, respectively), replaces the options by those
> specified
> -            as parameters, and stores 1 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a non-DHCP packet or a DHCP
> packet
> -            that is not DHCPDISCOVER or DHCPREQUEST, it leaves the packet
> -            unchanged and stores 0 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            The contents of the <ref table="DHCP_Option"/> table control
> the
> -            DHCP option names and values that this action supports.
> -          </p>
> -
> -          <p>
> -            <b>Example:</b>
> -            <code>
> -              reg0[0] = put_dhcp_opts(offerip = 10.0.0.2, router =
> 10.0.0.1,
> -              netmask = 255.255.255.0, dns_server = {8.8.8.8, 7.7.7.7});
> -            </code>
> -          </p>
> -        </dd>
> -
> -        <dt>
> -          <code><var>R</var> = put_dhcpv6_opts(<var>D1</var> =
> <var>V1</var>, <var>D2</var> = <var>V2</var>, ..., <var>Dn</var> =
> <var>Vn</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: one or more DHCPv6 option/value pairs.
> -          </p>
> -
> -          <p>
> -            <b>Result</b>: stored to a 1-bit subfield <var>R</var>.
> -          </p>
> -
> -          <p>
> -            Valid only in the ingress pipeline.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a DHCPv6 request packet, it
> changes
> -            the packet into a DHCPv6 reply, replaces the options by those
> -            specified as parameters, and stores 1 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a non-DHCPv6 packet or an
> invalid
> -            DHCPv6 request packet , it leaves the packet unchanged and
> stores
> -            0 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            The contents of the <ref table="DHCPv6_Options"/> table
> control the
> -            DHCPv6 option names and values that this action supports.
> -          </p>
> -
> -          <p>
> -            <b>Example:</b>
> -            <code>
> -              reg0[3] = put_dhcpv6_opts(ia_addr = aef0::4, server_id =
> 00:00:00:00:10:02,
> -              dns_server={ae70::1,ae70::2});
> -            </code>
> -          </p>
> -        </dd>
> -
> -        <dt>
> -          <code>set_queue(<var>queue_number</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: Queue number <var>queue_number</var>, in
> the range 0 to 61440.
> -          </p>
> -
> -          <p>
> -            This is a logical equivalent of the OpenFlow
> <code>set_queue</code>
> -            action.  It affects packets that egress a hypervisor through a
> -            physical interface.  For nonzero <var>queue_number</var>, it
> -            configures packet queuing to match the settings configured
> for the
> -            <ref table="Port_Binding"/> with
> -            <code>options:qdisc_queue_id</code> matching
> -            <var>queue_number</var>.  When <var>queue_number</var> is
> zero, it
> -            resets queuing to the default strategy.
> -          </p>
> -
> -          <p><b>Example:</b> <code>set_queue(10);</code></p>
> -        </dd>
> -
> -        <dt><code>ct_lb;</code></dt>
> -
> <dt><code>ct_lb(</code><var>ip</var>[<code>:</code><var>port</var>]...<code>);</code></dt>
> -        <dd>
> -          <p>
> -            With one or more arguments, <code>ct_lb</code> commits the
> packet
> -            to the connection tracking table and DNATs the packet's
> destination
> -            IP address (and port) to the IP address or addresses (and
> optional
> -            ports) specified in the string.  If multiple comma-separated
> IP
> -            addresses are specified, each is given equal weight for
> picking the
> -            DNAT address.  Processing automatically moves on to the next
> table,
> -            as if <code>next;</code> were specified, and later tables act
> on
> -            the packet as modified by the connection tracker.  Connection
> -            tracking state is scoped by the logical port when the action
> is
> -            used in a flow for a logical switch, so overlapping
> -            addresses may be used.  Connection tracking state is scoped
> by the
> -            logical topology when the action is used in a flow for a
> router.
> -          </p>
> -          <p>
> -            Without arguments, <code>ct_lb</code> sends the packet to the
> -            connection tracking table to NAT the packets.  If the packet
> is
> -            part of an established connection that was previously
> committed to
> -            the connection tracker via
> <code>ct_lb(</code>...<code>)</code>, it
> -            will automatically get DNATed to the same IP address as the
> first
> -            packet in that connection.
> -          </p>
> -        </dd>
> -
> -        <dt>
> -          <code><var>R</var> = dns_lookup();</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: No parameters.
> -          </p>
> -
> -          <p>
> -            <b>Result</b>: stored to a 1-bit subfield <var>R</var>.
> -          </p>
> -
> -          <p>
> -            Valid only in the ingress pipeline.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a valid DNS request (a UDP
> packet
> -            typically directed to port 53), it attempts to resolve the
> query
> -            using the contents of the <ref table="DNS"/> table.  If it is
> -            successful, it changes the packet into a DNS reply and stores
> 1 in
> -            <var>R</var>.  If the action is applied to a non-DNS packet,
> an
> -            invalid DNS request packet, or a valid DNS request for which
> the
> -            <ref table="DNS"/> table does not supply an answer, it leaves
> the
> -            packet unchanged and stores 0 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            Regardless of success, the action does not make any of the
> changes
> -            to the flow that are necessary to direct the packet back to
> the
> -            requester.  The logical pipeline can implement this behavior
> with
> -            matches and actions in later tables.
> -          </p>
> -
> -          <p>
> -            <b>Example:</b>
> -            <code>
> -              reg0[3] = dns_lookup();
> -            </code>
> -          </p>
> -
> -          <p>
> -            <b>Prerequisite:</b> <code>udp</code>
> -          </p>
> -        </dd>
> -
> -        <dt>
> -          <code><var>R</var> = put_nd_ra_opts(<var>D1</var> =
> <var>V1</var>, <var>D2</var> = <var>V2</var>, ..., <var>Dn</var> =
> <var>Vn</var>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: The following IPv6 ND Router Advertisement
> -               option/value pairs as defined in RFC 4861.
> -
> -            <ul>
> -              <li>
> -                <code>addr_mode</code>
> -                <p>
> -                  Mandatory parameter which specifies the address mode
> flag to
> -                  be set in the RA flag options field. The value of this
> option
> -                  is a string and the following values can be defined -
> -                  "slaac", "dhcpv6_stateful" and "dhcpv6_stateless".
> -                </p>
> -              </li>
> -
> -              <li>
> -                <code>slla</code>
> -                <p>
> -                  Mandatory parameter which specifies the link-layer
> address of
> -                  the interface from which the Router Advertisement is
> sent.
> -                </p>
> -              </li>
> -
> -              <li>
> -                <code>mtu</code>
> -                <p>
> -                  Optional parameter which specifies the MTU.
> -                </p>
> -              </li>
> -
> -              <li>
> -                <code>prefix</code>
> -                <p>
> -                  Optional parameter which should be specified if the
> addr_mode
> -                  is "slaac" or "dhcpv6_stateless". The value should be
> an IPv6
> -                  prefix which will be used for stateless IPv6 address
> -                  configuration. This option can be defined multiple
> times.
> -                </p>
> -              </li>
> -            </ul>
> -          </p>
> -
> -          <p>
> -            <b>Result</b>: stored to a 1-bit subfield <var>R</var>.
> -          </p>
> -
> -          <p>
> -            Valid only in the ingress pipeline.
> -          </p>
> -
> -          <p>
> -            When this action is applied to an IPv6 Router solicitation
> request
> -            packet, it changes the packet into an IPv6 Router
> Advertisement
> -            reply and adds the options specified in the parameters, and
> stores
> -            1 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            When this action is applied to a non-IPv6 Router solicitation
> -            packet or an invalid IPv6 request packet , it leaves the
> packet
> -            unchanged and stores 0 in <var>R</var>.
> -          </p>
> -
> -          <p>
> -            <b>Example:</b>
> -            <code>
> -              reg0[3] = put_nd_ra_opts(addr_mode = "slaac",
> -              slla = 00:00:00:00:10:02, prefix = aef0::/64, mtu = 1450);
> -            </code>
> -          </p>
> -        </dd>
> -
> -        <dt><code>set_meter(<var>rate</var>);</code></dt>
> -        <dt><code>set_meter(<var>rate</var>,
> <var>burst</var>);</code></dt>
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: rate limit int field <var>rate</var> in
> kbps,
> -            burst rate limits int field <var>burst</var> in kbps.
> -          </p>
> -
> -          <p>
> -            This action sets the rate limit for a flow.
> -          </p>
> -
> -          <p><b>Example:</b> <code>set_meter(100, 1000);</code></p>
> -        </dd>
> -
> -        <dt><code><var>R</var> =
> check_pkt_larger(<var>L</var>)</code></dt>
> -        <dd>
> -          <p>
> -            <b>Parameters</b>: packet length <var>L</var> to check for
> -            in bytes.
> -          </p>
> -
> -          <p>
> -            <b>Result</b>: stored to a 1-bit subfield <var>R</var>.
> -          </p>
> -
> -          <p>
> -            This is a logical equivalent of the OpenFlow
> -            <code>check_pkt_larger</code> action. If the packet is larger
> -            than the length specified in <var>L</var>, it stores 1 in the
> -            subfield <var>R</var>.
> -          </p>
> -
> -          <p><b>Example: </b><code>reg0[6] =
> check_pkt_larger(1000);</code></p>
> -        </dd>
> -      </dl>
> -
> -      <dl>
> -        <dt>
> -          <code>log(<var>key</var>=<var>value</var>,
> </code>...<code>);</code>
> -        </dt>
> -
> -        <dd>
> -          <p>
> -            Causes <code>ovn-controller</code> to log the packet on the
> chassis
> -            that processes it.  Packet logging currently uses the same
> logging
> -            mechanism as other Open vSwitch and OVN messages, which means
> that
> -            whether and where log messages appear depends on the local
> logging
> -            configuration that can be configured with
> <code>ovs-appctl</code>,
> -            etc.
> -          </p>
> -          <p>
> -            The <code>log</code> action takes zero or more of the
> following
> -            key-value pair arguments that control what is logged:
> -          </p>
> -          <dl>
> -            <dt><code>name=</code><var>string</var></dt>
> -            <dd>
> -              An optional name for the ACL.  The <var>string</var> is
> -              currently limited to 64 bytes.
> -            </dd>
> -            <dt><code>severity=</code><var>level</var></dt>
> -            <dd>
> -              Indicates the severity of the event.  The <var>level</var>
> is one
> -              of following (from more to less serious):
> <code>alert</code>,
> -              <code>warning</code>, <code>notice</code>,
> <code>info</code>, or
> -              <code>debug</code>.  If a severity is not provided, the
> default
> -              is <code>info</code>.
> -            </dd>
> -            <dt><code>verdict=</code><var>value</var></dt>
> -            <dd>
> -              The verdict for packets matching the flow.  The value must
> be one
> -              of <code>allow</code>, <code>deny</code>, or
> <code>reject</code>.
> -            </dd>
> -            <dt><code>meter=</code><var>string</var></dt>
> -            <dd>
> -              An optional rate-limiting meter to be applied to the logs.
> -              The <var>string</var> should reference a
> -              <ref column="name" table="Meter"/> entry from the
> -              <ref table="Meter"/> table.  The only meter
> -              <ref column="action" table="meter"/> that is appriopriate
> -              is <code>drop</code>.
> -            </dd>
> -          </dl>
> -        </dd>
> -      </dl>
> -
> -      <dl>
> -        <dt><code>icmp4 { <var>action</var>; </code>...<code>
> };</code></dt>
> -        <dt>
> -          <code>icmp4_error { <var>action</var>; </code>...<code>
> };</code>
> -        </dt>
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv4 packet being processed by an
> ICMPv4
> -            packet and executes each nested <var>action</var> on the
> ICMPv4
> -            packet.  Actions following these actions, if any,
> -            apply to the original, unmodified packet.
> -          </p>
> -
> -          <p>
> -            The ICMPv4 packet that these actions operates on is
> initialized
> -            based on the IPv4 packet being processed, as follows.  These
> are
> -            default values that the nested actions will probably want to
> -            change. Ethernet and IPv4 fields not listed here are not
> changed:
> -          </p>
> -
> -          <ul>
> -            <li><code>ip.proto = 1</code> (ICMPv4)</li>
> -            <li><code>ip.frag = 0</code> (not a fragment)</li>
> -            <li><code>ip.ttl = 255</code></li>
> -            <li><code>icmp4.type = 3</code> (destination unreachable)</li>
> -            <li><code>icmp4.code = 1</code> (host unreachable)</li>
> -          </ul>
> -
> -          <p>
> -              <code>icmp4_error</code> action is expected to be used to
> -              generate an ICMPv4 packet in response to an error in
> original
> -              IP packet. When this action generates the ICMPv4 packet, it
> -              also copies the original IP datagram following the ICMPv4
> header
> -              as per RFC 1122: 3.2.2.
> -          </p>
> -          <p><b>Prerequisite:</b> <code>ip4</code></p>
> -        </dd>
> -
> -        <dt><code>icmp6 { <var>action</var>; </code>...<code>
> };</code></dt>
> -        <dd>
> -          <p>
> -            Temporarily replaces the IPv6 packet being processed by an
> ICMPv6
> -            packet and executes each nested <var>action</var> on the
> ICMPv6
> -            packet. Actions following the <var>icmp6</var> action, if any,
> -            apply to the original, unmodified packet.
> -          </p>
> -
> -          <p>
> -            The ICMPv6 packet that this action operates on is initialized
> based
> -            on the IPv6 packet being processed, as follows. These are
> default
> -            values that the nested actions will probably want to change.
> -            Ethernet and IPv6 fields not listed here are not changed:
> -          </p>
> -
> -          <ul>
> -            <li><code>ip.proto = 58</code> (ICMPv6)</li>
> -            <li><code>ip.ttl = 255</code></li>
> -            <li><code>icmp6.type = 1</code> (destination unreachable)</li>
> -            <li><code>icmp6.code = 1</code> (administratively
> prohibited)</li>
> -          </ul>
> -
> -          <p><b>Prerequisite:</b> <code>ip6</code></p>
> -        </dd>
> -
> -        <dt><code>tcp_reset;</code></dt>
> -        <dd>
> -          <p>
> -            This action transforms the current TCP packet according to the
> -            following pseudocode:
> -          </p>
> -
> -          <pre>
> -if (tcp.ack) {
> -        tcp.seq = tcp.ack;
> -} else {
> -        tcp.ack = tcp.seq + length(tcp.payload);
> -        tcp.seq = 0;
> -}
> -tcp.flags = RST;
> -</pre>
> -
> -          <p>
> -            Then, the action drops all TCP options and payload data, and
> -            updates the TCP checksum. IP ttl is set to 255.
> -          </p>
> -
> -          <p><b>Prerequisite:</b> <code>tcp</code></p>
> -        </dd>
> -
> -        <dt><code>trigger_event;</code></dt>
> -        <dd>
> -          <p>
> -            This action is used to allow ovs-vswitchd to report CMS
> related
> -            events writing them in <ref table="Controller_Event"/> table.
> -            Supported event:
> -          </p>
> -
> -          <ul>
> -            <li>
> -              <p>
> -                <dfn>empty_lb_backends</dfn>. This event is raised if a
> -                received packet is destined for a load balancer VIP that
> has
> -                no configured backend destinations. For this event, the
> event
> -                info includes the load balancer VIP, the load balancer
> UUID,
> -                and the transport protocol.
> -              </p>
> -            </li>
> -          </ul>
> -        </dd>
> -        <dt><code>igmp;</code></dt>
> -        <dd>
> -          <p>
> -            This action sends the packet to <code>ovn-controller</code>
> for
> -            multicast snooping.
> -          </p>
> -          <p><b>Prerequisite:</b> <code>igmp</code></p>
> -        </dd>
> -      </dl>
> -    </column>
> -
> -    <column name="external_ids" key="stage-name">
> -      Human-readable name for this flow's stage in the pipeline.
> -    </column>
> -
> -    <column name="external_ids" key="stage-hint" type='{"type": "uuid"}'>
> -      UUID of a <ref db="OVN_Northbound"/> record that caused this
> logical flow
> -      to be created.  Currently used only for attribute of logical flows
> to
> -      northbound <ref db="OVN_Northbound" table="ACL"/> records.
> -    </column>
> -
> -    <column name="external_ids" key="source">
> -      Source file and line number of the code that added this flow to the
> -      pipeline.
> -    </column>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -  </table>
> -
> -  <table name="Multicast_Group" title="Logical Port Multicast Groups">
> -    <p>
> -      The rows in this table define multicast groups of logical ports.
> -      Multicast groups allow a single packet transmitted over a tunnel to
> a
> -      hypervisor to be delivered to multiple VMs on that hypervisor, which
> -      uses bandwidth more efficiently.
> -    </p>
> -
> -    <p>
> -      Each row in this table defines a logical multicast group numbered
> <ref
> -      column="tunnel_key"/> within <ref column="datapath"/>, whose logical
> -      ports are listed in the <ref column="ports"/> column.
> -    </p>
> -
> -    <column name="datapath">
> -      The logical datapath in which the multicast group resides.
> -    </column>
> -
> -    <column name="tunnel_key">
> -      The value used to designate this logical egress port in tunnel
> -      encapsulations.  An index forces the key to be unique within the
> <ref
> -      column="datapath"/>.  The unusual range ensures that multicast
> group IDs
> -      do not overlap with logical port IDs.
> -    </column>
> -
> -    <column name="name">
> -      <p>
> -        The logical multicast group's name.  An index forces the name to
> be
> -        unique within the <ref column="datapath"/>.  Logical flows in the
> -        ingress pipeline may output to the group just as for individual
> logical
> -        ports, by assigning the group's name to <code>outport</code> and
> -        executing an <code>output</code> action.
> -      </p>
> -
> -      <p>
> -        Multicast group names and logical port names share a single
> namespace
> -        and thus should not overlap (but the database schema cannot
> enforce
> -        this).  To try to avoid conflicts, <code>ovn-northd</code> uses
> names
> -        that begin with <code>_MC_</code>.
> -      </p>
> -    </column>
> -
> -    <column name="ports">
> -      The logical ports included in the multicast group.  All of these
> ports
> -      must be in the <ref column="datapath"/> logical datapath (but the
> -      database schema cannot enforce this).
> -    </column>
> -  </table>
> -
> -  <table name="Meter" title="Meter entry">
> -    <p>
> -      Each row in this table represents a meter that can be used for QoS
> or
> -      rate-limiting.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        A name for this meter.
> -      </p>
> -
> -      <p>
> -        Names that begin with "__" (two underscores) are reserved for
> -        OVN internal use and should not be added manually.
> -      </p>
> -    </column>
> -
> -    <column name="unit">
> -      <p>
> -        The unit for <ref column="rate" table="Meter_Band"/> and
> -        <ref column="burst_rate" table="Meter_Band"/> parameters in
> -        the <ref column="bands"/> entry.  <code>kbps</code> specifies
> -        kilobits per second, and <code>pktps</code> specifies packets
> -        per second.
> -      </p>
> -    </column>
> -
> -    <column name="bands">
> -      <p>
> -        The bands associated with this meter.  Each band specifies a
> -        rate above which the band is to take the action
> -        <code>action</code>.  If multiple bands' rates are exceeded,
> -        then the band with the highest rate among the exceeded bands is
> -        selected.
> -      </p>
> -    </column>
> -  </table>
> -
> -  <table name="Meter_Band" title="Band for meter entries">
> -    <p>
> -      Each row in this table represents a meter band which specifies the
> -      rate above which the configured action should be applied.  These
> bands
> -      are referenced by the <ref column="bands" table="Meter"/> column in
> -      the <ref table="Meter"/> table.
> -    </p>
> -
> -    <column name="action">
> -      <p>
> -        The action to execute when this band matches.  The only supported
> -        action is <code>drop</code>.
> -      </p>
> -    </column>
> -
> -    <column name="rate">
> -      <p>
> -        The rate limit for this band, in kilobits per second or bits per
> -        second, depending on whether the parent <ref table="Meter"/>
> -        entry's <ref column="unit" table="Meter"/> column specified
> -        <code>kbps</code> or <code>pktps</code>.
> -      </p>
> -    </column>
> -
> -    <column name="burst_size">
> -      <p>
> -        The maximum burst allowed for the band in kilobits or packets,
> -        depending on whether <code>kbps</code> or <code>pktps</code> was
> -        selected in the parent <ref table="Meter"/> entry's
> -        <ref column="unit" table="Meter"/> column.  If the size is zero,
> -        the switch is free to select some reasonable value depending on
> -        its configuration.
> -      </p>
> -    </column>
> -  </table>
> -
> -  <table name="Datapath_Binding" title="Physical-Logical Datapath
> Bindings">
> -    <p>
> -      Each row in this table represents a logical datapath, which
> implements a
> -      logical pipeline among the ports in the <ref table="Port_Binding"/>
> table
> -      associated with it.  In practice, the pipeline in a given logical
> -      datapath implements either a logical switch or a logical router.
> -    </p>
> -
> -    <p>
> -      The main purpose of a row in this table is provide a physical
> binding for
> -      a logical datapath.  A logical datapath does not have a physical
> -      location, so its physical binding information is limited: just <ref
> -      column="tunnel_key"/>.  The rest of the data in this table does not
> -      affect packet forwarding.
> -    </p>
> -
> -    <column name="tunnel_key">
> -      The tunnel key value to which the logical datapath is bound.
> -      The <code>Tunnel Encapsulation</code> section in
> -      <code>ovn-architecture</code>(7) describes how tunnel keys are
> -      constructed for each supported encapsulation.
> -    </column>
> -
> -    <group title="OVN_Northbound Relationship">
> -      <p>
> -        Each row in <ref table="Datapath_Binding"/> is associated with
> some
> -        logical datapath.  <code>ovn-northd</code> uses these keys to
> track the
> -        association of a logical datapath with concepts in the <ref
> -        db="OVN_Northbound"/> database.
> -      </p>
> -
> -      <column name="external_ids" key="logical-switch" type='{"type":
> "uuid"}'>
> -        For a logical datapath that represents a logical switch,
> -        <code>ovn-northd</code> stores in this key the UUID of the
> -        corresponding <ref table="Logical_Switch" db="OVN_Northbound"/>
> row in
> -        the <ref db="OVN_Northbound"/> database.
> -      </column>
> -
> -      <column name="external_ids" key="logical-router" type='{"type":
> "uuid"}'>
> -        For a logical datapath that represents a logical router,
> -        <code>ovn-northd</code> stores in this key the UUID of the
> -        corresponding <ref table="Logical_Router" db="OVN_Northbound"/>
> row in
> -        the <ref db="OVN_Northbound"/> database.
> -      </column>
> -
> -      <group title="Naming">
> -        <p>
> -          <code>ovn-northd</code> copies these from the name fields in
> the <ref
> -          db="OVN_Northbound"/> database, either from <ref
> -          table="Logical_Router" db="OVN_Northbound" column="name"/> and
> <ref
> -          table="Logical_Router" db="OVN_Northbound" column="external_ids"
> -          key="neutron:router_name"/> in the <ref table="Logical_Router"
> -          db="OVN_Northbound"/> table or from <ref table="Logical_Switch"
> -          db="OVN_Northbound" column="name"/> and <ref
> table="Logical_Switch"
> -          db="OVN_Northbound" column="external_ids"
> -          key="neutron:network_name"/> in the <ref table="Logical_Switch"
> -          db="OVN_Northbound"/> table.
> -        </p>
> -
> -        <column name="external_ids" key="name">
> -          A name for the logical datapath.
> -        </column>
> -
> -        <column name="external_ids" key="name2">
> -          Another name for the logical datapath.
> -        </column>
> -      </group>
> -    </group>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -  </table>
> -
> -  <table name="Port_Binding" title="Physical-Logical Port Bindings">
> -    <p>
> -      Each row in this table binds a logical port to a realization.  For
> most
> -      logical ports, this means binding to some physical location, for
> example
> -      by binding a logical port to a VIF that belongs to a VM running on a
> -      particular hypervisor.  Other logical ports, such as logical patch
> ports,
> -      can be realized without a specific physical location, but their
> bindings
> -      are still expressed through rows in this table.
> -    </p>
> -
> -    <p>
> -      For every <code>Logical_Switch_Port</code> record in
> -      <code>OVN_Northbound</code> database, <code>ovn-northd</code>
> -      creates a record in this table.  <code>ovn-northd</code> populates
> -      and maintains every column except the <code>chassis</code> column,
> -      which it leaves empty in new records.
> -    </p>
> -
> -    <p>
> -      <code>ovn-controller</code>/<code>ovn-controller-vtep</code>
> -      populates the <code>chassis</code> column for the records that
> -      identify the logical ports that are located on its
> hypervisor/gateway,
> -      which <code>ovn-controller</code>/<code>ovn-controller-vtep</code>
> in
> -      turn finds out by monitoring the local hypervisor's Open_vSwitch
> -      database, which identifies logical ports via the conventions
> described
> -      in <code>IntegrationGuide.rst</code>.  (The exceptions are for
> -      <code>Port_Binding</code> records with <code>type</code> of
> -      <code>l3gateway</code>, whose locations are identified by
> -      <code>ovn-northd</code> via the
> <code>options:l3gateway-chassis</code>
> -      column in this table.  <code>ovn-controller</code> is still
> responsible
> -      to populate the <code>chassis</code> column.)
> -    </p>
> -
> -    <p>
> -      When a chassis shuts down gracefully, it should clean up the
> -      <code>chassis</code> column that it previously had populated.
> -      (This is not critical because resources hosted on the chassis are
> equally
> -      unreachable regardless of whether their rows are present.)  To
> handle the
> -      case where a VM is shut down abruptly on one chassis, then brought
> up
> -      again on a different one,
> -      <code>ovn-controller</code>/<code>ovn-controller-vtep</code> must
> -      overwrite the <code>chassis</code> column with new information.
> -    </p>
> -
> -    <group title="Core Features">
> -      <column name="datapath">
> -        The logical datapath to which the logical port belongs.
> -      </column>
> -
> -      <column name="logical_port">
> -        A logical port, taken from <ref table="Logical_Switch_Port"
> -        column="name" db="OVN_Northbound"/> in the OVN_Northbound
> -        database's <ref table="Logical_Switch_Port" db="OVN_Northbound"/>
> -        table.  OVN does not prescribe a particular format for the
> -        logical port ID.
> -      </column>
> -
> -      <column name="encap">
> -        Points to supported encapsulation configurations to transmit
> -        logical dataplane packets to this chassis.  Each entry is a <ref
> -        table="Encap"/> record that describes the configuration.
> -      </column>
> -
> -      <column name="chassis">
> -        The meaning of this column depends on the value of the <ref
> column="type"/>
> -        column.  This is the meaning for each <ref column="type"/>
> -
> -        <dl>
> -          <dt>(empty string)</dt>
> -          <dd>
> -            The physical location of the logical port.  To successfully
> identify a
> -            chassis, this column must be a <ref table="Chassis"/>
> record.  This is
> -            populated by <code>ovn-controller</code>.
> -          </dd>
> -
> -          <dt>vtep</dt>
> -          <dd>
> -            The physical location of the hardware_vtep gateway.  To
> successfully
> -            identify a chassis, this column must be a <ref
> table="Chassis"/> record.
> -            This is populated by <code>ovn-controller-vtep</code>.
> -          </dd>
> -
> -          <dt>localnet</dt>
> -          <dd>
> -            Always empty.  A localnet port is realized on every chassis
> that has
> -            connectivity to the corresponding physical network.
> -          </dd>
> -
> -          <dt>localport</dt>
> -          <dd>
> -            Always empty.  A localport port is present on every chassis.
> -          </dd>
> -
> -          <dt>l3gateway</dt>
> -          <dd>
> -            The physical location of the L3 gateway.  To successfully
> identify a
> -            chassis, this column must be a <ref table="Chassis"/>
> record.  This is
> -            populated by <code>ovn-controller</code> based on the value of
> -            the <code>options:l3gateway-chassis</code> column in this
> table.
> -          </dd>
> -
> -          <dt>l2gateway</dt>
> -          <dd>
> -            The physical location of this L2 gateway.  To successfully
> identify a
> -            chassis, this column must be a <ref table="Chassis"/> record.
> -            This is populated by <code>ovn-controller</code> based on the
> value
> -            of the <code>options:l2gateway-chassis</code> column in this
> table.
> -          </dd>
> -        </dl>
> -
> -      </column>
> -
> -      <column name="gateway_chassis">
> -        <p>
> -          A list of <ref table="Gateway_Chassis"/>.
> -        </p>
> -        <p>
> -          This should only be populated for ports with
> -          <ref column="type"/> set to <code>chassisredirect</code>.
> -          This column defines the list of chassis used as gateways where
> -          traffic will be redirected through.
> -        </p>
> -      </column>
> -
> -      <column name="ha_chassis_group">
> -        <p>
> -          This should only be populated for ports with
> -          <ref column="type"/> set to <code>chassisredirect</code>.
> -          This column defines the HA chassis group with a list of
> -          HA chassis used as gateways where traffic will be redirected
> -          through.
> -        </p>
> -      </column>
> -
> -      <column name="tunnel_key">
> -        <p>
> -          A number that represents the logical port in the key (e.g. STT
> key or
> -          Geneve TLV) field carried within tunnel protocol packets.
> -        </p>
> -
> -        <p>
> -          The tunnel ID must be unique within the scope of a logical
> datapath.
> -        </p>
> -      </column>
> -
> -      <column name="mac">
> -        <p>
> -          The Ethernet address or addresses used as a source address on
> the
> -          logical port, each in the form
> -
> <var>xx</var>:<var>xx</var>:<var>xx</var>:<var>xx</var>:<var>xx</var>:<var>xx</var>.
> -          The string <code>unknown</code> is also allowed to indicate
> that the
> -          logical port has an unknown set of (additional) source
> addresses.
> -        </p>
> -
> -        <p>
> -          A VM interface would ordinarily have a single Ethernet
> address.  A
> -          gateway port might initially only have <code>unknown</code>,
> and then
> -          add MAC addresses to the set as it learns new source addresses.
> -        </p>
> -      </column>
> -
> -      <column name="type">
> -        <p>
> -          A type for this logical port.  Logical ports can be used to
> model other
> -          types of connectivity into an OVN logical switch.  The
> following types
> -          are defined:
> -        </p>
> -
> -        <dl>
> -          <dt>(empty string)</dt>
> -          <dd>VM (or VIF) interface.</dd>
> -
> -          <dt><code>patch</code></dt>
> -          <dd>
> -            One of a pair of logical ports that act as if connected by a
> patch
> -            cable.  Useful for connecting two logical datapaths, e.g. to
> connect
> -            a logical router to a logical switch or to another logical
> router.
> -          </dd>
> -
> -          <dt><code>l3gateway</code></dt>
> -          <dd>
> -            One of a pair of logical ports that act as if connected by a
> patch
> -            cable across multiple chassis.  Useful for connecting a
> logical
> -            switch with a Gateway router (which is only resident on a
> -            particular chassis).
> -          </dd>
> -
> -          <dt><code>localnet</code></dt>
> -          <dd>
> -            A connection to a locally accessible network from each
> -            <code>ovn-controller</code> instance.  A logical switch can
> only
> -            have a single <code>localnet</code> port attached.  This is
> used
> -            to model direct connectivity to an existing network.
> -          </dd>
> -
> -          <dt><code>localport</code></dt>
> -          <dd>
> -            A connection to a local VIF. Traffic that arrives on a
> -            <code>localport</code> is never forwarded over a tunnel to
> another
> -            chassis. These ports are present on every chassis and have
> the same
> -            address in all of them. This is used to model connectivity to
> local
> -            services that run on every hypervisor.
> -          </dd>
> -
> -          <dt><code>l2gateway</code></dt>
> -          <dd>
> -            An L2 connection to a physical network.  The chassis this
> -            <ref table="Port_Binding"/> is bound to will serve as
> -            an L2 gateway to the network named by
> -            <ref column="options"
> table="Port_Binding"/>:<code>network_name</code>.
> -          </dd>
> -
> -          <dt><code>vtep</code></dt>
> -          <dd>
> -            A port to a logical switch on a VTEP gateway chassis.  In
> order to
> -            get this port correctly recognized by the OVN controller, the
> <ref
> -            column="options"
> -            table="Port_Binding"/>:<code>vtep-physical-switch</code> and
> <ref
> -            column="options"
> -            table="Port_Binding"/>:<code>vtep-logical-switch</code> must
> also
> -            be defined.
> -          </dd>
> -
> -          <dt><code>chassisredirect</code></dt>
> -          <dd>
> -            A logical port that represents a particular instance, bound
> -            to a specific chassis, of an otherwise distributed parent
> -            port (e.g. of type <code>patch</code>).  A
> -            <code>chassisredirect</code> port should never be used as an
> -            <code>inport</code>.  When an ingress pipeline sets the
> -            <code>outport</code>, it may set the value to a logical port
> -            of type <code>chassisredirect</code>.  This will cause the
> -            packet to be directed to a specific chassis to carry out the
> -            egress pipeline.  At the beginning of the egress pipeline,
> -            the <code>outport</code> will be reset to the value of the
> -            distributed port.
> -          </dd>
> -        </dl>
> -      </column>
> -    </group>
> -
> -    <group title="Patch Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/> of
> -        <code>patch</code>.
> -      </p>
> -
> -      <column name="options" key="peer">
> -        The <ref column="logical_port"/> in the <ref
> table="Port_Binding"/>
> -        record for the other side of the patch.  The named <ref
> -        column="logical_port"/> must specify this <ref
> column="logical_port"/>
> -        in its own <code>peer</code> option.  That is, the two patch
> logical
> -        ports must have reversed <ref column="logical_port"/> and
> -        <code>peer</code> values.
> -      </column>
> -
> -      <column name="nat_addresses">
> -        MAC address followed by a list of SNAT and DNAT external IP
> -        addresses, followed by
> -        <code>is_chassis_resident("<var>lport</var>")</code>, where
> -        <var>lport</var> is the name of a logical port on the same chassis
> -        where the corresponding NAT rules are applied.  This is used to
> -        send gratuitous ARPs for SNAT and DNAT external IP addresses via
> -        <code>localnet</code>, from the chassis where <var>lport</var>
> -        resides.  Example: <code>80:fa:5b:06:72:b7 158.36.44.22
> -        158.36.44.24 is_chassis_resident("foo1")</code>.  This would
> result
> -        in generation of gratuitous ARPs for IP addresses 158.36.44.22 and
> -        158.36.44.24 with a MAC address of 80:fa:5b:06:72:b7 from the
> chassis
> -        where the logical port "foo1" resides.
> -      </column>
> -    </group>
> -
> -    <group title="L3 Gateway Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/> of
> -        <code>l3gateway</code>.
> -      </p>
> -
> -      <column name="options" key="peer">
> -        The <ref column="logical_port"/> in the <ref
> table="Port_Binding"/>
> -        record for the other side of the 'l3gateway' port.  The named <ref
> -        column="logical_port"/> must specify this <ref
> column="logical_port"/>
> -        in its own <code>peer</code> option.  That is, the two 'l3gateway'
> -        logical ports must have reversed <ref column="logical_port"/> and
> -        <code>peer</code> values.
> -      </column>
> -
> -      <column name="options" key="l3gateway-chassis">
> -        The <code>chassis</code> in which the port resides.
> -      </column>
> -
> -      <column name="options" key="nat-addresses">
> -        MAC address of the <code>l3gateway</code> port followed by a list
> of
> -        SNAT and DNAT external IP addresses.  This is used to send
> gratuitous
> -        ARPs for SNAT and DNAT external IP addresses via
> <code>localnet</code>.
> -        Example: <code>80:fa:5b:06:72:b7 158.36.44.22 158.36.44.24</code>.
> -        This would result in generation of gratuitous ARPs for IP
> addresses
> -        158.36.44.22 and 158.36.44.24 with a MAC address of
> 80:fa:5b:06:72:b7.
> -        This is used in OVS versions prior to 2.8.
> -      </column>
> -
> -      <column name="nat_addresses">
> -        MAC address of the <code>l3gateway</code> port followed by a list
> of
> -        SNAT and DNAT external IP addresses.  This is used to send
> gratuitous
> -        ARPs for SNAT and DNAT external IP addresses via
> <code>localnet</code>.
> -        Example: <code>80:fa:5b:06:72:b7 158.36.44.22 158.36.44.24</code>.
> -        This would result in generation of gratuitous ARPs for IP
> addresses
> -        158.36.44.22 and 158.36.44.24 with a MAC address of
> 80:fa:5b:06:72:b7.
> -        This is used in OVS version 2.8 and later versions.
> -      </column>
> -    </group>
> -
> -    <group title="Localnet Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/> of
> -        <code>localnet</code>.
> -      </p>
> -
> -      <column name="options" key="network_name">
> -        Required.  <code>ovn-controller</code> uses the configuration
> entry
> -        <code>ovn-bridge-mappings</code> to determine how to connect to
> this
> -        network.  <code>ovn-bridge-mappings</code> is a list of network
> names
> -        mapped to a local OVS bridge that provides access to that
> network.  An
> -        example of configuring <code>ovn-bridge-mappings</code> would be:
> -
> -        <pre>$ ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0,physnet2:br-eth1</pre>
> -
> -        <p>
> -          When a logical switch has a <code>localnet</code> port attached,
> -          every chassis that may have a local vif attached to that logical
> -          switch must have a bridge mapping configured to reach that
> -          <code>localnet</code>.  Traffic that arrives on a
> -          <code>localnet</code> port is never forwarded over a tunnel to
> -          another chassis.
> -        </p>
> -      </column>
> -
> -      <column name="tag">
> -        If set, indicates that the port represents a connection to a
> specific
> -        VLAN on a locally accessible network. The VLAN ID is used to match
> -        incoming traffic and is also added to outgoing traffic.
> -      </column>
> -    </group>
> -
> -    <group title="L2 Gateway Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/> of
> -        <code>l2gateway</code>.
> -      </p>
> -
> -      <column name="options" key="network_name">
> -        Required.  <code>ovn-controller</code> uses the configuration
> entry
> -        <code>ovn-bridge-mappings</code> to determine how to connect to
> this
> -        network.  <code>ovn-bridge-mappings</code> is a list of network
> names
> -        mapped to a local OVS bridge that provides access to that
> network.  An
> -        example of configuring <code>ovn-bridge-mappings</code> would be:
> -
> -        <pre>$ ovs-vsctl set open .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0,physnet2:br-eth1</pre>
> -
> -        <p>
> -          When a logical switch has a <code>l2gateway</code> port
> attached,
> -          the chassis that the <code>l2gateway</code> port is bound to
> -          must have a bridge mapping configured to reach the network
> -          identified by <code>network_name</code>.
> -        </p>
> -      </column>
> -
> -      <column name="options" key="l2gateway-chassis">
> -        Required. The <code>chassis</code> in which the port resides.
> -      </column>
> -
> -      <column name="tag">
> -        If set, indicates that the gateway is connected to a specific
> -        VLAN on the physical network. The VLAN ID is used to match
> -        incoming traffic and is also added to outgoing traffic.
> -      </column>
> -    </group>
> -
> -    <group title="VTEP Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/> of
> -        <code>vtep</code>.
> -      </p>
> -
> -      <column name="options" key="vtep-physical-switch">
> -        Required. The name of the VTEP gateway.
> -      </column>
> -
> -      <column name="options" key="vtep-logical-switch">
> -        Required.  A logical switch name connected by the VTEP gateway.
> Must
> -        be set when <ref column="type"/> is <code>vtep</code>.
> -      </column>
> -    </group>
> -
> -    <group title="VMI (or VIF) Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/>
> having
> -        (empty string)
> -      </p>
> -
> -      <column name="options" key="requested-chassis">
> -        If set, identifies a specific chassis (by name or hostname) that
> -        is allowed to bind this port. Using this option will prevent
> -        thrashing between two chassis trying to bind the same port during
> -        a live migration. It can also prevent similar thrashing due to a
> -        mis-configuration, if a port is accidentally created on more than
> -        one chassis.
> -      </column>
> -
> -      <column name="options" key="qos_max_rate">
> -        If set, indicates the maximum rate for data sent from this
> interface,
> -        in bit/s. The traffic will be shaped according to this limit.
> -      </column>
> -
> -      <column name="options" key="qos_burst">
> -        If set, indicates the maximum burst size for data sent from this
> -        interface, in bits.
> -      </column>
> -
> -      <column name="options" key="qdisc_queue_id"
> -              type='{"type": "integer", "minInteger": 1, "maxInteger":
> 61440}'>
> -        Indicates the queue number on the physical device. This is same
> as the
> -        <code>queue_id</code> used in OpenFlow in <code>struct
> -        ofp_action_enqueue</code>.
> -      </column>
> -    </group>
> -
> -    <group title="Chassis Redirect Options">
> -      <p>
> -        These options apply to logical ports with <ref column="type"/>
> -        of <code>chassisredirect</code>.
> -      </p>
> -
> -      <column name="options" key="distributed-port">
> -        The name of the distributed port for which this
> -        <code>chassisredirect</code> port represents a particular
> instance.
> -      </column>
> -
> -      <column name="options" key="redirect-chassis">
> -        The <code>chassis</code> that this <code>chassisredirect</code>
> port
> -        is bound to.  This is taken from <ref table="Logical_Router_Port"
> -        column="options" key="redirect-chassis" db="OVN_Northbound"/>
> -        in the OVN_Northbound database's <ref table="Logical_Router_Port"
> -        db="OVN_Northbound"/> table.
> -      </column>
> -    </group>
> -
> -    <group title="Nested Containers">
> -      <p>
> -        These columns support containers nested within a VM.
> Specifically,
> -        they are used when <ref column="type"/> is empty and <ref
> -        column="logical_port"/> identifies the interface of a container
> spawned
> -        inside a VM.  They are empty for containers or VMs that run
> directly on
> -        a hypervisor.
> -      </p>
> -
> -      <column name="parent_port">
> -        This is taken from
> -        <ref table="Logical_Switch_Port" column="parent_name"
> -        db="OVN_Northbound"/> in the OVN_Northbound database's
> -        <ref table="Logical_Switch_Port" db="OVN_Northbound"/> table.
> -      </column>
> -
> -      <column name="tag">
> -        <p>
> -          Identifies the VLAN tag in the network traffic associated with
> that
> -          container's network interface.
> -        </p>
> -
> -        <p>
> -          This column is used for a different purpose when <ref
> column="type"/>
> -          is <code>localnet</code> (see <code>Localnet Options</code>,
> above)
> -          or <code>l2gateway</code> (see <code>L2 Gateway Options</code>,
> above).
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Naming">
> -      <column name="external_ids" key="name">
> -        <p>
> -          For a logical switch port, <code>ovn-northd</code> copies this
> from
> -          <ref table="Logical_Switch_Port" db="OVN_Northbound"
> -          column="external_ids" key="neutron:port_name"/> in the <ref
> -          table="Logical_Switch_Port" db="OVN_Northbound"/> table in the
> -          OVN_Northbound database, if it is a nonempty string.
> -        </p>
> -
> -        <p>
> -          For a logical switch port, <code>ovn-northd</code> does not
> currently
> -          set this key.
> -        </p>
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        <p>
> -          See <em>External IDs</em> at the beginning of this document.
> -        </p>
> -
> -        <p>
> -          The <code>ovn-northd</code> program populates this column with
> -          all entries into the <ref column="external_ids"/> column of the
> -          <ref table="Logical_Switch_Port"/> table of the
> -          <ref db="OVN_Northbound"/> database.
> -        </p>
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="MAC_Binding" title="IP to MAC bindings">
> -    <p>
> -      Each row in this table specifies a binding from an IP address to an
> -      Ethernet address that has been discovered through ARP (for IPv4) or
> -      neighbor discovery (for IPv6).  This table is primarily used to
> discover
> -      bindings on physical networks, because IP-to-MAC bindings for
> virtual
> -      machines are usually populated statically into the <ref
> -      table="Port_Binding"/> table.
> -    </p>
> -
> -    <p>
> -      This table expresses a functional relationship: <ref
> -      table="MAC_Binding"/>(<ref column="logical_port"/>, <ref
> column="ip"/>) =
> -      <ref column="mac"/>.
> -    </p>
> -
> -    <p>
> -      In outline, the lifetime of a logical router's MAC binding looks
> like
> -      this:
> -    </p>
> -
> -    <ol>
> -      <li>
> -        On hypervisor 1, a logical router determines that a packet should
> be
> -        forwarded to IP address <var>A</var> on one of its router ports.
> It
> -        uses its logical flow table to determine that <var>A</var> lacks a
> -        static IP-to-MAC binding and the <code>get_arp</code> action to
> -        determine that it lacks a dynamic IP-to-MAC binding.
> -      </li>
> -
> -      <li>
> -        Using an OVN logical <code>arp</code> action, the logical router
> -        generates and sends a broadcast ARP request to the router port.
> It
> -        drops the IP packet.
> -      </li>
> -
> -      <li>
> -        The logical switch attached to the router port delivers the ARP
> request
> -        to all of its ports.  (It might make sense to deliver it only to
> ports
> -        that have no static IP-to-MAC bindings, but this could also be
> -        surprising behavior.)
> -      </li>
> -
> -      <li>
> -        A host or VM on hypervisor 2 (which might be the same as
> hypervisor 1)
> -        attached to the logical switch owns the IP address in question.
> It
> -        composes an ARP reply and unicasts it to the logical router port's
> -        Ethernet address.
> -      </li>
> -
> -      <li>
> -        The logical switch delivers the ARP reply to the logical router
> port.
> -      </li>
> -
> -      <li>
> -        The logical router flow table executes a <code>put_arp</code>
> action.
> -        To record the IP-to-MAC binding, <code>ovn-controller</code> adds
> a row
> -        to the <ref table="MAC_Binding"/> table.
> -      </li>
> -
> -      <li>
> -        On hypervisor 1, <code>ovn-controller</code> receives the updated
> <ref
> -        table="MAC_Binding"/> table from the OVN southbound database.
> The next
> -        packet destined to <var>A</var> through the logical router is sent
> -        directly to the bound Ethernet address.
> -      </li>
> -    </ol>
> -
> -    <column name="logical_port">
> -      The logical port on which the binding was discovered.
> -    </column>
> -
> -    <column name="ip">
> -      The bound IP address.
> -    </column>
> -
> -    <column name="mac">
> -      The Ethernet address to which the IP is bound.
> -    </column>
> -    <column name="datapath">
> -      The logical datapath to which the logical port belongs.
> -    </column>
> -  </table>
> -
> -  <table name="DHCP_Options" title="DHCP Options supported by native OVN
> DHCP">
> -    <p>
> -      Each row in this table stores the DHCP Options supported by native
> OVN
> -      DHCP. <code>ovn-northd</code> populates this table with the
> supported
> -      DHCP options. <code>ovn-controller</code> looks up this table to
> get the
> -      DHCP codes of the DHCP options defined in the "put_dhcp_opts"
> action.
> -      Please refer to the RFC 2132 <code>"
> https://tools.ietf.org/html/rfc2132"</code>
> -      for the possible list of DHCP options that can be defined here.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        Name of the DHCP option.
> -      </p>
> -
> -      <p>
> -        Example. name="router"
> -      </p>
> -    </column>
> -
> -    <column name="code">
> -      <p>
> -        DHCP option code for the DHCP option as defined in the RFC 2132.
> -      </p>
> -
> -      <p>
> -        Example. code=3
> -      </p>
> -    </column>
> -
> -    <column name="type">
> -      <p>
> -        Data type of the DHCP option code.
> -      </p>
> -
> -      <dl>
> -        <dt><code>value: bool</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is a bool.
> -          </p>
> -
> -          <p>
> -            Example. "name=ip_forward_enable", "code=19", "type=bool".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., ip_forward_enable = 1,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: uint8</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is an
> unsigned
> -            int8 (8 bits)
> -          </p>
> -
> -          <p>
> -            Example. "name=default_ttl", "code=23", "type=uint8".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., default_ttl = 50,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: uint16</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is an
> unsigned
> -            int16 (16 bits).
> -          </p>
> -
> -          <p>
> -            Example. "name=mtu", "code=26", "type=uint16".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., mtu = 1450,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: uint32</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is an
> unsigned
> -            int32 (32 bits).
> -          </p>
> -
> -          <p>
> -            Example. "name=lease_time", "code=51", "type=uint32".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., lease_time = 86400,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: ipv4</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is an IPv4
> -            address or addresses.
> -          </p>
> -
> -          <p>
> -            Example. "name=router", "code=3", "type=ipv4".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., router = 10.0.0.1,...)
> -          </p>
> -
> -          <p>
> -            Example. "name=dns_server", "code=6", "type=ipv4".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., dns_server = {8.8.8.8 7.7.7.7},...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: static_routes</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option contains a
> pair of
> -            IPv4 route and next hop addresses.
> -          </p>
> -
> -          <p>
> -            Example. "name=classless_static_route", "code=121",
> "type=static_routes".
> -          </p>
> -
> -          <p>
> -            put_dhcp_opts(..., classless_static_route = {
> 30.0.0.0/24,10.0.0.4,0.0.0.0/0,10.0.0.1}..
> <http://30.0.0.0/24,10.0.0.4,0.0.0.0/0,10.0.0.1%7D..>.)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: str</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCP option is a string.
> -          </p>
> -
> -          <p>
> -            Example. "name=host_name", "code=12", "type=str".
> -          </p>
> -        </dd>
> -      </dl>
> -    </column>
> -  </table>
> -
> -  <table name="DHCPv6_Options" title="DHCPv6 Options supported by native
> OVN DHCPv6">
> -    <p>
> -      Each row in this table stores the DHCPv6 Options supported by
> native OVN
> -      DHCPv6. <code>ovn-northd</code> populates this table with the
> supported
> -      DHCPv6 options. <code>ovn-controller</code> looks up this table to
> get
> -      the DHCPv6 codes of the DHCPv6 options defined in the
> -      <code>put_dhcpv6_opts</code> action. Please refer to RFC 3315 and
> RFC
> -      3646 for the list of DHCPv6 options that can be defined here.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        Name of the DHCPv6 option.
> -      </p>
> -
> -      <p>
> -        Example. name="ia_addr"
> -      </p>
> -    </column>
> -
> -    <column name="code">
> -      <p>
> -        DHCPv6 option code for the DHCPv6 option as defined in the
> appropriate
> -        RFC.
> -      </p>
> -
> -      <p>
> -        Example. code=3
> -      </p>
> -    </column>
> -
> -    <column name="type">
> -      <p>
> -        Data type of the DHCPv6 option code.
> -      </p>
> -
> -      <dl>
> -        <dt><code>value: ipv6</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCPv6 option is an IPv6
> -            address(es).
> -          </p>
> -
> -          <p>
> -            Example. "name=ia_addr", "code=5", "type=ipv6".
> -          </p>
> -
> -          <p>
> -            put_dhcpv6_opts(..., ia_addr = ae70::4,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: str</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCPv6 option is a
> string.
> -          </p>
> -
> -          <p>
> -            Example. "name=domain_search", "code=24", "type=str".
> -          </p>
> -
> -          <p>
> -            put_dhcpv6_opts(..., domain_search = ovn.domain,...)
> -          </p>
> -        </dd>
> -
> -        <dt><code>value: mac</code></dt>
> -        <dd>
> -          <p>
> -            This indicates that the value of the DHCPv6 option is a MAC
> address.
> -          </p>
> -
> -          <p>
> -            Example. "name=server_id", "code=2", "type=mac".
> -          </p>
> -
> -          <p>
> -            put_dhcpv6_opts(..., server_id = 01:02:03:04L05:06,...)
> -          </p>
> -        </dd>
> -      </dl>
> -    </column>
> -  </table>
> -  <table name="Connection" title="OVSDB client connections.">
> -    <p>
> -      Configuration for a database connection to an Open vSwitch database
> -      (OVSDB) client.
> -    </p>
> -
> -    <p>
> -      This table primarily configures the Open vSwitch database server
> -      (<code>ovsdb-server</code>).
> -    </p>
> -
> -    <p>
> -      The Open vSwitch database server can initiate and maintain active
> -      connections to remote clients.  It can also listen for database
> -      connections.
> -    </p>
> -
> -    <group title="Core Features">
> -      <column name="target">
> -        <p>Connection methods for clients.</p>
> -        <p>
> -          The following connection methods are currently supported:
> -        </p>
> -        <dl>
> -
> <dt><code>ssl:<var>host</var></code>[<code>:<var>port</var></code>]</dt>
> -          <dd>
> -            <p>
> -              The specified SSL <var>port</var> on the given
> <var>host</var>,
> -              which can either be a DNS name (if built with unbound
> library) or
> -              an IP address.  A valid SSL configuration must be provided
> when
> -              this form is used, this configuration can be specified via
> -              command-line options or the <ref table="SSL"/> table.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -            <p>
> -              SSL support is an optional feature that is not always
> -              built as part of Open vSwitch.
> -            </p>
> -          </dd>
> -
> -
> <dt><code>tcp:<var>host</var></code>[<code>:<var>port</var></code>]</dt>
> -          <dd>
> -            <p>
> -              The specified TCP <var>port</var> on the given
> <var>host</var>,
> -              which can either be a DNS name (if built with unbound
> library) or
> -              an IP address (IPv4 or IPv6).  If <var>host</var> is an IPv6
> -              address, wrap it in square brackets, e.g.
> <code>tcp:[::1]:6640</code>.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -          </dd>
> -
> <dt><code>pssl:</code>[<var>port</var>][<code>:<var>host</var></code>]</dt>
> -          <dd>
> -            <p>
> -              Listens for SSL connections on the specified TCP
> <var>port</var>.
> -              Specify 0 for <var>port</var> to have the kernel
> automatically
> -              choose an available port.  If <var>host</var>, which can
> either
> -              be a DNS name (if built with unbound library) or an IP
> address,
> -              is specified, then connections are restricted to the
> resolved or
> -              specified local IP address (either IPv4 or IPv6 address).
> If
> -              <var>host</var> is an IPv6 address, wrap in square brackets,
> -              e.g. <code>pssl:6640:[::1]</code>.  If <var>host</var> is
> not
> -              specified then it listens only on IPv4 (but not IPv6)
> addresses.
> -              A valid SSL configuration must be provided when this form
> is used,
> -              this can be specified either via command-line options or the
> -              <ref table="SSL"/> table.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -            <p>
> -              SSL support is an optional feature that is not always built
> as
> -              part of Open vSwitch.
> -            </p>
> -          </dd>
> -
> <dt><code>ptcp:</code>[<var>port</var>][<code>:<var>host</var></code>]</dt>
> -          <dd>
> -            <p>
> -              Listens for connections on the specified TCP
> <var>port</var>.
> -              Specify 0 for <var>port</var> to have the kernel
> automatically
> -              choose an available port.  If <var>host</var>, which can
> either
> -              be a DNS name (if built with unbound library) or an IP
> address,
> -              is specified, then connections are restricted to the
> resolved or
> -              specified local IP address (either IPv4 or IPv6 address).
> If
> -              <var>host</var> is an IPv6 address, wrap it in square
> brackets,
> -              e.g. <code>ptcp:6640:[::1]</code>.  If <var>host</var> is
> not
> -              specified then it listens only on IPv4 addresses.
> -            </p>
> -            <p>
> -              If <var>port</var> is not specified, it defaults to 6640.
> -            </p>
> -          </dd>
> -        </dl>
> -        <p>When multiple clients are configured, the <ref
> column="target"/>
> -        values must be unique.  Duplicate <ref column="target"/> values
> yield
> -        unspecified results.</p>
> -      </column>
> -
> -      <column name="read_only">
> -        <code>true</code> to restrict these connections to read-only
> -        transactions, <code>false</code> to allow them to modify the
> database.
> -      </column>
> -      <column name="role">
> -        String containing role name for this connection entry.
> -      </column>
> -    </group>
> -
> -    <group title="Client Failure Detection and Handling">
> -      <column name="max_backoff">
> -        Maximum number of milliseconds to wait between connection
> attempts.
> -        Default is implementation-specific.
> -      </column>
> -
> -      <column name="inactivity_probe">
> -        Maximum number of milliseconds of idle time on connection to the
> client
> -        before sending an inactivity probe message.  If Open vSwitch does
> not
> -        communicate with the client for the specified number of seconds,
> it
> -        will send a probe.  If a response is not received for the same
> -        additional amount of time, Open vSwitch assumes the connection
> has been
> -        broken and attempts to reconnect.  Default is
> implementation-specific.
> -        A value of 0 disables inactivity probes.
> -      </column>
> -    </group>
> -
> -    <group title="Status">
> -      <p>
> -        Key-value pair of <ref column="is_connected"/> is always updated.
> -        Other key-value pairs in the status columns may be updated depends
> -        on the <ref column="target"/> type.
> -      </p>
> -
> -      <p>
> -        When <ref column="target"/> specifies a connection method that
> -        listens for inbound connections (e.g. <code>ptcp:</code> or
> -        <code>punix:</code>), both <ref column="n_connections"/> and
> -        <ref column="is_connected"/> may also be updated while the
> -        remaining key-value pairs are omitted.
> -      </p>
> -
> -      <p>
> -        On the other hand, when <ref column="target"/> specifies an
> -        outbound connection, all key-value pairs may be updated, except
> -        the above-mentioned two key-value pairs associated with inbound
> -        connection targets. They are omitted.
> -      </p>
> -
> -      <column name="is_connected">
> -        <code>true</code> if currently connected to this client,
> -        <code>false</code> otherwise.
> -      </column>
> -
> -      <column name="status" key="last_error">
> -        A human-readable description of the last error on the connection
> -        to the manager; i.e. <code>strerror(errno)</code>.  This key
> -        will exist only if an error has occurred.
> -      </column>
> -
> -      <column name="status" key="state"
> -              type='{"type": "string", "enum": ["set", ["VOID",
> "BACKOFF", "CONNECTING", "ACTIVE", "IDLE"]]}'>
> -        <p>
> -          The state of the connection to the manager:
> -        </p>
> -        <dl>
> -          <dt><code>VOID</code></dt>
> -          <dd>Connection is disabled.</dd>
> -
> -          <dt><code>BACKOFF</code></dt>
> -          <dd>Attempting to reconnect at an increasing period.</dd>
> -
> -          <dt><code>CONNECTING</code></dt>
> -          <dd>Attempting to connect.</dd>
> -
> -          <dt><code>ACTIVE</code></dt>
> -          <dd>Connected, remote host responsive.</dd>
> -
> -          <dt><code>IDLE</code></dt>
> -          <dd>Connection is idle.  Waiting for response to
> keep-alive.</dd>
> -        </dl>
> -        <p>
> -          These values may change in the future.  They are provided only
> for
> -          human consumption.
> -        </p>
> -      </column>
> -
> -      <column name="status" key="sec_since_connect"
> -              type='{"type": "integer", "minInteger": 0}'>
> -        The amount of time since this client last successfully connected
> -        to the database (in seconds). Value is empty if client has never
> -        successfully been connected.
> -      </column>
> -
> -      <column name="status" key="sec_since_disconnect"
> -              type='{"type": "integer", "minInteger": 0}'>
> -        The amount of time since this client last disconnected from the
> -        database (in seconds). Value is empty if client has never
> -        disconnected.
> -      </column>
> -
> -      <column name="status" key="locks_held">
> -        Space-separated list of the names of OVSDB locks that the
> connection
> -        holds.  Omitted if the connection does not hold any locks.
> -      </column>
> -
> -      <column name="status" key="locks_waiting">
> -        Space-separated list of the names of OVSDB locks that the
> connection is
> -        currently waiting to acquire.  Omitted if the connection is not
> waiting
> -        for any locks.
> -      </column>
> -
> -      <column name="status" key="locks_lost">
> -        Space-separated list of the names of OVSDB locks that the
> connection
> -        has had stolen by another OVSDB client.  Omitted if no locks have
> been
> -        stolen from this connection.
> -      </column>
> -
> -      <column name="status" key="n_connections"
> -              type='{"type": "integer", "minInteger": 2}'>
> -        When <ref column="target"/> specifies a connection method that
> -        listens for inbound connections (e.g. <code>ptcp:</code> or
> -        <code>pssl:</code>) and more than one connection is actually
> active,
> -        the value is the number of active connections.  Otherwise, this
> -        key-value pair is omitted.
> -      </column>
> -
> -      <column name="status" key="bound_port" type='{"type": "integer"}'>
> -        When <ref column="target"/> is <code>ptcp:</code> or
> -        <code>pssl:</code>, this is the TCP port on which the OVSDB
> server is
> -        listening.  (This is particularly useful when <ref
> -        column="target"/> specifies a port of 0, allowing the kernel to
> -        choose any available port.)
> -      </column>
> -    </group>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -      <column name="other_config"/>
> -    </group>
> -  </table>
> -  <table name="SSL">
> -    SSL configuration for ovn-sb database access.
> -
> -    <column name="private_key">
> -      Name of a PEM file containing the private key used as the switch's
> -      identity for SSL connections to the controller.
> -    </column>
> -
> -    <column name="certificate">
> -      Name of a PEM file containing a certificate, signed by the
> -      certificate authority (CA) used by the controller and manager,
> -      that certifies the switch's private key, identifying a trustworthy
> -      switch.
> -    </column>
> -
> -    <column name="ca_cert">
> -      Name of a PEM file containing the CA certificate used to verify
> -      that the switch is connected to a trustworthy controller.
> -    </column>
> -
> -    <column name="bootstrap_ca_cert">
> -      If set to <code>true</code>, then Open vSwitch will attempt to
> -      obtain the CA certificate from the controller on its first SSL
> -      connection and save it to the named PEM file. If it is successful,
> -      it will immediately drop the connection and reconnect, and from then
> -      on all SSL connections must be authenticated by a certificate signed
> -      by the CA certificate thus obtained.  <em>This option exposes the
> -      SSL connection to a man-in-the-middle attack obtaining the initial
> -      CA certificate.</em>  It may still be useful for bootstrapping.
> -    </column>
> -
> -    <column name="ssl_protocols">
> -      List of SSL protocols to be enabled for SSL connections. The default
> -      when this option is omitted is <code>TLSv1,TLSv1.1,TLSv1.2</code>.
> -    </column>
> -
> -    <column name="ssl_ciphers">
> -      List of ciphers (in OpenSSL cipher string format) to be supported
> -      for SSL connections. The default when this option is omitted is
> -      <code>HIGH:!aNULL:!MD5</code>.
> -    </column>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -  </table>
> -  <table name="DNS" title="Native DNS resolution">
> -    <p>
> -      Each row in this table stores the DNS records. The OVN action
> -      <code>dns_lookup</code> uses this table for DNS resolution.
> -    </p>
> -
> -    <column name="records">
> -      Key-value pair of DNS records with <code>DNS query name</code> as
> the key
> -      and a string of IP address(es) separated by comma or space as the
> -      value.
> -
> -      <p><b>Example: </b> "vm1.ovn.org" = "10.0.0.4 aef0::4"</p>
> -    </column>
> -
> -    <column name="datapaths">
> -      The DNS records defined in the column <ref column="records"/> will
> be
> -      applied only to the DNS queries originating from the datapaths
> defined
> -      in this column.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="RBAC_Role">
> -    Role table for role-based access controls.
> -
> -    <column name="name">
> -        The role name, corresponding to the <code>role</code>
> -        column in the <code>Connection</code> table.
> -    </column>
> -
> -    <column name="permissions">
> -        A mapping of table names to rows in the
> -        <code>RBAC_Permission</code> table.
> -    </column>
> -  </table>
> -  <table name="RBAC_Permission">
> -    Permissions table for role-based access controls.
> -
> -    <column name="table">
> -      Name of table to which this row applies.
> -    </column>
> -
> -    <column name="authorization">
> -        Set of strings identifying columns and column:key pairs to be
> compared
> -        with client ID. At least one match is required in order to be
> -        authorized.  A zero-length string is treated as a special value
> -        indicating all clients should be considered authorized.
> -    </column>
> -
> -    <column name="insert_delete">
> -        When "true", row insertions and authorized row
> -        deletions are permitted.
> -    </column>
> -    <column name="update">
> -        Set of strings identifying columns and column:key pairs that
> authorized
> -        clients are allowed to modify.
> -    </column>
> -  </table>
> -  <table name="Gateway_Chassis">
> -    <p>
> -      Association of <ref table="Port_Binding"/> rows of
> -      <ref table="Port_Binding" column="type"/>
> <code>chassisredirect</code> to
> -      a <ref table="Chassis"/>. The traffic going out through a specific
> -      <code>chassisredirect</code> port will be redirected to a chassis,
> -      or a set of them in high availability configurations.
> -    </p>
> -
> -    <column name="name">
> -      <p>
> -        Name of the <ref table="Gateway_Chassis"/>.
> -      </p>
> -      <p>
> -        A suggested, but not required naming convention is
> -        <code>${port_name}_${chassis_name}</code>.
> -      </p>
> -    </column>
> -
> -    <column name="chassis">
> -      The <ref table="Chassis"/> to which we send the traffic.
> -    </column>
> -
> -    <column name="priority">
> -      This is the priority the specific <ref table="Chassis"/> among all
> -      Gateway_Chassis belonging to the same <ref table="Port_Binding"/>.
> -    </column>
> -
> -    <column name="options">
> -      Reserved for future use.
> -    </column>
> -
> -    <group title="Common Columns">
> -      The overall purpose of these columns is described under <code>Common
> -      Columns</code> at the beginning of this document.
> -
> -      <column name="external_ids"/>
> -    </group>
> -  </table>
> -
> -  <table name="HA_Chassis">
> -    <column name="chassis">
> -      <p>
> -        The <ref table="Chassis"/> which provides the HA functionality.
> -        </p>
> -    </column>
> -
> -    <column name="priority">
> -      <p>
> -        Priority of the HA chassis. Chassis with highest priority will be
> -        the master in the HA chassis group.
> -      </p>
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -
> -  <table name="HA_Chassis_Group">
> -    <p>
> -      Table representing a group of chassis which can provide High
> availability
> -      services. Each chassis in the group is represented by the table
> -      <ref table="HA_Chassis"/>. The HA chassis with highest priority will
> -      be the master of this group. If the master chassis failover is
> detected,
> -      the HA chassis with the next higher priority takes over the
> -      responsibility of providing the HA. If <ref db="OVN_Southbound"
> -      table="Port_Binding" column="ha_chassis_group"/> column of the table
> -      <ref db="OVN_Southbound" table="Port_Binding"/> references this
> table,
> -      then this HA chassis group provides the gateway functionality and
> -      redirects the gateway traffic to the master of this group.
> -    </p>
> -    <column name="name">
> -      Name of the <ref table="HA_Chassis_Group"/>. Name should be unique.
> -    </column>
> -
> -    <column name="ha_chassis">
> -      A list of <ref table="HA_Chassis"/> which belongs to this group.
> -    </column>
> -
> -    <column name="ref_chassis">
> -      A list of <ref table="chassis"/> which references this HA chassis
> group.
> -    </column>
> -
> -    <group title="Common Columns">
> -      <column name="external_ids">
> -        See <em>External IDs</em> at the beginning of this document.
> -      </column>
> -    </group>
> -  </table>
> -  <table name="Controller_Event" title="Controller Event table">
> -    <p>
> -      Database table used by <code>ovn-controller</code> to report CMS
> -      related events. Please note there is no guarantee a given event is
> -      written exactly once in the db. It is CMS responsibility to squash
> -      duplicated lines or to filter out duplicated events
> -    </p>
> -    <column name="event_type">
> -      Event type occurred
> -    </column>
> -    <column name="event_info">
> -    <p>
> -      Key-value pairs used to specify event info to the CMS.
> -      Possible values are:
> -    </p>
> -      <ul>
> -        <li>
> -         <code>vip</code>: VIP reported for the
> <code>empty_lb_backends</code>
> -         event
> -        </li>
> -        <li>
> -          <code>protocol</code>: Transport protocol reported for the
> -          <code>empty_lb_backends</code> event
> -        </li>
> -        <li>
> -          <code>load_balancer</code>: UUID of the load balancer reported
> for
> -          the <code>empty_lb_backends</code> event
> -        </li>
> -      </ul>
> -    </column>
> -    <column name="chassis">
> -      This column is a <ref table="Chassis"/> record to identify the
> chassis
> -      that has managed a given event.
> -    </column>
> -    <column name="seq_num">
> -      Event sequence number. Global counter for controller generated
> events.
> -      It can be used by the CMS to detect possible duplication of the same
> -      event.
> -    </column>
> -  </table>
> -  <table name="IP_Multicast">
> -    <p>
> -      IP Multicast configuration options. For now only applicable to IGMP.
> -    </p>
> -
> -    <column name="datapath">
> -      <ref table="Datapath_Binding"/> entry for which these configuration
> -      options are defined.
> -    </column>
> -    <column name="enabled">
> -      Enables/disables multicast snooping. Default: disabled.
> -    </column>
> -    <column name="querier">
> -      Enables/disables multicast querying. If
> -      <ref table="IP_Multicast" column="enabled"/> then multicast
> querying is
> -      enabled by default.
> -    </column>
> -    <column name="table_size">
> -      Limits the number of multicast groups that can be learned. Default:
> -      2048 groups per datapath.
> -    </column>
> -    <column name="idle_timeout">
> -      Configures the idle timeout (in seconds) for IP multicast groups if
> -      multicast snooping is enabled. Default: 300 seconds.
> -    </column>
> -    <column name="query_interval">
> -      Configures the interval (in seconds) for sending multicast queries
> if
> -      snooping and querier are enabled.
> -      Default: <ref table="IP_Multicast" column="idle_timeout"/>/2
> seconds.
> -    </column>
> -    <column name="seq_no">
> -      <code>ovn-controller</code> reads this value and flushes all learned
> -      multicast groups when it detects that <code>seq_no</code> was
> changed.
> -    </column>
> -
> -    <group title="Querier configuration options">
> -      The <code>ovn-controller</code> process that runs on OVN hypervisor
> -      nodes uses the following columns to determine field values in IGMP
> -      queries that it originates:
> -      <column name="eth_src">
> -        Source Ethernet address.
> -      </column>
> -      <column name="ip4_src">
> -        Source IPv4 address.
> -      </column>
> -      <column name="query_max_resp">
> -        Value (in seconds) to be used as "max-response" field in multicast
> -        queries. Default: 1 second.
> -      </column>
> -    </group>
> -  </table>
> -  <table name="IGMP_Group">
> -    <p>
> -      Contains learned IGMP groups indexed by address/datapath/chassis.
> -    </p>
> -
> -    <column name="address">
> -      Destination IPv4 address for the IGMP group.
> -    </column>
> -
> -    <column name="datapath">
> -      Datapath to which this IGMP group belongs.
> -    </column>
> -
> -    <column name="chassis">
> -      Chassis to which this IGMP group belongs.
> -    </column>
> -
> -    <column name="ports">
> -      The destination port bindings for this IGMP group.
> -    </column>
> -  </table>
> -</database>
> diff --git a/ovn/utilities/.gitignore b/ovn/utilities/.gitignore
> deleted file mode 100644
> index 1d01e0b28..000000000
> --- a/ovn/utilities/.gitignore
> +++ /dev/null
> @@ -1,11 +0,0 @@
> -/ovn-ctl.8
> -/ovn-nbctl
> -/ovn-nbctl.8
> -/ovn-sbctl
> -/ovn-sbctl.8
> -/ovn-trace
> -/ovn-trace.8
> -/ovn-detrace
> -/ovn-detrace.1
> -/ovn-docker-overlay-driver
> -/ovn-docker-underlay-driver
> diff --git a/ovn/utilities/automake.mk b/ovn/utilities/automake.mk
> deleted file mode 100644
> index e8c59a2eb..000000000
> --- a/ovn/utilities/automake.mk
> +++ /dev/null
> @@ -1,57 +0,0 @@
> -scripts_SCRIPTS += \
> -    ovn/utilities/ovn-ctl \
> -    ovn/utilities/ovndb-servers.ocf
> -
> -man_MANS += \
> -    ovn/utilities/ovn-ctl.8 \
> -    ovn/utilities/ovn-nbctl.8 \
> -    ovn/utilities/ovn-sbctl.8 \
> -    ovn/utilities/ovn-trace.8 \
> -    ovn/utilities/ovn-detrace.1
> -
> -MAN_ROOTS += \
> -    ovn/utilities/ovn-sbctl.8.in \
> -    ovn/utilities/ovn-detrace.1.in
> -
> -# Docker drivers
> -bin_SCRIPTS += \
> -    ovn/utilities/ovn-docker-overlay-driver \
> -    ovn/utilities/ovn-docker-underlay-driver \
> -    ovn/utilities/ovn-detrace
> -
> -EXTRA_DIST += \
> -    ovn/utilities/ovn-ctl \
> -    ovn/utilities/ovn-ctl.8.xml \
> -    ovn/utilities/ovn-docker-overlay-driver.in \
> -    ovn/utilities/ovn-docker-underlay-driver.in \
> -    ovn/utilities/ovn-nbctl.8.xml \
> -    ovn/utilities/ovn-trace.8.xml \
> -    ovn/utilities/ovn-detrace.in \
> -    ovn/utilities/ovndb-servers.ocf
> -
> -CLEANFILES += \
> -    ovn/utilities/ovn-ctl.8 \
> -    ovn/utilities/ovn-docker-overlay-driver \
> -    ovn/utilities/ovn-docker-underlay-driver \
> -    ovn/utilities/ovn-nbctl.8 \
> -    ovn/utilities/ovn-sbctl.8 \
> -    ovn/utilities/ovn-trace.8 \
> -    ovn/utilities/ovn-detrace.1 \
> -    ovn/utilities/ovn-detrace
> -
> -# ovn-nbctl
> -bin_PROGRAMS += ovn/utilities/ovn-nbctl
> -ovn_utilities_ovn_nbctl_SOURCES = ovn/utilities/ovn-nbctl.c
> -ovn_utilities_ovn_nbctl_LDADD = ovn/lib/libovn.la ovsdb/libovsdb.la lib/
> libopenvswitch.la
> -
> -# ovn-sbctl
> -bin_PROGRAMS += ovn/utilities/ovn-sbctl
> -ovn_utilities_ovn_sbctl_SOURCES = ovn/utilities/ovn-sbctl.c
> -ovn_utilities_ovn_sbctl_LDADD = ovn/lib/libovn.la ovsdb/libovsdb.la lib/
> libopenvswitch.la
> -
> -# ovn-trace
> -bin_PROGRAMS += ovn/utilities/ovn-trace
> -ovn_utilities_ovn_trace_SOURCES = ovn/utilities/ovn-trace.c
> -ovn_utilities_ovn_trace_LDADD = ovn/lib/libovn.la ovsdb/libovsdb.la lib/
> libopenvswitch.la
> -
> -include ovn/utilities/bugtool/automake.mk
> diff --git a/ovn/utilities/bugtool/automake.mk b/ovn/utilities/bugtool/
> automake.mk
> deleted file mode 100644
> index 8582074a7..000000000
> --- a/ovn/utilities/bugtool/automake.mk
> +++ /dev/null
> @@ -1,9 +0,0 @@
> -if HAVE_PYTHON2
> -bugtool_plugins += \
> -       ovn/utilities/bugtool/plugins/network-status/ovn.xml
> -
> -bugtool_scripts += \
> -       ovn/utilities/bugtool/ovn-bugtool-nbctl-show \
> -       ovn/utilities/bugtool/ovn-bugtool-sbctl-show \
> -       ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list
> -endif
> diff --git a/ovn/utilities/bugtool/ovn-bugtool-nbctl-show
> b/ovn/utilities/bugtool/ovn-bugtool-nbctl-show
> deleted file mode 100644
> index 927252745..000000000
> --- a/ovn/utilities/bugtool/ovn-bugtool-nbctl-show
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -#! /bin/sh
> -
> -# This library is free software; you can redistribute it and/or
> -# modify it under the terms of version 2.1 of the GNU Lesser General
> -# Public License as published by the Free Software Foundation.
> -#
> -# This library is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -# Lesser General Public License for more details.
> -#
> -# You should have received a copy of the GNU Lesser General Public
> -# License along with this library; if not, write to the Free Software
> -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
> -# USA
> -#
> -# Copyright (C) 2016 Nicira, Inc.
> -
> -ovn-nbctl --timeout=3 show
> diff --git a/ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list
> b/ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list
> deleted file mode 100644
> index 33a15d7d5..000000000
> --- a/ovn/utilities/bugtool/ovn-bugtool-sbctl-lflow-list
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -#! /bin/sh
> -
> -# This library is free software; you can redistribute it and/or
> -# modify it under the terms of version 2.1 of the GNU Lesser General
> -# Public License as published by the Free Software Foundation.
> -#
> -# This library is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -# Lesser General Public License for more details.
> -#
> -# You should have received a copy of the GNU Lesser General Public
> -# License along with this library; if not, write to the Free Software
> -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
> -# USA
> -#
> -# Copyright (C) 2016 Nicira, Inc.
> -
> -ovn-sbctl --timeout=3 lflow-list
> diff --git a/ovn/utilities/bugtool/ovn-bugtool-sbctl-show
> b/ovn/utilities/bugtool/ovn-bugtool-sbctl-show
> deleted file mode 100644
> index b6741bcc2..000000000
> --- a/ovn/utilities/bugtool/ovn-bugtool-sbctl-show
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -#! /bin/sh
> -
> -# This library is free software; you can redistribute it and/or
> -# modify it under the terms of version 2.1 of the GNU Lesser General
> -# Public License as published by the Free Software Foundation.
> -#
> -# This library is distributed in the hope that it will be useful,
> -# but WITHOUT ANY WARRANTY; without even the implied warranty of
> -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -# Lesser General Public License for more details.
> -#
> -# You should have received a copy of the GNU Lesser General Public
> -# License along with this library; if not, write to the Free Software
> -# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
> -# USA
> -#
> -# Copyright (C) 2016 Nicira, Inc.
> -
> -ovn-sbctl --timeout=3 show
> diff --git a/ovn/utilities/bugtool/plugins/network-status/ovn.xml
> b/ovn/utilities/bugtool/plugins/network-status/ovn.xml
> deleted file mode 100644
> index 3b399feb3..000000000
> --- a/ovn/utilities/bugtool/plugins/network-status/ovn.xml
> +++ /dev/null
> @@ -1,23 +0,0 @@
> -<!--
> - This library is free software; you can redistribute it and/or modify
> - it under the terms of version 2.1 of the GNU Lesser General Public
> - License as published by the Free Software Foundation.
> -
> - This library is distributed in the hope that it will be useful, but
> - WITHOUT ANY WARRANTY; without even the implied warranty of
> - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - Lesser General Public License for more details.
> -
> - You should have received a copy of the GNU Lesser General Public
> - License along with this library; if not, write to the Free Software
> - Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
> - USA.
> -
> - Copyright (C) 2016 Nicira, Inc.
> --->
> -
> -<collect>
> -  <command label="ovn-nbctl-show"
> filters="ovn">/usr/share/openvswitch/scripts/ovn-bugtool-nbctl-show</command>
> -  <command label="ovn-sbctl-show"
> filters="ovn">/usr/share/openvswitch/scripts/ovn-bugtool-sbctl-show</command>
> -  <command label="ovn-sbctl-lflow-list"
> filters="ovn">/usr/share/openvswitch/scripts/ovn-bugtool-sbctl-lflow-list</command>
> -</collect>
> diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl
> deleted file mode 100755
> index 7e5cd469c..000000000
> --- a/ovn/utilities/ovn-ctl
> +++ /dev/null
> @@ -1,822 +0,0 @@
> -#!/bin/sh
> -#
> -# Licensed under the Apache License, Version 2.0 (the "License");
> -# you may not use this file except in compliance with the License.
> -# You may obtain a copy of the License at:
> -#
> -#     http://www.apache.org/licenses/LICENSE-2.0
> -#
> -# Unless required by applicable law or agreed to in writing, software
> -# distributed under the License is distributed on an "AS IS" BASIS,
> -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> -# See the License for the specific language governing permissions and
> -# limitations under the License.
> -
> -case $0 in
> -    */*) dir0=`echo "$0" | sed 's,/[^/]*$,,'` ;;
> -    *) dir0=./ ;;
> -esac
> -. "$dir0/ovs-lib" || exit 1
> -
> -for dir in "$sbindir" "$bindir" /sbin /bin /usr/sbin /usr/bin; do
> -    case :$PATH: in
> -        *:$dir:*) ;;
> -        *) PATH=$PATH:$dir ;;
> -    esac
> -done
> -
> -
> -ovnnb_active_conf_file="$etcdir/ovnnb-active.conf"
> -ovnsb_active_conf_file="$etcdir/ovnsb-active.conf"
> -ovn_northd_db_conf_file="$etcdir/ovn-northd-db-params.conf"
> -## ----- ##
> -## start ##
> -## ----- ##
> -
> -pidfile_is_running () {
> -    pidfile=$1
> -    test -e "$pidfile" && pid=`cat "$pidfile"` && pid_exists "$pid"
> -} >/dev/null 2>&1
> -
> -stop_nb_ovsdb() {
> -    if pidfile_is_running $DB_NB_PID; then
> -        ovs-appctl -t $OVN_RUNDIR/ovnnb_db.ctl exit
> -    fi
> -}
> -
> -stop_sb_ovsdb() {
> -    if pidfile_is_running $DB_SB_PID; then
> -        ovs-appctl -t $OVN_RUNDIR/ovnsb_db.ctl exit
> -    fi
> -}
> -
> -stop_ovsdb () {
> -    stop_nb_ovsdb
> -    stop_sb_ovsdb
> -}
> -
> -demote_ovnnb() {
> -    if test ! -z "$DB_NB_SYNC_FROM_ADDR"; then
> -        echo
> "$DB_NB_SYNC_FROM_PROTO:$DB_NB_SYNC_FROM_ADDR:$DB_NB_SYNC_FROM_PORT" >
> $ovnnb_active_conf_file
> -    fi
> -
> -    if test -e $ovnnb_active_conf_file; then
> -        ovs-appctl -t $OVN_RUNDIR/ovnnb_db.ctl
> ovsdb-server/set-active-ovsdb-server `cat $ovnnb_active_conf_file`
> -        ovs-appctl -t $OVN_RUNDIR/ovnnb_db.ctl
> ovsdb-server/connect-active-ovsdb-server
> -    else
> -        echo >&2 "$0: active server details not set"
> -        exit 1
> -    fi
> -}
> -
> -demote_ovnsb() {
> -    if test ! -z "$DB_SB_SYNC_FROM_ADDR"; then
> -        echo
> "$DB_SB_SYNC_FROM_PROTO:$DB_SB_SYNC_FROM_ADDR:$DB_SB_SYNC_FROM_PORT" >
> $ovnsb_active_conf_file
> -    fi
> -
> -    if test -e $ovnsb_active_conf_file; then
> -        ovs-appctl -t $OVN_RUNDIR/ovnsb_db.ctl
> ovsdb-server/set-active-ovsdb-server `cat $ovnsb_active_conf_file`
> -        ovs-appctl -t $OVN_RUNDIR/ovnsb_db.ctl
> ovsdb-server/connect-active-ovsdb-server
> -    else
> -        echo >&2 "$0: active server details not set"
> -        exit 1
> -    fi
> -}
> -
> -promote_ovnnb() {
> -    rm -f $ovnnb_active_conf_file
> -    ovs-appctl -t $OVN_RUNDIR/ovnnb_db.ctl
> ovsdb-server/disconnect-active-ovsdb-server
> -}
> -
> -promote_ovnsb() {
> -    rm -f $ovnsb_active_conf_file
> -    ovs-appctl -t $OVN_RUNDIR/ovnsb_db.ctl
> ovsdb-server/disconnect-active-ovsdb-server
> -}
> -
> -start_ovsdb__() {
> -    local DB=$1 db=$2 schema_name=$3 table_name=$4
> -    local db_pid_file
> -    local cluster_local_addr
> -    local cluster_local_port
> -    local cluster_local_proto
> -    local cluster_remote_addr
> -    local cluster_remote_port
> -    local cluster_remote_proto
> -    local sync_from_proto
> -    local sync_from_addr
> -    local sync_from_port
> -    local file
> -    local schema
> -    local logfile
> -    local log
> -    local sock
> -    local detach
> -    local create_insecure_remote
> -    local port
> -    local addr
> -    local active_conf_file
> -    local use_remote_in_db
> -    local ovn_db_ssl_key
> -    local ovn_db_ssl_cert
> -    local ovn_db_ssl_cacert
> -    eval db_pid_file=\$DB_${DB}_PID
> -    eval cluster_local_addr=\$DB_${DB}_CLUSTER_LOCAL_ADDR
> -    eval cluster_local_port=\$DB_${DB}_CLUSTER_LOCAL_PORT
> -    eval cluster_local_proto=\$DB_${DB}_CLUSTER_LOCAL_PROTO
> -    eval cluster_remote_addr=\$DB_${DB}_CLUSTER_REMOTE_ADDR
> -    eval cluster_remote_port=\$DB_${DB}_CLUSTER_REMOTE_PORT
> -    eval cluster_remote_proto=\$DB_${DB}_CLUSTER_REMOTE_PROTO
> -    eval sync_from_proto=\$DB_${DB}_SYNC_FROM_PROTO
> -    eval sync_from_addr=\$DB_${DB}_SYNC_FROM_ADDR
> -    eval sync_from_port=\$DB_${DB}_SYNC_FROM_PORT
> -    eval file=\$DB_${DB}_FILE
> -    eval schema=\$DB_${DB}_SCHEMA
> -    eval logfile=\$OVN_${DB}_LOGFILE
> -    eval log=\$OVN_${DB}_LOG
> -    eval sock=\$DB_${DB}_SOCK
> -    eval detach=\$DB_${DB}_DETACH
> -    eval create_insecure_remote=\$DB_${DB}_CREATE_INSECURE_REMOTE
> -    eval port=\$DB_${DB}_PORT
> -    eval addr=\$DB_${DB}_ADDR
> -    eval active_conf_file=\$ovn${db}_active_conf_file
> -    eval use_remote_in_db=\$DB_${DB}_USE_REMOTE_IN_DB
> -    eval ovn_db_ssl_key=\$OVN_${DB}_DB_SSL_KEY
> -    eval ovn_db_ssl_cert=\$OVN_${DB}_DB_SSL_CERT
> -    eval ovn_db_ssl_cacert=\$OVN_${DB}_DB_SSL_CA_CERT
> -
> -    install_dir "$OVN_RUNDIR"
> -    # Check and eventually start ovsdb-server for DB
> -    if pidfile_is_running $db_pid_file; then
> -        return
> -    fi
> -
> -    if test ! -z "$cluster_local_addr"; then
> -        mode=cluster
> -    elif test ! -z "$sync_from_addr"; then
> -        mode=active_passive
> -        echo "$sync_from_proto:$sync_from_addr:\
> -$sync_from_port" > $active_conf_file
> -    else
> -        mode=standalone
> -    fi
> -
> -    if test $mode = cluster; then
> -        local local=$cluster_local_proto:$cluster_local_addr:\
> -$cluster_local_port
> -        local remote=$cluster_remote_proto:$cluster_remote_addr:\
> -$cluster_remote_port
> -        if test -n "$cluster_remote_addr"; then
> -            join_cluster "$file" "$schema_name" "$local" "$remote"
> -        else
> -            create_cluster "$file" "$schema" "$local"
> -        fi
> -    else
> -        upgrade_db "$file" "$schema"
> -    fi
> -
> -    set ovsdb-server
> -    set "$@" $log --log-file=$logfile
> -    set "$@" --remote=punix:$sock --pidfile=$db_pid_file
> -    set "$@" --unixctl=ovn${db}_db.ctl
> -
> -    [ "$OVS_USER" != "" ] && set "$@" --user "$OVS_USER"
> -
> -    if test X"$detach" != Xno; then
> -        set "$@" --detach --monitor
> -    else
> -        set exec "$@"
> -    fi
> -
> -    if test X"$use_remote_in_db" != Xno; then
> -        set "$@" --remote=db:$schema_name,$table_name,connections
> -    fi
> -
> -    if test X"$ovn_db_ssl_key" != X; then
> -        set "$@" --private-key=$ovn_db_ssl_key
> -    else
> -        set "$@" --private-key=db:$schema_name,SSL,private_key
> -    fi
> -    if test X"$ovn_db_ssl_cert" != X; then
> -        set "$@" --certificate=$ovn_db_ssl_cert
> -    else
> -        set "$@" --certificate=db:$schema_name,SSL,certificate
> -    fi
> -    if test X"$ovn_db_ssl_cacert" != X; then
> -        set "$@" --ca-cert=$ovn_db_ssl_cacert
> -    else
> -        set "$@" --ca-cert=db:$schema_name,SSL,ca_cert
> -    fi
> -
> -    set "$@" --ssl-protocols=db:$schema_name,SSL,ssl_protocols
> -    set "$@" --ssl-ciphers=db:$schema_name,SSL,ssl_ciphers
> -
> -    if test X"$create_insecure_remote" = Xyes; then
> -        set "$@" --remote=ptcp:$port:$addr
> -    fi
> -
> -    if test $mode = active_passive; then
> -        set "$@" --sync-from=`cat $active_conf_file`
> -    fi
> -
> -    "$@" "$file"
> -
> -    # Initialize the database if it's running standalone,
> -    # active-passive, or is the first server in a cluster.
> -    if test -z "$cluster_remote_addr"; then
> -        ovn-${db}ctl init
> -    fi
> -
> -    if test $mode = cluster; then
> -        upgrade_cluster "$schema" "unix:$sock"
> -    fi
> -}
> -
> -start_nb_ovsdb() {
> -    start_ovsdb__ NB nb OVN_Northbound NB_Global
> -}
> -
> -start_sb_ovsdb() {
> -    # Increase the limit on the number of open file descriptors, because
> -    # SB DB may connect to large number of chassises, on top of
> connections
> -    # for cluster members, northd, and serveral local unix sockets.
> -    MAXFD=8192
> -    if [ $(ulimit -n) -lt $MAXFD ]; then
> -        ulimit -n $MAXFD
> -    fi
> -
> -    start_ovsdb__ SB sb OVN_Southbound SB_Global
> -}
> -
> -start_ovsdb () {
> -    start_nb_ovsdb
> -    start_sb_ovsdb
> -}
> -
> -sync_status() {
> -    ovs-appctl -t $OVN_RUNDIR/ovn${1}_db.ctl ovsdb-server/sync-status |
> awk '{if(NR==1) print $2}'
> -}
> -
> -status_ovnnb() {
> -    if ! pidfile_is_running $DB_NB_PID; then
> -        echo "not-running"
> -    else
> -        echo "running/$(sync_status nb)"
> -    fi
> -}
> -
> -status_ovnsb() {
> -    if ! pidfile_is_running $DB_SB_PID; then
> -        echo "not-running"
> -    else
> -        echo "running/$(sync_status sb)"
> -    fi
> -}
> -
> -status_ovsdb () {
> -  if ! pidfile_is_running $DB_NB_PID; then
> -      log_success_msg "OVN Northbound DB is not running"
> -  else
> -      log_success_msg "OVN Northbound DB is running"
> -  fi
> -
> -  if ! pidfile_is_running $DB_SB_PID; then
> -      log_success_msg "OVN Southbound DB is not running"
> -  else
> -      log_success_msg "OVN Southbound DB is running"
> -  fi
> -}
> -
> -run_nb_ovsdb() {
> -    DB_NB_DETACH=no
> -    start_nb_ovsdb
> -}
> -
> -run_sb_ovsdb() {
> -    DB_SB_DETACH=no
> -    start_sb_ovsdb
> -}
> -
> -start_northd () {
> -    if [ ! -e $ovn_northd_db_conf_file ]; then
> -        if test X"$OVN_MANAGE_OVSDB" = Xyes; then
> -            start_ovsdb
> -
> -            if ! pidfile_is_running $DB_NB_PID; then
> -                log_failure_msg "OVN Northbound DB is not running"
> -                exit
> -            fi
> -            if ! pidfile_is_running $DB_SB_PID; then
> -                log_failure_msg "OVN Southbound DB is not running"
> -                exit
> -            fi
> -        fi
> -        ovn_northd_params="--ovnnb-db=$OVN_NORTHD_NB_DB \
> -        --ovnsb-db=$OVN_NORTHD_SB_DB"
> -    else
> -        ovn_northd_params="`cat $ovn_northd_db_conf_file`"
> -    fi
> -
> -    if daemon_is_running ovn-northd; then
> -        log_success_msg "ovn-northd is already running"
> -    else
> -        set ovn-northd
> -        if test X"$OVN_NORTHD_LOGFILE" != X; then
> -            set "$@" --log-file=$OVN_NORTHD_LOGFILE
> -        fi
> -
> -        [ "$OVN_USER" != "" ] && set "$@" --user "$OVN_USER"
> -
> -        set "$@" $OVN_NORTHD_LOG $ovn_northd_params
> -
> -        OVS_RUNDIR=${OVN_RUNDIR} start_daemon "$OVN_NORTHD_PRIORITY"
> "$OVN_NORTHD_WRAPPER" "$@"
> -    fi
> -}
> -
> -start_controller () {
> -    set ovn-controller "unix:$DB_SOCK"
> -    set "$@" $OVN_CONTROLLER_LOG
> -    if test X"$OVN_CONTROLLER_SSL_KEY" != X; then
> -        set "$@" --private-key=$OVN_CONTROLLER_SSL_KEY
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_CERT" != X; then
> -        set "$@" --certificate=$OVN_CONTROLLER_SSL_CERT
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_CA_CERT" != X; then
> -        set "$@" --ca-cert=$OVN_CONTROLLER_SSL_CA_CERT
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_BOOTSTRAP_CA_CERT" != X; then
> -        set "$@" --bootstrap-ca-cert=$OVN_CONTROLLER_SSL_BOOTSTRAP_CA_CERT
> -    fi
> -
> -    [ "$OVN_USER" != "" ] && set "$@" --user "$OVN_USER"
> -
> -    OVS_RUNDIR=${OVN_RUNDIR} start_daemon "$OVN_CONTROLLER_PRIORITY"
> "$OVN_CONTROLLER_WRAPPER" "$@"
> -}
> -
> -start_controller_vtep () {
> -    set ovn-controller-vtep
> -    set "$@" -vconsole:emer -vsyslog:err -vfile:info
> -    if test X"$OVN_CONTROLLER_SSL_KEY" != X; then
> -        set "$@" --private-key=$OVN_CONTROLLER_SSL_KEY
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_CERT" != X; then
> -        set "$@" --certificate=$OVN_CONTROLLER_SSL_CERT
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_CA_CERT" != X; then
> -        set "$@" --ca-cert=$OVN_CONTROLLER_SSL_CA_CERT
> -    fi
> -    if test X"$OVN_CONTROLLER_SSL_BOOTSTRAP_CA_CERT" != X; then
> -        set "$@" --bootstrap-ca-cert=$OVN_CONTROLLER_SSL_BOOTSTRAP_CA_CERT
> -    fi
> -    if test X"$DB_SOCK" != X; then
> -        set "$@" --vtep-db=$DB_SOCK
> -    fi
> -    if test X"$DB_SB_SOCK" != X; then
> -        set "$@" --ovnsb-db=$DB_SB_SOCK
> -    fi
> -
> -    [ "$OVN_USER" != "" ] && set "$@" --user "$OVN_USER"
> -
> -    OVS_RUNDIR=${OVN_RUNDIR} start_daemon "$OVN_CONTROLLER_PRIORITY"
> "$OVN_CONTROLLER_WRAPPER" "$@"
> -}
> -
> -## ---- ##
> -## stop ##
> -## ---- ##
> -
> -stop_northd () {
> -    OVS_RUNDIR=${OVN_RUNDIR} stop_daemon ovn-northd
> -
> -    if [ ! -e $ovn_northd_db_conf_file ]; then
> -        if test X"$OVN_MANAGE_OVSDB" = Xyes; then
> -            stop_ovsdb
> -        fi
> -    fi
> -}
> -
> -stop_controller () {
> -    OVS_RUNDIR=${OVN_RUNDIR} stop_daemon ovn-controller "$@"
> -}
> -
> -stop_controller_vtep () {
> -    OVS_RUNDIR=${OVN_RUNDIR} stop_daemon ovn-controller-vtep
> -}
> -
> -## ------- ##
> -## restart ##
> -## ------- ##
> -
> -restart_northd () {
> -    stop_northd
> -    start_northd
> -}
> -
> -restart_controller () {
> -    stop_controller --restart
> -    start_controller
> -}
> -
> -restart_controller_vtep () {
> -    stop_controller_vtep
> -    start_controller_vtep
> -}
> -
> -restart_ovsdb () {
> -    stop_ovsdb
> -    start_ovsdb
> -}
> -
> -restart_nb_ovsdb () {
> -    stop_nb_ovsdb
> -    start_nb_ovsdb
> -}
> -
> -restart_sb_ovsdb () {
> -    stop_sb_ovsdb
> -    start_sb_ovsdb
> -}
> -
> -## ---- ##
> -## main ##
> -## ---- ##
> -
> -set_defaults () {
> -    OVN_MANAGE_OVSDB=yes
> -
> -    OVS_RUNDIR=${OVS_RUNDIR:-${rundir}}
> -    OVN_RUNDIR=${OVN_RUNDIR:-${OVS_RUNDIR}}
> -
> -    DB_NB_SOCK=$OVN_RUNDIR/ovnnb_db.sock
> -    DB_NB_PID=$OVN_RUNDIR/ovnnb_db.pid
> -    DB_NB_FILE=$dbdir/ovnnb_db.db
> -    DB_NB_ADDR=0.0.0.0
> -    DB_NB_PORT=6641
> -    DB_NB_SYNC_FROM_PROTO=tcp
> -    DB_NB_SYNC_FROM_ADDR=
> -    DB_NB_SYNC_FROM_PORT=6641
> -
> -    DB_SB_SOCK=$OVN_RUNDIR/ovnsb_db.sock
> -    DB_SB_PID=$OVN_RUNDIR/ovnsb_db.pid
> -    DB_SB_FILE=$dbdir/ovnsb_db.db
> -    DB_SB_ADDR=0.0.0.0
> -    DB_SB_PORT=6642
> -    DB_SB_SYNC_FROM_PROTO=tcp
> -    DB_SB_SYNC_FROM_ADDR=
> -    DB_SB_SYNC_FROM_PORT=6642
> -
> -    DB_NB_SCHEMA=$datadir/ovn-nb.ovsschema
> -    DB_SB_SCHEMA=$datadir/ovn-sb.ovsschema
> -
> -    DB_SOCK=$OVN_RUNDIR/db.sock
> -    DB_CONF_FILE=$dbdir/conf.db
> -
> -    OVN_NORTHD_PRIORITY=-10
> -    OVN_NORTHD_WRAPPER=
> -    OVN_CONTROLLER_PRIORITY=-10
> -    OVN_CONTROLLER_WRAPPER=
> -
> -    OVN_USER=
> -    OVS_USER=
> -
> -    OVN_CONTROLLER_LOG="-vconsole:emer -vsyslog:err -vfile:info"
> -    OVN_NORTHD_LOG="-vconsole:emer -vsyslog:err -vfile:info"
> -    OVN_NORTHD_LOGFILE=""
> -    OVN_NB_LOG="-vconsole:off -vfile:info"
> -    OVN_SB_LOG="-vconsole:off -vfile:info"
> -    OVN_NB_LOGFILE="$logdir/ovsdb-server-nb.log"
> -    OVN_SB_LOGFILE="$logdir/ovsdb-server-sb.log"
> -
> -    OVN_CONTROLLER_SSL_KEY=""
> -    OVN_CONTROLLER_SSL_CERT=""
> -    OVN_CONTROLLER_SSL_CA_CERT=""
> -    OVN_CONTROLLER_SSL_BOOTSTRAP_CA_CERT=""
> -
> -    DB_SB_CREATE_INSECURE_REMOTE="no"
> -    DB_NB_CREATE_INSECURE_REMOTE="no"
> -
> -    MONITOR="yes"
> -
> -    DB_NB_DETACH="yes"
> -    DB_SB_DETACH="yes"
> -
> -    DB_NB_CLUSTER_LOCAL_ADDR=""
> -    DB_NB_CLUSTER_LOCAL_PROTO="tcp"
> -    DB_NB_CLUSTER_LOCAL_PORT=6643
> -    DB_NB_CLUSTER_REMOTE_ADDR=""
> -    DB_NB_CLUSTER_REMOTE_PROTO="tcp"
> -    DB_NB_CLUSTER_REMOTE_PORT=6643
> -
> -    DB_SB_CLUSTER_LOCAL_ADDR=""
> -    DB_SB_CLUSTER_LOCAL_PROTO="tcp"
> -    DB_SB_CLUSTER_LOCAL_PORT=6644
> -    DB_SB_CLUSTER_REMOTE_ADDR=""
> -    DB_SB_CLUSTER_REMOTE_PROTO="tcp"
> -    DB_SB_CLUSTER_REMOTE_PORT=6644
> -
> -    OVN_NORTHD_NB_DB="unix:$DB_NB_SOCK"
> -    OVN_NORTHD_SB_DB="unix:$DB_SB_SOCK"
> -    DB_NB_USE_REMOTE_IN_DB="yes"
> -    DB_SB_USE_REMOTE_IN_DB="yes"
> -
> -    OVN_NB_DB_SSL_KEY=""
> -    OVN_NB_DB_SSL_CERT=""
> -    OVN_NB_DB_SSL_CA_CERT=""
> -
> -    OVN_SB_DB_SSL_KEY=""
> -    OVN_SB_DB_SSL_CERT=""
> -    OVN_SB_DB_SSL_CA_CERT=""
> -
> -}
> -
> -set_option () {
> -    var=`echo "$option" | tr abcdefghijklmnopqrstuvwxyz-
> ABCDEFGHIJKLMNOPQRSTUVWXYZ_`
> -    eval set=\${$var+yes}
> -    eval old_value=\$$var
> -    if test X$set = X || \
> -        (test $type = bool && \
> -        test X"$old_value" != Xno && test X"$old_value" != Xyes); then
> -        echo >&2 "$0: unknown option \"$arg\" (use --help for help)"
> -        return
> -    fi
> -    eval $var=\$value
> -}
> -
> -usage () {
> -    set_defaults
> -    cat << EOF
> -$0: controls Open Virtual Network daemons
> -usage: $0 [OPTIONS] COMMAND
> -
> -This program is intended to be invoked internally by Open Virtual Network
> -startup scripts.  System administrators should not normally invoke it
> directly.
> -
> -Commands:
> -  start_northd                start ovn-northd
> -  start_ovsdb                 start ovn related ovsdb-server processes
> -  start_nb_ovsdb              start ovn northbound db ovsdb-server process
> -  start_sb_ovsdb              start ovn southbound db ovsdb-server process
> -  start_controller            start ovn-controller
> -  start_controller_vtep       start ovn-controller-vtep
> -  stop_northd                 stop ovn-northd
> -  stop_ovsdb                  stop ovn related ovsdb-server processes
> -  stop_nb_ovsdb               stop ovn northbound db ovsdb-server process
> -  stop_sb_ovsdb               stop ovn southbound db ovsdb-server process
> -  stop_controller             stop ovn-controller
> -  stop_controller_vtep        stop ovn-controller-vtep
> -  restart_northd              restart ovn-northd
> -  restart_ovsdb               restart ovn related ovsdb-server processes
> -  restart_nb_ovsdb            restart ovn northbound db ovsdb-server
> process
> -  restart_sb_ovsdb            restart ovn southbound db ovsdb-server
> process
> -  restart_controller          restart ovn-controller
> -  restart_controller_vtep     restart ovn-controller-vtep
> -  status_northd               status ovs-northd
> -  status_ovsdb                status related ovsdb-server processes
> -  status_controller           status ovn-controller
> -  status_controller_vtep      status ovn-controller-vtep
> -  promote_ovnnb               promote ovn northbound db backup server to
> active
> -  promote_ovnsb               promote ovn southbound db backup server to
> active
> -  demote_ovnnb                demote ovn northbound db active server to
> backup
> -  demote_ovnsb                demote ovn southbound db active server to
> backup
> -  run_nb_ovsdb                run ovn northbound db ovsdb-server process
> -  run_sb_ovsdb                run ovn southbound db ovsdb-server process
> -
> -Options:
> -  --ovn-northd-priority=NICE     set ovn-northd's niceness (default:
> $OVN_NORTHD_PRIORITY)
> -  --ovn-northd-wrapper=WRAPPER   run with a wrapper like valgrind for
> debugging
> -  --ovn-controller-priority=NICE     set ovn-controller's niceness
> (default: $OVN_CONTROLLER_PRIORITY)
> -  --ovn-controller-wrapper=WRAPPER   run with a wrapper like valgrind for
> debugging
> -  --ovn-controller-ssl-key=KEY OVN Southbound SSL private key file
> -  --ovn-controller-ssl-cert=CERT OVN Southbound SSL certificate file
> -  --ovn-controller-ssl-ca-cert=CERT OVN Southbound SSL CA certificate file
> -  --ovn-controller-ssl-bootstrap-ca-cert=CERT Bootstrapped OVN Southbound
> SSL CA certificate file
> -  --ovn-nb-db-ssl-key=KEY OVN Northbound DB SSL private key file
> -  --ovn-nb-db-ssl-cert=CERT OVN Northbound DB SSL certificate file
> -  --ovn-nb-db-ssl-ca-cert=CERT OVN Northbound DB SSL CA certificate file
> -  --ovn-sb-db-ssl-key=KEY OVN Southbound DB SSL private key file
> -  --ovn-sb-db-ssl-cert=CERT OVN Southbound DB SSL certificate file
> -  --ovn-sb-db-ssl-ca-cert=CERT OVN Southbound DB SSL CA certificate file
> -  --ovn-manage-ovsdb=yes|no        Whether or not the OVN databases
> should be
> -                                   automatically started and stopped along
> -                                   with ovn-northd. The default is "yes".
> If
> -                                   this is set to "no", the "start_ovsdb"
> and
> -                                   "stop_ovsdb" commands must be used to
> start
> -                                   and stop the OVN databases.
> -  --ovn-controller-log=STRING        ovn controller process logging
> params (default: $OVN_CONTROLLER_LOG)
> -  --ovn-northd-log=STRING            ovn northd process logging params
> (default: $OVN_NORTHD_LOG)
> -  --ovn-northd-logfile=STRING        ovn northd process log file
> (default: $OVN_NORTHD_LOGFILE)
> -  --ovn-nb-log=STRING             ovn NB ovsdb-server processes logging
> params (default: $OVN_NB_LOG)
> -  --ovn-sb-log=STRING             ovn SB ovsdb-server processes logging
> params (default: $OVN_SB_LOG)
> -  --ovn-user="user[:group]"      pass the --user flag to the ovn daemons
> -  --ovs-user="user[:group]"      pass the --user flag to ovs daemons
> -  -h, --help                     display this help message
> -
> -File location options:
> -  --db-sock=SOCKET     JSON-RPC socket name (default: $DB_SOCK)
> -  --db-nb-sock=SOCKET  OVN_Northbound db socket (default: $DB_NB_SOCK)
> -  --db-sb-scok=SOCKET  OVN_Southbound db socket (default: $DB_SB_SOCK)
> -  --db-nb-file=FILE    OVN_Northbound db file (default: $DB_NB_FILE)
> -  --db-sb-file=FILE    OVN_Southbound db file (default: $DB_SB_FILE)
> -  --db-nb-schema=FILE  OVN_Northbound db file (default: $DB_NB_SCHEMA)
> -  --db-sb-schema=FILE  OVN_Southbound db file (default: $DB_SB_SCHEMA)
> -  --db-nb-addr=ADDR    OVN Northbound db ptcp address (default:
> $DB_NB_ADDR)
> -  --db-nb-port=PORT    OVN Northbound db ptcp port (default: $DB_NB_PORT)
> -  --db-sb-addr=ADDR    OVN Southbound db ptcp address (default:
> $DB_SB_ADDR)
> -  --db-sb-port=PORT    OVN Southbound db ptcp port (default: $DB_SB_PORT)
> -  --ovn-nb-logfile=FILE OVN Northbound log file (default: $OVN_NB_LOGFILE)
> -  --ovn-sb-logfile=FILE OVN Southbound log file (default: $OVN_SB_LOGFILE)
> -  --db-nb-sync-from-addr=ADDR OVN Northbound active db tcp address
> (default: $DB_NB_SYNC_FROM_ADDR)
> -  --db-nb-sync-from-port=PORT OVN Northbound active db tcp port (default:
> $DB_NB_SYNC_FROM_PORT)
> -  --db-nb-sync-from-proto=PROTO OVN Northbound active db transport
> (default: $DB_NB_SYNC_FROM_PROTO)
> -  --db-nb-create-insecure-remote=yes|no Create ptcp OVN Northbound remote
> (default: $DB_NB_CREATE_INSECURE_REMOTE)
> -  --db-sb-sync-from-addr=ADDR OVN Southbound active db tcp address
> (default: $DB_SB_SYNC_FROM_ADDR)
> -  --db-sb-sync-from-port=ADDR OVN Southbound active db tcp port (default:
> $DB_SB_SYNC_FROM_PORT)
> -  --db-sb-sync-from-proto=PROTO OVN Southbound active db transport
> (default: $DB_SB_SYNC_FROM_PROTO)
> -  --db-sb-create-insecure-remote=yes|no Create ptcp OVN Southbound remote
> (default: $DB_SB_CREATE_INSECURE_REMOTE)
> -  --db-nb-cluster-local-addr=ADDR OVN_Northbound cluster local address \
> -  (default: $DB_NB_CLUSTER_LOCAL_ADDR)
> -  --db-nb-cluster-local-port=PORT OVN_Northbound cluster local tcp port \
> -  (default: $DB_NB_CLUSTER_LOCAL_PORT)
> -  --db-nb-cluster-local-proto=PROTO OVN_Northbound cluster local db
> transport \
> -  (default: $DB_NB_CLUSTER_LOCAL_PROTO)
> -  --db-nb-cluster-remote-addr=ADDR OVN_Northbound cluster remote address \
> -  (default: $DB_NB_CLUSTER_REMOTE_ADDR)
> -  --db-nb-cluster-remote-port=PORT OVN_Northbound cluster remote tcp port
> \
> -  (default: $DB_NB_CLUSTER_REMOTE_PORT)
> -  --db-nb-cluster-remote-proto=PROTO OVN_Northbound cluster remote db \
> -  transport (default: $DB_NB_CLUSTER_REMOTE_PROTO)
> -  --db-sb-cluster-local-addr=ADDR OVN_Southbound cluster local address \
> -  (default: $DB_SB_CLUSTER_LOCAL_ADDR)
> -  --db-sb-cluster-local-port=PORT OVN_Southbound cluster local tcp port \
> -  (default: $DB_SB_CLUSTER_LOCAL_PORT)
> -  --db-sb-cluster-local-proto=PROTO OVN_Southbound cluster local db
> transport \
> -  (default: $DB_SB_CLUSTER_LOCAL_PROTO)
> -  --db-sb-cluster-remote-addr=ADDR OVN_Southbound cluster remote address \
> -  (default: $DB_SB_CLUSTER_REMOTE_ADDR)
> -  --db-sb-cluster-remote-port=PORT OVN_Southbound cluster remote tcp port
> \
> -  (default: $DB_SB_CLUSTER_REMOTE_PORT)
> -  --db-sb-cluster-remote-proto=PROTO OVN_Southbound cluster remote db \
> -  transport (default: $DB_SB_CLUSTER_REMOTE_PROTO)
> -  --ovn-northd-nb-db=NB DB address(es) (default: $OVN_NORTHD_NB_DB)
> -  --ovn-northd-sb-db=SB DB address(es) (default: $OVN_NORTHD_SB_DB)
> -  --db-nb-use-remote-in-db=yes|no OVN_Northbound db listen on target
> connection table (default: $DB_NB_USE_REMOTE_IN_DB)
> -  --db-sb-use-remote-in-db=yes|no OVN_Southbound db listen on target
> connection table (default: $DB_SB_USE_REMOTE_IN_DB)
> -
> -Default directories with "configure" option and environment variable
> override:
> -  logs: /usr/local/var/log/openvswitch (--with-logdir, OVS_LOGDIR)
> -  pidfiles and sockets: /usr/local/var/run/openvswitch (--with-rundir,
> OVS_RUNDIR)
> -  ovn-nb.db: /usr/local/etc/openvswitch (--with-dbdir, OVS_DBDIR)
> -  ovn-sb.db: /usr/local/etc/openvswitch (--with-dbdir, OVS_DBDIR)
> -  system configuration: /usr/local/etc (--sysconfdir, OVS_SYSCONFDIR)
> -  data files: /usr/local/share/openvswitch (--pkgdatadir, OVS_PKGDATADIR)
> -  user binaries: /usr/local/bin (--bindir, OVS_BINDIR)
> -  system binaries: /usr/local/sbin (--sbindir, OVS_SBINDIR)
> -EOF
> -}
> -
> -set_defaults
> -command=
> -for arg
> -do
> -    case $arg in
> -        -h | --help)
> -            usage
> -            ;;
> -        --[a-z]*=*)
> -            option=`expr X"$arg" : 'X--\([^=]*\)'`
> -            value=`expr X"$arg" : 'X[^=]*=\(.*\)'`
> -            type=string
> -            set_option
> -            ;;
> -        --no-[a-z]*)
> -            option=`expr X"$arg" : 'X--no-\(.*\)'`
> -            value=no
> -            type=bool
> -            set_option
> -            ;;
> -        --[a-z]*)
> -            option=`expr X"$arg" : 'X--\(.*\)'`
> -            value=yes
> -            type=bool
> -            set_option
> -            ;;
> -        -*)
> -            echo >&2 "$0: unknown option \"$arg\" (use --help for help)"
> -            exit 1
> -            ;;
> -        *)
> -            if test X"$command" = X; then
> -                command=$arg
> -            else
> -                echo >&2 "$0: exactly one non-option argument required
> (use --help for help)"
> -                exit 1
> -            fi
> -            ;;
> -    esac
> -done
> -case $command in
> -    start_northd)
> -        start_northd
> -        ;;
> -    start_ovsdb)
> -        start_ovsdb
> -        ;;
> -    start_nb_ovsdb)
> -        start_nb_ovsdb
> -        ;;
> -    start_sb_ovsdb)
> -        start_sb_ovsdb
> -        ;;
> -    start_controller)
> -        start_controller
> -        ;;
> -    start_controller_vtep)
> -        start_controller_vtep
> -        ;;
> -    stop_northd)
> -        stop_northd
> -        ;;
> -    stop_ovsdb)
> -       stop_ovsdb
> -        ;;
> -    stop_nb_ovsdb)
> -       stop_nb_ovsdb
> -        ;;
> -    stop_sb_ovsdb)
> -       stop_sb_ovsdb
> -        ;;
> -    stop_controller)
> -        stop_controller
> -        ;;
> -    stop_controller_vtep)
> -        stop_controller_vtep
> -        ;;
> -    restart_northd)
> -        restart_northd
> -        ;;
> -    restart_ovsdb)
> -        restart_ovsdb
> -        ;;
> -    restart_nb_ovsdb)
> -        restart_nb_ovsdb
> -        ;;
> -    restart_sb_ovsdb)
> -        restart_sb_ovsdb
> -        ;;
> -    restart_controller)
> -        restart_controller
> -        ;;
> -    restart_controller_vtep)
> -        restart_controller_vtep
> -        ;;
> -    status_northd)
> -        daemon_status ovn-northd || exit 1
> -        ;;
> -    status_ovsdb)
> -        status_ovsdb
> -        ;;
> -    status_controller)
> -        daemon_status ovn-controller || exit 1
> -        ;;
> -    status_controller_vtep)
> -        daemon_status ovn-controller-vtep || exit 1
> -        ;;
> -    promote_ovnnb)
> -        promote_ovnnb
> -        ;;
> -    promote_ovnsb)
> -        promote_ovnsb
> -        ;;
> -    demote_ovnnb)
> -        demote_ovnnb
> -        ;;
> -    demote_ovnsb)
> -        demote_ovnsb
> -        ;;
> -    status_ovnnb)
> -        status_ovnnb
> -        ;;
> -    status_ovnsb)
> -        status_ovnsb
> -        ;;
> -    run_nb_ovsdb)
> -        run_nb_ovsdb
> -        ;;
> -    run_sb_ovsdb)
> -        run_sb_ovsdb
> -        ;;
> -    help)
> -        usage
> -        ;;
> -    preheat)
> -        echo >&2 "$0: preheating ovn to 350 degrees F."
> -        exit 1
> -        ;;
> -    '')
> -        echo >&2 "$0: missing command name (use --help for help)"
> -        exit 1
> -        ;;
> -    *)
> -        echo >&2 "$0: unknown command \"$command\" (use --help for help)"
> -        exit 1
> -        ;;
> -esac
> diff --git a/ovn/utilities/ovn-ctl.8.xml b/ovn/utilities/ovn-ctl.8.xml
> deleted file mode 100644
> index c5294d794..000000000
> --- a/ovn/utilities/ovn-ctl.8.xml
> +++ /dev/null
> @@ -1,215 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-ctl" section="8" title="ovn-ctl">
> -    <h1>Name</h1>
> -    <p>ovn-ctl -- Open Virtual Network northbound daemon lifecycle
> utility</p>
> -
> -    <h1>Synopsis</h1>
> -    <p><code>ovn-ctl</code> [<var>options</var>] <var>command</var></p>
> -
> -    <h1>Description</h1>
> -    <p>This program is intended to be invoked internally by Open Virtual
> Network
> -    startup scripts.  System administrators should not normally invoke it
> directly.</p>
> -
> -    <h1>Commands</h1>
> -
> -    <dl>
> -      <dt><code>start_northd</code></dt>
> -      <dt><code>start_controller</code></dt>
> -      <dt><code>start_controller_vtep</code></dt>
> -      <dt><code>stop_northd</code></dt>
> -      <dt><code>stop_controller</code></dt>
> -      <dt><code>stop_controller_vtep</code></dt>
> -      <dt><code>restart_northd</code></dt>
> -      <dt><code>restart_controller</code></dt>
> -      <dt><code>restart_controller_vtep</code></dt>
> -      <dt><code>promote_ovnnb</code></dt>
> -      <dt><code>promote_ovnsb</code></dt>
> -      <dt><code>demote_ovnnb</code></dt>
> -      <dt><code>demote_ovnsb</code></dt>
> -      <dt><code>status_ovnnb</code></dt>
> -      <dt><code>status_ovnsb</code></dt>
> -      <dt><code>start_ovsdb</code></dt>
> -      <dt><code>start_nb_ovsdb</code></dt>
> -      <dt><code>start_sb_ovsdb</code></dt>
> -      <dt><code>stop_ovsdb</code></dt>
> -      <dt><code>stop_nb_ovsdb</code></dt>
> -      <dt><code>stop_sb_ovsdb</code></dt>
> -      <dt><code>restart_ovsdb</code></dt>
> -      <dt><code>run_nb_ovsdb</code></dt>
> -      <dt><code>run_sb_ovsdb</code></dt>
> -    </dl>
> -
> -    <h1>Options</h1>
> -    <p><code>--ovn-northd-priority=<var>NICE</var></code></p>
> -    <p><code>--ovn-northd-wrapper=<var>WRAPPER</var></code></p>
> -    <p><code>--ovn-controller-priority=<var>NICE</var></code></p>
> -    <p><code>--ovn-controller-wrapper=<var>WRAPPER</var></code></p>
> -    <p><code>--ovn-user=<var>USER:GROUP</var></code></p>
> -    <p><code>--ovs-user=<var>USER:GROUP</var></code></p>
> -    <p><code>-h</code> | <code>--help</code></p>
> -
> -    <h1>File location options</h1>
> -    <p><code>--db-sock=<var>SOCKET</var></code></p>
> -    <p><code>--db-nb-file=<var>FILE</var></code></p>
> -    <p><code>--db-sb-file=<var>FILE</var></code></p>
> -    <p><code>--db-nb-schema=<var>FILE</var></code></p>
> -    <p><code>--db-sb-schema=<var>FILE</var></code></p>
> -    <p><code>--db-sb-create-insecure-remote=<var>yes|no</var></code></p>
> -    <p><code>--db-nb-create-insecure-remote=<var>yes|no</var></code></p>
> -    <p><code>--ovn-controller-ssl-key=<var>KEY</var></code></p>
> -    <p><code>--ovn-controller-ssl-cert=<var>CERT</var></code></p>
> -    <p><code>--ovn-controller-ssl-ca-cert=<var>CERT</var></code></p>
> -
> <p><code>--ovn-controller-ssl-bootstrap-ca-cert=<var>CERT</var></code></p>
> -
> -    <h1>Address and port options</h1>
> -    <p><code>--db-nb-sync-from-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-nb-sync-from-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-nb-sync-from-proto=<var>PROTO</var></code></p>
> -    <p><code>--db-sb-sync-from-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-sb-sync-from-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-sb-sync-from-proto=<var>PROTO</var></code></p>
> -    <p>
> -      <code>
> -        --ovn-northd-nb-db=<var>PROTO</var>:<var>IP ADDRESS</var>:
> -        <var>PORT</var>..
> -      </code>
> -    </p>
> -    <p>
> -      <code>
> -        --ovn-northd-sb-db=<var>PROTO</var>:<var>IP ADDRESS</var>:
> -        <var>PORT</var>..
> -      </code>
> -    </p>
> -    <h1> Clustering options </h1>
> -    <p><code>--db-nb-cluster-local-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-nb-cluster-local-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-nb-cluster-local-proto=<var>PROTO
> (tcp/ssl)</var></code></p>
> -    <p><code>--db-nb-cluster-remote-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-nb-cluster-remote-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-nb-cluster-remote-proto=<var>PROTO
> (tcp/ssl)</var></code></p>
> -    <p><code>--db-sb-cluster-local-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-sb-cluster-local-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-sb-cluster-local-proto=<var>PROTO
> (tcp/ssl)</var></code></p>
> -    <p><code>--db-sb-cluster-remote-addr=<var>IP ADDRESS</var></code></p>
> -    <p><code>--db-sb-cluster-remote-port=<var>PORT NUMBER</var></code></p>
> -    <p><code>--db-sb-cluster-remote-proto=<var>PROTO
> (tcp/ssl)</var></code></p>
> -
> -    <h1>Configuration files</h1>
> -    <p>Following are the optional configuration files. If present, it
> should be located in the etc dir</p>
> -
> -    <h2>ovnnb-active.conf</h2>
> -    <p>
> -    If present, this file should hold the url to connect to the active
> -    Northbound DB server
> -    </p>
> -    <p><code>tcp:x.x.x.x:6641</code></p>
> -
> -    <h2>ovnsb-active.conf</h2>
> -    <p>
> -    If present, this file should hold the url to connect to the active
> -    Southbound DB server
> -    </p>
> -    <p><code>tcp:x.x.x.x:6642</code></p>
> -
> -    <h2>ovn-northd-db-params.conf</h2>
> -    <p>
> -    If present, start_northd will not start the DB server even if
> -    <code>--ovn-manage-ovsdb=yes</code>. This file should hold the
> database url
> -    parameters to be passed to ovn-northd.
> -    </p>
> -    <p><code>--ovnnb-db=tcp:x.x.x.x:6641
> --ovnsb-db=tcp:x.x.x.x:6642</code></p>
> -
> -    <h1> Running OVN db servers without detaching </h1>
> -    <p><code># ovn-ctl run_nb_ovsdb</code></p>
> -    <p>
> -      This command runs the OVN nb ovsdb-server without passing the
> -      <code>detach</code> option, making it to block until ovsdb-server
> exits.
> -      This command will be useful for starting the OVN nb ovsdb-server in
> a
> -      container.
> -    </p>
> -    <p><code># ovn-ctl run_sb_ovsdb</code></p>
> -    <p>
> -      This command runs the OVN sb ovsdb-server without passing the
> -      <code>detach</code> option, making it to block until ovsdb-server
> exits.
> -      This command will be useful for starting the OVN sb ovsdb-server in
> a
> -      container.
> -    </p>
> -
> -    <h1>Example Usage</h1>
> -    <h2>Run ovn-controller on a host already running OVS</h2>
> -    <p><code># ovn-ctl start_controller</code></p>
> -
> -    <h2>Run ovn-northd on a host already running OVS</h2>
> -    <p><code># ovn-ctl start_northd</code></p>
> -
> -    <h2>All-in-one OVS+OVN for testing</h2>
> -    <p><code># ovs-ctl start --system-id="random"</code></p>
> -    <p><code># ovn-ctl start_northd</code></p>
> -    <p><code># ovn-ctl start_controller</code></p>
> -
> -    <h2>Promote and demote ovsdb servers</h2>
> -    <p><code># ovn-ctl promote_ovnnb</code></p>
> -    <p><code># ovn-ctl promote_ovnsb</code></p>
> -    <p><code># ovn-ctl --db-nb-sync-from-addr=x.x.x.x
> --db-nb-sync-from-port=6641 demote_ovnnb</code></p>
> -    <p><code># ovn-ctl --db-sb-sync-from-addr=x.x.x.x
> --db-sb-sync-from-port=6642 demote_ovnsb</code></p>
> -
> -    <h2>Creating a clustered db on 3 nodes with IPs x.x.x.x, y.y.y.y and
> z.z.z.z</h2>
> -    <h3>Starting OVN ovsdb servers and ovn-northd on the node with IP
> x.x.x.x</h3>
> -    <p>
> -      <code>
> -        # ovn-ctl --db-nb-addr=x.x.x.x --db-nb-create-insecure-remote=yes
> -        --db-sb-addr=x.x.x.x --db-sb-create-insecure-remote=yes
> -        --db-nb-cluster-local-addr=x.x.x.x
> -        --db-sb-cluster-local-addr=x.x.x.x
> -
> --ovn-northd-nb-db=tcp:x.x.x.x:6641,tcp:y.y.y.y:6641,tcp:z.z.z.z:6641
> -
> --ovn-northd-sb-db=tcp:x.x.x.x:6642,tcp:y.y.y.y:6642,tcp:z.z.z.z:6642
> -        start_northd
> -      </code>
> -    </p>
> -
> -    <h3>Starting OVN ovsdb-servers and ovn-northd on the node with IP
> y.y.y.y and joining the cluster started at x.x.x.x</h3>
> -    <p>
> -      <code>
> -        # ovn-ctl --db-nb-addr=y.y.y.y --db-nb-create-insecure-remote=yes
> -        --db-sb-addr=y.y.y.y --db-sb-create-insecure-remote=yes
> -        --db-nb-cluster-local-addr=y.y.y.y
> -        --db-sb-cluster-local-addr=y.y.y.y
> -        --db-nb-cluster-remote-addr=x.x.x.x
> -        --db-sb-cluster-remote-addr=x.x.x.x
> -
> --ovn-northd-nb-db=tcp:x.x.x.x:6641,tcp:y.y.y.y:6641,tcp:z.z.z.z:6641
> -
> --ovn-northd-sb-db=tcp:x.x.x.x:6642,tcp:y.y.y.y:6642,tcp:z.z.z.z:6642
> -        start_northd
> -      </code>
> -    </p>
> -
> -    <h3>Starting OVN ovsdb-servers and ovn-northd on the node with IP
> z.z.z.z and joining the cluster started at x.x.x.x</h3>
> -    <p>
> -      <code>
> -        # ovn-ctl --db-nb-addr=z.z.z.z
> -          --db-nb-create-insecure-remote=yes
> -          --db-nb-cluster-local-addr=z.z.z.z
> -          --db-sb-addr=z.z.z.z
> -          --db-sb-create-insecure-remote=yes
> -          --db-sb-cluster-local-addr=z.z.z.z
> -          --db-nb-cluster-remote-addr=x.x.x.x
> -          --db-sb-cluster-remote-addr=x.x.x.x
> -
> --ovn-northd-nb-db=tcp:x.x.x.x:6641,tcp:y.y.y.y:6641,tcp:z.z.z.z:6641
> -
> --ovn-northd-sb-db=tcp:x.x.x.x:6642,tcp:y.y.y.y:6642,tcp:z.z.z.z:6642
> -          start_northd
> -      </code>
> -    </p>
> -
> -    <h2>Passing ssl keys when starting OVN dbs will supercede the default
> ssl values in db</h2>
> -    <h3>Starting standalone ovn db server passing SSL certificates</h3>
> -    <p>
> -      <code>
> -        # ovn-ctl --ovn-nb-db-ssl-key=/etc/openvswitch/ovnnb-privkey.pem
> -          --ovn-nb-db-ssl-cert=/etc/openvswitch/ovnnb-cert.pem
> -          --ovn-nb-db-ssl-ca-cert=/etc/openvswitch/cacert.pem
> -          --ovn-sb-db-ssl-key=/etc/openvswitch/ovnsb-privkey.pem
> -          --ovn-sb-db-ssl-cert=/etc/openvswitch/ovnsb-cert.pem
> -          --ovn-sb-db-ssl-ca-cert=/etc/openvswitch/cacert.pem
> -           start_northd
> -      </code>
> -    </p>
> -</manpage>
> diff --git a/ovn/utilities/ovn-detrace.1.in b/ovn/utilities/
> ovn-detrace.1.in
> deleted file mode 100644
> index 2f662d4fe..000000000
> --- a/ovn/utilities/ovn-detrace.1.in
> +++ /dev/null
> @@ -1,38 +0,0 @@
> -.so lib/ovs.tmac
> -.TH ovn\-detrace 1 "@VERSION@" "Open vSwitch" "Open vSwitch Manual"
> -.
> -.SH NAME
> -ovn\-detrace \- convert ``ovs\-appctl ofproto/trace'' output to combine
> -OVN logical flow information.
> -.
> -.SH SYNOPSIS
> -\fBovn\-detrace < \fIfile\fR
> -.so lib/common-syn.man
> -.
> -.SH DESCRIPTION
> -The \fBovn\-detrace\fR program reads \fBovs\-appctl ofproto/trace\fR
> output on
> -stdin, looking for flow cookies, and expand each cookie with
> corresponding OVN
> -logical flows. It expands logical flow further with the north-bound
> information
> -e.g. the ACL that generated the logical flow, when relevant.
> -.PP
> -.
> -.SH "OPTIONS"
> -.so lib/common.man
> -.
> -.IP "\fB\-\-ovnsb=\fIserver\fR"
> -The OVN Southbound DB remote to contact.  If the \fBOVN_SB_DB\fR
> -environment variable is set, its value is used as the default.
> -Otherwise, the default is \fBunix:@RUNDIR@/ovnsb_db.sock\fR, but this
> -default is unlikely to be useful outside of single-machine OVN test
> -environments.
> -.
> -.IP "\fB\-\-ovnnb=\fIserver\fR"
> -The OVN Northbound DB remote to contact.  If the \fBOVN_NB_DB\fR
> -environment variable is set, its value is used as the default.
> -Otherwise, the default is \fBunix:@RUNDIR@/ovnnb_db.sock\fR, but this
> -default is unlikely to be useful outside of single-machine OVN test
> -environments.
> -.
> -.SH "SEE ALSO"
> -.
> -.BR ovs\-appctl (8), ovn\-sbctl (8), ovn-\-nbctl (8), ovn\-trace (8)
> diff --git a/ovn/utilities/ovn-detrace.in b/ovn/utilities/ovn-detrace.in
> deleted file mode 100755
> index c842adc32..000000000
> --- a/ovn/utilities/ovn-detrace.in
> +++ /dev/null
> @@ -1,215 +0,0 @@
> -#! @PYTHON@
> -#
> -# Copyright (c) 2017 eBay Inc.
> -#
> -# Licensed under the Apache License, Version 2.0 (the "License");
> -# you may not use this file except in compliance with the License.
> -# You may obtain a copy of the License at:
> -#
> -#     http://www.apache.org/licenses/LICENSE-2.0
> -#
> -# Unless required by applicable law or agreed to in writing, software
> -# distributed under the License is distributed on an "AS IS" BASIS,
> -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> -# See the License for the specific language governing permissions and
> -# limitations under the License.
> -
> -import getopt
> -import os
> -import re
> -import sys
> -import time
> -
> -try:
> -    from ovs.db import idl
> -    from ovs import jsonrpc
> -    from ovs.poller import Poller
> -    from ovs.stream import Stream
> -except Exception:
> -    print("ERROR: Please install the correct Open vSwitch python support")
> -    print("       libraries (@VERSION@).")
> -    print("       Alternatively, check that your PYTHONPATH is pointing
> to")
> -    print("       the correct location.")
> -    sys.exit(1)
> -
> -
> -argv0 = sys.argv[0]
> -
> -
> -def usage():
> -    print """\
> -%(argv0)s:
> -usage: %(argv0)s < FILE
> -where FILE is output from ovs-appctl ofproto/trace.
> -
> -The following options are also available:
> -  -h, --help                  display this help message
> -  -V, --version               display version information
> -  --ovnsb=DATABASE            use DATABASE as southbound DB
> -  --ovnnb=DATABASE            use DATABASE as northbound DB\
> -""" % {'argv0': argv0}
> -    sys.exit(0)
> -
> -
> -class OVSDB(object):
> -    @staticmethod
> -    def wait_for_db_change(idl):
> -        seq = idl.change_seqno
> -        stop = time.time() + 10
> -        while idl.change_seqno == seq and not idl.run():
> -            poller = Poller()
> -            idl.wait(poller)
> -            poller.block()
> -            if time.time() >= stop:
> -                raise Exception('Retry Timeout')
> -
> -    def __init__(self, db_sock, schema_name):
> -        self._db_sock = db_sock
> -        self._txn = None
> -        schema = self._get_schema(schema_name)
> -        schema.register_all()
> -        self._idl_conn = idl.Idl(db_sock, schema)
> -        OVSDB.wait_for_db_change(self._idl_conn)  # Initial Sync with DB
> -
> -    def _get_schema(self, schema_name):
> -        error, strm = Stream.open_block(Stream.open(self._db_sock))
> -        if error:
> -            raise Exception("Unable to connect to %s" % self._db_sock)
> -        rpc = jsonrpc.Connection(strm)
> -        req = jsonrpc.Message.create_request('get_schema', [schema_name])
> -        error, resp = rpc.transact_block(req)
> -        rpc.close()
> -
> -        if error or resp.error:
> -            raise Exception('Unable to retrieve schema.')
> -        return idl.SchemaHelper(None, resp.result)
> -
> -    def get_table(self, table_name):
> -        return self._idl_conn.tables[table_name]
> -
> -    def _find_row(self, table_name, find):
> -        return next(
> -            (row for row in self.get_table(table_name).rows.values()
> -             if find(row)), None)
> -
> -    def _find_row_by_name(self, table_name, value):
> -        return self._find_row(table_name, lambda row: row.name == value)
> -
> -    def find_row_by_partial_uuid(self, table_name, value):
> -        return self._find_row(table_name, lambda row: value in
> str(row.uuid))
> -
> -
> -def get_lflow_from_cookie(ovnsb_db, cookie):
> -    return ovnsb_db.find_row_by_partial_uuid('Logical_Flow', cookie)
> -
> -
> -def print_lflow(lflow, prefix):
> -    ldp_uuid = lflow.logical_datapath.uuid
> -    ldp_name = str(lflow.logical_datapath.external_ids.get('name'))
> -
> -    print '%sLogical datapath: "%s" (%s) [%s]' % (prefix,
> -                                                  ldp_name,
> -                                                  ldp_uuid,
> -                                                  lflow.pipeline)
> -    print "%sLogical flow: table=%s (%s), priority=%s, " \
> -          "match=(%s), actions=(%s)" % (prefix,
> -                                        lflow.table_id,
> -
> lflow.external_ids.get('stage-name'),
> -                                        lflow.priority,
> -                                        str(lflow.match).strip('"'),
> -                                        str(lflow.actions).strip('"'))
> -
> -
> -def print_lflow_nb_hint(lflow, prefix, ovnnb_db):
> -    external_ids = lflow.external_ids
> -    if external_ids.get('stage-name') in ['ls_in_acl',
> -                                          'ls_out_acl']:
> -        acl_hint = external_ids.get('stage-hint')
> -        if not acl_hint:
> -            return
> -        acl = ovnnb_db.find_row_by_partial_uuid('ACL', acl_hint)
> -        if not acl:
> -            return
> -        output = "%sACL: %s, priority=%s, " \
> -                 "match=(%s), %s" % (prefix,
> -                                     acl.direction,
> -                                     acl.priority,
> -                                     acl.match.strip('"'),
> -                                     acl.action)
> -        if acl.log:
> -            output += ' (log)'
> -        print output
> -
> -
> -def main():
> -    try:
> -        options, args = getopt.gnu_getopt(sys.argv[1:], 'hV',
> -                                          ['help', 'version', 'ovnsb=',
> 'ovnnb='])
> -    except getopt.GetoptError, geo:
> -        sys.stderr.write("%s: %s\n" % (argv0, geo.msg))
> -        sys.exit(1)
> -
> -    ovnsb_db = None
> -    ovnnb_db = None
> -
> -    for key, value in options:
> -        if key in ['-h', '--help']:
> -            usage()
> -        elif key in ['-V', '--version']:
> -            print "%s (Open vSwitch) @VERSION@" % argv0
> -        elif key in ['--ovnsb']:
> -            ovnsb_db = value
> -        elif key in ['--ovnnb']:
> -            ovnnb_db = value
> -        else:
> -            sys.exit(0)
> -
> -    if len(args) != 0:
> -        sys.stderr.write("%s: non-option argument not supported "
> -                         "(use --help for help)\n" % argv0)
> -        sys.exit(1)
> -
> -    ovs_rundir = os.getenv('OVS_RUNDIR', '@RUNDIR@')
> -    if not ovnsb_db:
> -        ovnsb_db = os.getenv('OVN_SB_DB')
> -        if not ovnsb_db:
> -            ovnsb_db = 'unix:%s/ovnsb_db.sock' % ovs_rundir
> -
> -    if not ovnnb_db:
> -        ovnnb_db = os.getenv('OVN_NB_DB')
> -        if not ovnnb_db:
> -            ovnnb_db = 'unix:%s/ovnnb_db.sock' % ovs_rundir
> -
> -    ovsdb_ovnsb = OVSDB(ovnsb_db, 'OVN_Southbound')
> -    ovsdb_ovnnb = OVSDB(ovnnb_db, 'OVN_Northbound')
> -
> -    regex_cookie = re.compile(r'^.*cookie 0x([0-9a-fA-F]+)')
> -    regex_table_id = re.compile(r'^[0-9]+\.')
> -    cookie = None
> -    while True:
> -        line = sys.stdin.readline()
> -        if cookie:
> -            # print lflow info when the current flow block ends
> -            if regex_table_id.match(line) or line.strip() == '':
> -                lflow = get_lflow_from_cookie(ovsdb_ovnsb, cookie)
> -                print_lflow(lflow, "  * ")
> -                print_lflow_nb_hint(lflow, "    * ", ovsdb_ovnnb)
> -                cookie = None
> -
> -        print line.strip()
> -        if line == "":
> -            break
> -
> -        m = regex_cookie.match(line)
> -        if not m:
> -            continue
> -        cookie = m.group(1)
> -
> -
> -if __name__ == "__main__":
> -    main()
> -
> -
> -# Local variables:
> -# mode: python
> -# End:
> diff --git a/ovn/utilities/ovn-docker-overlay-driver.in b/ovn/utilities/
> ovn-docker-overlay-driver.in
> deleted file mode 100755
> index 65edfcd9d..000000000
> --- a/ovn/utilities/ovn-docker-overlay-driver.in
> +++ /dev/null
> @@ -1,442 +0,0 @@
> -#! @PYTHON@
> -# Copyright (C) 2015 Nicira, Inc.
> -#
> -# Licensed under the Apache License, Version 2.0 (the "License");
> -# you may not use this file except in compliance with the License.
> -# You may obtain a copy of the License at:
> -#
> -#     http://www.apache.org/licenses/LICENSE-2.0
> -#
> -# Unless required by applicable law or agreed to in writing, software
> -# distributed under the License is distributed on an "AS IS" BASIS,
> -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> -# See the License for the specific language governing permissions and
> -# limitations under the License.
> -
> -import argparse
> -import ast
> -import atexit
> -import json
> -import os
> -import random
> -import re
> -import shlex
> -import subprocess
> -import sys
> -
> -import ovs.dirs
> -import ovs.util
> -import ovs.daemon
> -import ovs.vlog
> -
> -from flask import Flask, jsonify
> -from flask import request, abort
> -
> -app = Flask(__name__)
> -vlog = ovs.vlog.Vlog("ovn-docker-overlay-driver")
> -
> -OVN_BRIDGE = "br-int"
> -OVN_NB = ""
> -PLUGIN_DIR = "/etc/docker/plugins"
> -PLUGIN_FILE = "/etc/docker/plugins/openvswitch.spec"
> -
> -
> -def call_popen(cmd):
> -    child = subprocess.Popen(cmd, stdout=subprocess.PIPE)
> -    output = child.communicate()
> -    if child.returncode:
> -        raise RuntimeError("Fatal error executing %s" % (cmd))
> -    if len(output) == 0 or output[0] == None:
> -        output = ""
> -    else:
> -        output = output[0].strip()
> -    return output
> -
> -
> -def call_prog(prog, args_list):
> -    cmd = [prog, "--timeout=5", "-vconsole:off"] + args_list
> -    return call_popen(cmd)
> -
> -
> -def ovs_vsctl(*args):
> -    return call_prog("ovs-vsctl", list(args))
> -
> -
> -def ovn_nbctl(*args):
> -    args_list = list(args)
> -    database_option = "%s=%s" % ("--db", OVN_NB)
> -    args_list.insert(0, database_option)
> -    return call_prog("ovn-nbctl", args_list)
> -
> -
> -def cleanup():
> -    if os.path.isfile(PLUGIN_FILE):
> -        os.remove(PLUGIN_FILE)
> -
> -
> -def ovn_init_overlay():
> -    br_list = ovs_vsctl("list-br").split()
> -    if OVN_BRIDGE not in br_list:
> -        ovs_vsctl("--", "--may-exist", "add-br", OVN_BRIDGE,
> -                  "--", "set", "bridge", OVN_BRIDGE,
> -                  "external_ids:bridge-id=" + OVN_BRIDGE,
> -                  "other-config:disable-in-band=true", "fail-mode=secure")
> -
> -    global OVN_NB
> -    OVN_NB = ovs_vsctl("get", "Open_vSwitch", ".",
> -                           "external_ids:ovn-nb").strip('"')
> -    if not OVN_NB:
> -        sys.exit("OVN central database's ip address not set")
> -
> -    ovs_vsctl("set", "open_vswitch", ".",
> -              "external_ids:ovn-bridge=" + OVN_BRIDGE)
> -
> -
> -def prepare():
> -    parser = argparse.ArgumentParser()
> -
> -    ovs.vlog.add_args(parser)
> -    ovs.daemon.add_args(parser)
> -    args = parser.parse_args()
> -    ovs.vlog.handle_args(args)
> -    ovs.daemon.handle_args(args)
> -    ovn_init_overlay()
> -
> -    if not os.path.isdir(PLUGIN_DIR):
> -        os.makedirs(PLUGIN_DIR)
> -
> -    ovs.daemon.daemonize()
> -    try:
> -        fo = open(PLUGIN_FILE, "w")
> -        fo.write("tcp://0.0.0.0:5000")
> -        fo.close()
> -    except Exception as e:
> -        ovs.util.ovs_fatal(0, "Failed to write to spec file (%s)" %
> str(e),
> -                           vlog)
> -
> -    atexit.register(cleanup)
> -
> -
> - at app.route('/Plugin.Activate', methods=['POST'])
> -def plugin_activate():
> -    return jsonify({"Implements": ["NetworkDriver"]})
> -
> -
> - at app.route('/NetworkDriver.GetCapabilities', methods=['POST'])
> -def get_capability():
> -    return jsonify({"Scope": "global"})
> -
> -
> - at app.route('/NetworkDriver.DiscoverNew', methods=['POST'])
> -def new_discovery():
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.DiscoverDelete', methods=['POST'])
> -def delete_discovery():
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.CreateNetwork', methods=['POST'])
> -def create_network():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    # NetworkID will have docker generated network uuid and it
> -    # becomes 'name' in a OVN Logical switch record.
> -    network = data.get("NetworkID", "")
> -    if not network:
> -        abort(400)
> -
> -    # Limit subnet handling to ipv4 till ipv6 usecase is clear.
> -    ipv4_data = data.get("IPv4Data", "")
> -    if not ipv4_data:
> -        error = "create_network: No ipv4 subnet provided"
> -        return jsonify({'Err': error})
> -
> -    subnet = ipv4_data[0].get("Pool", "")
> -    if not subnet:
> -        error = "create_network: no subnet in ipv4 data from libnetwork"
> -        return jsonify({'Err': error})
> -
> -    gateway_ip = ipv4_data[0].get("Gateway", "").rsplit('/', 1)[0]
> -    if not gateway_ip:
> -        error = "create_network: no gateway in ipv4 data from libnetwork"
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ovn_nbctl("ls-add", network, "--", "set", "Logical_Switch",
> -                  network, "external_ids:subnet=" + subnet,
> -                  "external_ids:gateway_ip=" + gateway_ip)
> -    except Exception as e:
> -        error = "create_network: ls-add %s" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.DeleteNetwork', methods=['POST'])
> -def delete_network():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    try:
> -        ovn_nbctl("ls-del", nid)
> -    except Exception as e:
> -        error = "delete_network: ls-del %s" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.CreateEndpoint', methods=['POST'])
> -def create_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    interface = data.get("Interface", "")
> -    if not interface:
> -        error = "create_endpoint: no interfaces structure supplied by " \
> -                "libnetwork"
> -        return jsonify({'Err': error})
> -
> -    ip_address_and_mask = interface.get("Address", "")
> -    if not ip_address_and_mask:
> -        error = "create_endpoint: ip address not provided by libnetwork"
> -        return jsonify({'Err': error})
> -
> -    ip_address = ip_address_and_mask.rsplit('/', 1)[0]
> -    mac_address_input = interface.get("MacAddress", "")
> -    mac_address_output = ""
> -
> -    try:
> -        ovn_nbctl("lsp-add", nid, eid)
> -    except Exception as e:
> -        error = "create_endpoint: lsp-add (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    if not mac_address_input:
> -        mac_address = "02:%02x:%02x:%02x:%02x:%02x" % (random.randint(0,
> 255),
> -                                                       random.randint(0,
> 255),
> -                                                       random.randint(0,
> 255),
> -                                                       random.randint(0,
> 255),
> -                                                       random.randint(0,
> 255))
> -    else:
> -        mac_address = mac_address_input
> -
> -    try:
> -        ovn_nbctl("lsp-set-addresses", eid,
> -                  mac_address + " " + ip_address)
> -    except Exception as e:
> -        error = "create_endpoint: lsp-set-addresses (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    # Only return a mac address if one did not come as request.
> -    mac_address_output = ""
> -    if not mac_address_input:
> -        mac_address_output = mac_address
> -
> -    return jsonify({"Interface": {
> -                                    "Address": "",
> -                                    "AddressIPv6": "",
> -                                    "MacAddress": mac_address_output
> -                                    }})
> -
> -
> -def get_lsp_addresses(eid):
> -    ret = ovn_nbctl("--if-exists", "get", "Logical_Switch_Port", eid,
> -                    "addresses")
> -    if not ret:
> -        error = "endpoint not found in OVN database"
> -        return (None, None, error)
> -    addresses = ast.literal_eval(ret)
> -    if len(addresses) == 0:
> -        error = "unexpected return while fetching addresses"
> -        return (None, None, error)
> -    (mac_address, ip_address) = addresses[0].split()
> -    return (mac_address, ip_address, None)
> -
> -
> - at app.route('/NetworkDriver.EndpointOperInfo', methods=['POST'])
> -def show_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    try:
> -        (mac_address, ip_address, error) = get_lsp_addresses(eid)
> -        if error:
> -            jsonify({'Err': error})
> -    except Exception as e:
> -        error = "show_endpoint: get Logical_Switch_Port addresses. (%s)" \
> -                % (str(e))
> -        return jsonify({'Err': error})
> -
> -    veth_outside = eid[0:15]
> -    return jsonify({"Value": {"ip_address": ip_address,
> -                              "mac_address": mac_address,
> -                              "veth_outside": veth_outside
> -                              }})
> -
> -
> - at app.route('/NetworkDriver.DeleteEndpoint', methods=['POST'])
> -def delete_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    try:
> -        ovn_nbctl("lsp-del", eid)
> -    except Exception as e:
> -        error = "delete_endpoint: lsp-del %s" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.Join', methods=['POST'])
> -def network_join():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    sboxkey = data.get("SandboxKey", "")
> -    if not sboxkey:
> -        abort(400)
> -
> -    # sboxkey is of the form: /var/run/docker/netns/CONTAINER_ID
> -    vm_id = sboxkey.rsplit('/')[-1]
> -
> -    try:
> -        (mac_address, ip_address, error) = get_lsp_addresses(eid)
> -        if error:
> -            jsonify({'Err': error})
> -    except Exception as e:
> -        error = "network_join: %s" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    veth_outside = eid[0:15]
> -    veth_inside = eid[0:13] + "_c"
> -    command = "ip link add %s type veth peer name %s" \
> -              % (veth_inside, veth_outside)
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to create veth pair (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    command = "ip link set dev %s address %s" \
> -              % (veth_inside, mac_address)
> -
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to set veth mac address (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    command = "ip link set %s up" % (veth_outside)
> -
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to up the veth interface (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ovs_vsctl("add-port", OVN_BRIDGE, veth_outside)
> -        ovs_vsctl("set", "interface", veth_outside,
> -                  "external_ids:attached-mac=" + mac_address,
> -                  "external_ids:iface-id=" + eid,
> -                  "external_ids:vm-id=" + vm_id,
> -                  "external_ids:iface-status=active")
> -    except Exception as e:
> -        error = "network_join: failed to create a port (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({"InterfaceName": {
> -                                        "SrcName": veth_inside,
> -                                        "DstPrefix": "eth"
> -                                     },
> -                    "Gateway": "",
> -                    "GatewayIPv6": ""})
> -
> -
> - at app.route('/NetworkDriver.Leave', methods=['POST'])
> -def network_leave():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    veth_outside = eid[0:15]
> -    command = "ip link delete %s" % (veth_outside)
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_leave: failed to delete veth pair (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ovs_vsctl("--if-exists", "del-port", veth_outside)
> -    except Exception as e:
> -        error = "network_leave: failed to delete port (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -if __name__ == '__main__':
> -    prepare()
> -    app.run(host='0.0.0.0')
> diff --git a/ovn/utilities/ovn-docker-underlay-driver.in b/ovn/utilities/
> ovn-docker-underlay-driver.in
> deleted file mode 100755
> index d91ce9fca..000000000
> --- a/ovn/utilities/ovn-docker-underlay-driver.in
> +++ /dev/null
> @@ -1,677 +0,0 @@
> -#! @PYTHON@
> -# Copyright (C) 2015 Nicira, Inc.
> -#
> -# Licensed under the Apache License, Version 2.0 (the "License");
> -# you may not use this file except in compliance with the License.
> -# You may obtain a copy of the License at:
> -#
> -#     http://www.apache.org/licenses/LICENSE-2.0
> -#
> -# Unless required by applicable law or agreed to in writing, software
> -# distributed under the License is distributed on an "AS IS" BASIS,
> -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> -# See the License for the specific language governing permissions and
> -# limitations under the License.
> -
> -import argparse
> -import atexit
> -import getpass
> -import json
> -import os
> -import re
> -import shlex
> -import subprocess
> -import sys
> -import time
> -import uuid
> -
> -import ovs.dirs
> -import ovs.util
> -import ovs.daemon
> -import ovs.unixctl.server
> -import ovs.vlog
> -
> -from neutronclient.v2_0 import client
> -from flask import Flask, jsonify
> -from flask import request, abort
> -
> -app = Flask(__name__)
> -vlog = ovs.vlog.Vlog("ovn-docker-underlay-driver")
> -
> -AUTH_STRATEGY = ""
> -AUTH_URL = ""
> -ENDPOINT_URL = ""
> -OVN_BRIDGE = ""
> -PASSWORD = ""
> -PLUGIN_DIR = "/etc/docker/plugins"
> -PLUGIN_FILE = "/etc/docker/plugins/openvswitch.spec"
> -TENANT_ID = ""
> -USERNAME = ""
> -VIF_ID = ""
> -
> -
> -def call_popen(cmd):
> -    child = subprocess.Popen(cmd, stdout=subprocess.PIPE)
> -    output = child.communicate()
> -    if child.returncode:
> -        raise RuntimeError("Fatal error executing %s" % (cmd))
> -    if len(output) == 0 or output[0] == None:
> -        output = ""
> -    else:
> -        output = output[0].strip()
> -    return output
> -
> -
> -def call_prog(prog, args_list):
> -    cmd = [prog, "--timeout=5", "-vconsole:off"] + args_list
> -    return call_popen(cmd)
> -
> -
> -def ovs_vsctl(*args):
> -    return call_prog("ovs-vsctl", list(args))
> -
> -
> -def cleanup():
> -    if os.path.isfile(PLUGIN_FILE):
> -        os.remove(PLUGIN_FILE)
> -
> -
> -def ovn_init_underlay(args):
> -    global USERNAME, PASSWORD, TENANT_ID, AUTH_URL, AUTH_STRATEGY, VIF_ID
> -    global OVN_BRIDGE
> -
> -    if not args.bridge:
> -        sys.exit("OVS bridge name not provided")
> -    OVN_BRIDGE = args.bridge
> -
> -    VIF_ID = os.environ.get('OS_VIF_ID', '')
> -    if not VIF_ID:
> -        sys.exit("env OS_VIF_ID not set")
> -    USERNAME = os.environ.get('OS_USERNAME', '')
> -    if not USERNAME:
> -        sys.exit("env OS_USERNAME not set")
> -    TENANT_ID = os.environ.get('OS_TENANT_ID', '')
> -    if not TENANT_ID:
> -        sys.exit("env OS_TENANT_ID not set")
> -    AUTH_URL = os.environ.get('OS_AUTH_URL', '')
> -    if not AUTH_URL:
> -        sys.exit("env OS_AUTH_URL not set")
> -    AUTH_STRATEGY = "keystone"
> -
> -    PASSWORD = os.environ.get('OS_PASSWORD', '')
> -    if not PASSWORD:
> -        PASSWORD = getpass.getpass()
> -
> -
> -def prepare():
> -    parser = argparse.ArgumentParser()
> -    parser.add_argument('--bridge', help="The Bridge to which containers "
> -                        "interfaces connect to.")
> -
> -    ovs.vlog.add_args(parser)
> -    ovs.daemon.add_args(parser)
> -    args = parser.parse_args()
> -    ovs.vlog.handle_args(args)
> -    ovs.daemon.handle_args(args)
> -    ovn_init_underlay(args)
> -
> -    if not os.path.isdir(PLUGIN_DIR):
> -        os.makedirs(PLUGIN_DIR)
> -
> -    ovs.daemon.daemonize()
> -    try:
> -        fo = open(PLUGIN_FILE, "w")
> -        fo.write("tcp://127.0.0.1:5000")
> -        fo.close()
> -    except Exception as e:
> -        ovs.util.ovs_fatal(0, "Failed to write to spec file (%s)" %
> str(e),
> -                           vlog)
> -
> -    atexit.register(cleanup)
> -
> -
> - at app.route('/Plugin.Activate', methods=['POST'])
> -def plugin_activate():
> -    return jsonify({"Implements": ["NetworkDriver"]})
> -
> -
> - at app.route('/NetworkDriver.GetCapabilities', methods=['POST'])
> -def get_capability():
> -    return jsonify({"Scope": "global"})
> -
> -
> - at app.route('/NetworkDriver.DiscoverNew', methods=['POST'])
> -def new_discovery():
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.DiscoverDelete', methods=['POST'])
> -def delete_discovery():
> -    return jsonify({})
> -
> -
> -def neutron_login():
> -    try:
> -        neutron = client.Client(username=USERNAME,
> -                                password=PASSWORD,
> -                                tenant_id=TENANT_ID,
> -                                auth_url=AUTH_URL,
> -                                endpoint_url=ENDPOINT_URL,
> -                                auth_strategy=AUTH_STRATEGY)
> -    except Exception as e:
> -        raise RuntimeError("Failed to login into Neutron(%s)" % str(e))
> -    return neutron
> -
> -
> -def get_networkuuid_by_name(neutron, name):
> -    param = {'fields': 'id', 'name': name}
> -    ret = neutron.list_networks(**param)
> -    if len(ret['networks']) > 1:
> -        raise RuntimeError("More than one network for the given name")
> -    elif len(ret['networks']) == 0:
> -        network = None
> -    else:
> -        network = ret['networks'][0]['id']
> -    return network
> -
> -
> -def get_subnetuuid_by_name(neutron, name):
> -    param = {'fields': 'id', 'name': name}
> -    ret = neutron.list_subnets(**param)
> -    if len(ret['subnets']) > 1:
> -        raise RuntimeError("More than one subnet for the given name")
> -    elif len(ret['subnets']) == 0:
> -        subnet = None
> -    else:
> -        subnet = ret['subnets'][0]['id']
> -    return subnet
> -
> -
> - at app.route('/NetworkDriver.CreateNetwork', methods=['POST'])
> -def create_network():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    # NetworkID will have docker generated network uuid and it
> -    # becomes 'name' in a neutron network record.
> -    network = data.get("NetworkID", "")
> -    if not network:
> -        abort(400)
> -
> -    # Limit subnet handling to ipv4 till ipv6 usecase is clear.
> -    ipv4_data = data.get("IPv4Data", "")
> -    if not ipv4_data:
> -        error = "create_network: No ipv4 subnet provided"
> -        return jsonify({'Err': error})
> -
> -    subnet = ipv4_data[0].get("Pool", "")
> -    if not subnet:
> -        error = "create_network: no subnet in ipv4 data from libnetwork"
> -        return jsonify({'Err': error})
> -
> -    gateway_ip = ipv4_data[0].get("Gateway", "").rsplit('/', 1)[0]
> -    if not gateway_ip:
> -        error = "create_network: no gateway in ipv4 data from libnetwork"
> -        return jsonify({'Err': error})
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "create_network: neutron login. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        if get_networkuuid_by_name(neutron, network):
> -            error = "create_network: network has already been created"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "create_network: neutron network uuid by name. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        body = {'network': {'name': network, 'admin_state_up': True}}
> -        ret = neutron.create_network(body)
> -        network_id = ret['network']['id']
> -    except Exception as e:
> -        error = "create_network: neutron net-create call. (%s)" % str(e)
> -        return jsonify({'Err': error})
> -
> -    subnet_name = "docker-%s" % (network)
> -
> -    try:
> -        body = {'subnet': {'network_id': network_id,
> -                           'ip_version': 4,
> -                           'cidr': subnet,
> -                           'gateway_ip': gateway_ip,
> -                           'name': subnet_name}}
> -        created_subnet = neutron.create_subnet(body)
> -    except Exception as e:
> -        error = "create_network: neutron subnet-create call. (%s)" %
> str(e)
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.DeleteNetwork', methods=['POST'])
> -def delete_network():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "delete_network: neutron login. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        network = get_networkuuid_by_name(neutron, nid)
> -        if not network:
> -            error = "delete_network: failed in network by name. (%s)" %
> (nid)
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "delete_network: network uuid by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        neutron.delete_network(network)
> -    except Exception as e:
> -        error = "delete_network: neutron net-delete. (%s)" % str(e)
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> -def reserve_vlan():
> -    reserved_vlan = 0
> -    vlans = ovs_vsctl("--if-exists", "get", "Open_vSwitch", ".",
> -                      "external_ids:vlans").strip('"')
> -    if not vlans:
> -        reserved_vlan = 1
> -        ovs_vsctl("set", "Open_vSwitch", ".",
> -                  "external_ids:vlans=" + str(reserved_vlan))
> -        return reserved_vlan
> -
> -    vlan_set = str(vlans).split(',')
> -
> -    for vlan in range(1, 4095):
> -        if str(vlan) not in vlan_set:
> -            vlan_set.append(str(vlan))
> -            reserved_vlan = vlan
> -            vlans = re.sub(r'[ \[\]\']', '', str(vlan_set))
> -            ovs_vsctl("set", "Open_vSwitch", ".",
> -                      "external_ids:vlans=" + vlans)
> -            return reserved_vlan
> -
> -    if not reserved_vlan:
> -        raise RuntimeError("No more vlans available on this host")
> -
> -
> -def unreserve_vlan(reserved_vlan):
> -    vlans = ovs_vsctl("--if-exists", "get", "Open_vSwitch", ".",
> -                      "external_ids:vlans").strip('"')
> -    if not vlans:
> -        return
> -
> -    vlan_set = str(vlans).split(',')
> -    if str(reserved_vlan) not in vlan_set:
> -        return
> -
> -    vlan_set.remove(str(reserved_vlan))
> -    vlans = re.sub(r'[ \[\]\']', '', str(vlan_set))
> -    if vlans:
> -        ovs_vsctl("set", "Open_vSwitch", ".", "external_ids:vlans=" +
> vlans)
> -    else:
> -        ovs_vsctl("remove", "Open_vSwitch", ".", "external_ids", "vlans")
> -
> -
> -def create_port_underlay(neutron, network, eid, ip_address, mac_address):
> -    reserved_vlan = reserve_vlan()
> -    if mac_address:
> -        body = {'port': {'network_id': network,
> -                         'binding:profile': {'parent_name': VIF_ID,
> -                                             'tag': int(reserved_vlan)},
> -                         'mac_address': mac_address,
> -                         'fixed_ips': [{'ip_address': ip_address}],
> -                         'name': eid,
> -                         'admin_state_up': True}}
> -    else:
> -        body = {'port': {'network_id': network,
> -                         'binding:profile': {'parent_name': VIF_ID,
> -                                             'tag': int(reserved_vlan)},
> -                         'fixed_ips': [{'ip_address': ip_address}],
> -                         'name': eid,
> -                         'admin_state_up': True}}
> -
> -    try:
> -        ret = neutron.create_port(body)
> -        mac_address = ret['port']['mac_address']
> -    except Exception as e:
> -        unreserve_vlan(reserved_vlan)
> -        raise RuntimeError("Failed in creation of neutron port (%s)." %
> str(e))
> -
> -    ovs_vsctl("set", "Open_vSwitch", ".",
> -              "external_ids:" + eid + "_vlan=" + str(reserved_vlan))
> -
> -    return mac_address
> -
> -
> -def get_endpointuuid_by_name(neutron, name):
> -    param = {'fields': 'id', 'name': name}
> -    ret = neutron.list_ports(**param)
> -    if len(ret['ports']) > 1:
> -        raise RuntimeError("More than one endpoint for the given name")
> -    elif len(ret['ports']) == 0:
> -        endpoint = None
> -    else:
> -        endpoint = ret['ports'][0]['id']
> -    return endpoint
> -
> -
> - at app.route('/NetworkDriver.CreateEndpoint', methods=['POST'])
> -def create_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    interface = data.get("Interface", "")
> -    if not interface:
> -        error = "create_endpoint: no interfaces supplied by libnetwork"
> -        return jsonify({'Err': error})
> -
> -    ip_address_and_mask = interface.get("Address", "")
> -    if not ip_address_and_mask:
> -        error = "create_endpoint: ip address not provided by libnetwork"
> -        return jsonify({'Err': error})
> -
> -    ip_address = ip_address_and_mask.rsplit('/', 1)[0]
> -    mac_address_input = interface.get("MacAddress", "")
> -    mac_address_output = ""
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "create_endpoint: neutron login. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        endpoint = get_endpointuuid_by_name(neutron, eid)
> -        if endpoint:
> -            error = "create_endpoint: Endpoint has already been created"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "create_endpoint: endpoint uuid by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        network = get_networkuuid_by_name(neutron, nid)
> -        if not network:
> -            error = "Failed to get neutron network record for (%s)" %
> (nid)
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "create_endpoint: network uuid by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        mac_address = create_port_underlay(neutron, network, eid,
> ip_address,
> -                                           mac_address_input)
> -    except Exception as e:
> -        error = "create_endpoint: neutron port-create (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    if not mac_address_input:
> -        mac_address_output = mac_address
> -
> -    return jsonify({"Interface": {
> -                                    "Address": "",
> -                                    "AddressIPv6": "",
> -                                    "MacAddress": mac_address_output
> -                                    }})
> -
> -
> - at app.route('/NetworkDriver.EndpointOperInfo', methods=['POST'])
> -def show_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "%s" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        endpoint = get_endpointuuid_by_name(neutron, eid)
> -        if not endpoint:
> -            error = "show_endpoint: Failed to get endpoint by name"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "show_endpoint: get endpoint by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ret = neutron.show_port(endpoint)
> -        mac_address = ret['port']['mac_address']
> -        ip_address = ret['port']['fixed_ips'][0]['ip_address']
> -    except Exception as e:
> -        error = "show_endpoint: show port (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    veth_outside = eid[0:15]
> -    return jsonify({"Value": {"ip_address": ip_address,
> -                              "mac_address": mac_address,
> -                              "veth_outside": veth_outside
> -                              }})
> -
> -
> - at app.route('/NetworkDriver.DeleteEndpoint', methods=['POST'])
> -def delete_endpoint():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "delete_endpoint: neutron login (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    endpoint = get_endpointuuid_by_name(neutron, eid)
> -    if not endpoint:
> -        return jsonify({})
> -
> -    reserved_vlan = ovs_vsctl("--if-exists", "get", "Open_vSwitch", ".",
> -                              "external_ids:" + eid + "_vlan").strip('"')
> -    if reserved_vlan:
> -        unreserve_vlan(reserved_vlan)
> -        ovs_vsctl("remove", "Open_vSwitch", ".", "external_ids",
> -                  eid + "_vlan")
> -
> -    try:
> -        neutron.delete_port(endpoint)
> -    except Exception as e:
> -        error = "delete_endpoint: neutron port-delete. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -
> - at app.route('/NetworkDriver.Join', methods=['POST'])
> -def network_join():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    sboxkey = data.get("SandboxKey", "")
> -    if not sboxkey:
> -        abort(400)
> -
> -    # sboxkey is of the form: /var/run/docker/netns/CONTAINER_ID
> -    vm_id = sboxkey.rsplit('/')[-1]
> -
> -    try:
> -        neutron = neutron_login()
> -    except Exception as e:
> -        error = "network_join: neutron login. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    subnet_name = "docker-%s" % (nid)
> -    try:
> -        subnet = get_subnetuuid_by_name(neutron, subnet_name)
> -        if not subnet:
> -            error = "network_join: can't find subnet in neutron"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "network_join: subnet uuid by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ret = neutron.show_subnet(subnet)
> -        gateway_ip = ret['subnet']['gateway_ip']
> -        if not gateway_ip:
> -            error = "network_join: no gateway_ip for the subnet"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "network_join: neutron show subnet. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        endpoint = get_endpointuuid_by_name(neutron, eid)
> -        if not endpoint:
> -            error = "network_join: Failed to get endpoint by name"
> -            return jsonify({'Err': error})
> -    except Exception as e:
> -        error = "network_join: neutron endpoint by name. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ret = neutron.show_port(endpoint)
> -        mac_address = ret['port']['mac_address']
> -    except Exception as e:
> -        error = "network_join: neutron show port. (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    veth_outside = eid[0:15]
> -    veth_inside = eid[0:13] + "_c"
> -    command = "ip link add %s type veth peer name %s" \
> -              % (veth_inside, veth_outside)
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to create veth pair. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    command = "ip link set dev %s address %s" \
> -              % (veth_inside, mac_address)
> -
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to set veth mac address. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    command = "ip link set %s up" % (veth_outside)
> -
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_join: failed to up the veth iface. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        reserved_vlan = ovs_vsctl("--if-exists", "get", "Open_vSwitch",
> ".",
> -                                  "external_ids:" + eid +
> "_vlan").strip('"')
> -        if not reserved_vlan:
> -            error = "network_join: no reserved vlan for this endpoint"
> -            return jsonify({'Err': error})
> -        ovs_vsctl("add-port", OVN_BRIDGE, veth_outside, "tag=" +
> reserved_vlan)
> -    except Exception as e:
> -        error = "network_join: failed to create a OVS port. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({"InterfaceName": {
> -                                        "SrcName": veth_inside,
> -                                        "DstPrefix": "eth"
> -                                     },
> -                    "Gateway": gateway_ip,
> -                    "GatewayIPv6": ""})
> -
> -
> - at app.route('/NetworkDriver.Leave', methods=['POST'])
> -def network_leave():
> -    if not request.data:
> -        abort(400)
> -
> -    data = json.loads(request.data)
> -
> -    nid = data.get("NetworkID", "")
> -    if not nid:
> -        abort(400)
> -
> -    eid = data.get("EndpointID", "")
> -    if not eid:
> -        abort(400)
> -
> -    veth_outside = eid[0:15]
> -    command = "ip link delete %s" % (veth_outside)
> -    try:
> -        call_popen(shlex.split(command))
> -    except Exception as e:
> -        error = "network_leave: failed to delete veth pair. (%s)" %
> (str(e))
> -        return jsonify({'Err': error})
> -
> -    try:
> -        ovs_vsctl("--if-exists", "del-port", veth_outside)
> -    except Exception as e:
> -        error = "network_leave: Failed to delete port (%s)" % (str(e))
> -        return jsonify({'Err': error})
> -
> -    return jsonify({})
> -
> -if __name__ == '__main__':
> -    prepare()
> -    app.run(host='127.0.0.1')
> diff --git a/ovn/utilities/ovn-nbctl.8.xml b/ovn/utilities/ovn-nbctl.8.xml
> deleted file mode 100644
> index 41d50b694..000000000
> --- a/ovn/utilities/ovn-nbctl.8.xml
> +++ /dev/null
> @@ -1,1228 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-nbctl" section="8" title="ovn-nbctl">
> -    <h1>Name</h1>
> -    <p>ovn-nbctl -- Open Virtual Network northbound db management
> utility</p>
> -
> -    <h1>Synopsis</h1>
> -    <p><code>ovn-nbctl</code> [<var>options</var>] <var>command</var>
> [<var>arg</var>...]</p>
> -
> -    <h1>Description</h1>
> -    <p>This utility can be used to manage the OVN northbound database.</p>
> -
> -    <h1>General Commands</h1>
> -
> -    <dl>
> -      <dt><code>init</code></dt>
> -      <dd>
> -        Initializes the database, if it is empty.  If the database has
> already
> -        been initialized, this command has no effect.
> -      </dd>
> -
> -      <dt><code>show [<var>switch</var> | <var>router</var>]</code></dt>
> -      <dd>
> -        Prints a brief overview of the database contents.  If
> -        <var>switch</var> is provided, only records related to that
> -        logical switch are shown. If
> -        <var>router</var> is provided, only records related to that
> -        logical router are shown.
> -      </dd>
> -    </dl>
> -
> -    <h1>Logical Switch Commands</h1>
> -
> -    <dl>
> -      <dt><code>ls-add</code></dt>
> -      <dd>
> -        <p>
> -          Creates a new, unnamed logical switch, which initially has no
> ports.
> -          The switch does not have a name, other commands must refer to
> this
> -          switch by its UUID.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--may-exist</code> | <code>--add-duplicate</code>]
> <code>ls-add</code> <var>switch</var></dt>
> -      <dd>
> -        <p>
> -          Creates a new logical switch named <var>switch</var>, which
> -          initially has no ports.
> -        </p>
> -
> -        <p>
> -          The OVN northbound database schema does not require logical
> switch
> -          names to be unique, but the whole point to the names is to
> provide an
> -          easy way for humans to refer to the switches, making duplicate
> names
> -          unhelpful.  Thus, without any options, this command regards it
> as an
> -          error if <var>switch</var> is a duplicate name.  With
> -          <code>--may-exist</code>, adding a duplicate name succeeds but
> does
> -          not create a new logical switch.  With
> <code>--add-duplicate</code>,
> -          the command really creates a new logical switch with a duplicate
> -          name.  It is an error to specify both options.  If there are
> multiple
> -          logical switches with a duplicate name, configure the logical
> switches
> -          using the UUID instead of the <var>switch</var> name.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>ls-del</code>
> <var>switch</var></dt>
> -      <dd>
> -        Deletes <var>switch</var>.  It is an error if <var>switch</var>
> does
> -        not exist, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>ls-list</code></dt>
> -      <dd>
> -        Lists all existing switches on standard output, one per line.
> -      </dd>
> -    </dl>
> -
> -    <h1>ACL Commands</h1>
> -    <p>
> -      These commands operates on ACL objects for a given
> <var>entity</var>.
> -      The <var>entity</var> can be either a logical switch or a port
> group.
> -      The <var>entity</var> can be specified as uuid or name.  The
> -      <code>--type</code> option can be used to specify the type of the
> -      <var>entity</var>, in case both a logical switch and a port groups
> exist
> -      with the same name specified for <var>entity</var>.
> <code>type</code>
> -      must be either <code>switch</code> or <code>port-group</code>.
> -    </p>
> -    <dl>
> -      <dt>[<code>--type=</code>{<code>switch</code> |
> <code>port-group</code>}] [<code>--log</code>]
> [<code>--meter=</code><var>meter</var>]
> [<code>--severity=</code><var>severity</var>]
> [<code>--name=</code><var>name</var>] [<code>--may-exist</code>]
> <code>acl-add</code> <var>entity</var> <var>direction</var>
> <var>priority</var> <var>match</var> <var>verdict</var></dt>
> -      <dd>
> -        <p>
> -          Adds the specified ACL to <var>entity</var>.
> <var>direction</var>
> -          must be either <code>from-lport</code> or <code>to-lport</code>.
> -          <var>priority</var> must be between <code>0</code> and
> -          <code>32767</code>, inclusive.  A full description of the
> fields are
> -          in <code>ovn-nb</code>(5).  If <code>--may-exist</code> is
> specified,
> -          adding a duplicated ACL succeeds but the ACL is not really
> created.
> -          Without <code>--may-exist</code>, adding a duplicated ACL
> results in
> -          error.
> -        </p>
> -
> -        <p>
> -          The <code>--log</code> option enables packet logging for the
> ACL.
> -          The options <code>--severity</code> and <code>--name</code>
> specify a
> -          severity and name, respectively, for log entries (and also
> enable
> -          logging).  The severity must be one of <code>alert</code>,
> -          <code>warning</code>, <code>notice</code>, <code>info</code>, or
> -          <code>debug</code>.  If a severity is not specified, the
> default is
> -          <code>info</code>.  The <code>--meter=<var>meter</var></code>
> option
> -          is used to rate-limit packet logging.  The <var>meter</var>
> argument
> -          names a meter configured by <code>meter-add</code>.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--type=</code>{<code>switch</code> |
> <code>port-group</code>}] <code>acl-del</code> <var>entity</var>
> [<var>direction</var> [<var>priority</var> <var>match</var>]]</dt>
> -      <dd>
> -        Deletes ACLs from <var>entity</var>.  If only <var>entity</var> is
> -        supplied, all the ACLs from the <var>entity</var> are deleted.  If
> -        <var>direction</var> is also specified, then all the flows in that
> -        direction will be deleted from the <var>entity</var>.  If all the
> -        fields are given, then a single flow that matches all the fields
> will
> -        be deleted.
> -      </dd>
> -
> -      <dt>[<code>--type=</code>{<code>switch</code> |
> <code>port-group</code>}] <code>acl-list</code> <var>entity</var> </dt>
> -      <dd>
> -        Lists the ACLs on <var>entity</var>.
> -      </dd>
> -    </dl>
> -
> -    <h1>Logical Switch QoS Rule Commands</h1>
> -    <dl>
> -      <dt>[<code>--may-exist</code>] <code>qos-add</code>
> <var>switch</var> <var>direction</var> <var>priority</var> <var>match</var>
> [<code>dscp=</code><var>dscp</var>] [<code>rate=</code><var>rate</var>
> [<code>burst=</code><var>burst</var>]]</dt>
> -      <dd>
> -        <p>
> -          Adds QoS marking and metering rules to <var>switch</var>.
> -          <var>direction</var> must be either <code>from-lport</code> or
> -          <code>to-lport</code>.  <var>priority</var> must be between
> -          <code>0</code> and <code>32767</code>, inclusive.
> -        </p>
> -
> -        <p>
> -          If <code>dscp=</code><var>dscp</var> is specified, then
> -          matching packets will have DSCP marking applied.
> -          <var>dscp</var> must be between <code>0</code> and
> -          <code>63</code>, inclusive.  If
> <code>rate=</code><var>rate</var>
> -          is specified then matching packets will have metering applied
> -          at <var>rate</var> kbps.  If metering is configured, then
> -          <code>burst=</code><var>burst</var> specifies the burst rate
> -          limit in kilobits.  <code>dscp</code> and/or <code>rate</code>
> -          are required arguments.
> -        </p>
> -
> -        <p>
> -          If <code>--may-exist</code> is specified, adding a duplicated
> -          QoS rule succeeds but the QoS rule is not really created.
> -          Without <code>--may-exist</code>, adding a duplicated QoS rule
> -          results in error.
> -        </p>
> -      </dd>
> -
> -      <dt><code>qos-del</code> <var>switch</var> [<var>direction</var>
> [<var>priority</var> <var>match</var>]]</dt>
> -      <dd>
> -        Deletes QoS rules from <var>switch</var>.  If only
> -        <var>switch</var> is supplied, all the QoS rules from the logical
> -        switch are deleted.  If <var>direction</var> is also specified,
> -        then all the flows in that direction will be deleted from the
> -        logical switch.  If all the fields are supplied, then a single
> -        flow that matches the given fields will be deleted.
> -      </dd>
> -
> -      <dt><code>qos-list</code> <var>switch</var></dt>
> -      <dd>
> -        Lists the QoS rules on <var>switch</var>.
> -      </dd>
> -    </dl>
> -
> -    <h1>Meter Commands</h1>
> -    <dl>
> -        <dt><code>meter-add</code> <var>name</var> <var>action</var>
> <var>rate</var> <var>unit</var> [<var>burst</var>]</dt>
> -      <dd>
> -        <p>
> -          Adds the specified meter.  <var>name</var> must be a unique
> -          name to identify this meter.  The <var>action</var> argument
> -          specifies what should happen when this meter is exceeded.
> -          The only supported action is <code>drop</code>.
> -        </p>
> -
> -        <p>
> -          The <var>unit</var> specifies the unit for the <var>rate</var>
> -          argument; valid values are <code>kbps</code> and
> -          <code>pktps</code> for kilobits per second and packets per
> -          second, respectively.  The <var>burst</var> option
> -          configures the maximum burst allowed for the band in kilobits
> -          or packets depending on whether the <var>unit</var> chosen was
> -          <code>kbps</code> or <code>pktps</code>, respectively.  If a
> -          burst is not supplied, the switch is free to select some
> -          reasonable value depending on its configuration.
> -        </p>
> -
> -        <p>
> -          <code>ovn-nbctl</code> only supports adding a meter with a
> -          single band, but the other commands support meters with
> -          multiple bands.
> -        </p>
> -
> -        <p>
> -          Names that start with "__" (two underscores) are reserved for
> -          internal use by OVN, so <code>ovn-nbctl</code> does not allow
> -          adding them.
> -        </p>
> -      </dd>
> -
> -      <dt><code>meter-del</code> [<var>name</var>]</dt>
> -      <dd>
> -        <p>
> -          Deletes meters.  By default, all meters are deleted.  If
> -          <var>name</var> is supplied, only the meter with that name
> -          will be deleted.
> -      </p>
> -      </dd>
> -
> -      <dt><code>meter-list</code></dt>
> -      <dd>
> -        <p>
> -          Lists all meters.
> -        </p>
> -      </dd>
> -    </dl>
> -
> -    <h1>Logical Switch Port Commands</h1>
> -    <dl>
> -      <dt>[<code>--may-exist</code>] <code>lsp-add</code>
> <var>switch</var> <var>port</var></dt>
> -      <dd>
> -        <p>
> -          Creates on <var>lswitch</var> a new logical switch port named
> -          <var>port</var>.
> -        </p>
> -
> -        <p>
> -          It is an error if a logical port named <var>port</var> already
> -          exists, unless <code>--may-exist</code> is specified.
> Regardless of
> -          <code>--may-exist</code>, it is an error if the existing port
> is in
> -          some logical switch other than <var>switch</var> or if it has a
> -          parent port.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--may-exist</code>] <code>lsp-add</code>
> <var>switch</var> <var>port</var> <var>parent</var>
> <var>tag_request</var></dt>
> -      <dd>
> -        <p>
> -          Creates on <var>switch</var> a logical switch port named
> -          <var>port</var> that is a child of <var>parent</var> that is
> -          identified with VLAN ID <var>tag_request</var>,
> -          which must be between <code>0</code> and
> -          <code>4095</code>, inclusive. If
> -          <var>tag_request</var> is <code>0</code>,
> <code>ovn-northd</code>
> -          generates a tag that is unique in the scope of
> <var>parent</var>.
> -          This is useful in cases such as virtualized container
> environments
> -          where Open vSwitch does not have a direct connection to the
> -          container's port and it must be shared with the virtual
> machine's
> -          port.
> -        </p>
> -
> -        <p>
> -          It is an error if a logical port named <var>port</var> already
> -          exists, unless <code>--may-exist</code> is specified.
> Regardless of
> -          <code>--may-exist</code>, it is an error if the existing port
> is not
> -          in <var>switch</var> or if it does not have the specified
> -          <var>parent</var> and <var>tag_request</var>.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lsp-del</code>
> <var>port</var></dt>
> -      <dd>
> -        Deletes <var>port</var>.  It is an error if <var>port</var> does
> -        not exist, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>lsp-list</code> <var>switch</var></dt>
> -      <dd>
> -        Lists all the logical switch ports within <var>switch</var> on
> -        standard output, one per line.
> -      </dd>
> -
> -      <dt><code>lsp-get-parent</code> <var>port</var></dt>
> -      <dd>
> -        If set, get the parent port of <var>port</var>.  If not set, print
> -        nothing.
> -      </dd>
> -
> -      <dt><code>lsp-get-tag</code> <var>port</var></dt>
> -      <dd>
> -        If set, get the tag for <var>port</var> traffic.  If not set,
> print
> -        nothing.
> -      </dd>
> -
> -      <dt><code>lsp-set-addresses</code> <var>port</var>
> [<var>address</var>]...</dt>
> -      <dd>
> -        <p>
> -          Sets the addresses associated with <var>port</var> to
> -          <var>address</var>.  Each <var>address</var> should be one of
> the
> -          following:
> -        </p>
> -
> -        <dl>
> -          <dt>an Ethernet address, optionally followed by a space and one
> or more IP addresses</dt>
> -          <dd>
> -            OVN delivers packets for the Ethernet address to this port.
> -          </dd>
> -
> -          <dt><code>unknown</code></dt>
> -          <dd>
> -            OVN delivers unicast Ethernet packets whose destination MAC
> address
> -            is not in any logical port's addresses column to ports with
> address
> -            <code>unknown</code>.
> -          </dd>
> -
> -          <dt><code>dynamic</code></dt>
> -          <dd>
> -            Use this keyword to make <code>ovn-northd</code> generate a
> -            globally unique MAC address and choose an unused IPv4 address
> with
> -            the logical port's subnet and store them in the port's
> -            <code>dynamic_addresses</code> column.
> -          </dd>
> -
> -          <dt><code>router</code></dt>
> -          <dd>
> -            Accepted only when the <code>type</code> of the logical switch
> -            port is <code>router</code>.  This indicates that the
> Ethernet,
> -            IPv4, and IPv6 addresses for this logical switch port should
> be
> -            obtained from the connected logical router port, as specified
> by
> -            <code>router-port</code> in <code>lsp-set-options</code>.
> -          </dd>
> -        </dl>
> -
> -        <p>
> -          Multiple addresses may be set.  If no <var>address</var>
> argument is
> -          given, <var>port</var> will have no addresses associated with
> it.
> -        </p>
> -      </dd>
> -
> -      <dt><code>lsp-get-addresses</code> <var>port</var></dt>
> -      <dd>
> -        Lists all the addresses associated with <var>port</var> on
> standard
> -        output, one per line.
> -      </dd>
> -
> -      <dt><code>lsp-set-port-security</code> <var>port</var>
> [<var>addrs</var>]...</dt>
> -      <dd>
> -        <p>
> -          Sets the port security addresses associated with
> <var>port</var> to
> -          <var>addrs</var>.  Multiple sets of addresses may be set by
> using
> -          multiple <var>addrs</var> arguments.  If no <var>addrs</var>
> argument
> -          is given, <var>port</var> will not have port security enabled.
> -        </p>
> -
> -        <p>
> -          Port security limits the addresses from which a logical port
> may send
> -          packets and to which it may receive packets.  See the
> -          <code>ovn-nb</code>(5) documentation for the <ref
> -          column="port_security" table="Logical_Switch_Port"/> column in
> -          the <ref table="Logical_Switch_Port"/> table for details.
> -        </p>
> -      </dd>
> -
> -      <dt><code>lsp-get-port-security</code> <var>port</var></dt>
> -      <dd>
> -        Lists all the port security addresses associated with
> <var>port</var>
> -        on standard output, one per line.
> -      </dd>
> -
> -      <dt><code>lsp-get-up</code> <var>port</var></dt>
> -      <dd>
> -        Prints the state of <var>port</var>, either <code>up</code> or
> -        <code>down</code>.
> -      </dd>
> -
> -      <dt><code>lsp-set-enabled</code> <var>port</var>
> <var>state</var></dt>
> -      <dd>
> -        Set the administrative state of <var>port</var>, either
> <code>enabled</code>
> -        or <code>disabled</code>.  When a port is disabled, no traffic is
> allowed into
> -        or out of the port.
> -      </dd>
> -
> -      <dt><code>lsp-get-enabled</code> <var>port</var></dt>
> -      <dd>
> -        Prints the administrative state of <var>port</var>, either
> <code>enabled</code>
> -        or <code>disabled</code>.
> -      </dd>
> -
> -      <dt><code>lsp-set-type</code> <var>port</var> <var>type</var></dt>
> -      <dd>
> -        <p>
> -          Set the type for the logical port.  The type must be one of the
> following:
> -        </p>
> -
> -        <dl>
> -          <dt><code>(empty string)</code></dt>
> -          <dd>
> -            A VM (or VIF) interface.
> -          </dd>
> -
> -          <dt><code>router</code></dt>
> -          <dd>
> -            A connection to a logical router.
> -          </dd>
> -
> -          <dt><code>localnet</code></dt>
> -          <dd>
> -            A connection to a locally accessible network from each
> ovn-controller
> -            instance. A logical switch can only have a single localnet
> port
> -            attached. This is used to model direct connectivity to an
> existing
> -            network.
> -          </dd>
> -
> -          <dt><code>localport</code></dt>
> -          <dd>
> -            A connection to a local VIF. Traffic that arrives on a
> localport is
> -            never forwarded over a tunnel to another chassis. These ports
> are
> -            present on every chassis and have the same address in all of
> them.
> -            This is used to model connectivity to local services that run
> on
> -            every hypervisor.
> -          </dd>
> -
> -          <dt><code>l2gateway</code></dt>
> -          <dd>
> -            A connection to a physical network.
> -          </dd>
> -
> -          <dt><code>vtep</code></dt>
> -          <dd>
> -            A port to a logical switch on a VTEP gateway.
> -          </dd>
> -        </dl>
> -
> -      </dd>
> -
> -      <dt><code>lsp-get-type</code> <var>port</var></dt>
> -      <dd>
> -        Get the type for the logical port.
> -      </dd>
> -
> -      <dt><code>lsp-set-options</code> <var>port</var>
> [<var>key=value</var>]...</dt>
> -      <dd>
> -        Set type-specific key-value options for the logical port.
> -      </dd>
> -
> -      <dt><code>lsp-get-options</code> <var>port</var></dt>
> -      <dd>
> -        Get the type-specific options for the logical port.
> -      </dd>
> -
> -      <dt><code>lsp-set-dhcpv4-options</code> <var>port</var>
> -          <var>dhcp_options</var></dt>
> -      <dd>
> -        Set the DHCPv4 options for the logical port.  The
> -        <var>dhcp_options</var> is a UUID referring to a set of DHCP
> options in
> -        the <ref table="DHCP_Options" /> table.
> -      </dd>
> -
> -      <dt><code>lsp-get-dhcpv4-options</code> <var>port</var></dt>
> -      <dd>
> -        Get the configured DHCPv4 options for the logical port.
> -      </dd>
> -
> -      <dt><code>lsp-set-dhcpv6-options</code> <var>port</var>
> -          <var>dhcp_options</var></dt>
> -      <dd>
> -          Set the DHCPv6 options for the logical port.  The
> -          <var>dhcp_options</var> is a UUID referring to a set of DHCP
> options
> -          in the <ref table="DHCP_Options" /> table.
> -      </dd>
> -
> -      <dt><code>lsp-get-dhcpv6-options</code> <var>port</var></dt>
> -      <dd>
> -        Get the configured DHCPv6 options for the logical port.
> -      </dd>
> -
> -      <dt><code>lsp-get-ls</code> <var>port</var></dt>
> -      <dd>
> -        Get the logical switch which the <var>port</var> belongs to.
> -      </dd>
> -
> -    </dl>
> -
> -    <h1>Logical Router Commands</h1>
> -
> -    <dl>
> -      <dt><code>lr-add</code></dt>
> -      <dd>
> -        <p>
> -          Creates a new, unnamed logical router, which initially has no
> ports.
> -          The router does not have a name, other commands must refer to
> this
> -          router by its UUID.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--may-exist</code> | <code>--add-duplicate</code>]
> <code>lr-add</code> <var>router</var></dt>
> -      <dd>
> -        <p>
> -          Creates a new logical router named <var>router</var>, which
> -          initially has no ports.
> -        </p>
> -
> -        <p>
> -          The OVN northbound database schema does not require logical
> router
> -          names to be unique, but the whole point to the names is to
> provide an
> -          easy way for humans to refer to the routers, making duplicate
> names
> -          unhelpful.  Thus, without any options, this command regards it
> as an
> -          error if <var>router</var> is a duplicate name.  With
> -          <code>--may-exist</code>, adding a duplicate name succeeds but
> does
> -          not create a new logical router.  With
> <code>--add-duplicate</code>,
> -          the command really creates a new logical router with a duplicate
> -          name.  It is an error to specify both options.  If there are
> multiple
> -          logical routers with a duplicate name, configure the logical
> routers
> -          using the UUID instead of the <var>router</var> name.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lr-del</code>
> <var>router</var></dt>
> -      <dd>
> -        Deletes <var>router</var>.  It is an error if <var>router</var>
> does
> -        not exist, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>lr-list</code></dt>
> -      <dd>
> -        Lists all existing routers on standard output, one per line.
> -      </dd>
> -    </dl>
> -
> -    <h1>Logical Router Port Commands</h1>
> -
> -    <dl>
> -      <dt>[<code>--may-exist</code>] <code>lrp-add</code>
> <var>router</var> <var>port</var> <var>mac</var> <var>network</var>...
> [<code>peer=</code><var>peer</var>]</dt>
> -      <dd>
> -        <p>
> -          Creates on <var>router</var> a new logical router port named
> -          <var>port</var> with Ethernet address <var>mac</var> and one
> -          or more IP address/netmask for each <var>network</var>.
> -        </p>
> -
> -        <p>
> -          The optional argument <code>peer</code> identifies a logical
> -          router port that connects to this one.  The following example
> -          adds a router port with an IPv4 and IPv6 address with peer
> -          <code>lr1</code>:
> -        </p>
> -
> -        <p>
> -          <code>lrp-add lr0 lrp0 00:11:22:33:44:55 192.168.0.1/24
> 2001:db8::1/64 peer=lr1</code>
> -        </p>
> -
> -        <p>
> -          It is an error if a logical router port named <var>port</var>
> -          already exists, unless <code>--may-exist</code> is specified.
> -          Regardless of <code>--may-exist</code>, it is an error if the
> -          existing router port is in some logical router other than
> -          <var>router</var>.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lrp-del</code>
> <var>port</var></dt>
> -      <dd>
> -        Deletes <var>port</var>.  It is an error if <var>port</var> does
> -        not exist, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>lrp-list</code> <var>router</var></dt>
> -      <dd>
> -        Lists all the logical router ports within <var>router</var> on
> -        standard output, one per line.
> -      </dd>
> -
> -      <dt><code>lrp-set-enabled</code> <var>port</var>
> <var>state</var></dt>
> -      <dd>
> -        Set the administrative state of <var>port</var>, either
> -        <code>enabled</code> or <code>disabled</code>.  When a port is
> -        disabled, no traffic is allowed into or out of the port.
> -      </dd>
> -
> -      <dt><code>lrp-get-enabled</code> <var>port</var></dt>
> -      <dd>
> -        Prints the administrative state of <var>port</var>, either
> -        <code>enabled</code> or <code>disabled</code>.
> -      </dd>
> -
> -      <dt><code>lrp-set-gateway-chassis</code> <var>port</var>
> -          <var>chassis</var> [<var>priority</var>]</dt>
> -      <dd>
> -        Set gateway chassis for <var>port</var>. <var>chassis</var>
> -        is the name of the chassis. This creates a gateway chassis entry
> -        in Gateway_Chassis table. It won't check if chassis really exists
> -        in OVN_Southbound database. Priority will be set to 0
> -        if <var>priority</var> is not provided by user.
> <var>priority</var>
> -        must be between <code>0</code> and <code>32767</code>, inclusive.
> -      </dd>
> -      <dt><code>lrp-del-gateway-chassis</code> <var>port</var>
> -          <var>chassis</var></dt>
> -      <dd>
> -        Deletes gateway chassis from <var>port</var>.  It is an error if
> -        gateway chassis with <var>chassis</var> for <var>port</var> does
> -        not exist.
> -      </dd>
> -      <dt><code>lrp-get-gateway-chassis</code> <var>port</var></dt>
> -      <dd>
> -        Lists all the gateway chassis with priority within
> <var>port</var> on
> -        standard output, one per line, ordered based on priority.
> -      </dd>
> -    </dl>
> -
> -    <h1>Logical Router Static Route Commands</h1>
> -
> -    <dl>
> -      <dt>[<code>--may-exist</code>]
> [<code>--policy</code>=<var>POLICY</var>] <code>lr-route-add</code>
> <var>router</var> <var>prefix</var> <var>nexthop</var>
> [<var>port</var>]</dt>
> -      <dd>
> -        <p>
> -          Adds the specified route to <var>router</var>.
> -          <var>prefix</var> describes an IPv4 or IPv6 prefix for this
> -          route, such as <code>192.168.100.0/24</code>.
> -          <var>nexthop</var> specifies the gateway to use for this
> -          route, which should be the IP address of one of
> -          <var>router</var> logical router ports or the IP address of a
> -          logical port.  If <var>port</var> is specified, packets that
> -          match this route will be sent out that port.  When
> -          <var>port</var> is omitted, OVN infers the output port based
> -          on <var>nexthop</var>.
> -        </p>
> -
> -        <p>
> -          <code>--policy</code> describes the policy used to make routing
> -          decisions.  This should be one of "dst-ip" or "src-ip".  If not
> -          specified, the default is "dst-ip".
> -        </p>
> -
> -        <p>
> -          It is an error if a route with <var>prefix</var> already exists,
> -          unless <code>--may-exist</code> is specified.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lr-route-del</code>
> <var>router</var> [<var>prefix</var>]</dt>
> -      <dd>
> -        <p>
> -          Deletes routes from <var>router</var>.  If only
> <var>router</var>
> -          is supplied, all the routes from the logical router are
> -          deleted.  If <var>prefix</var> is also specified, then all the
> -          routes that match the prefix will be deleted from the logical
> -          router.
> -        </p>
> -
> -        <p>
> -          It is an error if <var>prefix</var> is specified and there
> -          is no matching route entry, unless <code>--if-exists</code> is
> -          specified.
> -        </p>
> -      </dd>
> -
> -      <dt><code>lr-route-list</code> <var>router</var></dt>
> -      <dd>
> -        Lists the routes on <var>router</var>.
> -      </dd>
> -    </dl>
> -
> -    <h1>NAT Commands</h1>
> -
> -    <dl>
> -      <dt>[<code>--may-exist</code>] <code>lr-nat-add</code>
> <var>router</var> <var>type</var> <var>external_ip</var>
> <var>logical_ip</var> [<var>logical_port</var> <var>external_mac</var>]</dt>
> -      <dd>
> -        <p>
> -          Adds the specified NAT to <var>router</var>.
> -          The <var>type</var> must be one of <code>snat</code>,
> -          <code>dnat</code>, or <code>dnat_and_snat</code>.
> -          The <var>external_ip</var> is an IPv4 address.
> -          The <var>logical_ip</var> is an IPv4 network (e.g
> 192.168.1.0/24)
> -          or an IPv4 address.
> -          The <var>logical_port</var> and <var>external_mac</var> are only
> -          accepted when <var>router</var> is a distributed router (rather
> -          than a gateway router) and <var>type</var> is
> -          <code>dnat_and_snat</code>.
> -          The <var>logical_port</var> is the name of an existing logical
> -          switch port where the <var>logical_ip</var> resides.
> -          The <var>external_mac</var> is an Ethernet address.
> -        </p>
> -        <p>
> -          When <var>type</var> is <code>dnat</code>, the externally
> -          visible IP address <var>external_ip</var> is DNATted to the
> -          IP address <var>logical_ip</var> in the logical space.
> -        </p>
> -        <p>
> -          When <var>type</var> is <code>snat</code>, IP packets with their
> -          source IP address that either matches the IP address in
> -          <var>logical_ip</var> or is in the network provided by
> -          <var>logical_ip</var> is SNATed into the IP address in
> -          <var>external_ip</var>.
> -        </p>
> -        <p>
> -          When <var>type</var> is <code>dnat_and_snat</code>,
> -          the externally visible IP address <var>external_ip</var>
> -          is DNATted to the IP address <var>logical_ip</var> in
> -          the logical space.  In addition, IP packets with the source
> -          IP address that matches <var>logical_ip</var> is SNATed into
> -          the IP address in <var>external_ip</var>.
> -        </p>
> -        <p>
> -          When the <var>logical_port</var> and <var>external_mac</var>
> -          are specified, the NAT rule will be programmed on the chassis
> -          where the <var>logical_port</var> resides.  This includes
> -          ARP replies for the <var>external_ip</var>, which return the
> -          value of <var>external_mac</var>.  All packets transmitted
> -          with source IP address equal to <var>external_ip</var> will
> -          be sent using the <var>external_mac</var>.
> -        </p>
> -        <p>
> -          It is an error if a NAT already exists with the same values
> -          of <var>router</var>, <var>type</var>, <var>external_ip</var>,
> -          and <var>logical_ip</var>, unless <code>--may-exist</code> is
> -          specified.  When <code>--may-exist</code>,
> -          <var>logical_port</var>, and <var>external_mac</var> are all
> -          specified, the existing values of <var>logical_port</var> and
> -          <var>external_mac</var> are overwritten.
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lr-nat-del</code>
> <var>router</var> [<var>type</var> [<var>ip</var>]]</dt>
> -      <dd>
> -        <p>
> -          Deletes NATs from <var>router</var>.  If only <var>router</var>
> -          is supplied, all the NATs from the logical router are
> -          deleted.  If <var>type</var> is also specified, then all the
> -          NATs that match the <var>type</var> will be deleted from the
> logical
> -          router.  If all the fields are given, then a single NAT rule
> -          that matches all the fields will be deleted.  When
> <var>type</var>
> -          is <code>snat</code>, the <var>ip</var> should be logical_ip.
> -          When <var>type</var> is <code>dnat</code> or
> -          <code>dnat_and_snat</code>, the <var>ip</var> shoud be
> external_ip.
> -        </p>
> -
> -        <p>
> -          It is an error if <var>ip</var> is specified and there
> -          is no matching NAT entry, unless <code>--if-exists</code> is
> -          specified.
> -        </p>
> -      </dd>
> -
> -      <dt><code>lr-nat-list</code> <var>router</var></dt>
> -      <dd>
> -        Lists the NATs on <var>router</var>.
> -      </dd>
> -    </dl>
> -
> -    <h1>Load Balancer Commands</h1>
> -    <dl>
> -      <dt>[<code>--may-exist</code> | <code>--add-duplicate</code>]
> <code>lb-add</code> <var>lb</var> <var>vip</var> <var>ips</var>
> [<var>protocol</var>]</dt>
> -      <dd>
> -        <p>
> -         Creates a new load balancer named <var>lb</var> with the provided
> -         <var>vip</var> and <var>ips</var> or adds the <var>vip</var> to
> -         an existing <var>lb</var>.  <var>vip</var> should be a
> -         virtual IP address (or an IP address and a port number with
> -         <code>:</code> as a separator).  Examples for <var>vip</var> are
> -         <code>192.168.1.4</code>, <code>fd0f::1</code>, and
> -         <code>192.168.1.5:8080</code>. <var>ips</var> should be comma
> -         separated IP endpoints (or comma separated IP addresses and port
> -         numbers with <code>:</code> as a separator).  <var>ips</var> must
> -         be the same address family as <var>vip</var>.  Examples for
> -         <var>ips</var> are <code>10.0.0.1,10.0.0.2</code>or
> -         <code>[fdef::1]:8800,[fdef::2]:8800</code>.
> -        </p>
> -
> -        <p>
> -         The optional argument <var>protocol</var> must be either
> -         <code>tcp</code> or <code>udp</code>.  This argument is useful
> when
> -         a port number is provided as part of the <var>vip</var>.  If the
> -         <var>protocol</var> is unspecified and a port number is provided
> as
> -         part of the <var>vip</var>, OVN assumes the <var>protocol</var>
> to
> -         be <code>tcp</code>.
> -        </p>
> -
> -        <p>
> -         It is an error if the <var>vip</var> already exists in the load
> -         balancer named <var>lb</var>, unless <code>--may-exist</code> is
> -         specified.  With <code>--add-duplicate</code>, the command really
> -         creates a new load balancer with a duplicate name.
> -        </p>
> -
> -        <p>
> -         The following example adds a load balancer.
> -        </p>
> -
> -        <p>
> -         <code>lb-add lb0 30.0.0.10:80
> -         192.168.10.10:80,192.168.10.20:80,192.168.10.30:80 udp</code>
> -        </p>
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lb-del</code> <var>lb</var>
> [<var>vip</var>]</dt>
> -      <dd>
> -        Deletes <var>lb</var> or the <var>vip</var> from <var>lb</var>.
> -        If <var>vip</var> is supplied, only the <var>vip</var> will be
> -        deleted from the <var>lb</var>.  If only the <var>lb</var> is
> supplied,
> -        the <var>lb</var> will be deleted.  It is an error if
> <var>vip</var>
> -        does not already exist in <var>lb</var>, unless
> -        <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>lb-list</code> [<var>lb</var>]</dt>
> -      <dd>
> -        Lists the LBs.  If <var>lb</var> is also specified, then only the
> -        specified <var>lb</var> will be listed.
> -      </dd>
> -
> -      <dt>[<code>--may-exist</code>] <code>ls-lb-add</code>
> <var>switch</var> <var>lb</var></dt>
> -      <dd>
> -        Adds the specified <var>lb</var> to <var>switch</var>.
> -        It is an error if a load balancer named <var>lb</var> already
> exists
> -        in the <var>switch</var>, unless <code>--may-exist</code> is
> specified.
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>ls-lb-del</code>
> <var>switch</var> [<var>lb</var>]</dt>
> -      <dd>
> -        Removes <var>lb</var> from <var>switch</var>.  If only
> -        <var>switch</var> is supplied, all the LBs from the logical
> switch are
> -        removed.  If <var>lb</var> is also specified, then only the
> -        <var>lb</var> will be removed from the logical switch.
> -        It is an error if <var>lb</var> does not exist in the
> -        <var>switch</var>, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>ls-lb-list</code> <var>switch</var></dt>
> -      <dd>
> -        Lists the LBs for the given <var>switch</var>.
> -      </dd>
> -
> -      <dt>[<code>--may-exist</code>] <code>lr-lb-add</code>
> <var>router</var> <var>lb</var></dt>
> -      <dd>
> -        Adds the specified <var>lb</var> to <var>router</var>.
> -        It is an error if a load balancer named <var>lb</var> already
> exists
> -        in the <var>router</var>, unless <code>--may-exist</code> is
> specified.
> -      </dd>
> -
> -      <dt>[<code>--if-exists</code>] <code>lr-lb-del</code>
> <var>router</var> [<var>lb</var>]</dt>
> -      <dd>
> -        Removes <var>lb</var> from <var>router</var>.  If only
> -        <var>router</var> is supplied, all the LBs from the logical
> router are
> -        removed.  If <var>lb</var> is also specified, then only the
> -        <var>lb</var> will be removed from the logical router.
> -        It is an error if <var>lb</var> does not exist in the
> -        <var>router</var>, unless <code>--if-exists</code> is specified.
> -      </dd>
> -
> -      <dt><code>lr-lb-list</code> <var>router</var></dt>
> -      <dd>
> -        Lists the LBs for the given <var>router</var>.
> -      </dd>
> -    </dl>
> -
> -
> -    <h1>DHCP Options commands</h1>
> -
> -    <dl>
> -      <dt><code>dhcp-options-create</code> <var>cidr</var>
> [<var>key=value</var>]</dt>
> -      <dd>
> -        Creates a new DHCP Options entry in the <code>DHCP_Options</code>
> table
> -        with the specified <code>cidr</code> and optional
> <code>external-ids</code>.
> -      </dd>
> -
> -      <dt><code>dhcp-options-list</code></dt>
> -      <dd>
> -        Lists the DHCP Options entries.
> -      </dd>
> -
> -      <dt><code>dhcp-options-del</code> <var>dhcp-option</var></dt>
> -      <dd>
> -        Deletes the DHCP Options entry referred by <var>dhcp-option</var>
> UUID.
> -      </dd>
> -
> -      <dt><code>dhcp-options-set-options</code> <var>dhcp-option</var>
> [<var>key=value</var>]...</dt>
> -      <dd>
> -        Set the DHCP Options for the <var>dhcp-option</var> UUID.
> -      </dd>
> -
> -      <dt><code>dhcp-options-get-options</code>
> <var>dhcp-option</var></dt>
> -      <dd>
> -        Lists the DHCP Options for the <var>dhcp-option</var> UUID.
> -      </dd>
> -    </dl>
> -
> -    <h1>Port Group commands</h1>
> -
> -    <dl>
> -      <dt><code>pg-add</code> <var>group</var> [<var>port</var>]...</dt>
> -      <dd>
> -        Creates a new port group in the <code>Port_Group</code> table
> named
> -        <code>group</code> with optional <code>ports</code> added to the
> group.
> -      </dd>
> -
> -      <dt><code>pg-set-ports</code> <var>group</var>
> <var>port</var>...</dt>
> -      <dd>
> -        Sets <code>ports</code> on the port group named
> <code>group</code>. It
> -        is an error if <code>group</code> does not exist.
> -      </dd>
> -
> -      <dt><code>pg-del</code> <var>group</var></dt>
> -      <dd>
> -        Deletes port group <code>group</code>. It is an error if
> -        <code>group</code> does not exist.
> -      </dd>
> -    </dl>
> -
> -    <h1> HA Chassis Group commands</h1>
> -
> -    <dl>
> -      <dt><code>ha-chassis-group-add</code> <var>group</var></dt>
> -      <dd>
> -        Creates a new HA chassis group in the
> <code>HA_Chassis_Group</code>
> -        table named <code>group</code>.
> -      </dd>
> -
> -      <dt><code>ha-chassis-group-del</code> <var>group</var></dt>
> -      <dd>
> -        Deletes the HA chassis group <code>group</code>. It is an error if
> -        <code>group</code> does not exist.
> -      </dd>
> -
> -      <dt><code>ha-chassis-group-list</code></dt>
> -      <dd>
> -        Lists the HA chassis group <code>group</code> along with the
> -        <code>HA chassis</code> if any associated with it.
> -      </dd>
> -
> -      <dt><code>ha-chassis-group-add-chassis</code> <var>group</var>
> -      <var>chassis</var> <var>priority</var></dt>
> -      <dd>
> -        Adds a new HA chassis <code>chassis</code> to the
> -        HA Chassis group <code>group</code> with the specified priority.
> -        If the <code>chassis</code> already exists, then the
> -        <code>priority</code> is updated.
> -        The <code>chassis</code> should be the name of the chassis in the
> -        <code>OVN_Southbound</code>.
> -      </dd>
> -
> -      <dt><code>ha-chassis-group-remove-chassis</code> <var>group</var>
> -      <var>chassis</var></dt>
> -      <dd>
> -        Removes the HA chassis <code>chassis</code> from the HA chassis
> -        group <code>group</code>. It is an error if <code>chassis</code>
> does
> -        not exist.
> -      </dd>
> -    </dl>
> -
> -    <h1>Database Commands</h1>
> -    <p>These commands query and modify the contents of <code>ovsdb</code>
> tables.
> -    They are a slight abstraction of the <code>ovsdb</code> interface and
> -    as such they operate at a lower level than other
> <code>ovn-nbctl</code> commands.</p>
> -    <p><var>Identifying Tables, Records, and Columns</var></p>
> -    <p>Each of these commands has a <var>table</var> parameter to
> identify a table
> -    within the database.  Many of them also take a <var>record</var>
> parameter
> -    that identifies a particular record within a table.  The
> <var>record</var>
> -    parameter may be the UUID for a record, which may be abbreviated to
> its
> -    first 4 (or more) hex digits, as long as that is unique.  Many tables
> offer
> -    additional ways to identify records.  Some commands also take
> -    <var>column</var> parameters that identify a particular field within
> the
> -    records in a table.</p>
> -
> -    <p>
> -      For a list of tables and their columns, see <code>ovn-nb</code>(5)
> or
> -      see the table listing from the <code>--help</code> option.
> -    </p>
> -
> -    <p>
> -      Record names must be specified in full and with correct
> capitalization,
> -      except that UUIDs may be abbreviated to their first 4 (or more) hex
> -      digits, as long as that is unique within the table.  Names of
> tables and
> -      columns are not case-sensitive, and <code>-</code> and
> <code>_</code> are
> -      treated interchangeably.  Unique abbreviations of table and column
> names
> -      are acceptable, e.g. <code>d</code> or <code>dhcp</code> is
> sufficient
> -      to identify the <code>DHCP_Options</code> table.
> -    </p>
> -
> -    <xi:include href="lib/db-ctl-base.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h1>Synchronization Commands</h1>
> -
> -    <dl>
> -      <dt>sync</dt>
> -      <dd>
> -        Ordinarily, <code>--wait=sb</code> or <code>--wait=hv</code> only
> waits
> -        for changes by the current <code>ovn-nbctl</code> invocation to
> take
> -        effect.  This means that, if none of the commands supplied to
> -        <code>ovn-nbctl</code> change the database, then the command does
> not
> -        wait at all.  With the <code>sync</code> command, however,
> -        <code>ovn-nbctl</code> waits even for earlier changes to the
> database
> -        to propagate down to the southbound database or all of the OVN
> chassis,
> -        according to the argument to <code>--wait</code>.
> -      </dd>
> -    </dl>
> -
> -    <h1>Remote Connectivity Commands</h1>
> -    <dl>
> -      <dt><code>get-connection</code></dt>
> -      <dd>
> -      Prints the configured connection(s).
> -      </dd>
> -
> -      <dt><code>del-connection</code></dt>
> -      <dd>
> -      Deletes the configured connection(s).
> -      </dd>
> -
> -      <dt>[<code>--inactivity-probe=</code><var>msecs</var>]
> <code>set-connection</code> <var>target</var>...</dt>
> -      <dd>
> -        Sets the configured manager target or targets.  Use
> -        <code>--inactivity-probe=</code><var>msecs</var> to override the
> default
> -        idle connection inactivity probe time.  Use 0 to disable
> inactivity probes.
> -      </dd>
> -    </dl>
> -
> -    <h1>SSL Configuration Commands</h1>
> -    <dl>
> -      <dt><code>get-ssl</code></dt>
> -      <dd>
> -      Prints the SSL configuration.
> -      </dd>
> -
> -      <dt><code>del-ssl</code></dt>
> -      <dd>
> -      Deletes the current SSL configuration.
> -      </dd>
> -
> -      <dt>[<code>--bootstrap</code>] <code>set-ssl</code>
> -         <var>private-key</var> <var>certificate</var> <var>ca-cert</var>
> -         [<var>ssl-protocol-list</var> [<var>ssl-cipher-list</var>]]</dt>
> -      <dd>
> -      Sets the SSL configuration.
> -      </dd>
> -    </dl>
> -
> -    <h1>Daemon Mode</h1>
> -
> -    <p>
> -      When it is invoked in the most ordinary way, <code>ovn-nbctl</code>
> -      connects to an OVSDB server that hosts the northbound database,
> retrieves
> -      a partial copy of the database that is complete enough to do its
> work,
> -      sends a transaction request to the server, and receives and
> processes the
> -      server's reply.  In common interactive use, this is fine, but if the
> -      database is large, the step in which <code>ovn-nbctl</code>
> retrieves a
> -      partial copy of the database can take a long time, which yields poor
> -      performance overall.
> -    </p>
> -
> -    <p>
> -      To improve performance in such a case, <code>ovn-nbctl</code>
> offers a
> -      "daemon mode," in which the user first starts <code>ovn-nbctl</code>
> -      running in the background and afterward uses the daemon to execute
> -      operations.  Over several <code>ovn-nbctl</code> command
> invocations,
> -      this performs better overall because it retrieves a copy of the
> database
> -      only once at the beginning, not once per program run.
> -    </p>
> -
> -    <p>
> -      Use the <code>--detach</code> option to start an
> <code>ovn-nbctl</code>
> -      daemon.  With this option, <code>ovn-nbctl</code> prints the name
> of a
> -      control socket to stdout.  The client should save this name in
> -      environment variable <env>OVN_NB_DAEMON</env>.  Under the Bourne
> shell
> -      this might be done like this:
> -    </p>
> -
> -    <pre fixed="yes">
> -      export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
> -    </pre>
> -
> -    <p>
> -      When <env>OVN_NB_DAEMON</env> is set, <code>ovn-nbctl</code>
> -      automatically and transparently uses the daemon to execute its
> commands.
> -    </p>
> -
> -    <p>
> -      When the daemon is no longer needed, kill it and unset the
> environment
> -      variable, e.g.:
> -    </p>
> -
> -    <pre fixed="yes">
> -      kill $(cat /var/run/ovn-nbctl.pid)
> -      unset OVN_NB_DAEMON
> -    </pre>
> -
> -    <p>
> -      Daemon mode is experimental.
> -    </p>
> -
> -    <h2>Daemon Commands</h2>
> -
> -    <p>
> -      Daemon mode is internally implemented using the same mechanism used
> by
> -      <code>ovs-appctl</code>.  One may also use <code>ovs-appctl</code>
> -      directly with the following commands:
> -    </p>
> -
> -    <dl>
> -      <dt>
> -        <code>run</code> [<var>options</var>] <var>command</var>
> -        [<var>arg</var>...] [<code>--</code> [<var>options</var>]
> -        <var>command</var> [<var>arg</var>...] ...]
> -      </dt>
> -      <dd>
> -        Instructs the daemon process to run one or more
> <code>ovn-nbctl</code>
> -        commands described above and reply with the results of running
> these
> -        commands. Accepts the <code>--no-wait</code>, <code>--wait</code>,
> -        <code>--timeout</code>, <code>--dry-run</code>,
> <code>--oneline</code>,
> -        and the options described under <code>Table Formatting
> Options</code>
> -        in addition to the the command-specific options.
> -      </dd>
> -
> -      <dt><code>exit</code></dt>
> -      <dd>Causes <code>ovn-nbctl</code> to gracefully terminate.</dd>
> -    </dl>
> -
> -    <h1>Options</h1>
> -
> -    <dl>
> -      <dt><code>--no-wait</code> | <code>--wait=none</code></dt>
> -      <dt><code>--wait=sb</code></dt>
> -      <dt><code>--wait=hv</code></dt>
> -
> -      <dd>
> -        <p>
> -          These options control whether and how <code>ovn-nbctl</code>
> waits
> -          for the OVN system to become up-to-date with changes made in an
> -          <code>ovn-nbctl</code> invocation.
> -        </p>
> -
> -        <p>
> -          By default, or if <code>--no-wait</code> or
> <code>--wait=none</code>,
> -          <code>ovn-nbctl</code> exits immediately after confirming that
> -          changes have been committed to the northbound database, without
> -          waiting.
> -        </p>
> -
> -        <p>
> -          With <code>--wait=sb</code>, before <code>ovn-nbctl</code>
> exits, it
> -          waits for <code>ovn-northd</code> to bring the southbound
> database
> -          up-to-date with the northbound database updates.
> -        </p>
> -
> -        <p>
> -          With <code>--wait=hv</code>, before <code>ovn-nbctl</code>
> exits, it
> -          additionally waits for all OVN chassis (hypervisors and
> gateways) to
> -          become up-to-date with the northbound database updates.  (This
> can
> -          become an indefinite wait if any chassis is malfunctioning.)
> -        </p>
> -
> -        <p>
> -          Ordinarily, <code>--wait=sb</code> or <code>--wait=hv</code>
> only
> -          waits for changes by the current <code>ovn-nbctl</code>
> invocation to
> -          take effect.  This means that, if none of the commands supplied
> to
> -          <code>ovn-nbctl</code> change the database, then the command
> does not
> -          wait at all.  Use the <code>sync</code> command to override this
> -          behavior.
> -        </p>
> -      </dd>
> -
> -    <dt><code>--db</code> <var>database</var></dt>
> -    <dd>
> -      The OVSDB database remote to contact.  If the <env>OVN_NB_DB</env>
> -      environment variable is set, its value is used as the default.
> -      Otherwise, the default is <code>unix:@RUNDIR@/ovnnb_db.sock</code>,
> but this
> -      default is unlikely to be useful outside of single-machine OVN test
> -      environments.
> -    </dd>
> -
> -    <dt><code>--leader-only</code></dt>
> -    <dt><code>--no-leader-only</code></dt>
> -    <dd>
> -      By default, or with <code>--leader-only</code>, when the database
> server
> -      is a clustered database, <code>ovn-nbctl</code> will avoid servers
> other
> -      than the cluster leader.  This ensures that any data that
> -      <code>ovn-nbctl</code> reads and reports is up-to-date.  With
> -      <code>--no-leader-only</code>, <code>ovn-nbctl</code> will use any
> server
> -      in the cluster, which means that for read-only transactions it can
> report
> -      and act on stale data (transactions that modify the database are
> always
> -      serialized even with <code>--no-leader-only</code>).  Refer to
> -      <code>Understanding Cluster Consistency</code> in
> <code>ovsdb</code>(7)
> -      for more information.
> -    </dd>
> -
> -    <dt><code>--shuffle-remotes</code></dt>
> -    <dt><code>--no-shuffle-remotes</code></dt>
> -    <dd>
> -      By default, or with <code>--shuffle-remotes</code>, when there are
> -      multiple remotes specified in the OVSDB connection string specified
> by
> -      <code>--db</code> or the <env>OVN_NB_DB</env> environment variable,
> -      the order of the remotes will be shuffled before the client tries to
> -      connect.  The remotes will be shuffled only once to a new order
> before
> -      the first connection attempt.  The following retries, if any, will
> -      follow the same new order.  The default behavior is to make sure
> -      clients of a clustered database can distribute evenly to all
> memembers
> -      of the cluster.  With <code>--no-shuffle-remotes</code>,
> -      <code>ovn-nbctl</code> will use the original order specified in the
> -      connection string to connect.  This allows user to specify the
> -      preferred order, which is particularly useful for testing.
> -    </dd>
> -    </dl>
> -
> -    <h2>Daemon Options</h2>
> -    <xi:include href="lib/daemon.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h1>Logging options</h1>
> -    <xi:include href="lib/vlog.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h1>Table Formatting Options</h1>
> -    These options control the format of output from the <code>list</code>
> and
> -    <code>find</code> commands.
> -    <xi:include href="lib/table.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>PKI Options</h2>
> -    <p>
> -      PKI configuration is required to use SSL for the connection to the
> -      database.
> -    </p>
> -    <xi:include href="lib/ssl.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -    <xi:include href="lib/ssl-bootstrap.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -    <h2>Other Options</h2>
> -
> -    <xi:include href="lib/common.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -</manpage>
> diff --git a/ovn/utilities/ovn-nbctl.c b/ovn/utilities/ovn-nbctl.c
> deleted file mode 100644
> index 98a8faa0b..000000000
> --- a/ovn/utilities/ovn-nbctl.c
> +++ /dev/null
> @@ -1,6061 +0,0 @@
> -/*
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <getopt.h>
> -#include <inttypes.h>
> -#include <stdlib.h>
> -#include <stdio.h>
> -
> -#include "command-line.h"
> -#include "daemon.h"
> -#include "db-ctl-base.h"
> -#include "dirs.h"
> -#include "fatal-signal.h"
> -#include "jsonrpc.h"
> -#include "openvswitch/json.h"
> -#include "ovn/lib/acl-log.h"
> -#include "ovn/lib/ovn-nb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "packets.h"
> -#include "openvswitch/poll-loop.h"
> -#include "process.h"
> -#include "smap.h"
> -#include "sset.h"
> -#include "stream.h"
> -#include "stream-ssl.h"
> -#include "svec.h"
> -#include "table.h"
> -#include "timeval.h"
> -#include "timer.h"
> -#include "unixctl.h"
> -#include "util.h"
> -#include "openvswitch/vlog.h"
> -
> -VLOG_DEFINE_THIS_MODULE(nbctl);
> -
> -/* --db: The database server to contact. */
> -static const char *db;
> -
> -/* --oneline: Write each command's output as a single line? */
> -static bool oneline;
> -
> -/* --dry-run: Do not commit any changes. */
> -static bool dry_run;
> -
> -/* --wait=TYPE: Wait for configuration change to take effect? */
> -enum nbctl_wait_type {
> -    NBCTL_WAIT_NONE,            /* Do not wait. */
> -    NBCTL_WAIT_SB,              /* Wait for southbound database updates.
> */
> -    NBCTL_WAIT_HV               /* Wait for hypervisors to catch up. */
> -};
> -static enum nbctl_wait_type wait_type = NBCTL_WAIT_NONE;
> -
> -/* Should we wait (if specified by 'wait_type') even if the commands don't
> - * change the database at all? */
> -static bool force_wait = false;
> -
> -/* --timeout: Time to wait for a connection to 'db'. */
> -static unsigned int timeout;
> -
> -/* Format for table output. */
> -static struct table_style table_style = TABLE_STYLE_DEFAULT;
> -
> -/* The IDL we're using and the current transaction, if any.
> - * This is for use by nbctl_exit() only, to allow it to clean up.
> - * Other code should use its context arguments. */
> -static struct ovsdb_idl *the_idl;
> -static struct ovsdb_idl_txn *the_idl_txn;
> -OVS_NO_RETURN static void nbctl_exit(int status);
> -
> -/* --leader-only, --no-leader-only: Only accept the leader in a cluster.
> */
> -static int leader_only = true;
> -
> -/* --shuffle-remotes, --no-shuffle-remotes: Shuffle the order of remotes
> that
> - * are specified in the connetion method string. */
> -static int shuffle_remotes = true;
> -
> -/* --unixctl-path: Path to use for unixctl server, for "monitor" and
> "snoop"
> -     commands. */
> -static char *unixctl_path;
> -
> -static unixctl_cb_func server_cmd_exit;
> -static unixctl_cb_func server_cmd_run;
> -
> -static void nbctl_cmd_init(void);
> -OVS_NO_RETURN static void usage(void);
> -static struct option *get_all_options(void);
> -static bool has_option(const struct ovs_cmdl_parsed_option *, size_t n,
> -                       int option);
> -static void nbctl_client(const char *socket_name,
> -                         const struct ovs_cmdl_parsed_option *, size_t n,
> -                         int argc, char *argv[]);
> -static bool will_detach(const struct ovs_cmdl_parsed_option *, size_t n);
> -static void apply_options_direct(const struct ovs_cmdl_parsed_option *,
> -                                 size_t n, struct shash *local_options);
> -static char * OVS_WARN_UNUSED_RESULT run_prerequisites(struct
> ctl_command[],
> -                                                       size_t n_commands,
> -                                                       struct ovsdb_idl
> *);
> -static char * OVS_WARN_UNUSED_RESULT do_nbctl(const char *args,
> -                                              struct ctl_command *,
> size_t n,
> -                                              struct ovsdb_idl *,
> -                                              const struct timer *,
> -                                              bool *retry);
> -static char * OVS_WARN_UNUSED_RESULT dhcp_options_get(
> -    struct ctl_context *ctx, const char *id, bool must_exist,
> -    const struct nbrec_dhcp_options **);
> -static char * OVS_WARN_UNUSED_RESULT main_loop(const char *args,
> -                                               struct ctl_command
> *commands,
> -                                               size_t n_commands,
> -                                               struct ovsdb_idl *idl,
> -                                               const struct timer *);
> -static void server_loop(struct ovsdb_idl *idl, int argc, char *argv[]);
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    struct ovsdb_idl *idl;
> -    struct shash local_options;
> -
> -    set_program_name(argv[0]);
> -    fatal_ignore_sigpipe();
> -    vlog_set_levels(NULL, VLF_CONSOLE, VLL_WARN);
> -    vlog_set_levels_from_string_assert("reconnect:warn");
> -
> -    nbctl_cmd_init();
> -
> -    /* ovn-nbctl has three operation modes:
> -     *
> -     *    - Direct: Executes commands by contacting ovsdb-server directly.
> -     *
> -     *    - Server: Runs in the background as a daemon waiting for
> requests
> -     *      from ovn-nbctl running in client mode.
> -     *
> -     *    - Client: Executes commands by passing them to an ovn-nbctl
> running
> -     *      in the server mode.
> -     *
> -     * At this point we don't know what mode we're running in.  The mode
> partly
> -     * depends on the command line.  So, for now we transform the command
> line
> -     * into a parsed form, and figure out what to do with it later.
> -     */
> -    char *args = process_escape_args(argv);
> -    struct ovs_cmdl_parsed_option *parsed_options;
> -    size_t n_parsed_options;
> -    char *error_s = ovs_cmdl_parse_all(argc, argv, get_all_options(),
> -                                       &parsed_options,
> &n_parsed_options);
> -    if (error_s) {
> -        free(args);
> -        ctl_fatal("%s", error_s);
> -    }
> -
> -    /* Now figure out the operation mode:
> -     *
> -     *    - A --detach option implies server mode.
> -     *
> -     *    - An OVN_NB_DAEMON environment variable implies client mode.
> -     *
> -     *    - Otherwise, we're in direct mode. */
> -    char *socket_name = getenv("OVN_NB_DAEMON");
> -    if (socket_name && socket_name[0]
> -        && !will_detach(parsed_options, n_parsed_options)) {
> -        nbctl_client(socket_name, parsed_options, n_parsed_options,
> -                     argc, argv);
> -    }
> -
> -    /* Parse command line. */
> -    shash_init(&local_options);
> -    apply_options_direct(parsed_options, n_parsed_options,
> &local_options);
> -    free(parsed_options);
> -
> -    bool daemon_mode = false;
> -    if (get_detach()) {
> -        if (argc != optind) {
> -            free(args);
> -            ctl_fatal("non-option arguments not supported with --detach "
> -                      "(use --help for help)");
> -        }
> -        daemon_mode = true;
> -    }
> -    /* Initialize IDL. */
> -    idl = the_idl = ovsdb_idl_create_unconnected(&nbrec_idl_class, true);
> -    ovsdb_idl_set_shuffle_remotes(idl, shuffle_remotes);
> -    /* "retry" is true iff in daemon mode. */
> -    ovsdb_idl_set_remote(idl, db, daemon_mode);
> -    ovsdb_idl_set_leader_only(idl, leader_only);
> -
> -    if (daemon_mode) {
> -        server_loop(idl, argc, argv);
> -    } else {
> -        struct ctl_command *commands;
> -        size_t n_commands;
> -        char *error;
> -
> -        error = ctl_parse_commands(argc - optind, argv + optind,
> -                                   &local_options, &commands,
> &n_commands);
> -        if (error) {
> -            free(args);
> -            ctl_fatal("%s", error);
> -        }
> -        VLOG(ctl_might_write_to_db(commands, n_commands) ? VLL_INFO :
> VLL_DBG,
> -             "Called as %s", args);
> -
> -        ctl_timeout_setup(timeout);
> -
> -        error = run_prerequisites(commands, n_commands, idl);
> -        if (error) {
> -            free(args);
> -            ctl_fatal("%s", error);
> -        }
> -
> -        error = main_loop(args, commands, n_commands, idl, NULL);
> -        if (error) {
> -            free(args);
> -            ctl_fatal("%s", error);
> -        }
> -
> -        struct ctl_command *c;
> -        for (c = commands; c < &commands[n_commands]; c++) {
> -            ds_destroy(&c->output);
> -            table_destroy(c->table);
> -            free(c->table);
> -            shash_destroy_free_data(&c->options);
> -        }
> -        free(commands);
> -    }
> -
> -    ovsdb_idl_destroy(idl);
> -    idl = the_idl = NULL;
> -
> -    free(args);
> -    exit(EXIT_SUCCESS);
> -}
> -
> -static char *
> -main_loop(const char *args, struct ctl_command *commands, size_t
> n_commands,
> -          struct ovsdb_idl *idl, const struct timer *wait_timeout)
> -{
> -    unsigned int seqno;
> -    bool idl_ready;
> -
> -    /* Execute the commands.
> -     *
> -     * 'seqno' is the database sequence number for which we last tried to
> -     * execute our transaction.  There's no point in trying to commit
> more than
> -     * once for any given sequence number, because if the transaction
> fails
> -     * it's because the database changed and we need to obtain an
> up-to-date
> -     * view of the database before we try the transaction again. */
> -    seqno = ovsdb_idl_get_seqno(idl);
> -
> -    /* IDL might have already obtained the database copy during previous
> -     * invocation. If so, we can't expect the sequence number to change
> before
> -     * we issue any new requests. */
> -    idl_ready = ovsdb_idl_has_ever_connected(idl);
> -    for (;;) {
> -        ovsdb_idl_run(idl);
> -        if (!ovsdb_idl_is_alive(idl)) {
> -            int retval = ovsdb_idl_get_last_error(idl);
> -            ctl_fatal("%s: database connection failed (%s)",
> -                      db, ovs_retval_to_string(retval));
> -        }
> -
> -        if (idl_ready || seqno != ovsdb_idl_get_seqno(idl)) {
> -            idl_ready = false;
> -            seqno = ovsdb_idl_get_seqno(idl);
> -
> -            bool retry;
> -            char *error = do_nbctl(args, commands, n_commands, idl,
> -                                   wait_timeout, &retry);
> -            if (error) {
> -                return error;
> -            }
> -            if (!retry) {
> -                return NULL;
> -            }
> -        }
> -
> -        if (seqno == ovsdb_idl_get_seqno(idl)) {
> -            ovsdb_idl_wait(idl);
> -            poll_block();
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -/* All options that affect the main loop and are not external. */
> -#define MAIN_LOOP_OPTION_ENUMS                  \
> -        OPT_NO_WAIT,                            \
> -        OPT_WAIT,                               \
> -        OPT_DRY_RUN,                            \
> -        OPT_ONELINE
> -
> -#define MAIN_LOOP_LONG_OPTIONS                           \
> -        {"no-wait", no_argument, NULL, OPT_NO_WAIT},     \
> -        {"wait", required_argument, NULL, OPT_WAIT},     \
> -        {"dry-run", no_argument, NULL, OPT_DRY_RUN},     \
> -        {"oneline", no_argument, NULL, OPT_ONELINE},     \
> -        {"timeout", required_argument, NULL, 't'}
> -
> -enum {
> -    OPT_DB = UCHAR_MAX + 1,
> -    OPT_NO_SYSLOG,
> -    OPT_LOCAL,
> -    OPT_COMMANDS,
> -    OPT_OPTIONS,
> -    OPT_LEADER_ONLY,
> -    OPT_NO_LEADER_ONLY,
> -    OPT_SHUFFLE_REMOTES,
> -    OPT_NO_SHUFFLE_REMOTES,
> -    OPT_BOOTSTRAP_CA_CERT,
> -    MAIN_LOOP_OPTION_ENUMS,
> -    DAEMON_OPTION_ENUMS,
> -    VLOG_OPTION_ENUMS,
> -    TABLE_OPTION_ENUMS,
> -    SSL_OPTION_ENUMS,
> -};
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -handle_main_loop_option(int opt, const char *arg, bool *handled)
> -{
> -    ovs_assert(handled);
> -    *handled = true;
> -
> -    switch (opt) {
> -    case OPT_ONELINE:
> -        oneline = true;
> -        break;
> -
> -    case OPT_NO_WAIT:
> -        wait_type = NBCTL_WAIT_NONE;
> -        break;
> -
> -    case OPT_WAIT:
> -        if (!strcmp(arg, "none")) {
> -            wait_type = NBCTL_WAIT_NONE;
> -        } else if (!strcmp(arg, "sb")) {
> -            wait_type = NBCTL_WAIT_SB;
> -        } else if (!strcmp(arg, "hv")) {
> -            wait_type = NBCTL_WAIT_HV;
> -        } else {
> -            return xstrdup("argument to --wait must be "
> -                           "\"none\", \"sb\", or \"hv\"");
> -        }
> -        break;
> -
> -    case OPT_DRY_RUN:
> -        dry_run = true;
> -        break;
> -
> -    case 't':
> -        if (!str_to_uint(arg, 10, &timeout) || !timeout) {
> -            return xasprintf("value %s on -t or --timeout is invalid",
> arg);
> -        }
> -        break;
> -
> -    default:
> -        *handled = false;
> -        break;
> -    }
> -
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -build_short_options(const struct option *long_options, bool print_errors)
> -{
> -    char *tmp, *short_options;
> -
> -    tmp = ovs_cmdl_long_options_to_short_options(long_options);
> -    short_options = xasprintf("+%s%s", print_errors ? "" : ":", tmp);
> -    free(tmp);
> -
> -    return short_options;
> -}
> -
> -static struct option * OVS_WARN_UNUSED_RESULT
> -append_command_options(const struct option *options, int opt_val)
> -{
> -    struct option *o;
> -    size_t n_allocated;
> -    size_t n_existing;
> -    int i;
> -
> -    for (i = 0; options[i].name; i++) {
> -        ;
> -    }
> -    n_allocated = i + 1;
> -    n_existing = i;
> -
> -    /* We want to parse both global and command-specific options here, but
> -     * getopt_long() isn't too convenient for the job.  We copy our global
> -     * options into a dynamic array, then append all of the
> command-specific
> -     * options. */
> -    o = xmemdup(options, n_allocated * sizeof *options);
> -    ctl_add_cmd_options(&o, &n_existing, &n_allocated, opt_val);
> -
> -    return o;
> -}
> -
> -static struct option *
> -get_all_options(void)
> -{
> -    static const struct option global_long_options[] = {
> -        {"db", required_argument, NULL, OPT_DB},
> -        {"no-syslog", no_argument, NULL, OPT_NO_SYSLOG},
> -        {"help", no_argument, NULL, 'h'},
> -        {"commands", no_argument, NULL, OPT_COMMANDS},
> -        {"options", no_argument, NULL, OPT_OPTIONS},
> -        {"leader-only", no_argument, NULL, OPT_LEADER_ONLY},
> -        {"no-leader-only", no_argument, NULL, OPT_NO_LEADER_ONLY},
> -        {"shuffle-remotes", no_argument, NULL, OPT_SHUFFLE_REMOTES},
> -        {"no-shuffle-remotes", no_argument, NULL, OPT_NO_SHUFFLE_REMOTES},
> -        {"version", no_argument, NULL, 'V'},
> -        MAIN_LOOP_LONG_OPTIONS,
> -        DAEMON_LONG_OPTIONS,
> -        VLOG_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {"bootstrap-ca-cert", required_argument, NULL,
> OPT_BOOTSTRAP_CA_CERT},
> -        TABLE_LONG_OPTIONS,
> -        {NULL, 0, NULL, 0},
> -    };
> -
> -    static struct option *options;
> -    if (!options) {
> -        options = append_command_options(global_long_options, OPT_LOCAL);
> -    }
> -
> -    return options;
> -}
> -
> -static bool
> -has_option(const struct ovs_cmdl_parsed_option *parsed_options, size_t n,
> -           int option)
> -{
> -    for (const struct ovs_cmdl_parsed_option *po = parsed_options;
> -         po < &parsed_options[n]; po++) {
> -        if (po->o->val == option) {
> -            return true;
> -        }
> -    }
> -    return false;
> -}
> -
> -static bool
> -will_detach(const struct ovs_cmdl_parsed_option *parsed_options, size_t n)
> -{
> -    return has_option(parsed_options, n, OPT_DETACH);
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -add_local_option(const char *name, const char *arg,
> -                 struct shash *local_options)
> -{
> -    char *full_name = xasprintf("--%s", name);
> -    if (shash_find(local_options, full_name)) {
> -        char *error = xasprintf("'%s' option specified multiple times",
> -                                full_name);
> -        free(full_name);
> -        return error;
> -    }
> -    shash_add_nocopy(local_options, full_name, nullable_xstrdup(arg));
> -    return NULL;
> -}
> -
> -static void
> -apply_options_direct(const struct ovs_cmdl_parsed_option *parsed_options,
> -                     size_t n, struct shash *local_options)
> -{
> -    for (const struct ovs_cmdl_parsed_option *po = parsed_options;
> -         po < &parsed_options[n]; po++) {
> -        bool handled;
> -        char *error = handle_main_loop_option(po->o->val, po->arg,
> &handled);
> -        if (error) {
> -            ctl_fatal("%s", error);
> -        }
> -        if (handled) {
> -            continue;
> -        }
> -
> -        optarg = po->arg;
> -        switch (po->o->val) {
> -        case OPT_DB:
> -            db = po->arg;
> -            break;
> -
> -        case OPT_NO_SYSLOG:
> -            vlog_set_levels(&this_module, VLF_SYSLOG, VLL_WARN);
> -            break;
> -
> -        case OPT_LOCAL:
> -            error = add_local_option(po->o->name, po->arg, local_options);
> -            if (error) {
> -                ctl_fatal("%s", error);
> -            }
> -            break;
> -
> -        case 'h':
> -            usage();
> -            exit(EXIT_SUCCESS);
> -
> -        case OPT_COMMANDS:
> -            ctl_print_commands();
> -            /* fall through */
> -
> -        case OPT_OPTIONS:
> -            ctl_print_options(get_all_options());
> -            /* fall through */
> -
> -        case OPT_LEADER_ONLY:
> -            leader_only = true;
> -            break;
> -
> -        case OPT_NO_LEADER_ONLY:
> -            leader_only = false;
> -            break;
> -
> -        case OPT_SHUFFLE_REMOTES:
> -            shuffle_remotes = true;
> -            break;
> -
> -        case OPT_NO_SHUFFLE_REMOTES:
> -            shuffle_remotes = false;
> -            break;
> -
> -        case 'V':
> -            ovs_print_version(0, 0);
> -            printf("DB Schema %s\n", nbrec_get_db_version());
> -            exit(EXIT_SUCCESS);
> -
> -        DAEMON_OPTION_HANDLERS
> -        VLOG_OPTION_HANDLERS
> -        TABLE_OPTION_HANDLERS(&table_style)
> -        STREAM_SSL_OPTION_HANDLERS
> -
> -        case OPT_BOOTSTRAP_CA_CERT:
> -            stream_ssl_set_ca_cert_file(po->arg, true);
> -            break;
> -
> -        case '?':
> -            exit(EXIT_FAILURE);
> -
> -        default:
> -            abort();
> -
> -        case 0:
> -            break;
> -        }
> -    }
> -
> -    if (!db) {
> -        db = default_nb_db();
> -    }
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN northbound DB management utility\n\
> -usage: %s [OPTIONS] COMMAND [ARG...]\n\
> -\n\
> -General commands:\n\
> -  init                      initialize the database\n\
> -  show                      print overview of database contents\n\
> -  show SWITCH               print overview of database contents for
> SWITCH\n\
> -  show ROUTER               print overview of database contents for
> ROUTER\n\
> -\n\
> -Logical switch commands:\n\
> -  ls-add [SWITCH]           create a logical switch named SWITCH\n\
> -  ls-del SWITCH             delete SWITCH and all its ports\n\
> -  ls-list                   print the names of all logical switches\n\
> -\n\
> -ACL commands:\n\
> -  [--type={switch | port-group}] [--log] [--severity=SEVERITY]
> [--name=NAME] [--may-exist]\n\
> -  acl-add {SWITCH | PORTGROUP} DIRECTION PRIORITY MATCH ACTION\n\
> -                            add an ACL to SWITCH/PORTGROUP\n\
> -  [--type={switch | port-group}]\n\
> -  acl-del {SWITCH | PORTGROUP} [DIRECTION [PRIORITY MATCH]]\n\
> -                            remove ACLs from SWITCH/PORTGROUP\n\
> -  [--type={switch | port-group}]\n\
> -  acl-list {SWITCH | PORTGROUP}\n\
> -                            print ACLs for SWITCH\n\
> -\n\
> -QoS commands:\n\
> -  qos-add SWITCH DIRECTION PRIORITY MATCH [rate=RATE [burst=BURST]]
> [dscp=DSCP]\n\
> -                            add an QoS rule to SWITCH\n\
> -  qos-del SWITCH [DIRECTION [PRIORITY MATCH]]\n\
> -                            remove QoS rules from SWITCH\n\
> -  qos-list SWITCH           print QoS rules for SWITCH\n\
> -\n\
> -Meter commands:\n\
> -  meter-add NAME ACTION RATE UNIT [BURST]\n\
> -                            add a meter\n\
> -  meter-del [NAME]          remove meters\n\
> -  meter-list                print meters\n\
> -\n\
> -Logical switch port commands:\n\
> -  lsp-add SWITCH PORT       add logical port PORT on SWITCH\n\
> -  lsp-add SWITCH PORT PARENT TAG\n\
> -                            add logical port PORT on SWITCH with PARENT\n\
> -                            on TAG\n\
> -  lsp-del PORT              delete PORT from its attached switch\n\
> -  lsp-list SWITCH           print the names of all logical ports on
> SWITCH\n\
> -  lsp-get-parent PORT       get the parent of PORT if set\n\
> -  lsp-get-tag PORT          get the PORT's tag if set\n\
> -  lsp-set-addresses PORT [ADDRESS]...\n\
> -                            set MAC or MAC+IP addresses for PORT.\n\
> -  lsp-get-addresses PORT    get a list of MAC or MAC+IP addresses on
> PORT\n\
> -  lsp-set-port-security PORT [ADDRS]...\n\
> -                            set port security addresses for PORT.\n\
> -  lsp-get-port-security PORT    get PORT's port security addresses\n\
> -  lsp-get-up PORT           get state of PORT ('up' or 'down')\n\
> -  lsp-set-enabled PORT STATE\n\
> -                            set administrative state PORT\n\
> -                            ('enabled' or 'disabled')\n\
> -  lsp-get-enabled PORT      get administrative state PORT\n\
> -                            ('enabled' or 'disabled')\n\
> -  lsp-set-type PORT TYPE    set the type for PORT\n\
> -  lsp-get-type PORT         get the type for PORT\n\
> -  lsp-set-options PORT KEY=VALUE [KEY=VALUE]...\n\
> -                            set options related to the type of PORT\n\
> -  lsp-get-options PORT      get the type specific options for PORT\n\
> -  lsp-set-dhcpv4-options PORT [DHCP_OPTIONS_UUID]\n\
> -                            set dhcpv4 options for PORT\n\
> -  lsp-get-dhcpv4-options PORT  get the dhcpv4 options for PORT\n\
> -  lsp-set-dhcpv6-options PORT [DHCP_OPTIONS_UUID]\n\
> -                            set dhcpv6 options for PORT\n\
> -  lsp-get-dhcpv6-options PORT  get the dhcpv6 options for PORT\n\
> -  lsp-get-ls PORT           get the logical switch which the port belongs
> to\n\
> -\n\
> -Logical router commands:\n\
> -  lr-add [ROUTER]           create a logical router named ROUTER\n\
> -  lr-del ROUTER             delete ROUTER and all its ports\n\
> -  lr-list                   print the names of all logical routers\n\
> -\n\
> -Logical router port commands:\n\
> -  lrp-add ROUTER PORT MAC NETWORK... [peer=PEER]\n\
> -                            add logical port PORT on ROUTER\n\
> -  lrp-set-gateway-chassis PORT CHASSIS [PRIORITY]\n\
> -                            set gateway chassis for port PORT\n\
> -  lrp-del-gateway-chassis PORT CHASSIS\n\
> -                            delete gateway chassis from port PORT\n\
> -  lrp-get-gateway-chassis PORT\n\
> -                            print the names of all gateway chassis on
> PORT\n\
> -                            with PRIORITY\n\
> -  lrp-del PORT              delete PORT from its attached router\n\
> -  lrp-list ROUTER           print the names of all ports on ROUTER\n\
> -  lrp-set-enabled PORT STATE\n\
> -                            set administrative state PORT\n\
> -                            ('enabled' or 'disabled')\n\
> -  lrp-get-enabled PORT      get administrative state PORT\n\
> -                            ('enabled' or 'disabled')\n\
> -\n\
> -Route commands:\n\
> -  [--policy=POLICY] lr-route-add ROUTER PREFIX NEXTHOP [PORT]\n\
> -                            add a route to ROUTER\n\
> -  lr-route-del ROUTER [PREFIX]\n\
> -                            remove routes from ROUTER\n\
> -  lr-route-list ROUTER      print routes for ROUTER\n\
> -\n\
> -Policy commands:\n\
> -  lr-policy-add ROUTER PRIORITY MATCH ACTION [NEXTHOP]\n\
> -                            add a policy to router\n\
> -  lr-policy-del ROUTER [PRIORITY [MATCH]]\n\
> -                            remove policies from ROUTER\n\
> -  lr-policy-list ROUTER     print policies for ROUTER\n\
> -\n\
> -NAT commands:\n\
> -  lr-nat-add ROUTER TYPE EXTERNAL_IP LOGICAL_IP [LOGICAL_PORT
> EXTERNAL_MAC]\n\
> -                            add a NAT to ROUTER\n\
> -  lr-nat-del ROUTER [TYPE [IP]]\n\
> -                            remove NATs from ROUTER\n\
> -  lr-nat-list ROUTER        print NATs for ROUTER\n\
> -\n\
> -LB commands:\n\
> -  lb-add LB VIP[:PORT] IP[:PORT]... [PROTOCOL]\n\
> -                            create a load-balancer or add a VIP to an\n\
> -                            existing load balancer\n\
> -  lb-del LB [VIP]           remove a load-balancer or just the VIP from\n\
> -                            the load balancer\n\
> -  lb-list [LB]              print load-balancers\n\
> -  lr-lb-add ROUTER LB       add a load-balancer to ROUTER\n\
> -  lr-lb-del ROUTER [LB]     remove load-balancers from ROUTER\n\
> -  lr-lb-list ROUTER         print load-balancers\n\
> -  ls-lb-add SWITCH LB       add a load-balancer to SWITCH\n\
> -  ls-lb-del SWITCH [LB]     remove load-balancers from SWITCH\n\
> -  ls-lb-list SWITCH         print load-balancers\n\
> -\n\
> -DHCP Options commands:\n\
> -  dhcp-options-create CIDR [EXTERNAL_IDS]\n\
> -                           create a DHCP options row with CIDR\n\
> -  dhcp-options-del DHCP_OPTIONS_UUID\n\
> -                           delete DHCP_OPTIONS_UUID\n\
> -  dhcp-options-list        \n\
> -                           lists the DHCP_Options rows\n\
> -  dhcp-options-set-options DHCP_OPTIONS_UUID  KEY=VALUE [KEY=VALUE]...\n\
> -                           set DHCP options for DHCP_OPTIONS_UUID\n\
> -  dhcp-options-get-options DHCO_OPTIONS_UUID \n\
> -                           displays the DHCP options for
> DHCP_OPTIONS_UUID\n\
> -\n\
> -Connection commands:\n\
> -  get-connection             print the connections\n\
> -  del-connection             delete the connections\n\
> -  [--inactivity-probe=MSECS]\n\
> -  set-connection TARGET...   set the list of connections to TARGET...\n\
> -\n\
> -SSL commands:\n\
> -  get-ssl                     print the SSL configuration\n\
> -  del-ssl                     delete the SSL configuration\n\
> -  set-ssl PRIV-KEY CERT CA-CERT [SSL-PROTOS [SSL-CIPHERS]] \
> -set the SSL configuration\n\
> -Port group commands:\n\
> -  pg-add PG [PORTS]           Create port group PG with optional PORTS\n\
> -  pg-set-ports PG PORTS       Set PORTS on port group PG\n\
> -  pg-del PG                   Delete port group PG\n\n",
> -            program_name, program_name);
> -    printf("\
> -HA chassis group commands:\n\
> -  ha-chassis-group-add GRP  Create an HA chassis group GRP\n\
> -  ha-chassis-group-del GRP  Delete the HA chassis group GRP\n\
> -  ha-chassis-group-list     List the HA chassis groups\n\
> -  ha-chassis-group-add-chassis GRP CHASSIS [PRIORITY] Adds an HA\
> -chassis with optional PRIORITY to the HA chassis group GRP\n\
> -  ha-chassis-group-del-chassis GRP CHASSIS Deletes the HA chassis\
> -CHASSIS from the HA chassis group GRP\n\
> -\n\
> -%s\
> -%s\
> -\n\
> -Synchronization command (use with --wait=sb|hv):\n\
> -  sync                     wait even for earlier changes to take effect\n\
> -\n\
> -Options:\n\
> -  --db=DATABASE               connect to DATABASE\n\
> -                              (default: %s)\n\
> -  --no-wait, --wait=none      do not wait for OVN reconfiguration
> (default)\n\
> -  --no-leader-only            accept any cluster member, not just the
> leader\n\
> -  --no-shuffle-remotes        do not shuffle the order of remotes\n\
> -  --wait=sb                   wait for southbound database update\n\
> -  --wait=hv                   wait for all chassis to catch up\n\
> -  -t, --timeout=SECS          wait at most SECS seconds\n\
> -  --dry-run                   do not commit changes to database\n\
> -  --oneline                   print exactly one line of output per
> command\n",
> -           ctl_get_db_cmd_usage(),
> -           ctl_list_db_tables_usage(), default_nb_db());
> -    table_usage();
> -    daemon_usage();
> -    vlog_usage();
> -    printf("\
> -  --no-syslog             equivalent to --verbose=nbctl:syslog:warn\n");
> -    printf("\n\
> -Other options:\n\
> -  -h, --help                  display this help message\n\
> -  -V, --version               display version information\n");
> -    stream_usage("database", true, true, true);
> -    exit(EXIT_SUCCESS);
> -}
> -
> -/* One should not use ctl_fatal() within commands because it will kill the
> - * daemon if we're in daemon mode.  Use ctl_error() instead and return
> - * gracefully.  */
> -#define ctl_fatal dont_use_ctl_fatal_use_ctl_error_and_return
> -
> -/* Find a logical router given its id. */
> -static char * OVS_WARN_UNUSED_RESULT
> -lr_by_name_or_uuid(struct ctl_context *ctx, const char *id,
> -                   bool must_exist, const struct nbrec_logical_router
> **lr_p)
> -{
> -    const struct nbrec_logical_router *lr = NULL;
> -    bool is_uuid = false;
> -    struct uuid lr_uuid;
> -
> -    *lr_p = NULL;
> -    if (uuid_from_string(&lr_uuid, id)) {
> -        is_uuid = true;
> -        lr = nbrec_logical_router_get_for_uuid(ctx->idl, &lr_uuid);
> -    }
> -
> -    if (!lr) {
> -        const struct nbrec_logical_router *iter;
> -
> -        NBREC_LOGICAL_ROUTER_FOR_EACH(iter, ctx->idl) {
> -            if (strcmp(iter->name, id)) {
> -                continue;
> -            }
> -            if (lr) {
> -                return xasprintf("Multiple logical routers named '%s'.  "
> -                                 "Use a UUID.", id);
> -            }
> -            lr = iter;
> -        }
> -    }
> -
> -    if (!lr && must_exist) {
> -        return xasprintf("%s: router %s not found",
> -                         id, is_uuid ? "UUID" : "name");
> -    }
> -
> -    *lr_p = lr;
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -ls_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool
> must_exist,
> -                   const struct nbrec_logical_switch **ls_p)
> -{
> -    const struct nbrec_logical_switch *ls = NULL;
> -    *ls_p = NULL;
> -
> -    struct uuid ls_uuid;
> -    bool is_uuid = uuid_from_string(&ls_uuid, id);
> -    if (is_uuid) {
> -        ls = nbrec_logical_switch_get_for_uuid(ctx->idl, &ls_uuid);
> -    }
> -
> -    if (!ls) {
> -        const struct nbrec_logical_switch *iter;
> -
> -        NBREC_LOGICAL_SWITCH_FOR_EACH(iter, ctx->idl) {
> -            if (strcmp(iter->name, id)) {
> -                continue;
> -            }
> -            if (ls) {
> -                return xasprintf("Multiple logical switches named '%s'.  "
> -                                 "Use a UUID.", id);
> -            }
> -            ls = iter;
> -        }
> -    }
> -
> -    if (!ls && must_exist) {
> -        return xasprintf("%s: switch %s not found",
> -                         id, is_uuid ? "UUID" : "name");
> -    }
> -
> -    *ls_p = ls;
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -lb_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool
> must_exist,
> -                   const struct nbrec_load_balancer **lb_p)
> -{
> -    const struct nbrec_load_balancer *lb = NULL;
> -
> -    struct uuid lb_uuid;
> -    bool is_uuid = uuid_from_string(&lb_uuid, id);
> -    if (is_uuid) {
> -        lb = nbrec_load_balancer_get_for_uuid(ctx->idl, &lb_uuid);
> -    }
> -
> -    if (!lb) {
> -        const struct nbrec_load_balancer *iter;
> -
> -        NBREC_LOAD_BALANCER_FOR_EACH(iter, ctx->idl) {
> -            if (strcmp(iter->name, id)) {
> -                continue;
> -            }
> -            if (lb) {
> -                return xasprintf("Multiple load balancers named '%s'.  "
> -                                 "Use a UUID.", id);
> -            }
> -            lb = iter;
> -        }
> -    }
> -
> -    if (!lb && must_exist) {
> -        return xasprintf("%s: load balancer %s not found", id,
> -                         is_uuid ? "UUID" : "name");
> -    }
> -
> -    if (lb_p) {
> -        *lb_p = lb;
> -    }
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -pg_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool
> must_exist,
> -                   const struct nbrec_port_group **pg_p)
> -{
> -    const struct nbrec_port_group *pg = NULL;
> -    *pg_p = NULL;
> -
> -    struct uuid pg_uuid;
> -    bool is_uuid = uuid_from_string(&pg_uuid, id);
> -    if (is_uuid) {
> -        pg = nbrec_port_group_get_for_uuid(ctx->idl, &pg_uuid);
> -    }
> -
> -    if (!pg) {
> -        const struct nbrec_port_group *iter;
> -
> -        NBREC_PORT_GROUP_FOR_EACH (iter, ctx->idl) {
> -            if (!strcmp(iter->name, id)) {
> -                pg = iter;
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!pg && must_exist) {
> -        return xasprintf("%s: port group %s not found", id,
> -                         is_uuid ? "UUID" : "name");
> -    }
> -
> -    *pg_p = pg;
> -    return NULL;
> -}
> -
> -static void
> -print_alias(const struct smap *external_ids, const char *key, struct ds
> *s)
> -{
> -    const char *alias = smap_get(external_ids, key);
> -    if (alias && alias[0]) {
> -        ds_put_format(s, " (aka %s)", alias);
> -    }
> -}
> -
> -/* gateway_chassis ordering
> - *  */
> -static int
> -compare_chassis_prio_(const void *gc1_, const void *gc2_)
> -{
> -    const struct nbrec_gateway_chassis *const *gc1p = gc1_;
> -    const struct nbrec_gateway_chassis *const *gc2p = gc2_;
> -    const struct nbrec_gateway_chassis *gc1 = *gc1p;
> -    const struct nbrec_gateway_chassis *gc2 = *gc2p;
> -
> -    int prio_diff = gc2->priority - gc1->priority;
> -    if (!prio_diff) {
> -        return strcmp(gc2->name, gc1->name);
> -    }
> -    return prio_diff;
> -}
> -
> -static const struct nbrec_gateway_chassis **
> -get_ordered_gw_chassis_prio_list(const struct nbrec_logical_router_port
> *lrp)
> -{
> -    const struct nbrec_gateway_chassis **gcs;
> -    int i;
> -
> -    gcs = xmalloc(sizeof *gcs * lrp->n_gateway_chassis);
> -    for (i = 0; i < lrp->n_gateway_chassis; i++) {
> -        gcs[i] = lrp->gateway_chassis[i];
> -    }
> -
> -    qsort(gcs, lrp->n_gateway_chassis, sizeof *gcs,
> compare_chassis_prio_);
> -    return gcs;
> -}
> -
> -/* Given pointer to logical router, this routine prints the router
> - * information.  */
> -static void
> -print_lr(const struct nbrec_logical_router *lr, struct ds *s)
> -{
> -    ds_put_format(s, "router "UUID_FMT" (%s)",
> -                  UUID_ARGS(&lr->header_.uuid), lr->name);
> -    print_alias(&lr->external_ids, "neutron:router_name", s);
> -    ds_put_char(s, '\n');
> -
> -    for (size_t i = 0; i < lr->n_ports; i++) {
> -        const struct nbrec_logical_router_port *lrp = lr->ports[i];
> -        ds_put_format(s, "    port %s\n", lrp->name);
> -        if (lrp->mac) {
> -            ds_put_cstr(s, "        mac: ");
> -            ds_put_format(s, "\"%s\"\n", lrp->mac);
> -        }
> -        if (lrp->n_networks) {
> -            ds_put_cstr(s, "        networks: [");
> -            for (size_t j = 0; j < lrp->n_networks; j++) {
> -                ds_put_format(s, "%s\"%s\"",
> -                        j == 0 ? "" : ", ",
> -                        lrp->networks[j]);
> -            }
> -            ds_put_cstr(s, "]\n");
> -        }
> -
> -        if (lrp->n_gateway_chassis) {
> -            const struct nbrec_gateway_chassis **gcs;
> -
> -            gcs = get_ordered_gw_chassis_prio_list(lrp);
> -            ds_put_cstr(s, "        gateway chassis: [");
> -            for (size_t j = 0; j < lrp->n_gateway_chassis; j++) {
> -                const struct nbrec_gateway_chassis *gc = gcs[j];
> -                ds_put_format(s, "%s ", gc->chassis_name);
> -            }
> -            ds_chomp(s, ' ');
> -            ds_put_cstr(s, "]\n");
> -            free(gcs);
> -        }
> -    }
> -
> -    for (size_t i = 0; i < lr->n_nat; i++) {
> -        const struct nbrec_nat *nat = lr->nat[i];
> -        ds_put_format(s, "    nat "UUID_FMT"\n",
> -                  UUID_ARGS(&nat->header_.uuid));
> -        ds_put_cstr(s, "        external ip: ");
> -        ds_put_format(s, "\"%s\"\n", nat->external_ip);
> -        ds_put_cstr(s, "        logical ip: ");
> -        ds_put_format(s, "\"%s\"\n", nat->logical_ip);
> -        ds_put_cstr(s, "        type: ");
> -        ds_put_format(s, "\"%s\"\n", nat->type);
> -    }
> -}
> -
> -static void
> -print_ls(const struct nbrec_logical_switch *ls, struct ds *s)
> -{
> -    ds_put_format(s, "switch "UUID_FMT" (%s)",
> -                  UUID_ARGS(&ls->header_.uuid), ls->name);
> -    print_alias(&ls->external_ids, "neutron:network_name", s);
> -    ds_put_char(s, '\n');
> -
> -    for (size_t i = 0; i < ls->n_ports; i++) {
> -        const struct nbrec_logical_switch_port *lsp = ls->ports[i];
> -
> -        ds_put_format(s, "    port %s", lsp->name);
> -        print_alias(&lsp->external_ids, "neutron:port_name", s);
> -        ds_put_char(s, '\n');
> -
> -        if (lsp->type[0]) {
> -            ds_put_format(s, "        type: %s\n", lsp->type);
> -        }
> -        if (lsp->parent_name) {
> -            ds_put_format(s, "        parent: %s\n", lsp->parent_name);
> -        }
> -        if (lsp->n_tag) {
> -            ds_put_format(s, "        tag: %"PRIu64"\n", lsp->tag[0]);
> -        }
> -
> -        /* Print the addresses, but not if there's just a single "router"
> -         * address because that's just clutter. */
> -        if (lsp->n_addresses
> -            && !(lsp->n_addresses == 1
> -                 && !strcmp(lsp->addresses[0], "router"))) {
> -            ds_put_cstr(s, "        addresses: [");
> -            for (size_t j = 0; j < lsp->n_addresses; j++) {
> -                ds_put_format(s, "%s\"%s\"",
> -                        j == 0 ? "" : ", ",
> -                        lsp->addresses[j]);
> -            }
> -            ds_put_cstr(s, "]\n");
> -        }
> -
> -        const char *router_port = smap_get(&lsp->options, "router-port");
> -        if (router_port) {
> -            ds_put_format(s, "        router-port: %s\n", router_port);
> -        }
> -    }
> -}
> -
> -static void
> -nbctl_init(struct ctl_context *ctx OVS_UNUSED)
> -{
> -}
> -
> -static void
> -nbctl_pre_sync(struct ctl_context *ctx OVS_UNUSED)
> -{
> -    if (wait_type != NBCTL_WAIT_NONE) {
> -        force_wait = true;
> -    } else {
> -        VLOG_INFO("\"sync\" command has no effect without --wait");
> -    }
> -}
> -
> -static void
> -nbctl_sync(struct ctl_context *ctx OVS_UNUSED)
> -{
> -}
> -
> -static void
> -nbctl_show(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -
> -    if (ctx->argc == 2) {
> -        char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], false, &ls);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -        if (ls) {
> -            print_ls(ls, &ctx->output);
> -        }
> -    } else {
> -        NBREC_LOGICAL_SWITCH_FOR_EACH(ls, ctx->idl) {
> -            print_ls(ls, &ctx->output);
> -        }
> -    }
> -    const struct nbrec_logical_router *lr;
> -
> -    if (ctx->argc == 2) {
> -        char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], false, &lr);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -        if (lr) {
> -            print_lr(lr, &ctx->output);
> -        }
> -    } else {
> -        NBREC_LOGICAL_ROUTER_FOR_EACH(lr, ctx->idl) {
> -            print_lr(lr, &ctx->output);
> -        }
> -    }
> -}
> -
> -static void
> -nbctl_ls_add(struct ctl_context *ctx)
> -{
> -    const char *ls_name = ctx->argc == 2 ? ctx->argv[1] : NULL;
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") !=
> NULL;
> -    if (may_exist && add_duplicate) {
> -        ctl_error(ctx, "--may-exist and --add-duplicate may not be used "
> -                  "together");
> -        return;
> -    }
> -
> -    if (ls_name) {
> -        if (!add_duplicate) {
> -            const struct nbrec_logical_switch *ls;
> -            NBREC_LOGICAL_SWITCH_FOR_EACH (ls, ctx->idl) {
> -                if (!strcmp(ls->name, ls_name)) {
> -                    if (may_exist) {
> -                        return;
> -                    }
> -                    ctl_error(ctx, "%s: a switch with this name already "
> -                              "exists", ls_name);
> -                    return;
> -                }
> -            }
> -        }
> -    } else if (may_exist) {
> -        ctl_error(ctx, "--may-exist requires specifying a name");
> -        return;
> -    } else if (add_duplicate) {
> -        ctl_error(ctx, "--add-duplicate requires specifying a name");
> -        return;
> -    }
> -
> -    struct nbrec_logical_switch *ls;
> -    ls = nbrec_logical_switch_insert(ctx->txn);
> -    if (ls_name) {
> -        nbrec_logical_switch_set_name(ls, ls_name);
> -    }
> -}
> -
> -static void
> -nbctl_ls_del(struct ctl_context *ctx)
> -{
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch *ls = NULL;
> -
> -    char *error = ls_by_name_or_uuid(ctx, id, must_exist, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!ls) {
> -        return;
> -    }
> -
> -    nbrec_logical_switch_delete(ls);
> -}
> -
> -static void
> -nbctl_ls_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    struct smap switches;
> -
> -    smap_init(&switches);
> -    NBREC_LOGICAL_SWITCH_FOR_EACH(ls, ctx->idl) {
> -        smap_add_format(&switches, ls->name, UUID_FMT " (%s)",
> -                        UUID_ARGS(&ls->header_.uuid), ls->name);
> -    }
> -    const struct smap_node **nodes = smap_sort(&switches);
> -    for (size_t i = 0; i < smap_count(&switches); i++) {
> -        const struct smap_node *node = nodes[i];
> -        ds_put_format(&ctx->output, "%s\n", node->value);
> -    }
> -    smap_destroy(&switches);
> -    free(nodes);
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -lsp_by_name_or_uuid(struct ctl_context *ctx, const char *id,
> -                    bool must_exist,
> -                    const struct nbrec_logical_switch_port **lsp_p)
> -{
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -    *lsp_p = NULL;
> -
> -    struct uuid lsp_uuid;
> -    bool is_uuid = uuid_from_string(&lsp_uuid, id);
> -    if (is_uuid) {
> -        lsp = nbrec_logical_switch_port_get_for_uuid(ctx->idl, &lsp_uuid);
> -    }
> -
> -    if (!lsp) {
> -        NBREC_LOGICAL_SWITCH_PORT_FOR_EACH(lsp, ctx->idl) {
> -            if (!strcmp(lsp->name, id)) {
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!lsp && must_exist) {
> -        return xasprintf("%s: port %s not found",
> -                         id, is_uuid ? "UUID" : "name");
> -    }
> -
> -    *lsp_p = lsp;
> -    return NULL;
> -}
> -
> -/* Returns the logical switch that contains 'lsp'. */
> -static char * OVS_WARN_UNUSED_RESULT
> -lsp_to_ls(const struct ovsdb_idl *idl,
> -          const struct nbrec_logical_switch_port *lsp,
> -          const struct nbrec_logical_switch **ls_p)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    *ls_p = NULL;
> -
> -    NBREC_LOGICAL_SWITCH_FOR_EACH (ls, idl) {
> -        for (size_t i = 0; i < ls->n_ports; i++) {
> -            if (ls->ports[i] == lsp) {
> -                *ls_p = ls;
> -                return NULL;
> -            }
> -        }
> -    }
> -
> -    /* Can't happen because of the database schema */
> -    return xasprintf("logical port %s is not part of any logical switch",
> -                     lsp->name);
> -}
> -
> -static const char *
> -ls_get_name(const struct nbrec_logical_switch *ls,
> -                 char uuid_s[UUID_LEN + 1], size_t uuid_s_size)
> -{
> -    if (ls->name[0]) {
> -        return ls->name;
> -    }
> -    snprintf(uuid_s, uuid_s_size, UUID_FMT, UUID_ARGS(&ls->header_.uuid));
> -    return uuid_s;
> -}
> -
> -static void
> -nbctl_lsp_add(struct ctl_context *ctx)
> -{
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -
> -    const struct nbrec_logical_switch *ls = NULL;
> -    char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const char *parent_name;
> -    int64_t tag;
> -    if (ctx->argc == 3) {
> -        parent_name = NULL;
> -        tag = -1;
> -    } else if (ctx->argc == 5) {
> -        /* Validate tag. */
> -        parent_name = ctx->argv[3];
> -        if (!ovs_scan(ctx->argv[4], "%"SCNd64, &tag)
> -            || tag < 0 || tag > 4095) {
> -            ctl_error(ctx, "%s: invalid tag (must be in range 0 to 4095)",
> -                      ctx->argv[4]);
> -            return;
> -        }
> -    } else {
> -        ctl_error(ctx, "lsp-add with parent must also specify a tag");
> -        return;
> -    }
> -
> -    const char *lsp_name = ctx->argv[2];
> -    const struct nbrec_logical_switch_port *lsp;
> -    error = lsp_by_name_or_uuid(ctx, lsp_name, false, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lsp) {
> -        if (!may_exist) {
> -            ctl_error(ctx, "%s: a port with this name already exists",
> -                      lsp_name);
> -            return;
> -        }
> -
> -        const struct nbrec_logical_switch *lsw;
> -        error = lsp_to_ls(ctx->idl, lsp, &lsw);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -        if (lsw != ls) {
> -            char uuid_s[UUID_LEN + 1];
> -            ctl_error(ctx, "%s: port already exists but in switch %s",
> -                      lsp_name, ls_get_name(lsw, uuid_s, sizeof uuid_s));
> -            return;
> -        }
> -
> -        if (parent_name) {
> -            if (!lsp->parent_name) {
> -                ctl_error(ctx, "%s: port already exists but has no
> parent",
> -                          lsp_name);
> -                return;
> -            } else if (strcmp(parent_name, lsp->parent_name)) {
> -                ctl_error(ctx, "%s: port already exists with different
> parent "
> -                          "%s", lsp_name, lsp->parent_name);
> -                return;
> -            }
> -
> -            if (!lsp->n_tag_request) {
> -                ctl_error(ctx, "%s: port already exists but has no "
> -                          "tag_request", lsp_name);
> -                return;
> -            } else if (lsp->tag_request[0] != tag) {
> -                ctl_error(ctx, "%s: port already exists with different "
> -                          "tag_request %"PRId64, lsp_name,
> -                          lsp->tag_request[0]);
> -                return;
> -            }
> -        } else {
> -            if (lsp->parent_name) {
> -                ctl_error(ctx, "%s: port already exists but has parent
> %s",
> -                          lsp_name, lsp->parent_name);
> -                return;
> -            }
> -        }
> -
> -        return;
> -    }
> -
> -    /* Create the logical port. */
> -    lsp = nbrec_logical_switch_port_insert(ctx->txn);
> -    nbrec_logical_switch_port_set_name(lsp, lsp_name);
> -    if (tag >= 0) {
> -        nbrec_logical_switch_port_set_parent_name(lsp, parent_name);
> -        nbrec_logical_switch_port_set_tag_request(lsp, &tag, 1);
> -    }
> -
> -    /* Insert the logical port into the logical switch. */
> -    nbrec_logical_switch_verify_ports(ls);
> -    struct nbrec_logical_switch_port **new_ports = xmalloc(sizeof
> *new_ports *
> -                                                    (ls->n_ports + 1));
> -    nullable_memcpy(new_ports, ls->ports, sizeof *new_ports *
> ls->n_ports);
> -    new_ports[ls->n_ports] = CONST_CAST(struct nbrec_logical_switch_port
> *,
> -                                             lsp);
> -    nbrec_logical_switch_set_ports(ls, new_ports, ls->n_ports + 1);
> -    free(new_ports);
> -}
> -
> -/* Removes logical switch port 'ls->ports[idx]'. */
> -static void
> -remove_lsp(const struct nbrec_logical_switch *ls, size_t idx)
> -{
> -    const struct nbrec_logical_switch_port *lsp = ls->ports[idx];
> -
> -    /* First remove 'lsp' from the array of ports.  This is what will
> -     * actually cause the logical port to be deleted when the transaction
> is
> -     * sent to the database server (due to garbage collection). */
> -    struct nbrec_logical_switch_port **new_ports
> -        = xmemdup(ls->ports, sizeof *new_ports * ls->n_ports);
> -    new_ports[idx] = new_ports[ls->n_ports - 1];
> -    nbrec_logical_switch_verify_ports(ls);
> -    nbrec_logical_switch_set_ports(ls, new_ports, ls->n_ports - 1);
> -    free(new_ports);
> -
> -    /* Delete 'lsp' from the IDL.  This won't have a real effect on the
> -     * database server (the IDL will suppress it in fact) but it means
> that it
> -     * won't show up when we iterate with
> NBREC_LOGICAL_SWITCH_PORT_FOR_EACH
> -     * later. */
> -    nbrec_logical_switch_port_delete(lsp);
> -}
> -
> -static void
> -nbctl_lsp_del(struct ctl_context *ctx)
> -{
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, ctx->argv[1], must_exist,
> &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lsp) {
> -        return;
> -    }
> -
> -    /* Find the switch that contains 'lsp', then delete it. */
> -    const struct nbrec_logical_switch *ls;
> -    NBREC_LOGICAL_SWITCH_FOR_EACH (ls, ctx->idl) {
> -        for (size_t i = 0; i < ls->n_ports; i++) {
> -            if (ls->ports[i] == lsp) {
> -                remove_lsp(ls, i);
> -                return;
> -            }
> -        }
> -    }
> -
> -    /* Can't happen because of the database schema. */
> -    ctl_error(ctx, "logical port %s is not part of any logical switch",
> -              ctx->argv[1]);
> -}
> -
> -static void
> -nbctl_lsp_list(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch *ls;
> -    struct smap lsps;
> -    size_t i;
> -
> -    char *error = ls_by_name_or_uuid(ctx, id, true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    smap_init(&lsps);
> -    for (i = 0; i < ls->n_ports; i++) {
> -        const struct nbrec_logical_switch_port *lsp = ls->ports[i];
> -        smap_add_format(&lsps, lsp->name, UUID_FMT " (%s)",
> -                        UUID_ARGS(&lsp->header_.uuid), lsp->name);
> -    }
> -    const struct smap_node **nodes = smap_sort(&lsps);
> -    for (i = 0; i < smap_count(&lsps); i++) {
> -        const struct smap_node *node = nodes[i];
> -        ds_put_format(&ctx->output, "%s\n", node->value);
> -    }
> -    smap_destroy(&lsps);
> -    free(nodes);
> -}
> -
> -static void
> -nbctl_lsp_get_parent(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, ctx->argv[1], true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lsp->parent_name) {
> -        ds_put_format(&ctx->output, "%s\n", lsp->parent_name);
> -    }
> -}
> -
> -static void
> -nbctl_lsp_get_tag(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, ctx->argv[1], true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lsp->n_tag > 0) {
> -        ds_put_format(&ctx->output, "%"PRId64"\n", lsp->tag[0]);
> -    }
> -}
> -
> -static char *
> -lsp_contains_duplicate_ip(struct lport_addresses *laddrs1,
> -                          struct lport_addresses *laddrs2)
> -{
> -    for (size_t i = 0; i < laddrs1->n_ipv4_addrs; i++) {
> -        for (size_t j = 0; j < laddrs2->n_ipv4_addrs; j++) {
> -            if (laddrs1->ipv4_addrs[i].addr ==
> laddrs2->ipv4_addrs[j].addr) {
> -                return xasprintf("duplicate IPv4 address %s",
> -                                 laddrs1->ipv4_addrs[i].addr_s);
> -            }
> -        }
> -    }
> -
> -    for (size_t i = 0; i < laddrs1->n_ipv6_addrs; i++) {
> -        for (size_t j = 0; j < laddrs2->n_ipv6_addrs; j++) {
> -            if (IN6_ARE_ADDR_EQUAL(&laddrs1->ipv6_addrs[i].addr,
> -                                   &laddrs2->ipv6_addrs[j].addr)) {
> -                return xasprintf("duplicate IPv6 address %s",
> -                                 laddrs1->ipv6_addrs[i].addr_s);
> -            }
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static char *
> -lsp_contains_duplicates(const struct nbrec_logical_switch *ls,
> -                        const struct nbrec_logical_switch_port *lsp,
> -                        const char *address)
> -{
> -    struct lport_addresses laddrs;
> -    if (!extract_lsp_addresses(address, &laddrs)) {
> -        return NULL;
> -    }
> -
> -    char *sub_error = NULL;
> -    for (size_t i = 0; i < ls->n_ports; i++) {
> -        struct nbrec_logical_switch_port *lsp_test = ls->ports[i];
> -        if (lsp_test == lsp) {
> -            continue;
> -        }
> -        for (size_t j = 0; j < lsp_test->n_addresses; j++) {
> -            struct lport_addresses laddrs_test;
> -            char *addr = lsp_test->addresses[j];
> -            if (is_dynamic_lsp_address(addr) &&
> lsp_test->dynamic_addresses) {
> -                addr = lsp_test->dynamic_addresses;
> -            }
> -            if (extract_lsp_addresses(addr, &laddrs_test)) {
> -                sub_error = lsp_contains_duplicate_ip(&laddrs,
> &laddrs_test);
> -                destroy_lport_addresses(&laddrs_test);
> -                if (sub_error) {
> -                    goto err_out;
> -                }
> -            }
> -        }
> -    }
> -
> -err_out: ;
> -    char *error = NULL;
> -    if (sub_error) {
> -        error = xasprintf("Error on switch %s: %s", ls->name, sub_error);
> -        free(sub_error);
> -    }
> -    destroy_lport_addresses(&laddrs);
> -    return error;
> -}
> -
> -static void
> -nbctl_lsp_set_addresses(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const struct nbrec_logical_switch *ls;
> -    error = lsp_to_ls(ctx->idl, lsp, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    int i;
> -    for (i = 2; i < ctx->argc; i++) {
> -        char ipv6_s[IPV6_SCAN_LEN + 1];
> -        struct eth_addr ea;
> -        ovs_be32 ip;
> -
> -        if (strcmp(ctx->argv[i], "unknown") && strcmp(ctx->argv[i],
> "dynamic")
> -            && strcmp(ctx->argv[i], "router")
> -            && !ovs_scan(ctx->argv[i], ETH_ADDR_SCAN_FMT,
> -                         ETH_ADDR_SCAN_ARGS(ea))
> -            && !ovs_scan(ctx->argv[i], "dynamic "IPV6_SCAN_FMT, ipv6_s)
> -            && !ovs_scan(ctx->argv[i], "dynamic "IP_SCAN_FMT,
> -                         IP_SCAN_ARGS(&ip))) {
> -            ctl_error(ctx, "%s: Invalid address format. See ovn-nb(5). "
> -                      "Hint: An Ethernet address must be "
> -                      "listed before an IP address, together as a single "
> -                      "argument.", ctx->argv[i]);
> -            return;
> -        }
> -
> -        error = lsp_contains_duplicates(ls, lsp, ctx->argv[i]);
> -        if (error) {
> -            ctl_error(ctx, "%s", error);
> -            return;
> -        }
> -    }
> -
> -    nbrec_logical_switch_port_set_addresses(lsp,
> -            (const char **) ctx->argv + 2, ctx->argc - 2);
> -}
> -
> -static void
> -nbctl_lsp_get_addresses(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -    struct svec addresses;
> -    const char *mac;
> -    size_t i;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    svec_init(&addresses);
> -    for (i = 0; i < lsp->n_addresses; i++) {
> -        svec_add(&addresses, lsp->addresses[i]);
> -    }
> -    svec_sort(&addresses);
> -    SVEC_FOR_EACH(i, mac, &addresses) {
> -        ds_put_format(&ctx->output, "%s\n", mac);
> -    }
> -    svec_destroy(&addresses);
> -}
> -
> -static void
> -nbctl_lsp_set_port_security(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    nbrec_logical_switch_port_set_port_security(lsp,
> -            (const char **) ctx->argv + 2, ctx->argc - 2);
> -}
> -
> -static void
> -nbctl_lsp_get_port_security(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -    struct svec addrs;
> -    const char *addr;
> -    size_t i;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    svec_init(&addrs);
> -    for (i = 0; i < lsp->n_port_security; i++) {
> -        svec_add(&addrs, lsp->port_security[i]);
> -    }
> -    svec_sort(&addrs);
> -    SVEC_FOR_EACH(i, addr, &addrs) {
> -        ds_put_format(&ctx->output, "%s\n", addr);
> -    }
> -    svec_destroy(&addrs);
> -}
> -
> -static void
> -nbctl_lsp_get_up(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    ds_put_format(&ctx->output,
> -                  "%s\n", (lsp->up && *lsp->up) ? "up" : "down");
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -parse_enabled(const char *state, bool *enabled_p)
> -{
> -    ovs_assert(enabled_p);
> -
> -    if (!strcasecmp(state, "enabled")) {
> -        *enabled_p = true;
> -    } else if (!strcasecmp(state, "disabled")) {
> -        *enabled_p = false;
> -    } else {
> -        return xasprintf("%s: state must be \"enabled\" or \"disabled\"",
> -                         state);
> -    }
> -    return NULL;
> -}
> -
> -static void
> -nbctl_lsp_set_enabled(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const char *state = ctx->argv[2];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    bool enabled;
> -    error = parse_enabled(state, &enabled);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    nbrec_logical_switch_port_set_enabled(lsp, &enabled, 1);
> -}
> -
> -static void
> -nbctl_lsp_get_enabled(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    ds_put_format(&ctx->output, "%s\n",
> -                  !lsp->enabled || *lsp->enabled ? "enabled" :
> "disabled");
> -}
> -
> -static void
> -nbctl_lsp_set_type(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const char *type = ctx->argv[2];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (ovn_is_known_nb_lsp_type(type)) {
> -        nbrec_logical_switch_port_set_type(lsp, type);
> -    } else {
> -        ctl_error(ctx, "Logical switch port type '%s' is unrecognized. "
> -                  "Not setting type.", type);
> -        return;
> -    }
> -}
> -
> -static void
> -nbctl_lsp_get_type(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    ds_put_format(&ctx->output, "%s\n", lsp->type);
> -}
> -
> -static void
> -nbctl_lsp_set_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -    size_t i;
> -    struct smap options = SMAP_INITIALIZER(&options);
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    for (i = 2; i < ctx->argc; i++) {
> -        char *key, *value;
> -        value = xstrdup(ctx->argv[i]);
> -        key = strsep(&value, "=");
> -        if (value) {
> -            smap_add(&options, key, value);
> -        }
> -        free(key);
> -    }
> -
> -    nbrec_logical_switch_port_set_options(lsp, &options);
> -
> -    smap_destroy(&options);
> -}
> -
> -static void
> -nbctl_lsp_get_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -    struct smap_node *node;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    SMAP_FOR_EACH(node, &lsp->options) {
> -        ds_put_format(&ctx->output, "%s=%s\n", node->key, node->value);
> -    }
> -}
> -
> -static void
> -nbctl_lsp_set_dhcpv4_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    const struct nbrec_dhcp_options *dhcp_opt = NULL;
> -    if (ctx->argc == 3 ) {
> -        error = dhcp_options_get(ctx, ctx->argv[2], true, &dhcp_opt);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -    }
> -
> -    if (dhcp_opt) {
> -        ovs_be32 ip;
> -        unsigned int plen;
> -        error = ip_parse_cidr(dhcp_opt->cidr, &ip, &plen);
> -        if (error){
> -            free(error);
> -            ctl_error(ctx, "DHCP options cidr '%s' is not IPv4",
> -                      dhcp_opt->cidr);
> -            return;
> -        }
> -    }
> -    nbrec_logical_switch_port_set_dhcpv4_options(lsp, dhcp_opt);
> -}
> -
> -static void
> -nbctl_lsp_set_dhcpv6_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    const struct nbrec_dhcp_options *dhcp_opt = NULL;
> -    if (ctx->argc == 3) {
> -        error = dhcp_options_get(ctx, ctx->argv[2], true, &dhcp_opt);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -    }
> -
> -    if (dhcp_opt) {
> -        struct in6_addr ip;
> -        unsigned int plen;
> -        error = ipv6_parse_cidr(dhcp_opt->cidr, &ip, &plen);
> -        if (error) {
> -            free(error);
> -            ctl_error(ctx, "DHCP options cidr '%s' is not IPv6",
> -                      dhcp_opt->cidr);
> -            return;
> -        }
> -    }
> -    nbrec_logical_switch_port_set_dhcpv6_options(lsp, dhcp_opt);
> -}
> -
> -static void
> -nbctl_lsp_get_dhcpv4_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lsp->dhcpv4_options) {
> -        ds_put_format(&ctx->output, UUID_FMT " (%s)\n",
> -                      UUID_ARGS(&lsp->dhcpv4_options->header_.uuid),
> -                      lsp->dhcpv4_options->cidr);
> -    }
> -}
> -
> -static void
> -nbctl_lsp_get_dhcpv6_options(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lsp->dhcpv6_options) {
> -        ds_put_format(&ctx->output, UUID_FMT " (%s)\n",
> -                      UUID_ARGS(&lsp->dhcpv6_options->header_.uuid),
> -                      lsp->dhcpv6_options->cidr);
> -    }
> -}
> -
> -static void
> -nbctl_lsp_get_ls(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_switch_port *lsp = NULL;
> -
> -    char *error = lsp_by_name_or_uuid(ctx, id, true, &lsp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const struct nbrec_logical_switch *ls;
> -    NBREC_LOGICAL_SWITCH_FOR_EACH(ls, ctx->idl) {
> -        for (size_t i = 0; i < ls->n_ports; i++) {
> -            if (ls->ports[i] == lsp) {
> -                ds_put_format(&ctx->output, UUID_FMT " (%s)\n",
> -                      UUID_ARGS(&ls->header_.uuid), ls->name);
> -                break;
> -            }
> -        }
> -    }
> -}
> -
> -enum {
> -    DIR_FROM_LPORT,
> -    DIR_TO_LPORT
> -};
> -
> -static int
> -dir_encode(const char *dir)
> -{
> -    if (!strcmp(dir, "from-lport")) {
> -        return DIR_FROM_LPORT;
> -    } else if (!strcmp(dir, "to-lport")) {
> -        return DIR_TO_LPORT;
> -    }
> -
> -    OVS_NOT_REACHED();
> -}
> -
> -static int
> -acl_cmp(const void *acl1_, const void *acl2_)
> -{
> -    const struct nbrec_acl *const *acl1p = acl1_;
> -    const struct nbrec_acl *const *acl2p = acl2_;
> -    const struct nbrec_acl *acl1 = *acl1p;
> -    const struct nbrec_acl *acl2 = *acl2p;
> -
> -    int dir1 = dir_encode(acl1->direction);
> -    int dir2 = dir_encode(acl2->direction);
> -
> -    if (dir1 != dir2) {
> -        return dir1 < dir2 ? -1 : 1;
> -    } else if (acl1->priority != acl2->priority) {
> -        return acl1->priority > acl2->priority ? -1 : 1;
> -    } else {
> -        return strcmp(acl1->match, acl2->match);
> -    }
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -acl_cmd_get_pg_or_ls(struct ctl_context *ctx,
> -                     const struct nbrec_logical_switch **ls,
> -                     const struct nbrec_port_group **pg)
> -{
> -    const char *opt_type = shash_find_data(&ctx->options, "--type");
> -    char *error;
> -
> -    if (!opt_type) {
> -        error = pg_by_name_or_uuid(ctx, ctx->argv[1], false, pg);
> -        if (error) {
> -            return error;
> -        }
> -        error = ls_by_name_or_uuid(ctx, ctx->argv[1], false, ls);
> -        if (error) {
> -            return error;
> -        }
> -        if (*pg && *ls) {
> -            return xasprintf("Same name '%s' exists in both port-groups
> and "
> -                             "logical switches. Specify --type=port-group
> or "
> -                             "switch, or use a UUID.", ctx->argv[1]);
> -        }
> -        if (!*pg && !*ls) {
> -            return xasprintf("'%s' is not found for port-group or
> switch.",
> -                             ctx->argv[1]);
> -        }
> -    } else if (!strcmp(opt_type, "port-group")) {
> -        error = pg_by_name_or_uuid(ctx, ctx->argv[1], true, pg);
> -        if (error) {
> -            return error;
> -        }
> -        *ls = NULL;
> -    } else if (!strcmp(opt_type, "switch")) {
> -        error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, ls);
> -        if (error) {
> -            return error;
> -        }
> -        *pg = NULL;
> -    } else {
> -        return xasprintf("Invalid value '%s' for option --type",
> opt_type);
> -    }
> -
> -    return NULL;
> -}
> -
> -static void
> -nbctl_acl_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls = NULL;
> -    const struct nbrec_port_group *pg = NULL;
> -    const struct nbrec_acl **acls;
> -    size_t i;
> -
> -    char *error = acl_cmd_get_pg_or_ls(ctx, &ls, &pg);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    size_t n_acls = pg ? pg->n_acls : ls->n_acls;
> -    struct nbrec_acl **nb_acls = pg ? pg->acls : ls->acls;
> -
> -    acls = xmalloc(sizeof *acls * n_acls);
> -    for (i = 0; i < n_acls; i++) {
> -        acls[i] = nb_acls[i];
> -    }
> -
> -    qsort(acls, n_acls, sizeof *acls, acl_cmp);
> -
> -    for (i = 0; i < n_acls; i++) {
> -        const struct nbrec_acl *acl = acls[i];
> -        ds_put_format(&ctx->output, "%10s %5"PRId64" (%s) %s",
> -                      acl->direction, acl->priority, acl->match,
> -                      acl->action);
> -        if (acl->log) {
> -            ds_put_cstr(&ctx->output, " log(");
> -            if (acl->name) {
> -                ds_put_format(&ctx->output, "name=%s,", acl->name);
> -            }
> -            if (acl->severity) {
> -                ds_put_format(&ctx->output, "severity=%s,",
> acl->severity);
> -            }
> -            if (acl->meter) {
> -                ds_put_format(&ctx->output, "meter=\"%s\",", acl->meter);
> -            }
> -            ds_chomp(&ctx->output, ',');
> -            ds_put_cstr(&ctx->output, ")");
> -        }
> -        ds_put_cstr(&ctx->output, "\n");
> -    }
> -
> -    free(acls);
> -}
> -
> -static int
> -qos_cmp(const void *qos1_, const void *qos2_)
> -{
> -    const struct nbrec_qos *const *qos1p = qos1_;
> -    const struct nbrec_qos *const *qos2p = qos2_;
> -    const struct nbrec_qos *qos1 = *qos1p;
> -    const struct nbrec_qos *qos2 = *qos2p;
> -
> -    int dir1 = dir_encode(qos1->direction);
> -    int dir2 = dir_encode(qos2->direction);
> -
> -    if (dir1 != dir2) {
> -        return dir1 < dir2 ? -1 : 1;
> -    } else if (qos1->priority != qos2->priority) {
> -        return qos1->priority > qos2->priority ? -1 : 1;
> -    } else {
> -        return strcmp(qos1->match, qos2->match);
> -    }
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -parse_direction(const char *arg, const char **direction_p)
> -{
> -    /* Validate direction.  Only require the first letter. */
> -    if (arg[0] == 't') {
> -        *direction_p = "to-lport";
> -    } else if (arg[0] == 'f') {
> -        *direction_p = "from-lport";
> -    } else {
> -        *direction_p = NULL;
> -        return xasprintf("%s: direction must be \"to-lport\" or "
> -                         "\"from-lport\"", arg);
> -    }
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -parse_priority(const char *arg, int64_t *priority_p)
> -{
> -    /* Validate priority. */
> -    int64_t priority;
> -    if (!ovs_scan(arg, "%"SCNd64, &priority)
> -        || priority < 0 || priority > 32767) {
> -        /* Priority_p could be uninitialized as no valid priority was
> -         * input, initialize it to a valid value of 0 before returning */
> -        *priority_p = 0;
> -        return xasprintf("%s: priority must in range 0...32767", arg);
> -    }
> -    *priority_p = priority;
> -    return NULL;
> -}
> -
> -static void
> -nbctl_acl_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls = NULL;
> -    const struct nbrec_port_group *pg = NULL;
> -    const char *action = ctx->argv[5];
> -
> -    char *error = acl_cmd_get_pg_or_ls(ctx, &ls, &pg);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const char *direction;
> -    error = parse_direction(ctx->argv[2], &direction);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    int64_t priority;
> -    error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    /* Validate action. */
> -    if (strcmp(action, "allow") && strcmp(action, "allow-related")
> -        && strcmp(action, "drop") && strcmp(action, "reject")) {
> -        ctl_error(ctx, "%s: action must be one of \"allow\", "
> -                  "\"allow-related\", \"drop\", and \"reject\"", action);
> -        return;
> -    }
> -
> -    /* Create the acl. */
> -    struct nbrec_acl *acl = nbrec_acl_insert(ctx->txn);
> -    nbrec_acl_set_priority(acl, priority);
> -    nbrec_acl_set_direction(acl, direction);
> -    nbrec_acl_set_match(acl, ctx->argv[4]);
> -    nbrec_acl_set_action(acl, action);
> -
> -    /* Logging options. */
> -    bool log = shash_find(&ctx->options, "--log") != NULL;
> -    const char *severity = shash_find_data(&ctx->options, "--severity");
> -    const char *name = shash_find_data(&ctx->options, "--name");
> -    const char *meter = shash_find_data(&ctx->options, "--meter");
> -    if (log || severity || name || meter) {
> -        nbrec_acl_set_log(acl, true);
> -    }
> -    if (severity) {
> -        if (log_severity_from_string(severity) == UINT8_MAX) {
> -            ctl_error(ctx, "bad severity: %s", severity);
> -            return;
> -        }
> -        nbrec_acl_set_severity(acl, severity);
> -    }
> -    if (name) {
> -        nbrec_acl_set_name(acl, name);
> -    }
> -    if (meter) {
> -        nbrec_acl_set_meter(acl, meter);
> -    }
> -
> -    /* Check if same acl already exists for the ls/portgroup */
> -    size_t n_acls = pg ? pg->n_acls : ls->n_acls;
> -    struct nbrec_acl **acls = pg ? pg->acls : ls->acls;
> -    for (size_t i = 0; i < n_acls; i++) {
> -        if (!acl_cmp(&acls[i], &acl)) {
> -            bool may_exist = shash_find(&ctx->options, "--may-exist") !=
> NULL;
> -            if (!may_exist) {
> -                ctl_error(ctx, "Same ACL already existed on the ls %s.",
> -                          ctx->argv[1]);
> -                return;
> -            }
> -            return;
> -        }
> -    }
> -
> -    /* Insert the acl into the logical switch/port group. */
> -    struct nbrec_acl **new_acls = xmalloc(sizeof *new_acls * (n_acls +
> 1));
> -    nullable_memcpy(new_acls, acls, sizeof *new_acls * n_acls);
> -    new_acls[n_acls] = acl;
> -    if (pg) {
> -        nbrec_port_group_verify_acls(pg);
> -        nbrec_port_group_set_acls(pg, new_acls, n_acls + 1);
> -    } else {
> -        nbrec_logical_switch_verify_acls(ls);
> -        nbrec_logical_switch_set_acls(ls, new_acls, n_acls + 1);
> -    }
> -    free(new_acls);
> -}
> -
> -static void
> -nbctl_acl_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls = NULL;
> -    const struct nbrec_port_group *pg = NULL;
> -
> -    char *error = acl_cmd_get_pg_or_ls(ctx, &ls, &pg);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If direction, priority, and match are not specified, delete
> -         * all ACLs. */
> -        if (pg) {
> -            nbrec_port_group_verify_acls(pg);
> -            nbrec_port_group_set_acls(pg, NULL, 0);
> -        } else {
> -            nbrec_logical_switch_verify_acls(ls);
> -            nbrec_logical_switch_set_acls(ls, NULL, 0);
> -        }
> -        return;
> -    }
> -
> -    const char *direction;
> -    error = parse_direction(ctx->argv[2], &direction);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    size_t n_acls = pg ? pg->n_acls : ls->n_acls;
> -    struct nbrec_acl **acls = pg ? pg->acls : ls->acls;
> -    /* If priority and match are not specified, delete all ACLs with the
> -     * specified direction. */
> -    if (ctx->argc == 3) {
> -        struct nbrec_acl **new_acls = xmalloc(sizeof *new_acls * n_acls);
> -
> -        int n_new_acls = 0;
> -        for (size_t i = 0; i < n_acls; i++) {
> -            if (strcmp(direction, acls[i]->direction)) {
> -                new_acls[n_new_acls++] = acls[i];
> -            }
> -        }
> -
> -        if (pg) {
> -            nbrec_port_group_verify_acls(pg);
> -            nbrec_port_group_set_acls(pg, new_acls, n_new_acls);
> -        } else {
> -            nbrec_logical_switch_verify_acls(ls);
> -            nbrec_logical_switch_set_acls(ls, new_acls, n_new_acls);
> -        }
> -        free(new_acls);
> -        return;
> -    }
> -
> -    int64_t priority;
> -    error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 4) {
> -        ctl_error(ctx, "cannot specify priority without match");
> -        return;
> -    }
> -
> -    /* Remove the matching rule. */
> -    for (size_t i = 0; i < n_acls; i++) {
> -        struct nbrec_acl *acl = acls[i];
> -
> -        if (priority == acl->priority && !strcmp(ctx->argv[4],
> acl->match) &&
> -             !strcmp(direction, acl->direction)) {
> -            struct nbrec_acl **new_acls
> -                = xmemdup(acls, sizeof *new_acls * n_acls);
> -            new_acls[i] = acls[n_acls - 1];
> -            if (pg) {
> -                nbrec_port_group_verify_acls(pg);
> -                nbrec_port_group_set_acls(pg, new_acls,
> -                                          n_acls - 1);
> -            } else {
> -                nbrec_logical_switch_verify_acls(ls);
> -                nbrec_logical_switch_set_acls(ls, new_acls,
> -                                              n_acls - 1);
> -            }
> -            free(new_acls);
> -            return;
> -        }
> -    }
> -}
> -
> -static void
> -nbctl_qos_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    const struct nbrec_qos **qos_rules;
> -    size_t i;
> -
> -    char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    qos_rules = xmalloc(sizeof *qos_rules * ls->n_qos_rules);
> -    for (i = 0; i < ls->n_qos_rules; i++) {
> -        qos_rules[i] = ls->qos_rules[i];
> -    }
> -
> -    qsort(qos_rules, ls->n_qos_rules, sizeof *qos_rules, qos_cmp);
> -
> -    for (i = 0; i < ls->n_qos_rules; i++) {
> -        const struct nbrec_qos *qos_rule = qos_rules[i];
> -        ds_put_format(&ctx->output, "%10s %5"PRId64" (%s)",
> -                      qos_rule->direction, qos_rule->priority,
> -                      qos_rule->match);
> -        for (size_t j = 0; j < qos_rule->n_bandwidth; j++) {
> -            if (!strcmp(qos_rule->key_bandwidth[j], "rate")) {
> -                ds_put_format(&ctx->output, " rate=%"PRId64"",
> -                              qos_rule->value_bandwidth[j]);
> -            }
> -        }
> -        for (size_t j = 0; j < qos_rule->n_bandwidth; j++) {
> -            if (!strcmp(qos_rule->key_bandwidth[j], "burst")) {
> -                ds_put_format(&ctx->output, " burst=%"PRId64"",
> -                              qos_rule->value_bandwidth[j]);
> -            }
> -        }
> -        for (size_t j = 0; j < qos_rule->n_action; j++) {
> -            if (!strcmp(qos_rule->key_action[j], "dscp")) {
> -                ds_put_format(&ctx->output, " dscp=%"PRId64"",
> -                              qos_rule->value_action[j]);
> -            }
> -        }
> -        ds_put_cstr(&ctx->output, "\n");
> -    }
> -
> -    free(qos_rules);
> -}
> -
> -static void
> -nbctl_qos_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    const char *direction;
> -    int64_t priority;
> -    int64_t dscp = -1;
> -    int64_t rate = 0;
> -    int64_t burst = 0;
> -    char *error;
> -
> -    error = parse_direction(ctx->argv[2], &direction);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    for (int i = 5; i < ctx->argc; i++) {
> -        if (!strncmp(ctx->argv[i], "dscp=", 5)) {
> -            if (!ovs_scan(ctx->argv[i] + 5, "%"SCNd64, &dscp)
> -                || dscp < 0 || dscp > 63) {
> -                ctl_error(ctx, "%s: dscp must be in the range 0...63",
> -                          ctx->argv[i] + 5);
> -                return;
> -            }
> -        }
> -        else if (!strncmp(ctx->argv[i], "rate=", 5)) {
> -            if (!ovs_scan(ctx->argv[i] + 5, "%"SCNd64, &rate)
> -                || rate < 1 || rate > UINT32_MAX) {
> -                ctl_error(ctx, "%s: rate must be in the range
> 1...4294967295",
> -                          ctx->argv[i] + 5);
> -                return;
> -            }
> -        }
> -        else if (!strncmp(ctx->argv[i], "burst=", 6)) {
> -            if (!ovs_scan(ctx->argv[i] + 6, "%"SCNd64, &burst)
> -                || burst < 1 || burst > UINT32_MAX) {
> -                ctl_error(ctx, "%s: burst must be in the range
> 1...4294967295",
> -                          ctx->argv[i] + 6);
> -                return;
> -            }
> -        } else {
> -            ctl_error(ctx, "%s: supported arguments are \"dscp=\",
> \"rate=\", "
> -                      "and \"burst=\"", ctx->argv[i]);
> -            return;
> -        }
> -    }
> -
> -    /* Validate rate and dscp. */
> -    if (-1 == dscp && !rate) {
> -        ctl_error(ctx, "Either \"rate\" and/or \"dscp\" must be
> specified");
> -        return;
> -    }
> -
> -    /* Create the qos. */
> -    struct nbrec_qos *qos = nbrec_qos_insert(ctx->txn);
> -    nbrec_qos_set_priority(qos, priority);
> -    nbrec_qos_set_direction(qos, direction);
> -    nbrec_qos_set_match(qos, ctx->argv[4]);
> -    if (-1 != dscp) {
> -        const char *dscp_key = "dscp";
> -        nbrec_qos_set_action(qos, &dscp_key, &dscp, 1);
> -    }
> -    if (rate) {
> -        const char *bandwidth_key[2] = {"rate", "burst"};
> -        const int64_t bandwidth_value[2] = {rate, burst};
> -        size_t n_bandwidth = 1;
> -        if (burst) {
> -            n_bandwidth = 2;
> -        }
> -        nbrec_qos_set_bandwidth(qos, bandwidth_key, bandwidth_value,
> -                                n_bandwidth);
> -    }
> -
> -    /* Check if same qos rule already exists for the ls */
> -    for (size_t i = 0; i < ls->n_qos_rules; i++) {
> -        if (!qos_cmp(&ls->qos_rules[i], &qos)) {
> -            bool may_exist = shash_find(&ctx->options, "--may-exist") !=
> NULL;
> -            if (!may_exist) {
> -                ctl_error(ctx, "Same qos already existed on the ls %s.",
> -                          ctx->argv[1]);
> -                return;
> -            }
> -            return;
> -        }
> -    }
> -
> -    /* Insert the qos rule the logical switch. */
> -    nbrec_logical_switch_verify_qos_rules(ls);
> -    struct nbrec_qos **new_qos_rules
> -        = xmalloc(sizeof *new_qos_rules * (ls->n_qos_rules + 1));
> -    nullable_memcpy(new_qos_rules,
> -                    ls->qos_rules, sizeof *new_qos_rules *
> ls->n_qos_rules);
> -    new_qos_rules[ls->n_qos_rules] = qos;
> -    nbrec_logical_switch_set_qos_rules(ls, new_qos_rules,
> -                                       ls->n_qos_rules + 1);
> -    free(new_qos_rules);
> -}
> -
> -static void
> -nbctl_qos_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If direction, priority, and match are not specified, delete
> -         * all QoS rules. */
> -        nbrec_logical_switch_verify_qos_rules(ls);
> -        nbrec_logical_switch_set_qos_rules(ls, NULL, 0);
> -        return;
> -    }
> -
> -    const char *direction;
> -    error = parse_direction(ctx->argv[2], &direction);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    /* If priority and match are not specified, delete all qos_rules with
> the
> -     * specified direction. */
> -    if (ctx->argc == 3) {
> -        struct nbrec_qos **new_qos_rules
> -            = xmalloc(sizeof *new_qos_rules * ls->n_qos_rules);
> -
> -        int n_qos_rules = 0;
> -        for (size_t i = 0; i < ls->n_qos_rules; i++) {
> -            if (strcmp(direction, ls->qos_rules[i]->direction)) {
> -                new_qos_rules[n_qos_rules++] = ls->qos_rules[i];
> -            }
> -        }
> -
> -        nbrec_logical_switch_verify_qos_rules(ls);
> -        nbrec_logical_switch_set_qos_rules(ls, new_qos_rules,
> n_qos_rules);
> -        free(new_qos_rules);
> -        return;
> -    }
> -
> -    int64_t priority;
> -    error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 4) {
> -        ctl_error(ctx, "cannot specify priority without match");
> -        return;
> -    }
> -
> -    /* Remove the matching rule. */
> -    for (size_t i = 0; i < ls->n_qos_rules; i++) {
> -        struct nbrec_qos *qos = ls->qos_rules[i];
> -
> -        if (priority == qos->priority && !strcmp(ctx->argv[4],
> qos->match) &&
> -             !strcmp(direction, qos->direction)) {
> -            struct nbrec_qos **new_qos_rules
> -                = xmemdup(ls->qos_rules,
> -                          sizeof *new_qos_rules * ls->n_qos_rules);
> -            new_qos_rules[i] = ls->qos_rules[ls->n_qos_rules - 1];
> -            nbrec_logical_switch_verify_qos_rules(ls);
> -            nbrec_logical_switch_set_qos_rules(ls, new_qos_rules,
> -                                          ls->n_qos_rules - 1);
> -            free(new_qos_rules);
> -            return;
> -        }
> -    }
> -}
> -
> -static int
> -meter_cmp(const void *meter1_, const void *meter2_)
> -{
> -    struct nbrec_meter *const *meter1p = meter1_;
> -    struct nbrec_meter *const *meter2p = meter2_;
> -    const struct nbrec_meter *meter1 = *meter1p;
> -    const struct nbrec_meter *meter2 = *meter2p;
> -
> -    return strcmp(meter1->name, meter2->name);
> -}
> -
> -static void
> -nbctl_meter_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_meter **meters = NULL;
> -    const struct nbrec_meter *meter;
> -    size_t n_capacity = 0;
> -    size_t n_meters = 0;
> -
> -    NBREC_METER_FOR_EACH (meter, ctx->idl) {
> -        if (n_meters == n_capacity) {
> -            meters = x2nrealloc(meters, &n_capacity, sizeof *meters);
> -        }
> -
> -        meters[n_meters] = meter;
> -        n_meters++;
> -    }
> -
> -    if (n_meters) {
> -        qsort(meters, n_meters, sizeof *meters, meter_cmp);
> -    }
> -
> -    for (size_t i = 0; i < n_meters; i++) {
> -        meter = meters[i];
> -        ds_put_format(&ctx->output, "%s: bands:\n", meter->name);
> -
> -        for (size_t j = 0; j < meter->n_bands; j++) {
> -            const struct nbrec_meter_band *band = meter->bands[j];
> -
> -            ds_put_format(&ctx->output, "  %s: %"PRId64" %s",
> -                          band->action, band->rate, meter->unit);
> -            if (band->burst_size) {
> -                ds_put_format(&ctx->output, ", %"PRId64" %s burst",
> -                              band->burst_size,
> -                              !strcmp(meter->unit, "kbps") ? "kb" :
> "packet" );
> -            }
> -        }
> -
> -        ds_put_cstr(&ctx->output, "\n");
> -    }
> -
> -    free(meters);
> -}
> -
> -static void
> -nbctl_meter_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_meter *meter;
> -
> -    const char *name = ctx->argv[1];
> -    NBREC_METER_FOR_EACH (meter, ctx->idl) {
> -        if (!strcmp(meter->name, name)) {
> -            ctl_error(ctx, "meter with name \"%s\" already exists", name);
> -            return;
> -        }
> -    }
> -
> -    if (!strncmp(name, "__", 2)) {
> -        ctl_error(ctx, "meter names that begin with \"__\" are reserved");
> -        return;
> -    }
> -
> -    const char *action = ctx->argv[2];
> -    if (strcmp(action, "drop")) {
> -        ctl_error(ctx, "action must be \"drop\"");
> -        return;
> -    }
> -
> -    int64_t rate;
> -    if (!ovs_scan(ctx->argv[3], "%"SCNd64, &rate)
> -        || rate < 1 || rate > UINT32_MAX) {
> -        ctl_error(ctx, "rate must be in the range 1...4294967295");
> -        return;
> -    }
> -
> -    const char *unit = ctx->argv[4];
> -    if (strcmp(unit, "kbps") && strcmp(unit, "pktps")) {
> -        ctl_error(ctx, "unit must be \"kbps\" or \"pktps\"");
> -        return;
> -    }
> -
> -    int64_t burst = 0;
> -    if (ctx->argc > 5) {
> -        if (!ovs_scan(ctx->argv[5], "%"SCNd64, &burst)
> -            || burst < 0 || burst > UINT32_MAX) {
> -            ctl_error(ctx, "burst must be in the range 0...4294967295");
> -            return;
> -        }
> -    }
> -
> -    /* Create the band.  We only support adding a single band. */
> -    struct nbrec_meter_band *band = nbrec_meter_band_insert(ctx->txn);
> -    nbrec_meter_band_set_action(band, action);
> -    nbrec_meter_band_set_rate(band, rate);
> -    nbrec_meter_band_set_burst_size(band, burst);
> -
> -    /* Create the meter. */
> -    meter = nbrec_meter_insert(ctx->txn);
> -    nbrec_meter_set_name(meter, name);
> -    nbrec_meter_set_unit(meter, unit);
> -    nbrec_meter_set_bands(meter, &band, 1);
> -}
> -
> -static void
> -nbctl_meter_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_meter *meter, *next;
> -
> -    /* If a name is not specified, delete all meters. */
> -    if (ctx->argc == 1) {
> -        NBREC_METER_FOR_EACH_SAFE (meter, next, ctx->idl) {
> -            nbrec_meter_delete(meter);
> -        }
> -        return;
> -    }
> -
> -    /* Remove the matching meter. */
> -    NBREC_METER_FOR_EACH (meter, ctx->idl) {
> -        if (strcmp(ctx->argv[1], meter->name)) {
> -            continue;
> -        }
> -
> -        nbrec_meter_delete(meter);
> -        return;
> -    }
> -}
> -
> -static void
> -nbctl_lb_add(struct ctl_context *ctx)
> -{
> -    const char *lb_name = ctx->argv[1];
> -    const char *lb_vip = ctx->argv[2];
> -    char *lb_ips = ctx->argv[3];
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") !=
> NULL;
> -
> -    const char *lb_proto;
> -    bool is_update_proto = false;
> -
> -    if (ctx->argc == 4) {
> -        /* Default protocol. */
> -        lb_proto = "tcp";
> -    } else {
> -        /* Validate protocol. */
> -        lb_proto = ctx->argv[4];
> -        is_update_proto = true;
> -        if (strcmp(lb_proto, "tcp") && strcmp(lb_proto, "udp")) {
> -            ctl_error(ctx, "%s: protocol must be one of \"tcp\",
> \"udp\".",
> -                      lb_proto);
> -            return;
> -        }
> -    }
> -
> -    struct sockaddr_storage ss_vip;
> -    if (!inet_parse_active(lb_vip, 0, &ss_vip, false)) {
> -        ctl_error(ctx, "%s: should be an IP address (or an IP address "
> -                  "and a port number with : as a separator).", lb_vip);
> -        return;
> -    }
> -
> -    struct ds lb_vip_normalized_ds = DS_EMPTY_INITIALIZER;
> -    uint16_t lb_vip_port = ss_get_port(&ss_vip);
> -    if (lb_vip_port) {
> -        ss_format_address(&ss_vip, &lb_vip_normalized_ds);
> -        ds_put_format(&lb_vip_normalized_ds, ":%d", lb_vip_port);
> -    } else {
> -        ss_format_address_nobracks(&ss_vip, &lb_vip_normalized_ds);
> -    }
> -    const char *lb_vip_normalized = ds_cstr(&lb_vip_normalized_ds);
> -
> -    if (!lb_vip_port && is_update_proto) {
> -        ds_destroy(&lb_vip_normalized_ds);
> -        ctl_error(ctx, "Protocol is unnecessary when no port of vip "
> -                  "is given.");
> -        return;
> -    }
> -
> -    char *token = NULL, *save_ptr = NULL;
> -    struct ds lb_ips_new = DS_EMPTY_INITIALIZER;
> -    for (token = strtok_r(lb_ips, ",", &save_ptr);
> -            token != NULL; token = strtok_r(NULL, ",", &save_ptr)) {
> -        struct sockaddr_storage ss_dst;
> -
> -        if (lb_vip_port) {
> -            if (!inet_parse_active(token, -1, &ss_dst, false)) {
> -                ctl_error(ctx, "%s: should be an IP address and a port "
> -                          "number with : as a separator.", token);
> -                goto out;
> -            }
> -        } else {
> -            if (!inet_parse_address(token, &ss_dst)) {
> -                ctl_error(ctx, "%s: should be an IP address.", token);
> -                goto out;
> -            }
> -        }
> -
> -        if (ss_vip.ss_family != ss_dst.ss_family) {
> -            ctl_error(ctx, "%s: IP address family is different from VIP
> %s.",
> -                      token, lb_vip_normalized);
> -            goto out;
> -        }
> -        ds_put_format(&lb_ips_new, "%s%s",
> -                lb_ips_new.length ? "," : "", token);
> -    }
> -
> -    const struct nbrec_load_balancer *lb = NULL;
> -    if (!add_duplicate) {
> -        char *error = lb_by_name_or_uuid(ctx, lb_name, false, &lb);
> -        if (error) {
> -            ctx->error = error;
> -            goto out;
> -        }
> -        if (lb) {
> -            if (smap_get(&lb->vips, lb_vip_normalized)) {
> -                if (!may_exist) {
> -                    ctl_error(ctx, "%s: a load balancer with this vip
> (%s) "
> -                              "already exists", lb_name,
> lb_vip_normalized);
> -                    goto out;
> -                }
> -                /* Update the vips. */
> -                smap_replace(CONST_CAST(struct smap *, &lb->vips),
> -                        lb_vip_normalized, ds_cstr(&lb_ips_new));
> -            } else {
> -                /* Add the new vips. */
> -                smap_add(CONST_CAST(struct smap *, &lb->vips),
> -                        lb_vip_normalized, ds_cstr(&lb_ips_new));
> -            }
> -
> -            /* Update the load balancer. */
> -            if (is_update_proto) {
> -                nbrec_load_balancer_verify_protocol(lb);
> -                nbrec_load_balancer_set_protocol(lb, lb_proto);
> -            }
> -            nbrec_load_balancer_verify_vips(lb);
> -            nbrec_load_balancer_set_vips(lb, &lb->vips);
> -            goto out;
> -        }
> -    }
> -
> -    /* Create the load balancer. */
> -    lb = nbrec_load_balancer_insert(ctx->txn);
> -    nbrec_load_balancer_set_name(lb, lb_name);
> -    nbrec_load_balancer_set_protocol(lb, lb_proto);
> -    smap_add(CONST_CAST(struct smap *, &lb->vips),
> -            lb_vip_normalized, ds_cstr(&lb_ips_new));
> -    nbrec_load_balancer_set_vips(lb, &lb->vips);
> -out:
> -    ds_destroy(&lb_ips_new);
> -
> -    ds_destroy(&lb_vip_normalized_ds);
> -}
> -
> -static void
> -nbctl_lb_del(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_load_balancer *lb = NULL;
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -
> -    char *error = lb_by_name_or_uuid(ctx, id, false, &lb);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lb) {
> -        return;
> -    }
> -
> -    if (ctx->argc == 3) {
> -        const char *lb_vip = ctx->argv[2];
> -        if (smap_get(&lb->vips, lb_vip)) {
> -            smap_remove(CONST_CAST(struct smap *, &lb->vips), lb_vip);
> -            if (smap_is_empty(&lb->vips)) {
> -                nbrec_load_balancer_delete(lb);
> -                return;
> -            }
> -
> -            /* Delete the vip of the load balancer. */
> -            nbrec_load_balancer_verify_vips(lb);
> -            nbrec_load_balancer_set_vips(lb, &lb->vips);
> -            return;
> -        }
> -        if (must_exist) {
> -            ctl_error(ctx, "vip %s is not part of the load balancer.",
> -                      lb_vip);
> -            return;
> -        }
> -        return;
> -    }
> -    nbrec_load_balancer_delete(lb);
> -}
> -
> -static void
> -lb_info_add_smap(const struct nbrec_load_balancer *lb,
> -                 struct smap *lbs, int vip_width)
> -{
> -    const struct smap_node **nodes = smap_sort(&lb->vips);
> -    if (nodes) {
> -        struct ds val = DS_EMPTY_INITIALIZER;
> -        for (int i = 0; i < smap_count(&lb->vips); i++) {
> -            const struct smap_node *node = nodes[i];
> -
> -            struct sockaddr_storage ss;
> -            if (!inet_parse_active(node->key, 0, &ss, false)) {
> -                continue;
> -            }
> -
> -            char *protocol = ss_get_port(&ss) ? lb->protocol : "tcp/udp";
> -            i == 0 ? ds_put_format(&val,
> -                        UUID_FMT "    %-20.16s%-11.7s%-*.*s%s",
> -                        UUID_ARGS(&lb->header_.uuid),
> -                        lb->name, protocol,
> -                        vip_width + 4, vip_width,
> -                        node->key, node->value)
> -                   : ds_put_format(&val, "\n%60s%-11.7s%-*.*s%s",
> -                        "", protocol,
> -                        vip_width + 4, vip_width,
> -                        node->key, node->value);
> -        }
> -
> -        smap_add_nocopy(lbs, xasprintf("%-20.16s", lb->name),
> -                        ds_steal_cstr(&val));
> -        free(nodes);
> -    }
> -}
> -
> -static void
> -lb_info_print(struct ctl_context *ctx, struct smap *lbs, int vip_width)
> -{
> -    const struct smap_node **nodes = smap_sort(lbs);
> -    if (nodes) {
> -        ds_put_format(&ctx->output, "%-40.36s%-20.16s%-11.7s%-*.*s%s\n",
> -                "UUID", "LB", "PROTO", vip_width + 4, vip_width, "VIP",
> "IPs");
> -        for (size_t i = 0; i < smap_count(lbs); i++) {
> -            const struct smap_node *node = nodes[i];
> -            ds_put_format(&ctx->output, "%s\n", node->value);
> -        }
> -
> -        free(nodes);
> -    }
> -}
> -
> -static int
> -lb_get_max_vip_length(const struct nbrec_load_balancer *lb, int vip_width)
> -{
> -    const struct smap_node *node;
> -    int max_length = vip_width;
> -
> -    SMAP_FOR_EACH (node, &lb->vips) {
> -        size_t keylen = strlen(node->key);
> -        if (max_length < keylen) {
> -            max_length = keylen;
> -        }
> -    }
> -
> -    return max_length;
> -}
> -
> -static void
> -lb_info_list_all(struct ctl_context *ctx,
> -                 const char *lb_name, bool lb_check)
> -{
> -    const struct nbrec_load_balancer *lb;
> -    struct smap lbs = SMAP_INITIALIZER(&lbs);
> -    int vip_width = 0;
> -
> -    NBREC_LOAD_BALANCER_FOR_EACH (lb, ctx->idl) {
> -        if (lb_check && strcmp(lb->name, lb_name)) {
> -            continue;
> -        }
> -        vip_width = lb_get_max_vip_length(lb, vip_width);
> -    }
> -
> -    NBREC_LOAD_BALANCER_FOR_EACH(lb, ctx->idl) {
> -        if (lb_check && strcmp(lb->name, lb_name)) {
> -            continue;
> -        }
> -        lb_info_add_smap(lb, &lbs, vip_width);
> -    }
> -
> -    lb_info_print(ctx, &lbs, vip_width);
> -    smap_destroy(&lbs);
> -}
> -
> -static void
> -nbctl_lb_list(struct ctl_context *ctx)
> -{
> -    if (ctx->argc == 1) {
> -        lb_info_list_all(ctx, NULL, false);
> -    } else if (ctx->argc == 2) {
> -        lb_info_list_all(ctx, ctx->argv[1], true);
> -    }
> -}
> -
> -static void
> -nbctl_lr_lb_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr = NULL;
> -    const struct nbrec_load_balancer *new_lb;
> -
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    error = lb_by_name_or_uuid(ctx, ctx->argv[2], true, &new_lb);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    for (int i = 0; i < lr->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = lr->load_balancer[i];
> -
> -        if (uuid_equals(&new_lb->header_.uuid, &lb->header_.uuid)) {
> -            if (may_exist) {
> -                return;
> -            }
> -            ctl_error(ctx, UUID_FMT " : a load balancer with this UUID "
> -                      "already exists", UUID_ARGS(&lb->header_.uuid));
> -            return;
> -        }
> -    }
> -
> -    /* Insert the load balancer into the logical router. */
> -    nbrec_logical_router_verify_load_balancer(lr);
> -    struct nbrec_load_balancer **new_lbs
> -        = xmalloc(sizeof *new_lbs * (lr->n_load_balancer + 1));
> -
> -    nullable_memcpy(new_lbs, lr->load_balancer,
> -                    sizeof *new_lbs * lr->n_load_balancer);
> -    new_lbs[lr->n_load_balancer] = CONST_CAST(struct nbrec_load_balancer
> *,
> -            new_lb);
> -    nbrec_logical_router_set_load_balancer(lr, new_lbs,
> -            lr->n_load_balancer + 1);
> -    free(new_lbs);
> -}
> -
> -static void
> -nbctl_lr_lb_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    const struct nbrec_load_balancer *del_lb;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If load-balancer is not specified, remove
> -         * all load-balancers from the logical router. */
> -        nbrec_logical_router_verify_load_balancer(lr);
> -        nbrec_logical_router_set_load_balancer(lr, NULL, 0);
> -        return;
> -    }
> -
> -    error = lb_by_name_or_uuid(ctx, ctx->argv[2], true, &del_lb);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    for (size_t i = 0; i < lr->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = lr->load_balancer[i];
> -
> -        if (uuid_equals(&del_lb->header_.uuid, &lb->header_.uuid)) {
> -            /* Remove the matching rule. */
> -            nbrec_logical_router_verify_load_balancer(lr);
> -
> -            struct nbrec_load_balancer **new_lbs
> -                = xmemdup(lr->load_balancer,
> -                    sizeof *new_lbs * lr->n_load_balancer);
> -            new_lbs[i] = lr->load_balancer[lr->n_load_balancer - 1];
> -            nbrec_logical_router_set_load_balancer(lr, new_lbs,
> -                                          lr->n_load_balancer - 1);
> -            free(new_lbs);
> -            return;
> -        }
> -    }
> -
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    if (must_exist) {
> -        ctl_error(ctx, "load balancer %s is not part of any logical
> router.",
> -                  del_lb->name);
> -        return;
> -    }
> -}
> -
> -static void
> -nbctl_lr_lb_list(struct ctl_context *ctx)
> -{
> -    const char *lr_name = ctx->argv[1];
> -    const struct nbrec_logical_router *lr;
> -    struct smap lbs = SMAP_INITIALIZER(&lbs);
> -    int vip_width = 0;
> -
> -    char *error = lr_by_name_or_uuid(ctx, lr_name, true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    for (int i = 0; i < lr->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = lr->load_balancer[i];
> -        vip_width = lb_get_max_vip_length(lb, vip_width);
> -    }
> -    for (int i = 0; i < lr->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = lr->load_balancer[i];
> -        lb_info_add_smap(lb, &lbs, vip_width);
> -    }
> -
> -    lb_info_print(ctx, &lbs, vip_width);
> -    smap_destroy(&lbs);
> -}
> -
> -static void
> -nbctl_ls_lb_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls = NULL;
> -    const struct nbrec_load_balancer *new_lb;
> -
> -    char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    error = lb_by_name_or_uuid(ctx, ctx->argv[2], true, &new_lb);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    for (int i = 0; i < ls->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = ls->load_balancer[i];
> -
> -        if (uuid_equals(&new_lb->header_.uuid, &lb->header_.uuid)) {
> -            if (may_exist) {
> -                return;
> -            }
> -            ctl_error(ctx, UUID_FMT " : a load balancer with this UUID "
> -                      "already exists", UUID_ARGS(&lb->header_.uuid));
> -            return;
> -        }
> -    }
> -
> -    /* Insert the load balancer into the logical switch. */
> -    nbrec_logical_switch_verify_load_balancer(ls);
> -    struct nbrec_load_balancer **new_lbs
> -        = xmalloc(sizeof *new_lbs * (ls->n_load_balancer + 1));
> -
> -    nullable_memcpy(new_lbs, ls->load_balancer,
> -                    sizeof *new_lbs * ls->n_load_balancer);
> -    new_lbs[ls->n_load_balancer] = CONST_CAST(struct nbrec_load_balancer
> *,
> -            new_lb);
> -    nbrec_logical_switch_set_load_balancer(ls, new_lbs,
> -            ls->n_load_balancer + 1);
> -    free(new_lbs);
> -}
> -
> -static void
> -nbctl_ls_lb_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_switch *ls;
> -    const struct nbrec_load_balancer *del_lb;
> -    char *error = ls_by_name_or_uuid(ctx, ctx->argv[1], true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If load-balancer is not specified, remove
> -         * all load-balancers from the logical switch. */
> -        nbrec_logical_switch_verify_load_balancer(ls);
> -        nbrec_logical_switch_set_load_balancer(ls, NULL, 0);
> -        return;
> -    }
> -
> -    error = lb_by_name_or_uuid(ctx, ctx->argv[2], true, &del_lb);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    for (size_t i = 0; i < ls->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = ls->load_balancer[i];
> -
> -        if (uuid_equals(&del_lb->header_.uuid, &lb->header_.uuid)) {
> -            /* Remove the matching rule. */
> -            nbrec_logical_switch_verify_load_balancer(ls);
> -
> -            struct nbrec_load_balancer **new_lbs
> -                = xmemdup(ls->load_balancer,
> -                        sizeof *new_lbs * ls->n_load_balancer);
> -            new_lbs[i] = ls->load_balancer[ls->n_load_balancer - 1];
> -            nbrec_logical_switch_set_load_balancer(ls, new_lbs,
> -                                          ls->n_load_balancer - 1);
> -            free(new_lbs);
> -            return;
> -        }
> -    }
> -
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    if (must_exist) {
> -        ctl_error(ctx, "load balancer %s is not part of any logical
> switch.",
> -                  del_lb->name);
> -        return;
> -    }
> -}
> -
> -static void
> -nbctl_ls_lb_list(struct ctl_context *ctx)
> -{
> -    const char *ls_name = ctx->argv[1];
> -    const struct nbrec_logical_switch *ls;
> -    struct smap lbs = SMAP_INITIALIZER(&lbs);
> -    int vip_width = 0;
> -
> -    char *error = ls_by_name_or_uuid(ctx, ls_name, true, &ls);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    for (int i = 0; i < ls->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = ls->load_balancer[i];
> -        vip_width = lb_get_max_vip_length(lb, vip_width);
> -    }
> -    for (int i = 0; i < ls->n_load_balancer; i++) {
> -        const struct nbrec_load_balancer *lb
> -            = ls->load_balancer[i];
> -        lb_info_add_smap(lb, &lbs, vip_width);
> -    }
> -
> -    lb_info_print(ctx, &lbs, vip_width);
> -    smap_destroy(&lbs);
> -}
> -
> -static void
> -nbctl_lr_add(struct ctl_context *ctx)
> -{
> -    const char *lr_name = ctx->argc == 2 ? ctx->argv[1] : NULL;
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") !=
> NULL;
> -    if (may_exist && add_duplicate) {
> -        ctl_error(ctx, "--may-exist and --add-duplicate may not be used "
> -                  "together");
> -        return;
> -    }
> -
> -    if (lr_name) {
> -        if (!add_duplicate) {
> -            const struct nbrec_logical_router *lr;
> -            NBREC_LOGICAL_ROUTER_FOR_EACH (lr, ctx->idl) {
> -                if (!strcmp(lr->name, lr_name)) {
> -                    if (may_exist) {
> -                        return;
> -                    }
> -                    ctl_error(ctx, "%s: a router with this name already "
> -                              "exists", lr_name);
> -                    return;
> -                }
> -            }
> -        }
> -    } else if (may_exist) {
> -        ctl_error(ctx, "--may-exist requires specifying a name");
> -        return;
> -    } else if (add_duplicate) {
> -        ctl_error(ctx, "--add-duplicate requires specifying a name");
> -        return;
> -    }
> -
> -    struct nbrec_logical_router *lr;
> -    lr = nbrec_logical_router_insert(ctx->txn);
> -    if (lr_name) {
> -        nbrec_logical_router_set_name(lr, lr_name);
> -    }
> -}
> -
> -static void
> -nbctl_lr_del(struct ctl_context *ctx)
> -{
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_router *lr = NULL;
> -
> -    char *error = lr_by_name_or_uuid(ctx, id, must_exist, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lr) {
> -        return;
> -    }
> -
> -    nbrec_logical_router_delete(lr);
> -}
> -
> -static void
> -nbctl_lr_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    struct smap lrs;
> -
> -    smap_init(&lrs);
> -    NBREC_LOGICAL_ROUTER_FOR_EACH(lr, ctx->idl) {
> -        smap_add_format(&lrs, lr->name, UUID_FMT " (%s)",
> -                        UUID_ARGS(&lr->header_.uuid), lr->name);
> -    }
> -    const struct smap_node **nodes = smap_sort(&lrs);
> -    for (size_t i = 0; i < smap_count(&lrs); i++) {
> -        const struct smap_node *node = nodes[i];
> -        ds_put_format(&ctx->output, "%s\n", node->value);
> -    }
> -    smap_destroy(&lrs);
> -    free(nodes);
> -}
> -
> -static char *
> -dhcp_options_get(struct ctl_context *ctx, const char *id, bool must_exist,
> -                 const struct nbrec_dhcp_options **dhcp_opts_p)
> -{
> -    struct uuid dhcp_opts_uuid;
> -    const struct nbrec_dhcp_options *dhcp_opts = NULL;
> -    if (uuid_from_string(&dhcp_opts_uuid, id)) {
> -        dhcp_opts = nbrec_dhcp_options_get_for_uuid(ctx->idl,
> &dhcp_opts_uuid);
> -    }
> -
> -    *dhcp_opts_p = dhcp_opts;
> -    if (!dhcp_opts && must_exist) {
> -        return xasprintf("%s: dhcp options UUID not found", id);
> -    }
> -
> -    return NULL;
> -}
> -
> -static void
> -nbctl_dhcp_options_create(struct ctl_context *ctx)
> -{
> -    /* Validate the cidr */
> -    ovs_be32 ip;
> -    unsigned int plen;
> -    char *error = ip_parse_cidr(ctx->argv[1], &ip, &plen);
> -    if (error){
> -        /* check if its IPv6 cidr */
> -        free(error);
> -        struct in6_addr ipv6;
> -        error = ipv6_parse_cidr(ctx->argv[1], &ipv6, &plen);
> -        if (error) {
> -            free(error);
> -            ctl_error(ctx, "Invalid cidr format '%s'", ctx->argv[1]);
> -            return;
> -        }
> -    }
> -
> -    struct nbrec_dhcp_options *dhcp_opts =
> nbrec_dhcp_options_insert(ctx->txn);
> -    nbrec_dhcp_options_set_cidr(dhcp_opts, ctx->argv[1]);
> -
> -    struct smap ext_ids = SMAP_INITIALIZER(&ext_ids);
> -    for (size_t i = 2; i < ctx->argc; i++) {
> -        char *key, *value;
> -        value = xstrdup(ctx->argv[i]);
> -        key = strsep(&value, "=");
> -        if (value) {
> -            smap_add(&ext_ids, key, value);
> -        }
> -        free(key);
> -    }
> -
> -    nbrec_dhcp_options_set_external_ids(dhcp_opts, &ext_ids);
> -    smap_destroy(&ext_ids);
> -}
> -
> -static void
> -nbctl_dhcp_options_set_options(struct ctl_context *ctx)
> -{
> -    const struct nbrec_dhcp_options *dhcp_opts;
> -    char *error = dhcp_options_get(ctx, ctx->argv[1], true, &dhcp_opts);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    struct smap dhcp_options = SMAP_INITIALIZER(&dhcp_options);
> -    for (size_t i = 2; i < ctx->argc; i++) {
> -        char *key, *value;
> -        value = xstrdup(ctx->argv[i]);
> -        key = strsep(&value, "=");
> -        if (value) {
> -            smap_add(&dhcp_options, key, value);
> -        }
> -        free(key);
> -    }
> -
> -    nbrec_dhcp_options_set_options(dhcp_opts, &dhcp_options);
> -    smap_destroy(&dhcp_options);
> -}
> -
> -static void
> -nbctl_dhcp_options_get_options(struct ctl_context *ctx)
> -{
> -    const struct nbrec_dhcp_options *dhcp_opts;
> -    char *error = dhcp_options_get(ctx, ctx->argv[1], true, &dhcp_opts);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    struct smap_node *node;
> -    SMAP_FOR_EACH(node, &dhcp_opts->options) {
> -        ds_put_format(&ctx->output, "%s=%s\n", node->key, node->value);
> -    }
> -}
> -
> -static void
> -nbctl_dhcp_options_del(struct ctl_context *ctx)
> -{
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_dhcp_options *dhcp_opts;
> -
> -    char *error = dhcp_options_get(ctx, id, must_exist, &dhcp_opts);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!dhcp_opts) {
> -        return;
> -    }
> -
> -    nbrec_dhcp_options_delete(dhcp_opts);
> -}
> -
> -static void
> -nbctl_dhcp_options_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_dhcp_options *dhcp_opts;
> -    struct smap dhcp_options;
> -
> -    smap_init(&dhcp_options);
> -    NBREC_DHCP_OPTIONS_FOR_EACH(dhcp_opts, ctx->idl) {
> -        smap_add_format(&dhcp_options, dhcp_opts->cidr, UUID_FMT,
> -                        UUID_ARGS(&dhcp_opts->header_.uuid));
> -    }
> -    const struct smap_node **nodes = smap_sort(&dhcp_options);
> -    for (size_t i = 0; i < smap_count(&dhcp_options); i++) {
> -        const struct smap_node *node = nodes[i];
> -        ds_put_format(&ctx->output, "%s\n", node->value);
> -    }
> -    smap_destroy(&dhcp_options);
> -    free(nodes);
> -}
> -
> -/* The caller must free the returned string. */
> -static char *
> -normalize_ipv4_prefix(ovs_be32 ipv4, unsigned int plen)
> -{
> -    ovs_be32 network = ipv4 & be32_prefix_mask(plen);
> -    if (plen == 32) {
> -        return xasprintf(IP_FMT, IP_ARGS(network));
> -    } else {
> -        return xasprintf(IP_FMT"/%d", IP_ARGS(network), plen);
> -    }
> -}
> -
> -/* The caller must free the returned string. */
> -static char *
> -normalize_ipv6_prefix(struct in6_addr ipv6, unsigned int plen)
> -{
> -    char network_s[INET6_ADDRSTRLEN];
> -
> -    struct in6_addr mask = ipv6_create_mask(plen);
> -    struct in6_addr network = ipv6_addr_bitand(&ipv6, &mask);
> -
> -    inet_ntop(AF_INET6, &network, network_s, INET6_ADDRSTRLEN);
> -    if (plen == 128) {
> -        return xasprintf("%s", network_s);
> -    } else {
> -        return xasprintf("%s/%d", network_s, plen);
> -    }
> -}
> -
> -/* The caller must free the returned string. */
> -static char *
> -normalize_prefix_str(const char *orig_prefix)
> -{
> -    unsigned int plen;
> -    ovs_be32 ipv4;
> -    char *error;
> -
> -    error = ip_parse_cidr(orig_prefix, &ipv4, &plen);
> -    if (!error) {
> -        return normalize_ipv4_prefix(ipv4, plen);
> -    } else {
> -        struct in6_addr ipv6;
> -        free(error);
> -
> -        error = ipv6_parse_cidr(orig_prefix, &ipv6, &plen);
> -        if (error) {
> -            free(error);
> -            return NULL;
> -        }
> -        return normalize_ipv6_prefix(ipv6, plen);
> -    }
> -}
> -
> -static void
> -nbctl_lr_policy_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    int64_t priority = 0;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    error = parse_priority(ctx->argv[2], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    const char *action = ctx->argv[4];
> -    char *next_hop = NULL;
> -
> -    /* Validate action. */
> -    if (strcmp(action, "allow") && strcmp(action, "drop")
> -        && strcmp(action, "reroute")) {
> -        ctl_error(ctx, "%s: action must be one of \"allow\", \"drop\", "
> -                  "and \"reroute\"", action);
> -    }
> -    if (!strcmp(action, "reroute")) {
> -        if (ctx->argc < 6) {
> -            ctl_error(ctx, "Nexthop is required when action is reroute.");
> -        }
> -    }
> -
> -    /* Check if same routing policy already exists.
> -     * A policy is uniquely identified by priority and match */
> -    for (int i = 0; i < lr->n_policies; i++) {
> -        const struct nbrec_logical_router_policy *policy =
> lr->policies[i];
> -        if (policy->priority == priority &&
> -            !strcmp(policy->match, ctx->argv[3])) {
> -            ctl_error(ctx, "Same routing policy already existed on the "
> -                      "logical router %s.", ctx->argv[1]);
> -        }
> -    }
> -    if (ctx->argc == 6) {
> -        next_hop = normalize_prefix_str(ctx->argv[5]);
> -        if (!next_hop) {
> -            ctl_error(ctx, "bad next hop argument: %s", ctx->argv[5]);
> -        }
> -    }
> -
> -    struct nbrec_logical_router_policy *policy;
> -    policy = nbrec_logical_router_policy_insert(ctx->txn);
> -    nbrec_logical_router_policy_set_priority(policy, priority);
> -    nbrec_logical_router_policy_set_match(policy, ctx->argv[3]);
> -    nbrec_logical_router_policy_set_action(policy, action);
> -    if (ctx->argc == 6) {
> -        nbrec_logical_router_policy_set_nexthop(policy, next_hop);
> -    }
> -    nbrec_logical_router_verify_policies(lr);
> -    struct nbrec_logical_router_policy **new_policies
> -        = xmalloc(sizeof *new_policies * (lr->n_policies + 1));
> -    memcpy(new_policies, lr->policies,
> -           sizeof *new_policies * lr->n_policies);
> -    new_policies[lr->n_policies] = policy;
> -    nbrec_logical_router_set_policies(lr, new_policies,
> -                                      lr->n_policies + 1);
> -    free(new_policies);
> -    if (next_hop != NULL) {
> -        free(next_hop);
> -    }
> -}
> -
> -static void
> -nbctl_lr_policy_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    int64_t priority = 0;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If a priority is not specified, delete all policies. */
> -        nbrec_logical_router_set_policies(lr, NULL, 0);
> -        return;
> -    }
> -
> -    error = parse_priority(ctx->argv[2], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    /* If match is not specified, delete all routing policies with the
> -     * specified priority. */
> -    if (ctx->argc == 3) {
> -        struct nbrec_logical_router_policy **new_policies
> -            = xmemdup(lr->policies,
> -                      sizeof *new_policies * lr->n_policies);
> -        int n_policies = 0;
> -        for (int i = 0; i < lr->n_policies; i++) {
> -            if (priority != lr->policies[i]->priority) {
> -                new_policies[n_policies++] = lr->policies[i];
> -            }
> -        }
> -        nbrec_logical_router_verify_policies(lr);
> -        nbrec_logical_router_set_policies(lr, new_policies, n_policies);
> -        free(new_policies);
> -        return;
> -    }
> -
> -    /* Delete policy that has the same priority and match string */
> -    for (int i = 0; i < lr->n_policies; i++) {
> -        struct nbrec_logical_router_policy *routing_policy =
> lr->policies[i];
> -        if (priority == routing_policy->priority &&
> -            !strcmp(ctx->argv[3], routing_policy->match)) {
> -            struct nbrec_logical_router_policy **new_policies
> -                = xmemdup(lr->policies,
> -                          sizeof *new_policies * lr->n_policies);
> -            new_policies[i] = lr->policies[lr->n_policies - 1];
> -            nbrec_logical_router_verify_policies(lr);
> -            nbrec_logical_router_set_policies(lr, new_policies,
> -                                              lr->n_policies - 1);
> -            free(new_policies);
> -            return;
> -        }
> -    }
> -}
> -
> - struct routing_policy {
> -    int priority;
> -    char *match;
> -    const struct nbrec_logical_router_policy *policy;
> -};
> -
> -static int
> -routing_policy_cmp(const void *policy1_, const void *policy2_)
> -{
> -    const struct routing_policy *policy1p = policy1_;
> -    const struct routing_policy *policy2p = policy2_;
> -    if (policy1p->priority != policy2p->priority) {
> -        return policy1p->priority > policy2p->priority ? -1 : 1;
> -    } else {
> -        return strcmp(policy1p->match, policy2p->match);
> -    }
> -}
> -
> -static void
> -print_routing_policy(const struct nbrec_logical_router_policy *policy,
> -                     struct ds *s)
> -{
> -    if (policy->nexthop != NULL) {
> -        char *next_hop = normalize_prefix_str(policy->nexthop);
> -        ds_put_format(s, "%10"PRId64" %50s %15s %25s", policy->priority,
> -                      policy->match, policy->action, next_hop);
> -        free(next_hop);
> -    } else {
> -        ds_put_format(s, "%10"PRId64" %50s %15s", policy->priority,
> -                      policy->match, policy->action);
> -    }
> -    ds_put_char(s, '\n');
> -}
> -
> -static void
> -nbctl_lr_policy_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    struct routing_policy *policies;
> -    size_t n_policies = 0;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    policies = xmalloc(sizeof *policies * lr->n_policies);
> -     for (int i = 0; i < lr->n_policies; i++) {
> -        const struct nbrec_logical_router_policy *policy
> -            = lr->policies[i];
> -        policies[n_policies].priority = policy->priority;
> -        policies[n_policies].match = policy->match;
> -        policies[n_policies].policy = policy;
> -        n_policies++;
> -    }
> -    qsort(policies, n_policies, sizeof *policies, routing_policy_cmp);
> -    if (n_policies) {
> -        ds_put_cstr(&ctx->output, "Routing Policies\n");
> -    }
> -    for (int i = 0; i < n_policies; i++) {
> -        print_routing_policy(policies[i].policy, &ctx->output);
> -    }
> -    free(policies);
> -}
> -
> -static void
> -nbctl_lr_route_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr = NULL;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    char *prefix, *next_hop;
> -
> -    const char *policy = shash_find_data(&ctx->options, "--policy");
> -    if (policy && strcmp(policy, "src-ip") && strcmp(policy, "dst-ip")) {
> -        ctl_error(ctx, "bad policy: %s", policy);
> -        return;
> -    }
> -
> -    prefix = normalize_prefix_str(ctx->argv[2]);
> -    if (!prefix) {
> -        ctl_error(ctx, "bad prefix argument: %s", ctx->argv[2]);
> -        return;
> -    }
> -
> -    next_hop = normalize_prefix_str(ctx->argv[3]);
> -    if (!next_hop) {
> -        free(prefix);
> -        ctl_error(ctx, "bad next hop argument: %s", ctx->argv[3]);
> -        return;
> -    }
> -
> -    if (strchr(prefix, '.')) {
> -        ovs_be32 hop_ipv4;
> -        if (!ip_parse(ctx->argv[3], &hop_ipv4)) {
> -            free(prefix);
> -            free(next_hop);
> -            ctl_error(ctx, "bad IPv4 nexthop argument: %s", ctx->argv[3]);
> -            return;
> -        }
> -    } else {
> -        struct in6_addr hop_ipv6;
> -        if (!ipv6_parse(ctx->argv[3], &hop_ipv6)) {
> -            free(prefix);
> -            free(next_hop);
> -            ctl_error(ctx, "bad IPv6 nexthop argument: %s", ctx->argv[3]);
> -            return;
> -        }
> -    }
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    for (int i = 0; i < lr->n_static_routes; i++) {
> -        const struct nbrec_logical_router_static_route *route
> -            = lr->static_routes[i];
> -        char *rt_prefix;
> -
> -        rt_prefix = normalize_prefix_str(lr->static_routes[i]->ip_prefix);
> -        if (!rt_prefix) {
> -            /* Ignore existing prefix we couldn't parse. */
> -            continue;
> -        }
> -
> -        if (strcmp(rt_prefix, prefix)) {
> -            free(rt_prefix);
> -            continue;
> -        }
> -
> -        if (!may_exist) {
> -            ctl_error(ctx, "duplicate prefix: %s", prefix);
> -            free(next_hop);
> -            free(rt_prefix);
> -            free(prefix);
> -            return;
> -        }
> -
> -        /* Update the next hop for an existing route. */
> -        nbrec_logical_router_verify_static_routes(lr);
> -        nbrec_logical_router_static_route_verify_ip_prefix(route);
> -        nbrec_logical_router_static_route_verify_nexthop(route);
> -        nbrec_logical_router_static_route_set_ip_prefix(route, prefix);
> -        nbrec_logical_router_static_route_set_nexthop(route, next_hop);
> -        if (ctx->argc == 5) {
> -            nbrec_logical_router_static_route_set_output_port(route,
> -
> ctx->argv[4]);
> -        }
> -        if (policy) {
> -             nbrec_logical_router_static_route_set_policy(route, policy);
> -        }
> -        free(rt_prefix);
> -        free(next_hop);
> -        free(prefix);
> -        return;
> -    }
> -
> -    struct nbrec_logical_router_static_route *route;
> -    route = nbrec_logical_router_static_route_insert(ctx->txn);
> -    nbrec_logical_router_static_route_set_ip_prefix(route, prefix);
> -    nbrec_logical_router_static_route_set_nexthop(route, next_hop);
> -    if (ctx->argc == 5) {
> -        nbrec_logical_router_static_route_set_output_port(route,
> ctx->argv[4]);
> -    }
> -    if (policy) {
> -        nbrec_logical_router_static_route_set_policy(route, policy);
> -    }
> -
> -    nbrec_logical_router_verify_static_routes(lr);
> -    struct nbrec_logical_router_static_route **new_routes
> -        = xmalloc(sizeof *new_routes * (lr->n_static_routes + 1));
> -    nullable_memcpy(new_routes, lr->static_routes,
> -               sizeof *new_routes * lr->n_static_routes);
> -    new_routes[lr->n_static_routes] = route;
> -    nbrec_logical_router_set_static_routes(lr, new_routes,
> -                                           lr->n_static_routes + 1);
> -    free(new_routes);
> -    free(next_hop);
> -    free(prefix);
> -}
> -
> -static void
> -nbctl_lr_route_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If a prefix is not specified, delete all routes. */
> -        nbrec_logical_router_set_static_routes(lr, NULL, 0);
> -        return;
> -    }
> -
> -    char *prefix = normalize_prefix_str(ctx->argv[2]);
> -    if (!prefix) {
> -        ctl_error(ctx, "bad prefix argument: %s", ctx->argv[2]);
> -        return;
> -    }
> -
> -    for (int i = 0; i < lr->n_static_routes; i++) {
> -        char *rt_prefix =
> normalize_prefix_str(lr->static_routes[i]->ip_prefix);
> -        if (!rt_prefix) {
> -            /* Ignore existing prefix we couldn't parse. */
> -            continue;
> -        }
> -
> -        if (!strcmp(prefix, rt_prefix)) {
> -            struct nbrec_logical_router_static_route **new_routes
> -                = xmemdup(lr->static_routes,
> -                          sizeof *new_routes * lr->n_static_routes);
> -
> -            new_routes[i] = lr->static_routes[lr->n_static_routes - 1];
> -            nbrec_logical_router_verify_static_routes(lr);
> -            nbrec_logical_router_set_static_routes(lr, new_routes,
> -                                                 lr->n_static_routes - 1);
> -            free(new_routes);
> -            free(rt_prefix);
> -            free(prefix);
> -            return;
> -        }
> -        free(rt_prefix);
> -    }
> -
> -    if (!shash_find(&ctx->options, "--if-exists")) {
> -        ctl_error(ctx, "no matching prefix: %s", prefix);
> -    }
> -    free(prefix);
> -}
> -
> -static void
> -nbctl_lr_nat_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr = NULL;
> -    const char *nat_type = ctx->argv[2];
> -    const char *external_ip = ctx->argv[3];
> -    const char *logical_ip = ctx->argv[4];
> -    char *new_logical_ip = NULL;
> -
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (strcmp(nat_type, "dnat") && strcmp(nat_type, "snat")
> -            && strcmp(nat_type, "dnat_and_snat")) {
> -        ctl_error(ctx, "%s: type must be one of \"dnat\", \"snat\" and "
> -                  "\"dnat_and_snat\".", nat_type);
> -        return;
> -    }
> -
> -    ovs_be32 ipv4 = 0;
> -    unsigned int plen;
> -    if (!ip_parse(external_ip, &ipv4)) {
> -        ctl_error(ctx, "%s: should be an IPv4 address.", external_ip);
> -        return;
> -    }
> -
> -    if (strcmp("snat", nat_type)) {
> -        if (!ip_parse(logical_ip, &ipv4)) {
> -            ctl_error(ctx, "%s: should be an IPv4 address.", logical_ip);
> -            return;
> -        }
> -        new_logical_ip = xstrdup(logical_ip);
> -    } else {
> -        error = ip_parse_cidr(logical_ip, &ipv4, &plen);
> -        if (error) {
> -            free(error);
> -            ctl_error(ctx, "%s: should be an IPv4 address or network.",
> -                      logical_ip);
> -            return;
> -        }
> -        new_logical_ip = normalize_ipv4_prefix(ipv4, plen);
> -    }
> -
> -    const char *logical_port;
> -    const char *external_mac;
> -    if (ctx->argc == 6) {
> -        ctl_error(ctx, "lr-nat-add with logical_port "
> -                  "must also specify external_mac.");
> -        free(new_logical_ip);
> -        return;
> -    } else if (ctx->argc == 7) {
> -        if (strcmp(nat_type, "dnat_and_snat")) {
> -            ctl_error(ctx, "logical_port and external_mac are only valid
> when "
> -                      "type is \"dnat_and_snat\".");
> -            free(new_logical_ip);
> -            return;
> -        }
> -
> -        logical_port = ctx->argv[5];
> -        const struct nbrec_logical_switch_port *lsp;
> -        error = lsp_by_name_or_uuid(ctx, logical_port, true, &lsp);
> -        if (error) {
> -            ctx->error = error;
> -            free(new_logical_ip);
> -            return;
> -        }
> -
> -        external_mac = ctx->argv[6];
> -        struct eth_addr ea;
> -        if (!eth_addr_from_string(external_mac, &ea)) {
> -            ctl_error(ctx, "invalid mac address %s.", external_mac);
> -            free(new_logical_ip);
> -            return;
> -        }
> -    } else {
> -        logical_port = NULL;
> -        external_mac = NULL;
> -    }
> -
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    int is_snat = !strcmp("snat", nat_type);
> -    for (size_t i = 0; i < lr->n_nat; i++) {
> -        const struct nbrec_nat *nat = lr->nat[i];
> -        if (!strcmp(nat_type, nat->type)) {
> -            if (!strcmp(is_snat ? new_logical_ip : external_ip,
> -                        is_snat ? nat->logical_ip : nat->external_ip)) {
> -                if (!strcmp(is_snat ? external_ip : new_logical_ip,
> -                            is_snat ? nat->external_ip :
> nat->logical_ip)) {
> -                        if (may_exist) {
> -                            nbrec_nat_verify_logical_port(nat);
> -                            nbrec_nat_verify_external_mac(nat);
> -                            nbrec_nat_set_logical_port(nat, logical_port);
> -                            nbrec_nat_set_external_mac(nat, external_mac);
> -                            free(new_logical_ip);
> -                            return;
> -                        }
> -                        ctl_error(ctx, "%s, %s: a NAT with this
> external_ip "
> -                                  "and logical_ip already exists",
> -                                  external_ip, new_logical_ip);
> -                        free(new_logical_ip);
> -                        return;
> -                } else {
> -                    ctl_error(ctx, "a NAT with this type (%s) and %s (%s)
> "
> -                              "already exists",
> -                              nat_type,
> -                              is_snat ? "logical_ip" : "external_ip",
> -                              is_snat ? new_logical_ip : external_ip);
> -                    free(new_logical_ip);
> -                    return;
> -                }
> -            }
> -        }
> -    }
> -
> -    /* Create the NAT. */
> -    struct nbrec_nat *nat = nbrec_nat_insert(ctx->txn);
> -    nbrec_nat_set_type(nat, nat_type);
> -    nbrec_nat_set_external_ip(nat, external_ip);
> -    nbrec_nat_set_logical_ip(nat, new_logical_ip);
> -    if (logical_port && external_mac) {
> -        nbrec_nat_set_logical_port(nat, logical_port);
> -        nbrec_nat_set_external_mac(nat, external_mac);
> -    }
> -    free(new_logical_ip);
> -
> -    /* Insert the NAT into the logical router. */
> -    nbrec_logical_router_verify_nat(lr);
> -    struct nbrec_nat **new_nats = xmalloc(sizeof *new_nats * (lr->n_nat +
> 1));
> -    nullable_memcpy(new_nats, lr->nat, sizeof *new_nats * lr->n_nat);
> -    new_nats[lr->n_nat] = nat;
> -    nbrec_logical_router_set_nat(lr, new_nats, lr->n_nat + 1);
> -    free(new_nats);
> -}
> -
> -static void
> -nbctl_lr_nat_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr = NULL;
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        /* If type, external_ip and logical_ip are not specified, delete
> -         * all NATs. */
> -        nbrec_logical_router_verify_nat(lr);
> -        nbrec_logical_router_set_nat(lr, NULL, 0);
> -        return;
> -    }
> -
> -    const char *nat_type = ctx->argv[2];
> -    if (strcmp(nat_type, "dnat") && strcmp(nat_type, "snat")
> -            && strcmp(nat_type, "dnat_and_snat")) {
> -        ctl_error(ctx, "%s: type must be one of \"dnat\", \"snat\" and "
> -                  "\"dnat_and_snat\".", nat_type);
> -        return;
> -    }
> -
> -    if (ctx->argc == 3) {
> -        /*Deletes all NATs with the specified type. */
> -        struct nbrec_nat **new_nats = xmalloc(sizeof *new_nats *
> lr->n_nat);
> -        int n_nat = 0;
> -        for (size_t i = 0; i < lr->n_nat; i++) {
> -            if (strcmp(nat_type, lr->nat[i]->type)) {
> -                new_nats[n_nat++] = lr->nat[i];
> -            }
> -        }
> -
> -        nbrec_logical_router_verify_nat(lr);
> -        nbrec_logical_router_set_nat(lr, new_nats, n_nat);
> -        free(new_nats);
> -        return;
> -    }
> -
> -    const char *nat_ip = ctx->argv[3];
> -    int is_snat = !strcmp("snat", nat_type);
> -    /* Remove the matching NAT. */
> -    for (size_t i = 0; i < lr->n_nat; i++) {
> -        struct nbrec_nat *nat = lr->nat[i];
> -        if (!strcmp(nat_type, nat->type) &&
> -             !strcmp(nat_ip, is_snat ? nat->logical_ip :
> nat->external_ip)) {
> -            struct nbrec_nat **new_nats
> -                = xmemdup(lr->nat, sizeof *new_nats * lr->n_nat);
> -            new_nats[i] = lr->nat[lr->n_nat - 1];
> -            nbrec_logical_router_verify_nat(lr);
> -            nbrec_logical_router_set_nat(lr, new_nats,
> -                                          lr->n_nat - 1);
> -            free(new_nats);
> -            return;
> -        }
> -    }
> -
> -    if (must_exist) {
> -        ctl_error(ctx, "no matching NAT with the type (%s) and %s (%s)",
> -                  nat_type, is_snat ? "logical_ip" : "external_ip",
> nat_ip);
> -        return;
> -    }
> -}
> -
> -static void
> -nbctl_lr_nat_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    struct smap lr_nats = SMAP_INITIALIZER(&lr_nats);
> -    for (size_t i = 0; i < lr->n_nat; i++) {
> -        const struct nbrec_nat *nat = lr->nat[i];
> -        char *key = xasprintf("%-17.13s%s", nat->type, nat->external_ip);
> -        if (nat->external_mac && nat->logical_port) {
> -            smap_add_format(&lr_nats, key, "%-22.18s%-21.17s%s",
> -                            nat->logical_ip, nat->external_mac,
> -                            nat->logical_port);
> -        } else {
> -            smap_add_format(&lr_nats, key, "%s", nat->logical_ip);
> -        }
> -        free(key);
> -    }
> -
> -    const struct smap_node **nodes = smap_sort(&lr_nats);
> -    if (nodes) {
> -        ds_put_format(&ctx->output,
> "%-17.13s%-19.15s%-22.18s%-21.17s%s\n",
> -                "TYPE", "EXTERNAL_IP", "LOGICAL_IP", "EXTERNAL_MAC",
> -                "LOGICAL_PORT");
> -        for (size_t i = 0; i < smap_count(&lr_nats); i++) {
> -            const struct smap_node *node = nodes[i];
> -            ds_put_format(&ctx->output, "%-36.32s%s\n",
> -                    node->key, node->value);
> -        }
> -        free(nodes);
> -    }
> -    smap_destroy(&lr_nats);
> -}
> -
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -lrp_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool
> must_exist,
> -                    const struct nbrec_logical_router_port **lrp_p)
> -{
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -    *lrp_p = NULL;
> -
> -    struct uuid lrp_uuid;
> -    bool is_uuid = uuid_from_string(&lrp_uuid, id);
> -    if (is_uuid) {
> -        lrp = nbrec_logical_router_port_get_for_uuid(ctx->idl, &lrp_uuid);
> -    }
> -
> -    if (!lrp) {
> -        NBREC_LOGICAL_ROUTER_PORT_FOR_EACH(lrp, ctx->idl) {
> -            if (!strcmp(lrp->name, id)) {
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!lrp && must_exist) {
> -        return xasprintf("%s: port %s not found",
> -                         id, is_uuid ? "UUID" : "name");
> -    }
> -
> -    *lrp_p = lrp;
> -    return NULL;
> -}
> -
> -/* Returns the logical router that contains 'lrp'. */
> -static char * OVS_WARN_UNUSED_RESULT
> -lrp_to_lr(const struct ovsdb_idl *idl,
> -          const struct nbrec_logical_router_port *lrp,
> -          const struct nbrec_logical_router **lr_p)
> -{
> -    const struct nbrec_logical_router *lr;
> -    *lr_p = NULL;
> -
> -    NBREC_LOGICAL_ROUTER_FOR_EACH (lr, idl) {
> -        for (size_t i = 0; i < lr->n_ports; i++) {
> -            if (lr->ports[i] == lrp) {
> -                *lr_p = lr;
> -                return NULL;
> -            }
> -        }
> -    }
> -
> -    /* Can't happen because of the database schema */
> -    return xasprintf("port %s is not part of any logical router",
> -                     lrp->name);
> -}
> -
> -static const char *
> -lr_get_name(const struct nbrec_logical_router *lr, char uuid_s[UUID_LEN +
> 1],
> -            size_t uuid_s_size)
> -{
> -    if (lr->name[0]) {
> -        return lr->name;
> -    }
> -    snprintf(uuid_s, uuid_s_size, UUID_FMT, UUID_ARGS(&lr->header_.uuid));
> -    return uuid_s;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -gc_by_name_or_uuid(struct ctl_context *ctx, const char *id, bool
> must_exist,
> -                   const struct nbrec_gateway_chassis **gc_p)
> -{
> -    const struct nbrec_gateway_chassis *gc = NULL;
> -    *gc_p = NULL;
> -
> -    struct uuid gc_uuid;
> -    bool is_uuid = uuid_from_string(&gc_uuid, id);
> -    if (is_uuid) {
> -        gc = nbrec_gateway_chassis_get_for_uuid(ctx->idl, &gc_uuid);
> -    }
> -
> -    if (!gc) {
> -        NBREC_GATEWAY_CHASSIS_FOR_EACH (gc, ctx->idl) {
> -            if (!strcmp(gc->name, id)) {
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!gc && must_exist) {
> -        return xasprintf("%s: gateway chassis %s not found", id,
> -                         is_uuid ? "UUID" : "name");
> -    }
> -
> -    *gc_p = gc;
> -    return NULL;
> -}
> -
> -static void
> -nbctl_lrp_set_gateway_chassis(struct ctl_context *ctx)
> -{
> -    char *gc_name;
> -    int64_t priority = 0;
> -    const char *lrp_name = ctx->argv[1];
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -    char *error = lrp_by_name_or_uuid(ctx, lrp_name, true, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lrp) {
> -        ctl_error(ctx, "router port %s is required", lrp_name);
> -        return;
> -    }
> -
> -    const char *chassis_name = ctx->argv[2];
> -    if (ctx->argv[3]) {
> -        error = parse_priority(ctx->argv[3], &priority);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -    }
> -
> -    gc_name = xasprintf("%s-%s", lrp_name, chassis_name);
> -    const struct nbrec_gateway_chassis *existing_gc;
> -    error = gc_by_name_or_uuid(ctx, gc_name, false, &existing_gc);
> -    if (error) {
> -        ctx->error = error;
> -        free(gc_name);
> -        return;
> -    }
> -    if (existing_gc) {
> -        nbrec_gateway_chassis_set_priority(existing_gc, priority);
> -        free(gc_name);
> -        return;
> -    }
> -
> -    /* Create the logical gateway chassis. */
> -    struct nbrec_gateway_chassis *gc
> -        = nbrec_gateway_chassis_insert(ctx->txn);
> -    nbrec_gateway_chassis_set_name(gc, gc_name);
> -    nbrec_gateway_chassis_set_chassis_name(gc, chassis_name);
> -    nbrec_gateway_chassis_set_priority(gc, priority);
> -
> -    /* Insert the logical gateway chassis into the logical router port. */
> -    nbrec_logical_router_port_verify_gateway_chassis(lrp);
> -    struct nbrec_gateway_chassis **new_gc = xmalloc(
> -        sizeof *new_gc * (lrp->n_gateway_chassis + 1));
> -    nullable_memcpy(new_gc, lrp->gateway_chassis,
> -                    sizeof *new_gc * lrp->n_gateway_chassis);
> -    new_gc[lrp->n_gateway_chassis] = gc;
> -    nbrec_logical_router_port_set_gateway_chassis(
> -        lrp, new_gc, lrp->n_gateway_chassis + 1);
> -    free(new_gc);
> -    free(gc_name);
> -}
> -
> -/* Removes logical router port 'lrp->gateway_chassis[idx]'. */
> -static void
> -remove_gc(const struct nbrec_logical_router_port *lrp, size_t idx)
> -{
> -    const struct nbrec_gateway_chassis *gc = lrp->gateway_chassis[idx];
> -
> -    if (lrp->n_gateway_chassis == 1) {
> -        nbrec_logical_router_port_set_gateway_chassis(lrp, NULL, 0);
> -    } else {
> -        /* First remove 'gc' from the array of gateway_chassis.  This is
> what
> -         * will actually cause the gateway chassis to be deleted when the
> -         * transaction is sent to the database server (due to garbage
> -         * collection). */
> -        struct nbrec_gateway_chassis **new_gc
> -            = xmemdup(lrp->gateway_chassis,
> -                      sizeof *new_gc * lrp->n_gateway_chassis);
> -        new_gc[idx] = new_gc[lrp->n_gateway_chassis - 1];
> -        nbrec_logical_router_port_verify_gateway_chassis(lrp);
> -        nbrec_logical_router_port_set_gateway_chassis(
> -            lrp, new_gc, lrp->n_gateway_chassis - 1);
> -        free(new_gc);
> -    }
> -
> -    /* Delete 'gc' from the IDL.  This won't have a real effect on
> -     * the database server (the IDL will suppress it in fact) but it
> -     * means that it won't show up when we iterate with
> -     * NBREC_GATEWAY_CHASSIS_FOR_EACH later. */
> -    nbrec_gateway_chassis_delete(gc);
> -}
> -
> -static void
> -nbctl_lrp_del_gateway_chassis(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -    char *error = lrp_by_name_or_uuid(ctx, ctx->argv[1], true, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lrp) {
> -        return;
> -    }
> -    /* Find the lrp that contains 'gc', then delete it. */
> -    const char *chassis_name = ctx->argv[2];
> -    for (size_t i = 0; i < lrp->n_gateway_chassis; i++) {
> -        if (!strncmp(lrp->gateway_chassis[i]->chassis_name,
> -                    chassis_name,
> -                    strlen(lrp->gateway_chassis[i]->chassis_name))) {
> -            remove_gc(lrp, i);
> -            return;
> -        }
> -    }
> -
> -    /* Can't happen because of the database schema. */
> -    ctl_error(ctx, "chassis %s is not added to logical port %s",
> -              chassis_name, ctx->argv[1]);
> -}
> -
> -/* Print a list of gateway chassis. */
> -static void
> -nbctl_lrp_get_gateway_chassis(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -    const struct nbrec_gateway_chassis **gcs;
> -    size_t i;
> -
> -    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    gcs = get_ordered_gw_chassis_prio_list(lrp);
> -
> -    for (i = 0; i < lrp->n_gateway_chassis; i++) {
> -        const struct nbrec_gateway_chassis *gc = gcs[i];
> -        ds_put_format(&ctx->output, "%s %5"PRId64"\n",
> -                      gc->name, gc->priority);
> -    }
> -
> -    free(gcs);
> -}
> -
> -static void
> -nbctl_lrp_add(struct ctl_context *ctx)
> -{
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -
> -    const struct nbrec_logical_router *lr = NULL;
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const char *lrp_name = ctx->argv[2];
> -    const char *mac = ctx->argv[3];
> -    const char **networks = (const char **) &ctx->argv[4];
> -
> -    int n_networks = ctx->argc - 4;
> -    for (int i = 4; i < ctx->argc; i++) {
> -        if (strchr(ctx->argv[i], '=')) {
> -            n_networks = i - 4;
> -            break;
> -        }
> -    }
> -
> -    if (!n_networks) {
> -        ctl_error(ctx, "%s: router port requires specifying a network",
> -                  lrp_name);
> -        return;
> -    }
> -
> -    char **settings = (char **) &ctx->argv[n_networks + 4];
> -    int n_settings = ctx->argc - 4 - n_networks;
> -
> -    const struct nbrec_logical_router_port *lrp;
> -    error = lrp_by_name_or_uuid(ctx, lrp_name, false, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (lrp) {
> -        if (!may_exist) {
> -            ctl_error(ctx, "%s: a port with this name already exists",
> -                      lrp_name);
> -            return;
> -        }
> -
> -        const struct nbrec_logical_router *bound_lr;
> -        error = lrp_to_lr(ctx->idl, lrp, &bound_lr);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -        if (bound_lr != lr) {
> -            char uuid_s[UUID_LEN + 1];
> -            ctl_error(ctx, "%s: port already exists but in router %s",
> -                      lrp_name, lr_get_name(bound_lr, uuid_s, sizeof
> uuid_s));
> -            return;
> -        }
> -
> -        if (strcmp(mac, lrp->mac)) {
> -            ctl_error(ctx, "%s: port already exists with mac %s",
> lrp_name,
> -                      lrp->mac);
> -            return;
> -        }
> -
> -        struct sset new_networks = SSET_INITIALIZER(&new_networks);
> -        for (int i = 0; i < n_networks; i++) {
> -            sset_add(&new_networks, networks[i]);
> -        }
> -
> -        struct sset orig_networks = SSET_INITIALIZER(&orig_networks);
> -        sset_add_array(&orig_networks, lrp->networks, lrp->n_networks);
> -
> -        bool same_networks = sset_equals(&orig_networks, &new_networks);
> -        sset_destroy(&orig_networks);
> -        sset_destroy(&new_networks);
> -        if (!same_networks) {
> -            ctl_error(ctx, "%s: port already exists with different
> network",
> -                      lrp_name);
> -            return;
> -        }
> -
> -        /* Special-case sanity-check of peer ports. */
> -        const char *peer = NULL;
> -        for (int i = 0; i < n_settings; i++) {
> -            if (!strncmp(settings[i], "peer=", 5)) {
> -                peer = settings[i] + 5;
> -                break;
> -            }
> -        }
> -
> -        if ((!peer != !lrp->peer) ||
> -                (lrp->peer && strcmp(peer, lrp->peer))) {
> -            ctl_error(ctx, "%s: port already exists with mismatching
> peer",
> -                      lrp_name);
> -            return;
> -        }
> -
> -        return;
> -    }
> -
> -    struct eth_addr ea;
> -    if (!eth_addr_from_string(mac, &ea)) {
> -        ctl_error(ctx, "%s: invalid mac address %s", lrp_name, mac);
> -        return;
> -    }
> -
> -    for (int i = 0; i < n_networks; i++) {
> -        ovs_be32 ipv4;
> -        unsigned int plen;
> -        error = ip_parse_cidr(networks[i], &ipv4, &plen);
> -        if (error) {
> -            free(error);
> -            struct in6_addr ipv6;
> -            error = ipv6_parse_cidr(networks[i], &ipv6, &plen);
> -            if (error) {
> -                free(error);
> -                ctl_error(ctx, "%s: invalid network address: %s",
> lrp_name,
> -                          networks[i]);
> -                return;
> -            }
> -        }
> -    }
> -
> -    /* Create the logical port. */
> -    lrp = nbrec_logical_router_port_insert(ctx->txn);
> -    nbrec_logical_router_port_set_name(lrp, lrp_name);
> -    nbrec_logical_router_port_set_mac(lrp, mac);
> -    nbrec_logical_router_port_set_networks(lrp, networks, n_networks);
> -
> -    for (int i = 0; i < n_settings; i++) {
> -        error = ctl_set_column("Logical_Router_Port", &lrp->header_,
> -                               settings[i], ctx->symtab);
> -        if (error) {
> -            ctx->error = error;
> -            return;
> -        }
> -    }
> -
> -    /* Insert the logical port into the logical router. */
> -    nbrec_logical_router_verify_ports(lr);
> -    struct nbrec_logical_router_port **new_ports = xmalloc(sizeof
> *new_ports *
> -                                                        (lr->n_ports +
> 1));
> -    nullable_memcpy(new_ports, lr->ports, sizeof *new_ports *
> lr->n_ports);
> -    new_ports[lr->n_ports] = CONST_CAST(struct nbrec_logical_router_port
> *,
> -                                             lrp);
> -    nbrec_logical_router_set_ports(lr, new_ports, lr->n_ports + 1);
> -    free(new_ports);
> -}
> -
> -/* Removes logical router port 'lr->ports[idx]'. */
> -static void
> -remove_lrp(const struct nbrec_logical_router *lr, size_t idx)
> -{
> -    const struct nbrec_logical_router_port *lrp = lr->ports[idx];
> -
> -    /* First remove 'lrp' from the array of ports.  This is what will
> -     * actually cause the logical port to be deleted when the transaction
> is
> -     * sent to the database server (due to garbage collection). */
> -    struct nbrec_logical_router_port **new_ports
> -        = xmemdup(lr->ports, sizeof *new_ports * lr->n_ports);
> -    new_ports[idx] = new_ports[lr->n_ports - 1];
> -    nbrec_logical_router_verify_ports(lr);
> -    nbrec_logical_router_set_ports(lr, new_ports, lr->n_ports - 1);
> -    free(new_ports);
> -
> -    /* Delete 'lrp' from the IDL.  This won't have a real effect on
> -     * the database server (the IDL will suppress it in fact) but it
> -     * means that it won't show up when we iterate with
> -     * NBREC_LOGICAL_ROUTER_PORT_FOR_EACH later. */
> -    nbrec_logical_router_port_delete(lrp);
> -}
> -
> -static void
> -nbctl_lrp_del(struct ctl_context *ctx)
> -{
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -
> -    char *error = lrp_by_name_or_uuid(ctx, ctx->argv[1], must_exist,
> &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lrp) {
> -        return;
> -    }
> -
> -    /* Find the router that contains 'lrp', then delete it. */
> -    const struct nbrec_logical_router *lr;
> -    NBREC_LOGICAL_ROUTER_FOR_EACH (lr, ctx->idl) {
> -        for (size_t i = 0; i < lr->n_ports; i++) {
> -            if (lr->ports[i] == lrp) {
> -                remove_lrp(lr, i);
> -                return;
> -            }
> -        }
> -    }
> -
> -    /* Can't happen because of the database schema. */
> -    ctl_error(ctx, "logical port %s is not part of any logical router",
> -              ctx->argv[1]);
> -}
> -
> -/* Print a list of logical router ports. */
> -static void
> -nbctl_lrp_list(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_router *lr;
> -    struct smap lrps;
> -    size_t i;
> -
> -    char *error = lr_by_name_or_uuid(ctx, id, true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    smap_init(&lrps);
> -    for (i = 0; i < lr->n_ports; i++) {
> -        const struct nbrec_logical_router_port *lrp = lr->ports[i];
> -        smap_add_format(&lrps, lrp->name, UUID_FMT " (%s)",
> -                        UUID_ARGS(&lrp->header_.uuid), lrp->name);
> -    }
> -    const struct smap_node **nodes = smap_sort(&lrps);
> -    for (i = 0; i < smap_count(&lrps); i++) {
> -        const struct smap_node *node = nodes[i];
> -        ds_put_format(&ctx->output, "%s\n", node->value);
> -    }
> -    smap_destroy(&lrps);
> -    free(nodes);
> -}
> -
> -/* Set the logical router port admin-enabled state. */
> -static void
> -nbctl_lrp_set_enabled(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const char *state = ctx->argv[2];
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -
> -    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lrp) {
> -        return;
> -    }
> -
> -    bool enabled;
> -    error = parse_enabled(state, &enabled);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    nbrec_logical_router_port_set_enabled(lrp, &enabled, 1);
> -}
> -
> -/* Print admin-enabled state for logical router port. */
> -static void
> -nbctl_lrp_get_enabled(struct ctl_context *ctx)
> -{
> -    const char *id = ctx->argv[1];
> -    const struct nbrec_logical_router_port *lrp = NULL;
> -
> -    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -    if (!lrp) {
> -        return;
> -    }
> -
> -    ds_put_format(&ctx->output, "%s\n",
> -                  !lrp->enabled ||
> -                  *lrp->enabled ? "enabled" : "disabled");
> -}
> -
> -struct ipv4_route {
> -    int priority;
> -    ovs_be32 addr;
> -    const struct nbrec_logical_router_static_route *route;
> -};
> -
> -static int
> -ipv4_route_cmp(const void *route1_, const void *route2_)
> -{
> -    const struct ipv4_route *route1p = route1_;
> -    const struct ipv4_route *route2p = route2_;
> -
> -    if (route1p->priority != route2p->priority) {
> -        return route1p->priority > route2p->priority ? -1 : 1;
> -    } else if (route1p->addr != route2p->addr) {
> -        return ntohl(route1p->addr) < ntohl(route2p->addr) ? -1 : 1;
> -    } else {
> -        return 0;
> -    }
> -}
> -
> -struct ipv6_route {
> -    int priority;
> -    struct in6_addr addr;
> -    const struct nbrec_logical_router_static_route *route;
> -};
> -
> -static int
> -ipv6_route_cmp(const void *route1_, const void *route2_)
> -{
> -    const struct ipv6_route *route1p = route1_;
> -    const struct ipv6_route *route2p = route2_;
> -
> -    if (route1p->priority != route2p->priority) {
> -        return route1p->priority > route2p->priority ? -1 : 1;
> -    }
> -    return memcmp(&route1p->addr, &route2p->addr, sizeof(route1p->addr));
> -}
> -
> -static void
> -print_route(const struct nbrec_logical_router_static_route *route, struct
> ds *s)
> -{
> -
> -    char *prefix = normalize_prefix_str(route->ip_prefix);
> -    char *next_hop = normalize_prefix_str(route->nexthop);
> -    ds_put_format(s, "%25s %25s", prefix, next_hop);
> -    free(prefix);
> -    free(next_hop);
> -
> -    if (route->policy) {
> -        ds_put_format(s, " %s", route->policy);
> -    } else {
> -        ds_put_format(s, " %s", "dst-ip");
> -    }
> -
> -    if (route->output_port) {
> -        ds_put_format(s, " %s", route->output_port);
> -    }
> -    ds_put_char(s, '\n');
> -}
> -
> -static void
> -nbctl_lr_route_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_logical_router *lr;
> -    struct ipv4_route *ipv4_routes;
> -    struct ipv6_route *ipv6_routes;
> -    size_t n_ipv4_routes = 0;
> -    size_t n_ipv6_routes = 0;
> -
> -    char *error = lr_by_name_or_uuid(ctx, ctx->argv[1], true, &lr);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    ipv4_routes = xmalloc(sizeof *ipv4_routes * lr->n_static_routes);
> -    ipv6_routes = xmalloc(sizeof *ipv6_routes * lr->n_static_routes);
> -
> -    for (int i = 0; i < lr->n_static_routes; i++) {
> -        const struct nbrec_logical_router_static_route *route
> -            = lr->static_routes[i];
> -        unsigned int plen;
> -        ovs_be32 ipv4;
> -        const char *policy = route->policy ? route->policy : "dst-ip";
> -        error = ip_parse_cidr(route->ip_prefix, &ipv4, &plen);
> -        if (!error) {
> -            ipv4_routes[n_ipv4_routes].priority = !strcmp(policy,
> "dst-ip")
> -                                                    ? (2 * plen) + 1
> -                                                    : 2 * plen;
> -            ipv4_routes[n_ipv4_routes].addr = ipv4;
> -            ipv4_routes[n_ipv4_routes].route = route;
> -            n_ipv4_routes++;
> -        } else {
> -            free(error);
> -
> -            struct in6_addr ipv6;
> -            error = ipv6_parse_cidr(route->ip_prefix, &ipv6, &plen);
> -            if (!error) {
> -                ipv6_routes[n_ipv6_routes].priority = !strcmp(policy,
> "dst-ip")
> -                                                        ? (2 * plen) + 1
> -                                                        : 2 * plen;
> -                ipv6_routes[n_ipv6_routes].addr = ipv6;
> -                ipv6_routes[n_ipv6_routes].route = route;
> -                n_ipv6_routes++;
> -            } else {
> -                /* Invalid prefix. */
> -                VLOG_WARN("router "UUID_FMT" (%s) has invalid prefix: %s",
> -                          UUID_ARGS(&lr->header_.uuid), lr->name,
> -                          route->ip_prefix);
> -                free(error);
> -                continue;
> -            }
> -        }
> -    }
> -
> -    qsort(ipv4_routes, n_ipv4_routes, sizeof *ipv4_routes,
> ipv4_route_cmp);
> -    qsort(ipv6_routes, n_ipv6_routes, sizeof *ipv6_routes,
> ipv6_route_cmp);
> -
> -    if (n_ipv4_routes) {
> -        ds_put_cstr(&ctx->output, "IPv4 Routes\n");
> -    }
> -    for (int i = 0; i < n_ipv4_routes; i++) {
> -        print_route(ipv4_routes[i].route, &ctx->output);
> -    }
> -
> -    if (n_ipv6_routes) {
> -        ds_put_format(&ctx->output, "%sIPv6 Routes\n",
> -                      n_ipv4_routes ?  "\n" : "");
> -    }
> -    for (int i = 0; i < n_ipv6_routes; i++) {
> -        print_route(ipv6_routes[i].route, &ctx->output);
> -    }
> -
> -    free(ipv4_routes);
> -    free(ipv6_routes);
> -}
> -
> -static void
> -verify_connections(struct ctl_context *ctx)
> -{
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    const struct nbrec_connection *conn;
> -
> -    nbrec_nb_global_verify_connections(nb_global);
> -
> -    NBREC_CONNECTION_FOR_EACH(conn, ctx->idl) {
> -        nbrec_connection_verify_target(conn);
> -    }
> -}
> -
> -static void
> -pre_connection(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_nb_global_col_connections);
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_connection_col_target);
> -    ovsdb_idl_add_column(ctx->idl,
> &nbrec_connection_col_inactivity_probe);
> -}
> -
> -static void
> -cmd_get_connection(struct ctl_context *ctx)
> -{
> -    const struct nbrec_connection *conn;
> -    struct svec targets;
> -    size_t i;
> -
> -    verify_connections(ctx);
> -
> -    /* Print the targets in sorted order for reproducibility. */
> -    svec_init(&targets);
> -
> -    NBREC_CONNECTION_FOR_EACH(conn, ctx->idl) {
> -        svec_add(&targets, conn->target);
> -    }
> -
> -    svec_sort_unique(&targets);
> -    for (i = 0; i < targets.n; i++) {
> -        ds_put_format(&ctx->output, "%s\n", targets.names[i]);
> -    }
> -    svec_destroy(&targets);
> -}
> -
> -static void
> -delete_connections(struct ctl_context *ctx)
> -{
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    const struct nbrec_connection *conn, *next;
> -
> -    /* Delete Manager rows pointed to by 'connection_options' column. */
> -    NBREC_CONNECTION_FOR_EACH_SAFE(conn, next, ctx->idl) {
> -        nbrec_connection_delete(conn);
> -    }
> -
> -    /* Delete 'Manager' row refs in 'manager_options' column. */
> -    nbrec_nb_global_set_connections(nb_global, NULL, 0);
> -}
> -
> -static void
> -cmd_del_connection(struct ctl_context *ctx)
> -{
> -    verify_connections(ctx);
> -    delete_connections(ctx);
> -}
> -
> -static void
> -insert_connections(struct ctl_context *ctx, char *targets[], size_t n)
> -{
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    struct nbrec_connection **connections;
> -    size_t i, conns=0;
> -    const char *inactivity_probe = shash_find_data(&ctx->options,
> -                                                   "--inactivity-probe");
> -
> -    /* Insert each connection in a new row in Connection table. */
> -    connections = xmalloc(n * sizeof *connections);
> -    for (i = 0; i < n; i++) {
> -        if (stream_verify_name(targets[i]) &&
> -                   pstream_verify_name(targets[i])) {
> -            VLOG_WARN("target type \"%s\" is possibly erroneous",
> targets[i]);
> -        }
> -
> -        connections[conns] = nbrec_connection_insert(ctx->txn);
> -        nbrec_connection_set_target(connections[conns], targets[i]);
> -        if (inactivity_probe) {
> -            int64_t msecs = atoll(inactivity_probe);
> -            nbrec_connection_set_inactivity_probe(connections[conns],
> -                                                  &msecs, 1);
> -        }
> -        conns++;
> -    }
> -
> -    /* Store uuids of new connection rows in 'connection' column. */
> -    nbrec_nb_global_set_connections(nb_global, connections, conns);
> -    free(connections);
> -}
> -
> -static void
> -cmd_set_connection(struct ctl_context *ctx)
> -{
> -    const size_t n = ctx->argc - 1;
> -
> -    verify_connections(ctx);
> -    delete_connections(ctx);
> -    insert_connections(ctx, &ctx->argv[1], n);
> -}
> -
> -static void
> -pre_cmd_get_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_nb_global_col_ssl);
> -
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_ssl_col_private_key);
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_ssl_col_certificate);
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_ssl_col_ca_cert);
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_ssl_col_bootstrap_ca_cert);
> -}
> -
> -static void
> -cmd_get_ssl(struct ctl_context *ctx)
> -{
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    const struct nbrec_ssl *ssl = nbrec_ssl_first(ctx->idl);
> -
> -    nbrec_nb_global_verify_ssl(nb_global);
> -    if (ssl) {
> -        nbrec_ssl_verify_private_key(ssl);
> -        nbrec_ssl_verify_certificate(ssl);
> -        nbrec_ssl_verify_ca_cert(ssl);
> -        nbrec_ssl_verify_bootstrap_ca_cert(ssl);
> -
> -        ds_put_format(&ctx->output, "Private key: %s\n",
> ssl->private_key);
> -        ds_put_format(&ctx->output, "Certificate: %s\n",
> ssl->certificate);
> -        ds_put_format(&ctx->output, "CA Certificate: %s\n", ssl->ca_cert);
> -        ds_put_format(&ctx->output, "Bootstrap: %s\n",
> -                ssl->bootstrap_ca_cert ? "true" : "false");
> -    }
> -}
> -
> -static void
> -pre_cmd_del_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_nb_global_col_ssl);
> -}
> -
> -static void
> -cmd_del_ssl(struct ctl_context *ctx)
> -{
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    const struct nbrec_ssl *ssl = nbrec_ssl_first(ctx->idl);
> -
> -    if (ssl) {
> -        nbrec_nb_global_verify_ssl(nb_global);
> -        nbrec_ssl_delete(ssl);
> -        nbrec_nb_global_set_ssl(nb_global, NULL);
> -    }
> -}
> -
> -static void
> -pre_cmd_set_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &nbrec_nb_global_col_ssl);
> -}
> -
> -static void
> -cmd_set_ssl(struct ctl_context *ctx)
> -{
> -    bool bootstrap = shash_find(&ctx->options, "--bootstrap");
> -    const struct nbrec_nb_global *nb_global =
> nbrec_nb_global_first(ctx->idl);
> -    const struct nbrec_ssl *ssl = nbrec_ssl_first(ctx->idl);
> -
> -    nbrec_nb_global_verify_ssl(nb_global);
> -    if (ssl) {
> -        nbrec_ssl_delete(ssl);
> -    }
> -    ssl = nbrec_ssl_insert(ctx->txn);
> -
> -    nbrec_ssl_set_private_key(ssl, ctx->argv[1]);
> -    nbrec_ssl_set_certificate(ssl, ctx->argv[2]);
> -    nbrec_ssl_set_ca_cert(ssl, ctx->argv[3]);
> -
> -    nbrec_ssl_set_bootstrap_ca_cert(ssl, bootstrap);
> -
> -    if (ctx->argc == 5) {
> -        nbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]);
> -    } else if (ctx->argc == 6) {
> -        nbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]);
> -        nbrec_ssl_set_ssl_ciphers(ssl, ctx->argv[5]);
> -    }
> -
> -    nbrec_nb_global_set_ssl(nb_global, ssl);
> -}
> -
> -static char *
> -set_ports_on_pg(struct ctl_context *ctx, const struct nbrec_port_group
> *pg,
> -                char **new_ports, size_t num_new_ports)
> -{
> -    struct nbrec_logical_switch_port **lports;
> -    lports = xmalloc(sizeof *lports * num_new_ports);
> -
> -    size_t i;
> -    char *error = NULL;
> -    for (i = 0; i < num_new_ports; i++) {
> -        const struct nbrec_logical_switch_port *lsp;
> -        error = lsp_by_name_or_uuid(ctx, new_ports[i], true, &lsp);
> -        if (error) {
> -            goto out;
> -        }
> -        lports[i] = (struct nbrec_logical_switch_port *) lsp;
> -    }
> -
> -    nbrec_port_group_set_ports(pg, lports, num_new_ports);
> -
> -out:
> -    free(lports);
> -    return error;
> -}
> -
> -static void
> -cmd_pg_add(struct ctl_context *ctx)
> -{
> -    const struct nbrec_port_group *pg;
> -
> -    pg = nbrec_port_group_insert(ctx->txn);
> -    nbrec_port_group_set_name(pg, ctx->argv[1]);
> -    if (ctx->argc > 2) {
> -        ctx->error = set_ports_on_pg(ctx, pg, ctx->argv + 2, ctx->argc -
> 2);
> -    }
> -}
> -
> -static void
> -cmd_pg_set_ports(struct ctl_context *ctx)
> -{
> -    const struct nbrec_port_group *pg;
> -
> -    char *error;
> -    error = pg_by_name_or_uuid(ctx, ctx->argv[1], true, &pg);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    ctx->error = set_ports_on_pg(ctx, pg, ctx->argv + 2, ctx->argc - 2);
> -}
> -
> -static void
> -cmd_pg_del(struct ctl_context *ctx)
> -{
> -    const struct nbrec_port_group *pg;
> -
> -    char *error;
> -    error = pg_by_name_or_uuid(ctx, ctx->argv[1], true, &pg);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    nbrec_port_group_delete(pg);
> -}
> -
> -static const struct nbrec_ha_chassis_group*
> -ha_chassis_group_by_name_or_uuid(struct ctl_context *ctx, const char *id,
> -                                 bool must_exist)
> -{
> -    struct uuid ch_grp_uuid;
> -    const struct nbrec_ha_chassis_group *ha_ch_grp = NULL;
> -    bool is_uuid = uuid_from_string(&ch_grp_uuid, id);
> -    if (is_uuid) {
> -        ha_ch_grp = nbrec_ha_chassis_group_get_for_uuid(ctx->idl,
> -                                                        &ch_grp_uuid);
> -    }
> -
> -    if (!ha_ch_grp) {
> -        const struct nbrec_ha_chassis_group *iter;
> -        NBREC_HA_CHASSIS_GROUP_FOR_EACH (iter, ctx->idl) {
> -            if (!strcmp(iter->name, id)) {
> -                ha_ch_grp = iter;
> -                break;
> -            }
> -        }
> -    }
> -
> -    if (!ha_ch_grp && must_exist) {
> -        ctx->error = xasprintf("%s: ha_chassi_group %s not found",
> -                               id, is_uuid ? "UUID" : "name");
> -    }
> -
> -    return ha_ch_grp;
> -}
> -
> -static void
> -cmd_ha_ch_grp_add(struct ctl_context *ctx)
> -{
> -    const char *name = ctx->argv[1];
> -    struct nbrec_ha_chassis_group *ha_ch_grp =
> -        nbrec_ha_chassis_group_insert(ctx->txn);
> -    nbrec_ha_chassis_group_set_name(ha_ch_grp, name);
> -}
> -
> -static void
> -cmd_ha_ch_grp_del(struct ctl_context *ctx)
> -{
> -    const char *name_or_id = ctx->argv[1];
> -
> -    const struct nbrec_ha_chassis_group *ha_ch_grp =
> -        ha_chassis_group_by_name_or_uuid(ctx, name_or_id, true);
> -
> -    if (ha_ch_grp) {
> -        nbrec_ha_chassis_group_delete(ha_ch_grp);
> -    }
> -}
> -
> -static void
> -cmd_ha_ch_grp_list(struct ctl_context *ctx)
> -{
> -    const struct nbrec_ha_chassis_group *ha_ch_grp;
> -
> -    NBREC_HA_CHASSIS_GROUP_FOR_EACH (ha_ch_grp, ctx->idl) {
> -        ds_put_format(&ctx->output, UUID_FMT " (%s)\n",
> -                      UUID_ARGS(&ha_ch_grp->header_.uuid),
> ha_ch_grp->name);
> -        const struct nbrec_ha_chassis *ha_ch;
> -        for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) {
> -            ha_ch = ha_ch_grp->ha_chassis[i];
> -            ds_put_format(&ctx->output,
> -                          "    "UUID_FMT " (%s)\n"
> -                          "    priority %"PRId64"\n\n",
> -                          UUID_ARGS(&ha_ch->header_.uuid),
> ha_ch->chassis_name,
> -                          ha_ch->priority);
> -        }
> -        ds_put_cstr(&ctx->output, "\n");
> -    }
> -}
> -
> -static void
> -cmd_ha_ch_grp_add_chassis(struct ctl_context *ctx)
> -{
> -    const struct nbrec_ha_chassis_group *ha_ch_grp =
> -        ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true);
> -
> -    if (!ha_ch_grp) {
> -        return;
> -    }
> -
> -    const char *chassis_name = ctx->argv[2];
> -    int64_t priority;
> -    char *error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    struct nbrec_ha_chassis *ha_chassis = NULL;
> -    for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) {
> -        if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name,
> chassis_name)) {
> -            ha_chassis = ha_ch_grp->ha_chassis[i];
> -            break;
> -        }
> -    }
> -
> -    if (ha_chassis) {
> -        nbrec_ha_chassis_set_priority(ha_chassis, priority);
> -        return;
> -    }
> -
> -    ha_chassis = nbrec_ha_chassis_insert(ctx->txn);
> -    nbrec_ha_chassis_set_chassis_name(ha_chassis, chassis_name);
> -    nbrec_ha_chassis_set_priority(ha_chassis, priority);
> -
> -    nbrec_ha_chassis_group_verify_ha_chassis(ha_ch_grp);
> -
> -    struct nbrec_ha_chassis **new_ha_chs =
> -        xmalloc(sizeof *new_ha_chs * (ha_ch_grp->n_ha_chassis + 1));
> -    nullable_memcpy(new_ha_chs, ha_ch_grp->ha_chassis,
> -                    sizeof *new_ha_chs * ha_ch_grp->n_ha_chassis);
> -    new_ha_chs[ha_ch_grp->n_ha_chassis] =
> -        CONST_CAST(struct nbrec_ha_chassis *, ha_chassis);
> -    nbrec_ha_chassis_group_set_ha_chassis(ha_ch_grp, new_ha_chs,
> -                                          ha_ch_grp->n_ha_chassis + 1);
> -    free(new_ha_chs);
> -}
> -
> -static void
> -cmd_ha_ch_grp_remove_chassis(struct ctl_context *ctx)
> -{
> -    const struct nbrec_ha_chassis_group *ha_ch_grp =
> -        ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true);
> -
> -    if (!ha_ch_grp) {
> -        return;
> -    }
> -
> -    const char *chassis_name = ctx->argv[2];
> -    struct nbrec_ha_chassis *ha_chassis = NULL;
> -    size_t idx = 0;
> -    for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) {
> -        if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name,
> chassis_name)) {
> -            ha_chassis = ha_ch_grp->ha_chassis[i];
> -            idx = i;
> -            break;
> -        }
> -    }
> -
> -    if (!ha_chassis) {
> -        ctx->error = xasprintf("%s: ha chassis not found in %s ha "
> -                               "chassis group", chassis_name,
> ctx->argv[1]);
> -        return;
> -    }
> -
> -    struct nbrec_ha_chassis **new_ha_ch
> -        = xmemdup(ha_ch_grp->ha_chassis,
> -                  sizeof *new_ha_ch * ha_ch_grp->n_ha_chassis);
> -    new_ha_ch[idx] = new_ha_ch[ha_ch_grp->n_ha_chassis - 1];
> -    nbrec_ha_chassis_group_verify_ha_chassis(ha_ch_grp);
> -    nbrec_ha_chassis_group_set_ha_chassis(ha_ch_grp, new_ha_ch,
> -                                          ha_ch_grp->n_ha_chassis - 1);
> -    free(new_ha_ch);
> -    nbrec_ha_chassis_delete(ha_chassis);
> -}
> -
> -static void
> -cmd_ha_ch_grp_set_chassis_prio(struct ctl_context *ctx)
> -{
> -    const struct nbrec_ha_chassis_group *ha_ch_grp =
> -        ha_chassis_group_by_name_or_uuid(ctx, ctx->argv[1], true);
> -
> -    if (!ha_ch_grp) {
> -        return;
> -    }
> -
> -    int64_t priority;
> -    char *error = parse_priority(ctx->argv[3], &priority);
> -    if (error) {
> -        ctx->error = error;
> -        return;
> -    }
> -
> -    const char *chassis_name = ctx->argv[2];
> -    struct nbrec_ha_chassis *ha_chassis = NULL;
> -
> -    for (size_t i = 0; i < ha_ch_grp->n_ha_chassis; i++) {
> -        if (!strcmp(ha_ch_grp->ha_chassis[i]->chassis_name,
> chassis_name)) {
> -            ha_chassis = ha_ch_grp->ha_chassis[i];
> -            break;
> -        }
> -    }
> -
> -    if (!ha_chassis) {
> -        ctx->error = xasprintf("%s: ha chassis not found in %s ha "
> -                               "chassis group", chassis_name,
> ctx->argv[1]);
> -        return;
> -    }
> -
> -    nbrec_ha_chassis_set_priority(ha_chassis, priority);
> -}
> -
> -static const struct ctl_table_class tables[NBREC_N_TABLES] = {
> -    [NBREC_TABLE_DHCP_OPTIONS].row_ids
> -    = {{&nbrec_logical_switch_port_col_name, NULL,
> -        &nbrec_logical_switch_port_col_dhcpv4_options},
> -       {&nbrec_logical_switch_port_col_external_ids,
> -        "neutron:port_name",
> &nbrec_logical_switch_port_col_dhcpv4_options},
> -       {&nbrec_logical_switch_port_col_name, NULL,
> -        &nbrec_logical_switch_port_col_dhcpv6_options},
> -       {&nbrec_logical_switch_port_col_external_ids,
> -        "neutron:port_name",
> &nbrec_logical_switch_port_col_dhcpv6_options}},
> -
> -    [NBREC_TABLE_LOGICAL_SWITCH].row_ids
> -    = {{&nbrec_logical_switch_col_name, NULL, NULL},
> -       {&nbrec_logical_switch_col_external_ids, "neutron:network_name",
> NULL}},
> -
> -    [NBREC_TABLE_LOGICAL_SWITCH_PORT].row_ids
> -    = {{&nbrec_logical_switch_port_col_name, NULL, NULL},
> -       {&nbrec_logical_switch_port_col_external_ids,
> -        "neutron:port_name", NULL}},
> -
> -    [NBREC_TABLE_LOGICAL_ROUTER].row_ids
> -    = {{&nbrec_logical_router_col_name, NULL, NULL},
> -       {&nbrec_logical_router_col_external_ids, "neutron:router_name",
> NULL}},
> -
> -    [NBREC_TABLE_LOGICAL_ROUTER_PORT].row_ids[0]
> -    = {&nbrec_logical_router_port_col_name, NULL, NULL},
> -
> -    [NBREC_TABLE_ADDRESS_SET].row_ids[0]
> -    = {&nbrec_address_set_col_name, NULL, NULL},
> -
> -    [NBREC_TABLE_PORT_GROUP].row_ids[0]
> -    = {&nbrec_port_group_col_name, NULL, NULL},
> -
> -    [NBREC_TABLE_ACL].row_ids[0] = {&nbrec_acl_col_name, NULL, NULL},
> -
> -    [NBREC_TABLE_HA_CHASSIS_GROUP].row_ids[0]
> -    = {&nbrec_ha_chassis_group_col_name, NULL, NULL},
> -};
> -
> -static char *
> -run_prerequisites(struct ctl_command *commands, size_t n_commands,
> -                  struct ovsdb_idl *idl)
> -{
> -    ovsdb_idl_add_table(idl, &nbrec_table_nb_global);
> -    if (wait_type == NBCTL_WAIT_SB) {
> -        ovsdb_idl_add_column(idl, &nbrec_nb_global_col_sb_cfg);
> -    } else if (wait_type == NBCTL_WAIT_HV) {
> -        ovsdb_idl_add_column(idl, &nbrec_nb_global_col_hv_cfg);
> -    }
> -
> -    for (struct ctl_command *c = commands; c < &commands[n_commands];
> c++) {
> -        if (c->syntax->prerequisites) {
> -            struct ctl_context ctx;
> -
> -            ds_init(&c->output);
> -            c->table = NULL;
> -
> -            ctl_context_init(&ctx, c, idl, NULL, NULL, NULL);
> -            (c->syntax->prerequisites)(&ctx);
> -            if (ctx.error) {
> -                char *error = xstrdup(ctx.error);
> -                ctl_context_done(&ctx, c);
> -                return error;
> -            }
> -            ctl_context_done(&ctx, c);
> -
> -            ovs_assert(!c->output.string);
> -            ovs_assert(!c->table);
> -        }
> -    }
> -
> -    return NULL;
> -}
> -
> -static void
> -oneline_format(struct ds *lines, struct ds *s)
> -{
> -    size_t j;
> -
> -    ds_chomp(lines, '\n');
> -    for (j = 0; j < lines->length; j++) {
> -        int ch = lines->string[j];
> -        switch (ch) {
> -        case '\n':
> -            ds_put_cstr(s, "\\n");
> -            break;
> -
> -        case '\\':
> -            ds_put_cstr(s, "\\\\");
> -            break;
> -
> -        default:
> -            ds_put_char(s, ch);
> -        }
> -    }
> -    ds_put_char(s, '\n');
> -}
> -
> -static void
> -oneline_print(struct ds *lines)
> -{
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    oneline_format(lines, &s);
> -    fputs(ds_cstr(&s), stdout);
> -    ds_destroy(&s);
> -}
> -
> -static char *
> -do_nbctl(const char *args, struct ctl_command *commands, size_t
> n_commands,
> -         struct ovsdb_idl *idl, const struct timer *wait_timeout, bool
> *retry)
> -{
> -    struct ovsdb_idl_txn *txn;
> -    enum ovsdb_idl_txn_status status;
> -    struct ovsdb_symbol_table *symtab;
> -    struct ctl_context ctx;
> -    struct ctl_command *c;
> -    struct shash_node *node;
> -    int64_t next_cfg = 0;
> -    char *error = NULL;
> -
> -    ovs_assert(retry);
> -
> -    txn = the_idl_txn = ovsdb_idl_txn_create(idl);
> -    if (dry_run) {
> -        ovsdb_idl_txn_set_dry_run(txn);
> -    }
> -
> -    ovsdb_idl_txn_add_comment(txn, "ovs-nbctl: %s", args);
> -
> -    const struct nbrec_nb_global *nb = nbrec_nb_global_first(idl);
> -    if (!nb) {
> -        /* XXX add verification that table is empty */
> -        nb = nbrec_nb_global_insert(txn);
> -    }
> -
> -    if (wait_type != NBCTL_WAIT_NONE) {
> -        ovsdb_idl_txn_increment(txn, &nb->header_,
> &nbrec_nb_global_col_nb_cfg,
> -                                force_wait);
> -    }
> -
> -    symtab = ovsdb_symbol_table_create();
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        ds_init(&c->output);
> -        c->table = NULL;
> -    }
> -    ctl_context_init(&ctx, NULL, idl, txn, symtab, NULL);
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        ctl_context_init_command(&ctx, c);
> -        if (c->syntax->run) {
> -            (c->syntax->run)(&ctx);
> -        }
> -        if (ctx.error) {
> -            error = xstrdup(ctx.error);
> -            ctl_context_done(&ctx, c);
> -            goto out_error;
> -        }
> -        ctl_context_done_command(&ctx, c);
> -
> -        if (ctx.try_again) {
> -            ctl_context_done(&ctx, NULL);
> -            goto try_again;
> -        }
> -    }
> -    ctl_context_done(&ctx, NULL);
> -
> -    SHASH_FOR_EACH (node, &symtab->sh) {
> -        struct ovsdb_symbol *symbol = node->data;
> -        if (!symbol->created) {
> -            error = xasprintf("row id \"%s\" is referenced but never
> created "
> -                              "(e.g. with \"-- --id=%s create ...\")",
> -                              node->name, node->name);
> -            goto out_error;
> -        }
> -        if (!symbol->strong_ref) {
> -            if (!symbol->weak_ref) {
> -                VLOG_WARN("row id \"%s\" was created but no reference to
> it "
> -                          "was inserted, so it will not actually appear
> in "
> -                          "the database", node->name);
> -            } else {
> -                VLOG_WARN("row id \"%s\" was created but only a weak "
> -                          "reference to it was inserted, so it will not "
> -                          "actually appear in the database", node->name);
> -            }
> -        }
> -    }
> -
> -    status = ovsdb_idl_txn_commit_block(txn);
> -    if (wait_type != NBCTL_WAIT_NONE && status == TXN_SUCCESS) {
> -        next_cfg = ovsdb_idl_txn_get_increment_new_value(txn);
> -    }
> -    if (status == TXN_UNCHANGED || status == TXN_SUCCESS) {
> -        for (c = commands; c < &commands[n_commands]; c++) {
> -            if (c->syntax->postprocess) {
> -                ctl_context_init(&ctx, c, idl, txn, symtab, NULL);
> -                (c->syntax->postprocess)(&ctx);
> -                if (ctx.error) {
> -                    error = xstrdup(ctx.error);
> -                    ctl_context_done(&ctx, c);
> -                    goto out_error;
> -                }
> -                ctl_context_done(&ctx, c);
> -            }
> -        }
> -    }
> -
> -    switch (status) {
> -    case TXN_UNCOMMITTED:
> -    case TXN_INCOMPLETE:
> -        OVS_NOT_REACHED();
> -
> -    case TXN_ABORTED:
> -        /* Should not happen--we never call ovsdb_idl_txn_abort(). */
> -        error = xstrdup("transaction aborted");
> -        goto out_error;
> -
> -    case TXN_UNCHANGED:
> -    case TXN_SUCCESS:
> -        break;
> -
> -    case TXN_TRY_AGAIN:
> -        goto try_again;
> -
> -    case TXN_ERROR:
> -        error = xasprintf("transaction error: %s",
> -                          ovsdb_idl_txn_get_error(txn));
> -        goto out_error;
> -
> -    case TXN_NOT_LOCKED:
> -        /* Should not happen--we never call ovsdb_idl_set_lock(). */
> -        error = xstrdup("database not locked");
> -        goto out_error;
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        struct ds *ds = &c->output;
> -
> -        if (c->table) {
> -            table_print(c->table, &table_style);
> -        } else if (oneline) {
> -            oneline_print(ds);
> -        } else {
> -            fputs(ds_cstr(ds), stdout);
> -        }
> -    }
> -
> -    if (wait_type != NBCTL_WAIT_NONE && status != TXN_UNCHANGED) {
> -        ovsdb_idl_enable_reconnect(idl);
> -        for (;;) {
> -            ovsdb_idl_run(idl);
> -            NBREC_NB_GLOBAL_FOR_EACH (nb, idl) {
> -                int64_t cur_cfg = (wait_type == NBCTL_WAIT_SB
> -                                   ? nb->sb_cfg
> -                                   : nb->hv_cfg);
> -                if (cur_cfg >= next_cfg) {
> -                    goto done;
> -                }
> -            }
> -            ovsdb_idl_wait(idl);
> -            if (wait_timeout) {
> -                timer_wait(wait_timeout);
> -            }
> -            poll_block();
> -            if (wait_timeout && timer_expired(wait_timeout)) {
> -                error = xstrdup("timeout expired");
> -                goto out_error;
> -            }
> -        }
> -    done: ;
> -    }
> -
> -    ovsdb_symbol_table_destroy(symtab);
> -    ovsdb_idl_txn_destroy(txn);
> -    the_idl_txn = NULL;
> -
> -    *retry = false;
> -    return NULL;
> -
> -try_again:
> -    /* Our transaction needs to be rerun, or a prerequisite was not met.
> Free
> -     * resources and return so that the caller can try again. */
> -    *retry = true;
> -
> -out_error:
> -    ovsdb_idl_txn_abort(txn);
> -    ovsdb_idl_txn_destroy(txn);
> -    the_idl_txn = NULL;
> -
> -    ovsdb_symbol_table_destroy(symtab);
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        ds_destroy(&c->output);
> -        table_destroy(c->table);
> -        free(c->table);
> -    }
> -
> -    return error;
> -}
> -
> -/* Frees the current transaction and the underlying IDL and then calls
> - * exit(status).
> - *
> - * Freeing the transaction and the IDL is not strictly necessary, but it
> makes
> - * for a clean memory leak report from valgrind in the normal case.  That
> makes
> - * it easier to notice real memory leaks. */
> -static void
> -nbctl_exit(int status)
> -{
> -    if (the_idl_txn) {
> -        ovsdb_idl_txn_abort(the_idl_txn);
> -        ovsdb_idl_txn_destroy(the_idl_txn);
> -    }
> -    ovsdb_idl_destroy(the_idl);
> -    exit(status);
> -}
> -
> -static const struct ctl_command_syntax nbctl_commands[] = {
> -    { "init", 0, 0, "", NULL, nbctl_init, NULL, "", RW },
> -    { "sync", 0, 0, "", nbctl_pre_sync, nbctl_sync, NULL, "", RO },
> -    { "show", 0, 1, "[SWITCH]", NULL, nbctl_show, NULL, "", RO },
> -
> -    /* logical switch commands. */
> -    { "ls-add", 0, 1, "[SWITCH]", NULL, nbctl_ls_add, NULL,
> -      "--may-exist,--add-duplicate", RW },
> -    { "ls-del", 1, 1, "SWITCH", NULL, nbctl_ls_del, NULL, "--if-exists",
> RW },
> -    { "ls-list", 0, 0, "", NULL, nbctl_ls_list, NULL, "", RO },
> -
> -    /* acl commands. */
> -    { "acl-add", 5, 6, "{SWITCH | PORTGROUP} DIRECTION PRIORITY MATCH
> ACTION",
> -      NULL, nbctl_acl_add, NULL,
> -      "--log,--may-exist,--type=,--name=,--severity=,--meter=", RW },
> -    { "acl-del", 1, 4, "{SWITCH | PORTGROUP} [DIRECTION [PRIORITY
> MATCH]]",
> -      NULL, nbctl_acl_del, NULL, "--type=", RW },
> -    { "acl-list", 1, 1, "{SWITCH | PORTGROUP}",
> -      NULL, nbctl_acl_list, NULL, "--type=", RO },
> -
> -    /* qos commands. */
> -    { "qos-add", 5, 7,
> -      "SWITCH DIRECTION PRIORITY MATCH [rate=RATE [burst=BURST]]
> [dscp=DSCP]",
> -      NULL, nbctl_qos_add, NULL, "--may-exist", RW },
> -    { "qos-del", 1, 4, "SWITCH [DIRECTION [PRIORITY MATCH]]", NULL,
> -      nbctl_qos_del, NULL, "", RW },
> -    { "qos-list", 1, 1, "SWITCH", NULL, nbctl_qos_list, NULL, "", RO },
> -
> -    /* meter commands. */
> -    { "meter-add", 4, 5, "NAME ACTION RATE UNIT [BURST]", NULL,
> -      nbctl_meter_add, NULL, "", RW },
> -    { "meter-del", 0, 1, "[NAME]", NULL, nbctl_meter_del, NULL, "", RW },
> -    { "meter-list", 0, 0, "", NULL, nbctl_meter_list, NULL, "", RO },
> -
> -    /* logical switch port commands. */
> -    { "lsp-add", 2, 4, "SWITCH PORT [PARENT] [TAG]", NULL, nbctl_lsp_add,
> -      NULL, "--may-exist", RW },
> -    { "lsp-del", 1, 1, "PORT", NULL, nbctl_lsp_del, NULL, "--if-exists",
> RW },
> -    { "lsp-list", 1, 1, "SWITCH", NULL, nbctl_lsp_list, NULL, "", RO },
> -    { "lsp-get-parent", 1, 1, "PORT", NULL, nbctl_lsp_get_parent, NULL,
> -      "", RO },
> -    { "lsp-get-tag", 1, 1, "PORT", NULL, nbctl_lsp_get_tag, NULL, "", RO
> },
> -    { "lsp-set-addresses", 1, INT_MAX, "PORT [ADDRESS]...", NULL,
> -      nbctl_lsp_set_addresses, NULL, "", RW },
> -    { "lsp-get-addresses", 1, 1, "PORT", NULL, nbctl_lsp_get_addresses,
> NULL,
> -      "", RO },
> -    { "lsp-set-port-security", 0, INT_MAX, "PORT [ADDRS]...", NULL,
> -      nbctl_lsp_set_port_security, NULL, "", RW },
> -    { "lsp-get-port-security", 1, 1, "PORT", NULL,
> -      nbctl_lsp_get_port_security, NULL, "", RO },
> -    { "lsp-get-up", 1, 1, "PORT", NULL, nbctl_lsp_get_up, NULL, "", RO },
> -    { "lsp-set-enabled", 2, 2, "PORT STATE", NULL, nbctl_lsp_set_enabled,
> -      NULL, "", RW },
> -    { "lsp-get-enabled", 1, 1, "PORT", NULL, nbctl_lsp_get_enabled, NULL,
> -      "", RO },
> -    { "lsp-set-type", 2, 2, "PORT TYPE", NULL, nbctl_lsp_set_type, NULL,
> -      "", RW },
> -    { "lsp-get-type", 1, 1, "PORT", NULL, nbctl_lsp_get_type, NULL, "",
> RO },
> -    { "lsp-set-options", 1, INT_MAX, "PORT KEY=VALUE [KEY=VALUE]...",
> NULL,
> -      nbctl_lsp_set_options, NULL, "", RW },
> -    { "lsp-get-options", 1, 1, "PORT", NULL, nbctl_lsp_get_options, NULL,
> -      "", RO },
> -    { "lsp-set-dhcpv4-options", 1, 2, "PORT [DHCP_OPT_UUID]", NULL,
> -      nbctl_lsp_set_dhcpv4_options, NULL, "", RW },
> -    { "lsp-get-dhcpv4-options", 1, 1, "PORT", NULL,
> -      nbctl_lsp_get_dhcpv4_options, NULL, "", RO },
> -    { "lsp-set-dhcpv6-options", 1, 2, "PORT [DHCP_OPT_UUID]", NULL,
> -      nbctl_lsp_set_dhcpv6_options, NULL, "", RW },
> -    { "lsp-get-dhcpv6-options", 1, 1, "PORT", NULL,
> -      nbctl_lsp_get_dhcpv6_options, NULL, "", RO },
> -    { "lsp-get-ls", 1, 1, "PORT", NULL, nbctl_lsp_get_ls, NULL, "", RO },
> -
> -    /* logical router commands. */
> -    { "lr-add", 0, 1, "[ROUTER]", NULL, nbctl_lr_add, NULL,
> -      "--may-exist,--add-duplicate", RW },
> -    { "lr-del", 1, 1, "ROUTER", NULL, nbctl_lr_del, NULL, "--if-exists",
> RW },
> -    { "lr-list", 0, 0, "", NULL, nbctl_lr_list, NULL, "", RO },
> -
> -    /* logical router port commands. */
> -    { "lrp-add", 4, INT_MAX,
> -      "ROUTER PORT MAC NETWORK... [COLUMN[:KEY]=VALUE]...",
> -      NULL, nbctl_lrp_add, NULL, "--may-exist", RW },
> -    { "lrp-set-gateway-chassis", 2, 3,
> -      "PORT CHASSIS [PRIORITY]",
> -      NULL, nbctl_lrp_set_gateway_chassis, NULL, "--may-exist", RW },
> -    { "lrp-del-gateway-chassis", 2, 2, "PORT CHASSIS", NULL,
> -      nbctl_lrp_del_gateway_chassis, NULL, "", RW },
> -    { "lrp-get-gateway-chassis", 1, 1, "PORT", NULL,
> -      nbctl_lrp_get_gateway_chassis, NULL, "", RO },
> -    { "lrp-del", 1, 1, "PORT", NULL, nbctl_lrp_del, NULL, "--if-exists",
> RW },
> -    { "lrp-list", 1, 1, "ROUTER", NULL, nbctl_lrp_list, NULL, "", RO },
> -    { "lrp-set-enabled", 2, 2, "PORT STATE", NULL, nbctl_lrp_set_enabled,
> -      NULL, "", RW },
> -    { "lrp-get-enabled", 1, 1, "PORT", NULL, nbctl_lrp_get_enabled,
> -      NULL, "", RO },
> -
> -    /* logical router route commands. */
> -    { "lr-route-add", 3, 4, "ROUTER PREFIX NEXTHOP [PORT]", NULL,
> -      nbctl_lr_route_add, NULL, "--may-exist,--policy=", RW },
> -    { "lr-route-del", 1, 2, "ROUTER [PREFIX]", NULL, nbctl_lr_route_del,
> -      NULL, "--if-exists", RW },
> -    { "lr-route-list", 1, 1, "ROUTER", NULL, nbctl_lr_route_list, NULL,
> -      "", RO },
> -
> -    /* Policy commands */
> -    { "lr-policy-add", 4, 5, "ROUTER PRIORITY MATCH ACTION [NEXTHOP]",
> NULL,
> -        nbctl_lr_policy_add, NULL, "", RW },
> -    { "lr-policy-del", 1, 3, "ROUTER [PRIORITY [MATCH]]", NULL,
> -        nbctl_lr_policy_del, NULL, "", RW },
> -    { "lr-policy-list", 1, 1, "ROUTER", NULL, nbctl_lr_policy_list, NULL,
> -       "", RO },
> -
> -    /* NAT commands. */
> -    { "lr-nat-add", 4, 6,
> -      "ROUTER TYPE EXTERNAL_IP LOGICAL_IP [LOGICAL_PORT EXTERNAL_MAC]",
> NULL,
> -      nbctl_lr_nat_add, NULL, "--may-exist", RW },
> -    { "lr-nat-del", 1, 3, "ROUTER [TYPE [IP]]", NULL,
> -        nbctl_lr_nat_del, NULL, "--if-exists", RW },
> -    { "lr-nat-list", 1, 1, "ROUTER", NULL, nbctl_lr_nat_list, NULL, "",
> RO },
> -
> -    /* load balancer commands. */
> -    { "lb-add", 3, 4, "LB VIP[:PORT] IP[:PORT]... [PROTOCOL]", NULL,
> -      nbctl_lb_add, NULL, "--may-exist,--add-duplicate", RW },
> -    { "lb-del", 1, 2, "LB [VIP]", NULL, nbctl_lb_del, NULL,
> -        "--if-exists", RW },
> -    { "lb-list", 0, 1, "[LB]", NULL, nbctl_lb_list, NULL, "", RO },
> -    { "lr-lb-add", 2, 2, "ROUTER LB", NULL, nbctl_lr_lb_add, NULL,
> -        "--may-exist", RW },
> -    { "lr-lb-del", 1, 2, "ROUTER [LB]", NULL, nbctl_lr_lb_del, NULL,
> -        "--if-exists", RW },
> -    { "lr-lb-list", 1, 1, "ROUTER", NULL, nbctl_lr_lb_list, NULL,
> -        "", RO },
> -    { "ls-lb-add", 2, 2, "SWITCH LB", NULL, nbctl_ls_lb_add, NULL,
> -        "--may-exist", RW },
> -    { "ls-lb-del", 1, 2, "SWITCH [LB]", NULL, nbctl_ls_lb_del, NULL,
> -        "--if-exists", RW },
> -    { "ls-lb-list", 1, 1, "SWITCH", NULL, nbctl_ls_lb_list, NULL,
> -        "", RO },
> -
> -    /* DHCP_Options commands */
> -    {"dhcp-options-create", 1, INT_MAX, "CIDR [EXTERNAL:IDS]", NULL,
> -     nbctl_dhcp_options_create, NULL, "", RW },
> -    {"dhcp-options-del", 1, 1, "DHCP_OPT_UUID", NULL,
> -     nbctl_dhcp_options_del, NULL, "", RW},
> -    {"dhcp-options-list", 0, 0, "", NULL, nbctl_dhcp_options_list, NULL,
> "", RO},
> -    {"dhcp-options-set-options", 1, INT_MAX, "DHCP_OPT_UUID KEY=VALUE
> [KEY=VALUE]...",
> -    NULL, nbctl_dhcp_options_set_options, NULL, "", RW },
> -    {"dhcp-options-get-options", 1, 1, "DHCP_OPT_UUID", NULL,
> -     nbctl_dhcp_options_get_options, NULL, "", RO },
> -
> -    /* Connection commands. */
> -    {"get-connection", 0, 0, "", pre_connection, cmd_get_connection,
> NULL, "", RO},
> -    {"del-connection", 0, 0, "", pre_connection, cmd_del_connection,
> NULL, "", RW},
> -    {"set-connection", 1, INT_MAX, "TARGET...", pre_connection,
> cmd_set_connection,
> -     NULL, "--inactivity-probe=", RW},
> -
> -    /* SSL commands. */
> -    {"get-ssl", 0, 0, "", pre_cmd_get_ssl, cmd_get_ssl, NULL, "", RO},
> -    {"del-ssl", 0, 0, "", pre_cmd_del_ssl, cmd_del_ssl, NULL, "", RW},
> -    {"set-ssl", 3, 5,
> -        "PRIVATE-KEY CERTIFICATE CA-CERT [SSL-PROTOS [SSL-CIPHERS]]",
> -        pre_cmd_set_ssl, cmd_set_ssl, NULL, "--bootstrap", RW},
> -
> -    /* Port Group Commands */
> -    {"pg-add", 1, INT_MAX, "", NULL, cmd_pg_add, NULL, "", RW },
> -    {"pg-set-ports", 2, INT_MAX, "", NULL, cmd_pg_set_ports, NULL, "", RW
> },
> -    {"pg-del", 1, 1, "", NULL, cmd_pg_del, NULL, "", RW },
> -
> -    /* HA chassis group commands. */
> -    {"ha-chassis-group-add", 1, 1, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_add, NULL, "", RW },
> -    {"ha-chassis-group-del", 1, 1, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_del, NULL, "", RW },
> -    {"ha-chassis-group-list", 0, 0, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_list, NULL, "", RO },
> -    {"ha-chassis-group-add-chassis", 3, 3, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_add_chassis, NULL, "", RW },
> -    {"ha-chassis-group-remove-chassis", 2, 2, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_remove_chassis, NULL, "", RW },
> -    {"ha-chassis-group-set-chassis-prio", 3, 3, "[CHASSIS GROUP]", NULL,
> -      cmd_ha_ch_grp_set_chassis_prio, NULL, "", RW },
> -
> -    {NULL, 0, 0, NULL, NULL, NULL, NULL, "", RO},
> -};
> -
> -/* Registers nbctl and common db commands. */
> -static void
> -nbctl_cmd_init(void)
> -{
> -    ctl_init(&nbrec_idl_class, nbrec_table_classes, tables, NULL,
> nbctl_exit);
> -    ctl_register_commands(nbctl_commands);
> -}
> -
> -/* Server implementation. */
> -
> -#undef ctl_fatal
> -
> -static const struct option *
> -find_option_by_value(const struct option *options, int value)
> -{
> -    const struct option *o;
> -
> -    for (o = options; o->name; o++) {
> -        if (o->val == value) {
> -            return o;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static char * OVS_WARN_UNUSED_RESULT
> -server_parse_options(int argc, char *argv[], struct shash *local_options,
> -                     int *n_options_p)
> -{
> -    static const struct option global_long_options[] = {
> -        VLOG_LONG_OPTIONS,
> -        MAIN_LOOP_LONG_OPTIONS,
> -        TABLE_LONG_OPTIONS,
> -        {NULL, 0, NULL, 0},
> -    };
> -    const int n_global_long_options = ARRAY_SIZE(global_long_options) - 1;
> -    char *short_options;
> -    struct option *options;
> -    char *error = NULL;
> -
> -    ovs_assert(n_options_p);
> -
> -    short_options = build_short_options(global_long_options, false);
> -    options = append_command_options(global_long_options, OPT_LOCAL);
> -
> -    optind = 0;
> -    opterr = 0;
> -    for (;;) {
> -        int idx;
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, options, &idx);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        bool handled;
> -        error = handle_main_loop_option(c, optarg, &handled);
> -        if (error) {
> -            goto out;
> -        }
> -        if (handled) {
> -            continue;
> -        }
> -
> -        switch (c) {
> -        case OPT_LOCAL:
> -            error = add_local_option(options[idx].name, optarg,
> local_options);
> -            if (error) {
> -                goto out;
> -            }
> -            break;
> -
> -        VLOG_OPTION_HANDLERS
> -        TABLE_OPTION_HANDLERS(&table_style)
> -
> -        case '?':
> -            if (find_option_by_value(options, optopt)) {
> -                error = xasprintf("option '%s' doesn't allow an argument",
> -                                  argv[optind-1]);
> -            } else if (optopt) {
> -                error = xasprintf("unrecognized option '%c'", optopt);
> -            } else {
> -                error = xasprintf("unrecognized option '%s'",
> argv[optind-1]);
> -            }
> -            goto out;
> -            break;
> -
> -        case ':':
> -            error = xasprintf("option '%s' requires an argument",
> -                              argv[optind-1]);
> -            goto out;
> -            break;
> -
> -        case 0:
> -            break;
> -
> -        default:
> -            error = xasprintf("unhandled option '%c'", c);
> -            goto out;
> -            break;
> -        }
> -    }
> -    *n_options_p = optind;
> -
> -out:
> -    for (int i = n_global_long_options; options[i].name; i++) {
> -        free(CONST_CAST(char *, options[i].name));
> -    }
> -    free(options);
> -    free(short_options);
> -
> -    return error;
> -}
> -
> -static void
> -server_cmd_exit(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -                const char *argv[] OVS_UNUSED, void *exiting_)
> -{
> -    bool *exiting = exiting_;
> -    *exiting = true;
> -    unixctl_command_reply(conn, NULL);
> -}
> -
> -static void
> -server_cmd_run(struct unixctl_conn *conn, int argc, const char **argv_,
> -               void *idl_)
> -{
> -    struct ovsdb_idl *idl = idl_;
> -    struct ctl_command *commands = NULL;
> -    struct shash local_options;
> -    size_t n_commands = 0;
> -    int n_options = 0;
> -    char *error = NULL;
> -
> -    /* Copy args so that getopt() can permute them. Leave last entry
> NULL. */
> -    char **argv = xcalloc(argc + 1, sizeof *argv);
> -    for (int i = 0; i < argc; i++) {
> -        argv[i] = xstrdup(argv_[i]);
> -    }
> -
> -    /* Reset global state. */
> -    oneline = false;
> -    dry_run = false;
> -    wait_type = NBCTL_WAIT_NONE;
> -    force_wait = false;
> -    timeout = 0;
> -    table_style = table_style_default;
> -
> -    /* Parse commands & options. */
> -    char *args = process_escape_args(argv);
> -    shash_init(&local_options);
> -    error = server_parse_options(argc, argv, &local_options, &n_options);
> -    if (error) {
> -        unixctl_command_reply_error(conn, error);
> -        goto out;
> -    }
> -    error = ctl_parse_commands(argc - n_options, argv + n_options,
> -                               &local_options, &commands, &n_commands);
> -    if (error) {
> -        unixctl_command_reply_error(conn, error);
> -        goto out;
> -    }
> -    VLOG(ctl_might_write_to_db(commands, n_commands) ? VLL_INFO : VLL_DBG,
> -         "Running command %s", args);
> -
> -    struct timer *wait_timeout = NULL;
> -    struct timer wait_timeout_;
> -    if (timeout) {
> -        wait_timeout = &wait_timeout_;
> -        timer_set_duration(wait_timeout, timeout * 1000);
> -    }
> -
> -    error = run_prerequisites(commands, n_commands, idl);
> -    if (error) {
> -        unixctl_command_reply_error(conn, error);
> -        goto out;
> -    }
> -    error = main_loop(args, commands, n_commands, idl, wait_timeout);
> -    if (error) {
> -        unixctl_command_reply_error(conn, error);
> -        goto out;
> -    }
> -
> -    struct ds output = DS_EMPTY_INITIALIZER;
> -    table_format_reset();
> -    for (struct ctl_command *c = commands; c < &commands[n_commands];
> c++) {
> -        if (c->table) {
> -            table_format(c->table, &table_style, &output);
> -        } else if (oneline) {
> -            oneline_format(&c->output, &output);
> -        } else {
> -            ds_put_cstr(&output, ds_cstr_ro(&c->output));
> -        }
> -
> -        ds_destroy(&c->output);
> -        table_destroy(c->table);
> -        free(c->table);
> -    }
> -    unixctl_command_reply(conn, ds_cstr_ro(&output));
> -    ds_destroy(&output);
> -
> -out:
> -    free(error);
> -    for (struct ctl_command *c = commands; c < &commands[n_commands];
> c++) {
> -        shash_destroy_free_data(&c->options);
> -    }
> -    free(commands);
> -    shash_destroy_free_data(&local_options);
> -    free(args);
> -    for (int i = 0; i < argc; i++) {
> -        free(argv[i]);
> -    }
> -    free(argv);
> -}
> -
> -static void
> -server_cmd_init(struct ovsdb_idl *idl, bool *exiting)
> -{
> -    unixctl_command_register("exit", "", 0, 0, server_cmd_exit, exiting);
> -    unixctl_command_register("run", "", 0, INT_MAX, server_cmd_run, idl);
> -}
> -
> -static void
> -server_loop(struct ovsdb_idl *idl, int argc, char *argv[])
> -{
> -    struct unixctl_server *server = NULL;
> -    bool exiting = false;
> -
> -    service_start(&argc, &argv);
> -    daemonize_start(false);
> -    int error = unixctl_server_create(unixctl_path, &server);
> -    if (error) {
> -        ctl_fatal("failed to create unixctl server (%s)",
> -                  ovs_retval_to_string(error));
> -    }
> -    puts(unixctl_server_get_path(server));
> -    fflush(stdout);
> -    server_cmd_init(idl, &exiting);
> -
> -    for (;;) {
> -        ovsdb_idl_run(idl);
> -        if (!ovsdb_idl_is_alive(idl)) {
> -            int retval = ovsdb_idl_get_last_error(idl);
> -            ctl_fatal("%s: database connection failed (%s)",
> -                      db, ovs_retval_to_string(retval));
> -        }
> -
> -        if (ovsdb_idl_has_ever_connected(idl)) {
> -            daemonize_complete();
> -            unixctl_server_run(server);
> -        }
> -        if (exiting) {
> -            break;
> -        }
> -
> -        ovsdb_idl_wait(idl);
> -        unixctl_server_wait(server);
> -        poll_block();
> -    }
> -
> -    unixctl_server_destroy(server);
> -}
> -
> -static void
> -nbctl_client(const char *socket_name,
> -             const struct ovs_cmdl_parsed_option *parsed_options, size_t
> n,
> -             int argc, char *argv[])
> -{
> -    struct svec args = SVEC_EMPTY_INITIALIZER;
> -
> -    for (const struct ovs_cmdl_parsed_option *po = parsed_options;
> -         po < &parsed_options[n]; po++) {
> -        optarg = po->arg;
> -        switch (po->o->val) {
> -        case OPT_DB:
> -            VLOG_WARN("not using ovn-nbctl daemon because of %s option",
> -                      po->o->name);
> -            svec_destroy(&args);
> -            return;
> -
> -        case OPT_NO_SYSLOG:
> -            vlog_set_levels(&this_module, VLF_SYSLOG, VLL_WARN);
> -            break;
> -
> -        case 'h':
> -            usage();
> -            exit(EXIT_SUCCESS);
> -
> -        case OPT_COMMANDS:
> -            ctl_print_commands();
> -            /* fall through */
> -
> -        case OPT_OPTIONS:
> -            ctl_print_options(get_all_options());
> -            /* fall through */
> -
> -        case OPT_LEADER_ONLY:
> -        case OPT_NO_LEADER_ONLY:
> -        case OPT_SHUFFLE_REMOTES:
> -        case OPT_NO_SHUFFLE_REMOTES:
> -        case OPT_BOOTSTRAP_CA_CERT:
> -        STREAM_SSL_CASES
> -        DAEMON_OPTION_CASES
> -            VLOG_INFO("using ovn-nbctl daemon, ignoring %s option",
> -                      po->o->name);
> -            break;
> -
> -        case 'V':
> -            ovs_print_version(0, 0);
> -            printf("DB Schema %s\n", nbrec_get_db_version());
> -            exit(EXIT_SUCCESS);
> -
> -        case 't':
> -            if (!str_to_uint(po->arg, 10, &timeout) || !timeout) {
> -                ctl_fatal("value %s on -t or --timeout is invalid",
> po->arg);
> -            }
> -            break;
> -
> -        VLOG_OPTION_HANDLERS
> -
> -        case OPT_LOCAL:
> -        default:
> -            if (po->arg) {
> -                svec_add_nocopy(&args,
> -                                xasprintf("--%s=%s", po->o->name,
> po->arg));
> -            } else {
> -                svec_add_nocopy(&args, xasprintf("--%s", po->o->name));
> -            }
> -            break;
> -        }
> -    }
> -    svec_add(&args, "--");
> -    for (int i = optind; i < argc; i++) {
> -        svec_add(&args, argv[i]);
> -    }
> -
> -    ctl_timeout_setup(timeout);
> -
> -    struct jsonrpc *client;
> -    int error = unixctl_client_create(socket_name, &client);
> -    if (error) {
> -        ctl_fatal("%s: could not connect to ovn-nb daemon (%s); "
> -                  "unset OVN_NB_DAEMON to avoid using daemon",
> -                  socket_name, ovs_strerror(error));
> -    }
> -
> -    char *cmd_result;
> -    char *cmd_error;
> -    error = unixctl_client_transact(client, "run",
> -                                    args.n, args.names,
> -                                    &cmd_result, &cmd_error);
> -    if (error) {
> -        ctl_fatal("%s: transaction error (%s)",
> -                  socket_name, ovs_strerror(error));
> -    }
> -    svec_destroy(&args);
> -
> -    int exit_status;
> -    if (cmd_error) {
> -        exit_status = EXIT_FAILURE;
> -        fprintf(stderr, "%s: %s", program_name, cmd_error);
> -    } else {
> -        exit_status = EXIT_SUCCESS;
> -        fputs(cmd_result, stdout);
> -    }
> -    free(cmd_result);
> -    free(cmd_error);
> -    jsonrpc_close(client);
> -    exit(exit_status);
> -}
> diff --git a/ovn/utilities/ovn-sbctl.8.in b/ovn/utilities/ovn-sbctl.8.in
> deleted file mode 100644
> index 2aaa457e8..000000000
> --- a/ovn/utilities/ovn-sbctl.8.in
> +++ /dev/null
> @@ -1,303 +0,0 @@
> -.\" -*- nroff -*-
> -.so lib/ovs.tmac
> -.TH ovn\-sbctl 8 "@VERSION@" "Open vSwitch" "Open vSwitch Manual"
> -.\" This program's name:
> -.ds PN ovn\-sbctl
> -.
> -.SH NAME
> -ovn\-sbctl \- utility for querying and configuring \fBOVN_Southbound\fR
> database
> -.
> -.SH SYNOPSIS
> -\fBovn\-sbctl\fR [\fIoptions\fR] \fB\-\-\fR [\fIoptions\fR] \fIcommand
> -\fR[\fIargs\fR] [\fB\-\-\fR [\fIoptions\fR] \fIcommand \fR[\fIargs\fR]]...
> -.
> -.SH DESCRIPTION
> -The \fBovn\-sbctl\fR program configures the \fBOVN_Southbound\fR database
> -by providing a high\-level interface to its configuration database.  See
> -\fBovn\-sb\fR(5) for comprehensive documentation of the database schema.
> -.PP
> -\fBovn\-sbctl\fR connects to an \fBovsdb\-server\fR process that
> -maintains an OVN_Southbound configuration database.  Using this
> -connection, it queries and possibly applies changes to the database,
> -depending on the supplied commands.
> -.PP
> -\fBovn\-sbctl\fR can perform any number of commands in a single run,
> -implemented as a single atomic transaction against the database.
> -.PP
> -The \fBovn\-sbctl\fR command line begins with global options (see
> -\fBOPTIONS\fR below for details).  The global options are followed by
> -one or more commands.  Each command should begin with \fB\-\-\fR by
> -itself as a command-line argument, to separate it from the following
> -commands.  (The \fB\-\-\fR before the first command is optional.)  The
> -command
> -itself starts with command-specific options, if any, followed by the
> -command name and any arguments.
> -.
> -.SH OPTIONS
> -.
> -The following options affect the behavior of \fBovn\-sbctl\fR as a
> -whole.  Some individual commands also accept their own options, which
> -are given just before the command name.  If the first command on the
> -command line has options, then those options must be separated from
> -the global options by \fB\-\-\fR.
> -.
> -.IP "\fB\-\-db=\fIserver\fR"
> -The OVSDB database remote to contact.  If the \fBOVN_SB_DB\fR
> -environment variable is set, its value is used as the default.
> -Otherwise, the default is \fBunix:@RUNDIR@/ovnsb_db.sock\fR, but this
> -default is unlikely to be useful outside of single-machine OVN test
> -environments.
> -.IP
> -\fIserver\fR may be an OVSDB active or passive connection method,
> -e.g. \fBssl:192.168.10.5:6640\fR, as described in \fBovsdb\fR(7).
> -.
> -.IP "\fB\-\-leader\-only\fR"
> -.IQ "\fB\-\-no\-leader\-only\fR"
> -By default, or with \fB\-\-leader\-only\fR, when the database server
> -is a clustered database, \fBovn\-sbctl\fR will avoid servers other
> -than the cluster leader.  This ensures that any data that
> -\fBovn\-sbctl\fR reads and reports is up-to-date.  With
> -\fB\-\-no\-leader\-only\fR, \fBovn\-sbctl\fR will use any server in
> -the cluster, which means that for read-only transactions it can report
> -and act on stale data (transactions that modify the database are
> -always serialized even with \fB\-\-no\-leader\-only\fR).  Refer to
> -\fBUnderstanding Cluster Consistency\fR in \fBovsdb\fR(7) for more
> -information.
> -.
> -.IP "\fB\-\-no\-syslog\fR"
> -By default, \fBovn\-sbctl\fR logs its arguments and the details of any
> -changes that it makes to the system log.  This option disables this
> -logging.
> -.IP
> -This option is equivalent to \fB\-\-verbose=sbctl:syslog:warn\fR.
> -.
> -.IP "\fB\-\-oneline\fR"
> -Modifies the output format so that the output for each command is printed
> -on a single line.  New-line characters that would otherwise separate
> -lines are printed as \fB\\n\fR, and any instances of \fB\\\fR that
> -would otherwise appear in the output are doubled.
> -Prints a blank line for each command that has no output.
> -This option does not affect the formatting of output from the
> -\fBlist\fR or \fBfind\fR commands; see \fBTable Formatting Options\fR
> -below.
> -.
> -.IP "\fB\-\-dry\-run\fR"
> -Prevents \fBovn\-sbctl\fR from actually modifying the database.
> -.
> -.IP "\fB\-t \fIsecs\fR"
> -.IQ "\fB\-\-timeout=\fIsecs\fR"
> -By default, or with a \fIsecs\fR of \fB0\fR, \fBovn\-sbctl\fR waits
> -forever for a response from the database.  This option limits runtime
> -to approximately \fIsecs\fR seconds.  If the timeout expires,
> -\fBovn\-sbctl\fR will exit with a \fBSIGALRM\fR signal.  (A timeout
> -would normally happen only if the database cannot be contacted, or if
> -the system is overloaded.)
> -.
> -.so lib/vlog.man
> -.so lib/common.man
> -.
> -.SS "Table Formatting Options"
> -These options control the format of output from the \fBlist\fR and
> -\fBfind\fR commands.
> -.so lib/table.man
> -.
> -.SS "Public Key Infrastructure Options"
> -.so lib/ssl-bootstrap.man
> -.so lib/ssl.man
> -.
> -.SH COMMANDS
> -The commands implemented by \fBovn\-sbctl\fR are described in the
> -sections below.
> -.SS "OVN_Southbound Commands"
> -These commands work with an \fBOVN_Southbound\fR database as a whole.
> -.
> -.IP "\fBinit\fR"
> -Initializes the database, if it is empty.  If the database has already
> -been initialized, this command has no effect.
> -.
> -.IP "\fBshow\fR"
> -Prints a brief overview of the database contents.
> -.
> -.SS "Chassis Commands"
> -These commands manipulate \fBOVN_Southbound\fR chassis.
> -.
> -.IP "[\fB\-\-may\-exist\fR] \fBchassis\-add \fIchassis\fR
> \fIencap-type\fR \fIencap-ip\fR"
> -Creates a new chassis named \fIchassis\fR.  \fIencap-type\fR is a
> -comma-separated list of tunnel types.  The chassis will have
> -one encap entry for each specified tunnel type with \fIencap-ip\fR
> -as the destination IP for each.
> -.IP
> -Without \fB\-\-may\-exist\fR, attempting to create a chassis that
> -exists is an error.  With \fB\-\-may\-exist\fR, this command does
> -nothing if \fIchassis\fR already exists.
> -.
> -.IP "[\fB\-\-if\-exists\fR] \fBchassis\-del \fIchassis\fR"
> -Deletes \fIchassis\fR and its \fIencaps\fR and \fIgateway_ports\fR.
> -.IP
> -Without \fB\-\-if\-exists\fR, attempting to delete a chassis that does
> -not exist is an error.  With \fB\-\-if\-exists\fR, attempting to
> -delete a chassis that does not exist has no effect.
> -.
> -.SS "Port binding Commands"
> -.
> -These commands manipulate \fBOVN_Southbound\fR port bindings.
> -.
> -.IP "[\fB\-\-may\-exist\fR] \fBlsp\-bind \fIlogical-port\fR \fIchassis\fR"
> -Binds the logical port named \fIlogical-port\fR to \fIchassis\fR.
> -.IP
> -Without \fB\-\-may\-exist\fR, attempting to bind a logical port that
> -has already been bound is an error.  With \fB\-\-may\-exist\fR, this
> -command does nothing if \fIlogical-port\fR has already been bound to
> -a chassis.
> -.
> -.IP "[\fB\-\-if\-exists\fR] \fBlsp\-unbind\fR \fIlogical-port\fR"
> -Resets the binding of \fIlogical-port\fR to \fINULL\fR.
> -.IP
> -Without \fB\-\-if\-exists\fR, attempting to unbind a logical port
> -that is not bound is an error.  With \fB\-\-if\-exists\fR, attempting
> -to unbind logical port that is not bound has no effect.
> -.
> -.SS "Logical Flow Commands"
> -.
> -.IP "[\fB\-\-uuid\fR] [\fB\-\-ovs\fR[\fB=\fIremote\fR]] [\fB\-\-stats\fR]
> \fBlflow\-list\fR [\fIlogical-datapath\fR] [\fIlflow\fR...]"
> -List logical flows.  If \fIlogical-datapath\fR is specified, only list
> -flows for that logical datapath.  The \fIlogical-datapath\fR may be
> -given as a UUID or as a datapath name (reporting an error if multiple
> -datapaths have the same name).
> -.IP
> -If at least one \fIlflow\fR is given, only matching logical flows, if
> -any, are listed.  Each \fIlflow\fR may be specified as a UUID or the
> -first few characters of a UUID, optionally prefixed by \fB0x\fR.
> -(Because \fBovn\-controller\fR sets OpenFlow flow cookies to the first
> -32 bits of the corresponding logical flow's UUID, this makes it easy
> -to look up the logical flow that generated a particular OpenFlow
> -flow.)
> -.IP
> -If \fB\-\-uuid\fR is specified, the output includes the first 32 bits
> -of each logical flow's UUID.  This makes it easier to find the
> -OpenFlow flows that correspond to a given logical flow.
> -.IP
> -If \fB\-\-ovs\fR is included, \fBovn\-sbctl\fR attempts to obtain and
> -display the OpenFlow flows that correspond to each OVN logical flow.
> -To do so, \fBovn\-sbctl\fR connects to \fIremote\fR (by default,
> -\fBunix:@RUNDIR@/br-int.mgmt\fR) over OpenFlow and retrieves the
> -flows.  If \fIremote\fR is specified, it must be an active OpenFlow
> -connection method described in \fBovsdb\fR(7).  Please see the
> -discussion of the similar \fB\-\-ovs\fR option in \fBovn-trace\fR(8)
> -for more information about the OpenFlow flow output.
> -.IP
> -By default, OpenFlow flow output includes only match and actions.  Add
> -\fB\-\-stats\fR to include all OpenFlow information, such as packet
> -and byte counters, duration, and timeouts.
> -.
> -.IP "[\fB\-\-uuid\fR] \fBdump\-flows\fR [\fIlogical-datapath\fR]"
> -Alias for \fBlflow\-list\fB.
> -.
> -.SS "Remote Connectivity Commands"
> -.
> -These commands manipulate the \fBconnections\fR column in the
> \fBSB_Global\fR
> -table and rows in the \fBConnection\fR table.  When \fBovsdb\-server\fR
> -is configured to use the \fBconnections\fR column for OVSDB connections,
> -this allows the administrator to use \fBovn\-sbctl\fR to configure
> database
> -connections.
> -.
> -.IP "\fBget\-connection\fR"
> -Prints the configured connection(s).
> -.
> -.IP "\fBdel\-connection\fR"
> -Deletes the configured connection(s).
> -.
> -.IP "\fBset\-connection\fR [\fIaccess\-specifier\fR] \fItarget\fR\&..."
> -Sets the configured manager target or targets.  Each \fItarget\fR may
> -may be an OVSDB active or passive connection method,
> -e.g. \fBpssl:6640\fR, as described in \fBovsdb\fR(7),
> -optionally preceded by an optional access-specifier (\fBread\-only\fR or
> -\fBread\-write\fR).
> -If provided, the effect of the access specifier persists for subsequent
> -targets until changed by another access specifier.
> -.
> -.SS "SSL Configuration"
> -When \fBovsdb\-server\fR is configured to connect using SSL, the
> -following parameters are required:
> -.TP
> -\fIprivate-key\fR
> -Specifies a PEM file containing the private key used for SSL connections.
> -.TP
> -\fIcertificate\fR
> -Specifies a PEM file containing a certificate, signed by the
> -certificate authority (CA) used by the connection peers, that
> -certifies the private key, identifying a trustworthy peer.
> -.TP
> -\fIca-cert\fR
> -Specifies a PEM file containing the CA certificate used to verify that
> -the connection peers are trustworthy.
> -.PP
> -These SSL settings apply to all SSL connections made by the southbound
> -database server.
> -.
> -.IP "\fBget\-ssl\fR"
> -Prints the SSL configuration.
> -.
> -.IP "\fBdel\-ssl\fR"
> -Deletes the current SSL configuration.
> -.
> -.IP "[\fB\-\-bootstrap\fR] \fBset\-ssl\fR \fIprivate-key\fR
> \fIcertificate\fR \fIca-cert\fR [\fIssl-protocol-list\fR
> [\fIssl-cipher-list\fR]]"
> -Sets the SSL configuration.  The \fB\-\-bootstrap\fR option is described
> -below.
> -.
> -.ST "CA Certificate Bootstrap"
> -.PP
> -Ordinarily, all of the files named in the SSL configuration must exist
> -before SSL connectivity can be used.  However, if the \fIca-cert\fR file
> -does not exist and the \fB\-\-bootstrap\fR
> -option is given, then \fBovsdb\-server\fR will attempt to obtain the
> -CA certificate from the target on its first SSL connection and
> -save it to the named PEM file.  If it is successful, it will
> -immediately drop the connection and reconnect, and from then on all
> -SSL connections must be authenticated by a certificate signed by the
> -CA certificate thus obtained.
> -.PP
> -\fBThis option exposes the SSL connection to a man-in-the-middle
> -attack obtaining the initial CA certificate\fR, but it may be useful
> -for bootstrapping.
> -.PP
> -This option is only useful if the SSL peer sends its CA certificate
> -as part of the SSL certificate chain.  The SSL protocol does not
> -require the controller to send the CA certificate.
> -.
> -.SS "Database Commands"
> -.
> -These commands query and modify the contents of \fBovsdb\fR tables.
> -They are a slight abstraction of the \fBovsdb\fR interface and as such
> -they operate at a lower level than other \fBovs\-sbctl\fR commands.
> -.PP
> -.ST "Identifying Tables, Records, and Columns"
> -.PP
> -Each of these commands has a \fItable\fR parameter to identify a table
> -within the database.  Many of them also take a \fIrecord\fR parameter
> -that identifies a particular record within a table.  The \fIrecord\fR
> -parameter may be the UUID for a record, and many tables offer
> -additional ways to identify records.  Some commands also take
> -\fIcolumn\fR parameters that identify a particular field within the
> -records in a table.
> -.PP
> -For a list of tables and their columns, see \fBovn\-sb\fR(5) or
> -see the table listing from the \fB--help\fR option.
> -.PP
> -Record names must be specified in full and with correct
> -capitalization, except that UUIDs may be abbreviated to their first 4
> -(or more) hex digits, as long as that is unique within the table.
> -Names of tables and columns are not case-sensitive, and \fB\-\fR and
> -\fB_\fR are treated interchangeably.  Unique abbreviations of table
> -and column names are acceptable, e.g. \fBaddr\fR or \fBa\fR is
> -sufficient to identify the \fBAddress_Set\fR table.
> -.
> -.so lib/db-ctl-base.man
> -.SH "EXIT STATUS"
> -.IP "0"
> -Successful program execution.
> -.IP "1"
> -Usage, syntax, or configuration file error.
> -.SH "SEE ALSO"
> -.
> -.BR ovn\-sb (5).
> diff --git a/ovn/utilities/ovn-sbctl.c b/ovn/utilities/ovn-sbctl.c
> deleted file mode 100644
> index dd6585a3e..000000000
> --- a/ovn/utilities/ovn-sbctl.c
> +++ /dev/null
> @@ -1,1541 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <ctype.h>
> -#include <errno.h>
> -#include <float.h>
> -#include <getopt.h>
> -#include <inttypes.h>
> -#include <signal.h>
> -#include <stdarg.h>
> -#include <stdlib.h>
> -#include <string.h>
> -#include <unistd.h>
> -
> -#include "colors.h"
> -#include "command-line.h"
> -#include "compiler.h"
> -#include "db-ctl-base.h"
> -#include "dirs.h"
> -#include "fatal-signal.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/json.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofp-flow.h"
> -#include "openvswitch/ofp-print.h"
> -#include "openvswitch/shash.h"
> -#include "openvswitch/vconn.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "ovsdb-data.h"
> -#include "ovsdb-idl.h"
> -#include "openvswitch/poll-loop.h"
> -#include "process.h"
> -#include "sset.h"
> -#include "stream-ssl.h"
> -#include "stream.h"
> -#include "table.h"
> -#include "timeval.h"
> -#include "util.h"
> -#include "svec.h"
> -
> -VLOG_DEFINE_THIS_MODULE(sbctl);
> -
> -struct sbctl_context;
> -
> -/* --db: The database server to contact. */
> -static const char *db;
> -
> -/* --oneline: Write each command's output as a single line? */
> -static bool oneline;
> -
> -/* --dry-run: Do not commit any changes. */
> -static bool dry_run;
> -
> -/* --timeout: Time to wait for a connection to 'db'. */
> -static unsigned int timeout;
> -
> -/* Format for table output. */
> -static struct table_style table_style = TABLE_STYLE_DEFAULT;
> -
> -/* The IDL we're using and the current transaction, if any.
> - * This is for use by sbctl_exit() only, to allow it to clean up.
> - * Other code should use its context arguments. */
> -static struct ovsdb_idl *the_idl;
> -static struct ovsdb_idl_txn *the_idl_txn;
> -OVS_NO_RETURN static void sbctl_exit(int status);
> -
> -/* --leader-only, --no-leader-only: Only accept the leader in a cluster.
> */
> -static int leader_only = true;
> -
> -static void sbctl_cmd_init(void);
> -OVS_NO_RETURN static void usage(void);
> -static void parse_options(int argc, char *argv[], struct shash
> *local_options);
> -static void run_prerequisites(struct ctl_command[], size_t n_commands,
> -                              struct ovsdb_idl *);
> -static bool do_sbctl(const char *args, struct ctl_command *, size_t n,
> -                     struct ovsdb_idl *);
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    struct ovsdb_idl *idl;
> -    struct ctl_command *commands;
> -    struct shash local_options;
> -    unsigned int seqno;
> -    size_t n_commands;
> -
> -    set_program_name(argv[0]);
> -    fatal_ignore_sigpipe();
> -    vlog_set_levels(NULL, VLF_CONSOLE, VLL_WARN);
> -    vlog_set_levels_from_string_assert("reconnect:warn");
> -
> -    sbctl_cmd_init();
> -
> -    /* Parse command line. */
> -    char *args = process_escape_args(argv);
> -    shash_init(&local_options);
> -    parse_options(argc, argv, &local_options);
> -    char *error = ctl_parse_commands(argc - optind, argv + optind,
> -                                     &local_options, &commands,
> &n_commands);
> -    if (error) {
> -        ctl_fatal("%s", error);
> -    }
> -    VLOG(ctl_might_write_to_db(commands, n_commands) ? VLL_INFO : VLL_DBG,
> -         "Called as %s", args);
> -
> -    ctl_timeout_setup(timeout);
> -
> -    /* Initialize IDL. */
> -    idl = the_idl = ovsdb_idl_create(db, &sbrec_idl_class, false, true);
> -    ovsdb_idl_set_leader_only(idl, leader_only);
> -    run_prerequisites(commands, n_commands, idl);
> -
> -    /* Execute the commands.
> -     *
> -     * 'seqno' is the database sequence number for which we last tried to
> -     * execute our transaction.  There's no point in trying to commit
> more than
> -     * once for any given sequence number, because if the transaction
> fails
> -     * it's because the database changed and we need to obtain an
> up-to-date
> -     * view of the database before we try the transaction again. */
> -    seqno = ovsdb_idl_get_seqno(idl);
> -    for (;;) {
> -        ovsdb_idl_run(idl);
> -        if (!ovsdb_idl_is_alive(idl)) {
> -            int retval = ovsdb_idl_get_last_error(idl);
> -            ctl_fatal("%s: database connection failed (%s)",
> -                        db, ovs_retval_to_string(retval));
> -        }
> -
> -        if (seqno != ovsdb_idl_get_seqno(idl)) {
> -            seqno = ovsdb_idl_get_seqno(idl);
> -            if (do_sbctl(args, commands, n_commands, idl)) {
> -                free(args);
> -                exit(EXIT_SUCCESS);
> -            }
> -        }
> -
> -        if (seqno == ovsdb_idl_get_seqno(idl)) {
> -            ovsdb_idl_wait(idl);
> -            poll_block();
> -        }
> -    }
> -}
> -
> -static void
> -parse_options(int argc, char *argv[], struct shash *local_options)
> -{
> -    enum {
> -        OPT_DB = UCHAR_MAX + 1,
> -        OPT_ONELINE,
> -        OPT_NO_SYSLOG,
> -        OPT_DRY_RUN,
> -        OPT_LOCAL,
> -        OPT_COMMANDS,
> -        OPT_OPTIONS,
> -        OPT_BOOTSTRAP_CA_CERT,
> -        VLOG_OPTION_ENUMS,
> -        TABLE_OPTION_ENUMS,
> -        SSL_OPTION_ENUMS,
> -    };
> -    static const struct option global_long_options[] = {
> -        {"db", required_argument, NULL, OPT_DB},
> -        {"no-syslog", no_argument, NULL, OPT_NO_SYSLOG},
> -        {"dry-run", no_argument, NULL, OPT_DRY_RUN},
> -        {"oneline", no_argument, NULL, OPT_ONELINE},
> -        {"timeout", required_argument, NULL, 't'},
> -        {"help", no_argument, NULL, 'h'},
> -        {"commands", no_argument, NULL, OPT_COMMANDS},
> -        {"options", no_argument, NULL, OPT_OPTIONS},
> -        {"leader-only", no_argument, &leader_only, true},
> -        {"no-leader-only", no_argument, &leader_only, false},
> -        {"version", no_argument, NULL, 'V'},
> -        VLOG_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {"bootstrap-ca-cert", required_argument, NULL,
> OPT_BOOTSTRAP_CA_CERT},
> -        TABLE_LONG_OPTIONS,
> -        {NULL, 0, NULL, 0},
> -    };
> -    const int n_global_long_options = ARRAY_SIZE(global_long_options) - 1;
> -    char *tmp, *short_options;
> -
> -    struct option *options;
> -    size_t allocated_options;
> -    size_t n_options;
> -    size_t i;
> -
> -    tmp = ovs_cmdl_long_options_to_short_options(global_long_options);
> -    short_options = xasprintf("+%s", tmp);
> -    free(tmp);
> -
> -    /* We want to parse both global and command-specific options here, but
> -     * getopt_long() isn't too convenient for the job.  We copy our global
> -     * options into a dynamic array, then append all of the
> command-specific
> -     * options. */
> -    options = xmemdup(global_long_options, sizeof global_long_options);
> -    allocated_options = ARRAY_SIZE(global_long_options);
> -    n_options = n_global_long_options;
> -    ctl_add_cmd_options(&options, &n_options, &allocated_options,
> OPT_LOCAL);
> -
> -    for (;;) {
> -        int idx;
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, options, &idx);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        switch (c) {
> -        case OPT_DB:
> -            db = optarg;
> -            break;
> -
> -        case OPT_ONELINE:
> -            oneline = true;
> -            break;
> -
> -        case OPT_NO_SYSLOG:
> -            vlog_set_levels(&this_module, VLF_SYSLOG, VLL_WARN);
> -            break;
> -
> -        case OPT_DRY_RUN:
> -            dry_run = true;
> -            break;
> -
> -        case OPT_LOCAL:
> -            if (shash_find(local_options, options[idx].name)) {
> -                ctl_fatal("'%s' option specified multiple times",
> -                            options[idx].name);
> -            }
> -            shash_add_nocopy(local_options,
> -                             xasprintf("--%s", options[idx].name),
> -                             nullable_xstrdup(optarg));
> -            break;
> -
> -        case 'h':
> -            usage();
> -
> -        case OPT_COMMANDS:
> -            ctl_print_commands();
> -            /* fall through */
> -
> -        case OPT_OPTIONS:
> -            ctl_print_options(global_long_options);
> -            /* fall through */
> -
> -        case 'V':
> -            ovs_print_version(0, 0);
> -            printf("DB Schema %s\n", sbrec_get_db_version());
> -            exit(EXIT_SUCCESS);
> -
> -        case 't':
> -            if (!str_to_uint(optarg, 10, &timeout) || !timeout) {
> -                ctl_fatal("value %s on -t or --timeout is invalid",
> optarg);
> -            }
> -            break;
> -
> -        VLOG_OPTION_HANDLERS
> -        TABLE_OPTION_HANDLERS(&table_style)
> -        STREAM_SSL_OPTION_HANDLERS
> -
> -        case OPT_BOOTSTRAP_CA_CERT:
> -            stream_ssl_set_ca_cert_file(optarg, true);
> -            break;
> -
> -        case '?':
> -            exit(EXIT_FAILURE);
> -
> -        default:
> -            abort();
> -
> -        case 0:
> -            break;
> -        }
> -    }
> -    free(short_options);
> -
> -    if (!db) {
> -        db = default_sb_db();
> -    }
> -
> -    for (i = n_global_long_options; options[i].name; i++) {
> -        free(CONST_CAST(char *, options[i].name));
> -    }
> -    free(options);
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN southbound DB management utility\n\
> -\n\
> -usage: %s [OPTIONS] COMMAND [ARG...]\n\
> -\n\
> -General commands:\n\
> -  show                        print overview of database contents\n\
> -\n\
> -Chassis commands:\n\
> -  chassis-add CHASSIS ENCAP-TYPE ENCAP-IP  create a new chassis named\n\
> -                                           CHASSIS with ENCAP-TYPE
> tunnels\n\
> -                                           and ENCAP-IP\n\
> -  chassis-del CHASSIS         delete CHASSIS and all of its encaps\n\
> -                              and gateway_ports\n\
> -\n\
> -Port binding commands:\n\
> -  lsp-bind PORT CHASSIS       bind logical port PORT to CHASSIS\n\
> -  lsp-unbind PORT             reset the port binding of logical port
> PORT\n\
> -\n\
> -Logical flow commands:\n\
> -  lflow-list [DATAPATH] [LFLOW...] List logical flows for DATAPATH\n\
> -  dump-flows [DATAPATH] [LFLOW...] Alias for lflow-list\n\
> -\n\
> -Connection commands:\n\
> -  get-connection             print the connections\n\
> -  del-connection             delete the connections\n\
> -  [--inactivity-probe=MSECS]\n\
> -  set-connection TARGET...   set the list of connections to TARGET...\n\
> -\n\
> -SSL commands:\n\
> -  get-ssl                     print the SSL configuration\n\
> -  del-ssl                     delete the SSL configuration\n\
> -  set-ssl PRIV-KEY CERT CA-CERT [SSL-PROTOS [SSL-CIPHERS]] \
> -set the SSL configuration\n\
> -\n\
> -%s\
> -%s\
> -\n\
> -Options:\n\
> -  --db=DATABASE               connect to DATABASE\n\
> -                              (default: %s)\n\
> -  --no-leader-only            accept any cluster member, not just the
> leader\n\
> -  -t, --timeout=SECS          wait at most SECS seconds\n\
> -  --dry-run                   do not commit changes to database\n\
> -  --oneline                   print exactly one line of output per
> command\n",
> -           program_name, program_name, ctl_get_db_cmd_usage(),
> -           ctl_list_db_tables_usage(), default_sb_db());
> -    table_usage();
> -    vlog_usage();
> -    printf("\
> -  --no-syslog             equivalent to --verbose=sbctl:syslog:warn\n");
> -    printf("\n\
> -Other options:\n\
> -  -h, --help                  display this help message\n\
> -  -V, --version               display version information\n");
> -    stream_usage("database", true, true, true);
> -    exit(EXIT_SUCCESS);
> -}
> -
> -
> -/* ovs-sbctl specific context.  Inherits the 'struct ctl_context' as
> base. */
> -struct sbctl_context {
> -    struct ctl_context base;
> -
> -    /* A cache of the contents of the database.
> -     *
> -     * A command that needs to use any of this information must first call
> -     * sbctl_context_populate_cache().  A command that changes anything
> that
> -     * could invalidate the cache must either call
> -     * sbctl_context_invalidate_cache() or manually update the cache to
> -     * maintain its correctness. */
> -    bool cache_valid;
> -    /* Maps from chassis name to struct sbctl_chassis. */
> -    struct shash chassis;
> -    /* Maps from lport name to struct sbctl_port_binding. */
> -    struct shash port_bindings;
> -};
> -
> -/* Casts 'base' into 'struct sbctl_context'. */
> -static struct sbctl_context *
> -sbctl_context_cast(struct ctl_context *base)
> -{
> -    return CONTAINER_OF(base, struct sbctl_context, base);
> -}
> -
> -struct sbctl_chassis {
> -    const struct sbrec_chassis *ch_cfg;
> -};
> -
> -struct sbctl_port_binding {
> -    const struct sbrec_port_binding *bd_cfg;
> -};
> -
> -static void
> -sbctl_context_invalidate_cache(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -
> -    if (!sbctl_ctx->cache_valid) {
> -        return;
> -    }
> -    sbctl_ctx->cache_valid = false;
> -    shash_destroy_free_data(&sbctl_ctx->chassis);
> -    shash_destroy_free_data(&sbctl_ctx->port_bindings);
> -}
> -
> -static void
> -sbctl_context_populate_cache(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -    const struct sbrec_chassis *chassis_rec;
> -    const struct sbrec_port_binding *port_binding_rec;
> -    struct sset chassis, port_bindings;
> -
> -    if (sbctl_ctx->cache_valid) {
> -        /* Cache is already populated. */
> -        return;
> -    }
> -    sbctl_ctx->cache_valid = true;
> -    shash_init(&sbctl_ctx->chassis);
> -    shash_init(&sbctl_ctx->port_bindings);
> -    sset_init(&chassis);
> -    SBREC_CHASSIS_FOR_EACH(chassis_rec, ctx->idl) {
> -        struct sbctl_chassis *ch;
> -
> -        if (!sset_add(&chassis, chassis_rec->name)) {
> -            VLOG_WARN("database contains duplicate chassis name (%s)",
> -                      chassis_rec->name);
> -            continue;
> -        }
> -
> -        ch = xmalloc(sizeof *ch);
> -        ch->ch_cfg = chassis_rec;
> -        shash_add(&sbctl_ctx->chassis, chassis_rec->name, ch);
> -    }
> -    sset_destroy(&chassis);
> -
> -    sset_init(&port_bindings);
> -    SBREC_PORT_BINDING_FOR_EACH(port_binding_rec, ctx->idl) {
> -        struct sbctl_port_binding *bd;
> -
> -        if (!sset_add(&port_bindings, port_binding_rec->logical_port)) {
> -            VLOG_WARN("database contains duplicate port binding for
> logical "
> -                      "port (%s)",
> -                      port_binding_rec->logical_port);
> -            continue;
> -        }
> -
> -        bd = xmalloc(sizeof *bd);
> -        bd->bd_cfg = port_binding_rec;
> -        shash_add(&sbctl_ctx->port_bindings,
> port_binding_rec->logical_port,
> -                  bd);
> -    }
> -    sset_destroy(&port_bindings);
> -}
> -
> -static void
> -check_conflicts(struct sbctl_context *sbctl_ctx, const char *name,
> -                char *msg)
> -{
> -    if (shash_find(&sbctl_ctx->chassis, name)) {
> -        ctl_fatal("%s because a chassis named %s already exists",
> -                    msg, name);
> -    }
> -    free(msg);
> -}
> -
> -static struct sbctl_chassis *
> -find_chassis(struct sbctl_context *sbctl_ctx, const char *name,
> -             bool must_exist)
> -{
> -    struct sbctl_chassis *sbctl_ch;
> -
> -    ovs_assert(sbctl_ctx->cache_valid);
> -
> -    sbctl_ch = shash_find_data(&sbctl_ctx->chassis, name);
> -    if (must_exist && !sbctl_ch) {
> -        ctl_fatal("no chassis named %s", name);
> -    }
> -
> -    return sbctl_ch;
> -}
> -
> -static struct sbctl_port_binding *
> -find_port_binding(struct sbctl_context *sbctl_ctx, const char *name,
> -                  bool must_exist)
> -{
> -    struct sbctl_port_binding *bd;
> -
> -    ovs_assert(sbctl_ctx->cache_valid);
> -
> -    bd = shash_find_data(&sbctl_ctx->port_bindings, name);
> -    if (must_exist && !bd) {
> -        ctl_fatal("no port named %s", name);
> -    }
> -
> -    return bd;
> -}
> -
> -static void
> -pre_get_info(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_chassis_col_name);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_chassis_col_encaps);
> -
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_encap_col_type);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_encap_col_ip);
> -
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_port_binding_col_logical_port);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_port_binding_col_chassis);
> -
> -    ovsdb_idl_add_column(ctx->idl,
> &sbrec_logical_flow_col_logical_datapath);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_pipeline);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_actions);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_priority);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_table_id);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_match);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_external_ids);
> -
> -    ovsdb_idl_add_column(ctx->idl,
> &sbrec_datapath_binding_col_external_ids);
> -
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ip_multicast_col_datapath);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ip_multicast_col_seq_no);
> -}
> -
> -static struct cmd_show_table cmd_show_tables[] = {
> -    {&sbrec_table_chassis,
> -     &sbrec_chassis_col_name,
> -     {&sbrec_chassis_col_hostname,
> -      &sbrec_chassis_col_encaps,
> -      NULL},
> -     {&sbrec_table_port_binding,
> -      &sbrec_port_binding_col_logical_port,
> -      &sbrec_port_binding_col_chassis}},
> -
> -    {&sbrec_table_encap,
> -     &sbrec_encap_col_type,
> -     {&sbrec_encap_col_ip,
> -      &sbrec_encap_col_options,
> -      NULL},
> -     {NULL, NULL, NULL}},
> -
> -    {NULL, NULL, {NULL, NULL, NULL}, {NULL, NULL, NULL}},
> -};
> -
> -static void
> -sbctl_init(struct ctl_context *ctx OVS_UNUSED)
> -{
> -}
> -
> -static void
> -cmd_chassis_add(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    const char *ch_name, *encap_types, *encap_ip;
> -
> -    ch_name = ctx->argv[1];
> -    encap_types = ctx->argv[2];
> -    encap_ip = ctx->argv[3];
> -
> -    sbctl_context_populate_cache(ctx);
> -    if (may_exist) {
> -        struct sbctl_chassis *sbctl_ch;
> -
> -        sbctl_ch = find_chassis(sbctl_ctx, ch_name, false);
> -        if (sbctl_ch) {
> -            return;
> -        }
> -    }
> -    check_conflicts(sbctl_ctx, ch_name,
> -                    xasprintf("cannot create a chassis named %s",
> ch_name));
> -
> -    struct sset encap_set;
> -    sset_from_delimited_string(&encap_set, encap_types, ",");
> -
> -    size_t n_encaps = sset_count(&encap_set);
> -    struct sbrec_encap **encaps = xmalloc(n_encaps * sizeof *encaps);
> -    const struct smap options = SMAP_CONST1(&options, "csum", "true");
> -    const char *encap_type;
> -    int i = 0;
> -    SSET_FOR_EACH (encap_type, &encap_set){
> -        encaps[i] = sbrec_encap_insert(ctx->txn);
> -
> -        sbrec_encap_set_type(encaps[i], encap_type);
> -        sbrec_encap_set_ip(encaps[i], encap_ip);
> -        sbrec_encap_set_options(encaps[i], &options);
> -        sbrec_encap_set_chassis_name(encaps[i], ch_name);
> -        i++;
> -    }
> -    sset_destroy(&encap_set);
> -
> -    struct sbrec_chassis *ch = sbrec_chassis_insert(ctx->txn);
> -    sbrec_chassis_set_name(ch, ch_name);
> -    sbrec_chassis_set_encaps(ch, encaps, n_encaps);
> -    free(encaps);
> -
> -    sbctl_context_invalidate_cache(ctx);
> -}
> -
> -static void
> -cmd_chassis_del(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    struct sbctl_chassis *sbctl_ch;
> -
> -    sbctl_context_populate_cache(ctx);
> -    sbctl_ch = find_chassis(sbctl_ctx, ctx->argv[1], must_exist);
> -    if (sbctl_ch) {
> -        if (sbctl_ch->ch_cfg) {
> -            size_t i;
> -
> -            for (i = 0; i < sbctl_ch->ch_cfg->n_encaps; i++) {
> -                sbrec_encap_delete(sbctl_ch->ch_cfg->encaps[i]);
> -            }
> -            sbrec_chassis_delete(sbctl_ch->ch_cfg);
> -        }
> -        shash_find_and_delete(&sbctl_ctx->chassis, ctx->argv[1]);
> -        free(sbctl_ch);
> -    }
> -}
> -
> -static void
> -cmd_lsp_bind(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
> -    struct sbctl_chassis *sbctl_ch;
> -    struct sbctl_port_binding *sbctl_bd;
> -    char *lport_name, *ch_name;
> -
> -    /* port_binding must exist, chassis must exist! */
> -    lport_name = ctx->argv[1];
> -    ch_name = ctx->argv[2];
> -
> -    sbctl_context_populate_cache(ctx);
> -    sbctl_bd = find_port_binding(sbctl_ctx, lport_name, true);
> -    sbctl_ch = find_chassis(sbctl_ctx, ch_name, true);
> -
> -    if (sbctl_bd->bd_cfg->chassis) {
> -        if (may_exist && sbctl_bd->bd_cfg->chassis == sbctl_ch->ch_cfg) {
> -            return;
> -        } else {
> -            ctl_fatal("lport (%s) has already been binded to chassis
> (%s)",
> -                      lport_name, sbctl_bd->bd_cfg->chassis->name);
> -        }
> -    }
> -    sbrec_port_binding_set_chassis(sbctl_bd->bd_cfg, sbctl_ch->ch_cfg);
> -    sbctl_context_invalidate_cache(ctx);
> -}
> -
> -static void
> -cmd_lsp_unbind(struct ctl_context *ctx)
> -{
> -    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
> -    bool must_exist = !shash_find(&ctx->options, "--if-exists");
> -    struct sbctl_port_binding *sbctl_bd;
> -    char *lport_name;
> -
> -    lport_name = ctx->argv[1];
> -    sbctl_context_populate_cache(ctx);
> -    sbctl_bd = find_port_binding(sbctl_ctx, lport_name, must_exist);
> -    if (sbctl_bd) {
> -        sbrec_port_binding_set_chassis(sbctl_bd->bd_cfg, NULL);
> -    }
> -}
> -
> -enum {
> -    PL_INGRESS,
> -    PL_EGRESS,
> -};
> -
> -/* Help ensure we catch any future pipeline values */
> -static int
> -pipeline_encode(const char *pl)
> -{
> -    if (!strcmp(pl, "ingress")) {
> -        return PL_INGRESS;
> -    } else if (!strcmp(pl, "egress")) {
> -        return PL_EGRESS;
> -    }
> -
> -    OVS_NOT_REACHED();
> -}
> -
> -static int
> -lflow_cmp(const void *lf1_, const void *lf2_)
> -{
> -    const struct sbrec_logical_flow *const *lf1p = lf1_;
> -    const struct sbrec_logical_flow *const *lf2p = lf2_;
> -    const struct sbrec_logical_flow *lf1 = *lf1p;
> -    const struct sbrec_logical_flow *lf2 = *lf2p;
> -
> -    int pl1 = pipeline_encode(lf1->pipeline);
> -    int pl2 = pipeline_encode(lf2->pipeline);
> -
> -#define CMP(expr) \
> -    do { \
> -        int res; \
> -        res = (expr); \
> -        if (res) { \
> -            return res; \
> -        } \
> -    } while (0)
> -
> -    CMP(uuid_compare_3way(&lf1->logical_datapath->header_.uuid,
> -                          &lf2->logical_datapath->header_.uuid));
> -    CMP(pl1 - pl2);
> -    CMP(lf1->table_id > lf2->table_id ? 1 :
> -            (lf1->table_id < lf2->table_id ? -1 : 0));
> -    CMP(lf1->priority > lf2->priority ? -1 :
> -            (lf1->priority < lf2->priority ? 1 : 0));
> -    CMP(strcmp(lf1->match, lf2->match));
> -
> -#undef CMP
> -
> -    return 0;
> -}
> -
> -static char *
> -parse_partial_uuid(char *s)
> -{
> -    /* Accept a full or partial UUID. */
> -    if (uuid_is_partial_string(s)) {
> -        return s;
> -    }
> -
> -    /* Accept a full or partial UUID prefixed by 0x, since "ovs-ofctl
> -     * dump-flows" prints cookies prefixed by 0x. */
> -    if (s[0] == '0' && (s[1] == 'x' || s[1] == 'X')
> -        && uuid_is_partial_string(s + 2)) {
> -        return s + 2;
> -    }
> -
> -    /* Not a (partial) UUID. */
> -    return NULL;
> -}
> -
> -static const char *
> -strip_leading_zero(const char *s)
> -{
> -    return s + strspn(s, "0");
> -}
> -
> -static bool
> -is_partial_uuid_match(const struct uuid *uuid, const char *match)
> -{
> -    char uuid_s[UUID_LEN + 1];
> -    snprintf(uuid_s, sizeof uuid_s, UUID_FMT, UUID_ARGS(uuid));
> -
> -    /* We strip leading zeros because we want to accept cookie values
> derived
> -     * from UUIDs, and cookie values are printed without leading zeros
> because
> -     * they're just numbers. */
> -    const char *s1 = strip_leading_zero(uuid_s);
> -    const char *s2 = strip_leading_zero(match);
> -
> -    return !strncmp(s1, s2, strlen(s2));
> -}
> -
> -static char *
> -default_ovs(void)
> -{
> -    return xasprintf("unix:%s/br-int.mgmt", ovs_rundir());
> -}
> -
> -static struct vconn *
> -sbctl_open_vconn(struct shash *options)
> -{
> -    struct shash_node *ovs = shash_find(options, "--ovs");
> -    if (!ovs) {
> -        return NULL;
> -    }
> -
> -    char *remote = ovs->data ? xstrdup(ovs->data) : default_ovs();
> -    struct vconn *vconn;
> -    int retval = vconn_open_block(remote, 1 << OFP13_VERSION, 0, -1,
> &vconn);
> -    if (retval) {
> -        VLOG_WARN("%s: connection failed (%s)", remote,
> ovs_strerror(retval));
> -    }
> -    free(remote);
> -    return vconn;
> -}
> -
> -static void
> -sbctl_dump_openflow(struct vconn *vconn, const struct uuid *uuid, bool
> stats)
> -{
> -    struct ofputil_flow_stats_request fsr = {
> -        .cookie = htonll(uuid->parts[0]),
> -        .cookie_mask = OVS_BE64_MAX,
> -        .out_port = OFPP_ANY,
> -        .out_group = OFPG_ANY,
> -        .table_id = OFPTT_ALL,
> -    };
> -
> -    struct ofputil_flow_stats *fses;
> -    size_t n_fses;
> -    int error = vconn_dump_flows(vconn, &fsr, OFPUTIL_P_OF13_OXM,
> -                                 &fses, &n_fses);
> -    if (error) {
> -        VLOG_WARN("%s: error obtaining flow stats (%s)",
> -                  vconn_get_name(vconn), ovs_strerror(error));
> -        return;
> -    }
> -
> -    if (n_fses) {
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        for (size_t i = 0; i < n_fses; i++) {
> -            const struct ofputil_flow_stats *fs = &fses[i];
> -
> -            ds_clear(&s);
> -            if (stats) {
> -                ofputil_flow_stats_format(&s, fs, NULL, NULL, true);
> -            } else {
> -                ds_put_format(&s, "%stable=%s%"PRIu8" ",
> -                              colors.special, colors.end, fs->table_id);
> -                match_format(&fs->match, NULL, &s, OFP_DEFAULT_PRIORITY);
> -                if (ds_last(&s) != ' ') {
> -                    ds_put_char(&s, ' ');
> -                }
> -
> -                ds_put_format(&s, "%sactions=%s", colors.actions,
> colors.end);
> -                struct ofpact_format_params fp = { .s = &s };
> -                ofpacts_format(fs->ofpacts, fs->ofpacts_len, &fp);
> -            }
> -            printf("    %s\n", ds_cstr(&s));
> -        }
> -        ds_destroy(&s);
> -    }
> -
> -    for (size_t i = 0; i < n_fses; i++) {
> -        free(CONST_CAST(struct ofpact *, fses[i].ofpacts));
> -    }
> -    free(fses);
> -}
> -
> -static void
> -cmd_lflow_list(struct ctl_context *ctx)
> -{
> -    const struct sbrec_datapath_binding *datapath = NULL;
> -    if (ctx->argc > 1) {
> -        const struct ovsdb_idl_row *row;
> -        char *error = ctl_get_row(ctx, &sbrec_table_datapath_binding,
> -                                  ctx->argv[1], false, &row);
> -        if (error) {
> -            ctl_fatal("%s", error);
> -        }
> -
> -        datapath = (const struct sbrec_datapath_binding *)row;
> -        if (datapath) {
> -            ctx->argc--;
> -            ctx->argv++;
> -        }
> -    }
> -
> -    for (size_t i = 1; i < ctx->argc; i++) {
> -        char *s = parse_partial_uuid(ctx->argv[i]);
> -        if (!s) {
> -            ctl_fatal("%s is not a UUID or the beginning of a UUID",
> -                      ctx->argv[i]);
> -        }
> -        ctx->argv[i] = s;
> -    }
> -
> -    struct vconn *vconn = sbctl_open_vconn(&ctx->options);
> -    bool stats = shash_find(&ctx->options, "--stats") != NULL;
> -
> -    const struct sbrec_logical_flow **lflows = NULL;
> -    size_t n_flows = 0;
> -    size_t n_capacity = 0;
> -    const struct sbrec_logical_flow *lflow;
> -    SBREC_LOGICAL_FLOW_FOR_EACH (lflow, ctx->idl) {
> -        if (datapath && lflow->logical_datapath != datapath) {
> -            continue;
> -        }
> -
> -        if (n_flows == n_capacity) {
> -            lflows = x2nrealloc(lflows, &n_capacity, sizeof *lflows);
> -        }
> -        lflows[n_flows] = lflow;
> -        n_flows++;
> -    }
> -
> -    if (n_flows) {
> -        qsort(lflows, n_flows, sizeof *lflows, lflow_cmp);
> -    }
> -
> -    bool print_uuid = shash_find(&ctx->options, "--uuid") != NULL;
> -
> -    const struct sbrec_logical_flow *prev = NULL;
> -    for (size_t i = 0; i < n_flows; i++) {
> -        lflow = lflows[i];
> -
> -        /* Figure out whether to print this particular flow.  By default,
> we
> -         * print all flows, but if any UUIDs were listed on the command
> line
> -         * then we only print the matching ones. */
> -        bool include;
> -        if (ctx->argc > 1) {
> -            include = false;
> -            for (size_t j = 1; j < ctx->argc; j++) {
> -                if (is_partial_uuid_match(&lflow->header_.uuid,
> -                                          ctx->argv[j])) {
> -                    include = true;
> -                    break;
> -                }
> -            }
> -        } else {
> -            include = true;
> -        }
> -        if (!include) {
> -            continue;
> -        }
> -
> -        /* Print a header line for this datapath or pipeline, if we
> haven't
> -         * already done so. */
> -        if (!prev
> -            || prev->logical_datapath != lflow->logical_datapath
> -            || strcmp(prev->pipeline, lflow->pipeline)) {
> -            printf("Datapath:");
> -
> -            const struct smap *ids =
> &lflow->logical_datapath->external_ids;
> -            const char *name = smap_get(ids, "name");
> -            const char *name2 = smap_get(ids, "name2");
> -            if (name && name2) {
> -                printf(" \"%s\" aka \"%s\"", name, name2);
> -            } else if (name || name2) {
> -                printf(" \"%s\"", name ? name : name2);
> -            }
> -            printf(" ("UUID_FMT")  Pipeline: %s\n",
> -                   UUID_ARGS(&lflow->logical_datapath->header_.uuid),
> -                   lflow->pipeline);
> -        }
> -
> -        /* Print the flow. */
> -        printf("  ");
> -        if (print_uuid) {
> -            printf("uuid=0x%08"PRIx32", ", lflow->header_.uuid.parts[0]);
> -        }
> -        printf("table=%-2"PRId64"(%-19s), priority=%-5"PRId64
> -               ", match=(%s), action=(%s)\n",
> -               lflow->table_id,
> -               smap_get_def(&lflow->external_ids, "stage-name", ""),
> -               lflow->priority, lflow->match, lflow->actions);
> -        if (vconn) {
> -            sbctl_dump_openflow(vconn, &lflow->header_.uuid, stats);
> -        }
> -        prev = lflow;
> -    }
> -
> -    vconn_close(vconn);
> -    free(lflows);
> -}
> -
> -static void
> -sbctl_ip_mcast_flush_switch(struct ctl_context *ctx,
> -                            const struct sbrec_datapath_binding *dp)
> -{
> -    const struct sbrec_ip_multicast *ip_mcast;
> -
> -    /* Lookup the corresponding IP_Multicast entry. */
> -    SBREC_IP_MULTICAST_FOR_EACH (ip_mcast, ctx->idl) {
> -        if (ip_mcast->datapath != dp) {
> -            continue;
> -        }
> -
> -        sbrec_ip_multicast_set_seq_no(ip_mcast, ip_mcast->seq_no + 1);
> -    }
> -}
> -
> -static void
> -sbctl_ip_mcast_flush(struct ctl_context *ctx)
> -{
> -    const struct sbrec_datapath_binding *dp;
> -
> -    if (ctx->argc > 2) {
> -        return;
> -    }
> -
> -    if (ctx->argc == 2) {
> -        const struct ovsdb_idl_row *row;
> -        char *error = ctl_get_row(ctx, &sbrec_table_datapath_binding,
> -                                  ctx->argv[1], false, &row);
> -        if (error) {
> -            ctl_fatal("%s", error);
> -        }
> -
> -        dp = (const struct sbrec_datapath_binding *)row;
> -        if (!dp) {
> -            ctl_fatal("%s is not a valid datapath", ctx->argv[1]);
> -        }
> -
> -        sbctl_ip_mcast_flush_switch(ctx, dp);
> -    } else {
> -        SBREC_DATAPATH_BINDING_FOR_EACH (dp, ctx->idl) {
> -            sbctl_ip_mcast_flush_switch(ctx, dp);
> -        }
> -    }
> -}
> -
> -static void
> -verify_connections(struct ctl_context *ctx)
> -{
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    const struct sbrec_connection *conn;
> -
> -    sbrec_sb_global_verify_connections(sb_global);
> -
> -    SBREC_CONNECTION_FOR_EACH(conn, ctx->idl) {
> -        sbrec_connection_verify_target(conn);
> -    }
> -}
> -
> -static void
> -pre_connection(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_sb_global_col_connections);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_connection_col_target);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_connection_col_read_only);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_connection_col_role);
> -    ovsdb_idl_add_column(ctx->idl,
> &sbrec_connection_col_inactivity_probe);
> -}
> -
> -static void
> -cmd_get_connection(struct ctl_context *ctx)
> -{
> -    const struct sbrec_connection *conn;
> -    struct svec targets;
> -    size_t i;
> -
> -    verify_connections(ctx);
> -
> -    /* Print the targets in sorted order for reproducibility. */
> -    svec_init(&targets);
> -
> -    SBREC_CONNECTION_FOR_EACH(conn, ctx->idl) {
> -        char *s;
> -
> -        s = xasprintf("%s role=\"%s\" %s",
> -                      conn->read_only ? "read-only" : "read-write",
> -                      conn->role,
> -                      conn->target);
> -        svec_add(&targets, s);
> -        free(s);
> -    }
> -
> -    svec_sort_unique(&targets);
> -    for (i = 0; i < targets.n; i++) {
> -        ds_put_format(&ctx->output, "%s\n", targets.names[i]);
> -    }
> -    svec_destroy(&targets);
> -}
> -
> -static void
> -delete_connections(struct ctl_context *ctx)
> -{
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    const struct sbrec_connection *conn, *next;
> -
> -    /* Delete Manager rows pointed to by 'connection_options' column. */
> -    SBREC_CONNECTION_FOR_EACH_SAFE(conn, next, ctx->idl) {
> -        sbrec_connection_delete(conn);
> -    }
> -
> -    /* Delete 'Manager' row refs in 'manager_options' column. */
> -    sbrec_sb_global_set_connections(sb_global, NULL, 0);
> -}
> -
> -static void
> -cmd_del_connection(struct ctl_context *ctx)
> -{
> -    verify_connections(ctx);
> -    delete_connections(ctx);
> -}
> -
> -static void
> -insert_connections(struct ctl_context *ctx, char *targets[], size_t n)
> -{
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    struct sbrec_connection **connections;
> -    size_t i, conns=0;
> -    bool read_only = false;
> -    char *role = "";
> -    const char *inactivity_probe = shash_find_data(&ctx->options,
> -                                                   "--inactivity-probe");
> -
> -    /* Insert each connection in a new row in Connection table. */
> -    connections = xmalloc(n * sizeof *connections);
> -    for (i = 0; i < n; i++) {
> -        if (!strcmp(targets[i], "read-only")) {
> -            read_only = true;
> -            continue;
> -        } else if (!strcmp(targets[i], "read-write")) {
> -            read_only = false;
> -            continue;
> -        } else if (!strncmp(targets[i], "role=", 5)) {
> -            role = targets[i] + 5;
> -            continue;
> -        } else if (stream_verify_name(targets[i]) &&
> -                   pstream_verify_name(targets[i])) {
> -            VLOG_WARN("target type \"%s\" is possibly erroneous",
> targets[i]);
> -        }
> -
> -        connections[conns] = sbrec_connection_insert(ctx->txn);
> -        sbrec_connection_set_target(connections[conns], targets[i]);
> -        sbrec_connection_set_read_only(connections[conns], read_only);
> -        sbrec_connection_set_role(connections[conns], role);
> -        if (inactivity_probe) {
> -            int64_t msecs = atoll(inactivity_probe);
> -            sbrec_connection_set_inactivity_probe(connections[conns],
> -                                                  &msecs, 1);
> -        }
> -        conns++;
> -    }
> -
> -    /* Store uuids of new connection rows in 'connection' column. */
> -    sbrec_sb_global_set_connections(sb_global, connections, conns);
> -    free(connections);
> -}
> -
> -static void
> -cmd_set_connection(struct ctl_context *ctx)
> -{
> -    const size_t n = ctx->argc - 1;
> -
> -    verify_connections(ctx);
> -    delete_connections(ctx);
> -    insert_connections(ctx, &ctx->argv[1], n);
> -}
> -
> -static void
> -pre_cmd_get_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_sb_global_col_ssl);
> -
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ssl_col_private_key);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ssl_col_certificate);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ssl_col_ca_cert);
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_ssl_col_bootstrap_ca_cert);
> -}
> -
> -static void
> -cmd_get_ssl(struct ctl_context *ctx)
> -{
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    const struct sbrec_ssl *ssl = sbrec_ssl_first(ctx->idl);
> -
> -    sbrec_sb_global_verify_ssl(sb_global);
> -    if (ssl) {
> -        sbrec_ssl_verify_private_key(ssl);
> -        sbrec_ssl_verify_certificate(ssl);
> -        sbrec_ssl_verify_ca_cert(ssl);
> -        sbrec_ssl_verify_bootstrap_ca_cert(ssl);
> -
> -        ds_put_format(&ctx->output, "Private key: %s\n",
> ssl->private_key);
> -        ds_put_format(&ctx->output, "Certificate: %s\n",
> ssl->certificate);
> -        ds_put_format(&ctx->output, "CA Certificate: %s\n", ssl->ca_cert);
> -        ds_put_format(&ctx->output, "Bootstrap: %s\n",
> -                ssl->bootstrap_ca_cert ? "true" : "false");
> -    }
> -}
> -
> -static void
> -pre_cmd_del_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_sb_global_col_ssl);
> -}
> -
> -static void
> -cmd_del_ssl(struct ctl_context *ctx)
> -{
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    const struct sbrec_ssl *ssl = sbrec_ssl_first(ctx->idl);
> -
> -    if (ssl) {
> -        sbrec_sb_global_verify_ssl(sb_global);
> -        sbrec_ssl_delete(ssl);
> -        sbrec_sb_global_set_ssl(sb_global, NULL);
> -    }
> -}
> -
> -static void
> -pre_cmd_set_ssl(struct ctl_context *ctx)
> -{
> -    ovsdb_idl_add_column(ctx->idl, &sbrec_sb_global_col_ssl);
> -}
> -
> -static void
> -cmd_set_ssl(struct ctl_context *ctx)
> -{
> -    bool bootstrap = shash_find(&ctx->options, "--bootstrap");
> -    const struct sbrec_sb_global *sb_global =
> sbrec_sb_global_first(ctx->idl);
> -    const struct sbrec_ssl *ssl = sbrec_ssl_first(ctx->idl);
> -
> -    sbrec_sb_global_verify_ssl(sb_global);
> -    if (ssl) {
> -        sbrec_ssl_delete(ssl);
> -    }
> -    ssl = sbrec_ssl_insert(ctx->txn);
> -
> -    sbrec_ssl_set_private_key(ssl, ctx->argv[1]);
> -    sbrec_ssl_set_certificate(ssl, ctx->argv[2]);
> -    sbrec_ssl_set_ca_cert(ssl, ctx->argv[3]);
> -
> -    sbrec_ssl_set_bootstrap_ca_cert(ssl, bootstrap);
> -
> -    if (ctx->argc == 5) {
> -        sbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]);
> -    } else if (ctx->argc == 6) {
> -        sbrec_ssl_set_ssl_protocols(ssl, ctx->argv[4]);
> -        sbrec_ssl_set_ssl_ciphers(ssl, ctx->argv[5]);
> -    }
> -
> -    sbrec_sb_global_set_ssl(sb_global, ssl);
> -}
> -
> -
> -static const struct ctl_table_class tables[SBREC_N_TABLES] = {
> -    [SBREC_TABLE_CHASSIS].row_ids[0] = {&sbrec_chassis_col_name, NULL,
> NULL},
> -
> -    [SBREC_TABLE_DATAPATH_BINDING].row_ids
> -     = {{&sbrec_datapath_binding_col_external_ids, "name", NULL},
> -        {&sbrec_datapath_binding_col_external_ids, "name2", NULL},
> -        {&sbrec_datapath_binding_col_external_ids, "logical-switch",
> NULL},
> -        {&sbrec_datapath_binding_col_external_ids, "logical-router",
> NULL}},
> -
> -    [SBREC_TABLE_PORT_BINDING].row_ids
> -     = {{&sbrec_port_binding_col_logical_port, NULL, NULL},
> -        {&sbrec_port_binding_col_external_ids, "name", NULL}},
> -
> -    [SBREC_TABLE_MAC_BINDING].row_ids[0] =
> -    {&sbrec_mac_binding_col_logical_port, NULL, NULL},
> -
> -    [SBREC_TABLE_ADDRESS_SET].row_ids[0]
> -    = {&sbrec_address_set_col_name, NULL, NULL},
> -
> -    [SBREC_TABLE_HA_CHASSIS_GROUP].row_ids[0]
> -    = {&sbrec_ha_chassis_group_col_name, NULL, NULL},
> -
> -    [SBREC_TABLE_HA_CHASSIS].row_ids[0]
> -    = {&sbrec_ha_chassis_col_chassis, NULL, NULL},
> -};
> -
> -
> -static void
> -sbctl_context_init_command(struct sbctl_context *sbctl_ctx,
> -                           struct ctl_command *command)
> -{
> -    ctl_context_init_command(&sbctl_ctx->base, command);
> -}
> -
> -static void
> -sbctl_context_init(struct sbctl_context *sbctl_ctx,
> -                   struct ctl_command *command, struct ovsdb_idl *idl,
> -                   struct ovsdb_idl_txn *txn,
> -                   struct ovsdb_symbol_table *symtab)
> -{
> -    ctl_context_init(&sbctl_ctx->base, command, idl, txn, symtab,
> -                     sbctl_context_invalidate_cache);
> -    sbctl_ctx->cache_valid = false;
> -}
> -
> -static void
> -sbctl_context_done_command(struct sbctl_context *sbctl_ctx,
> -                           struct ctl_command *command)
> -{
> -    ctl_context_done_command(&sbctl_ctx->base, command);
> -}
> -
> -static void
> -sbctl_context_done(struct sbctl_context *sbctl_ctx,
> -                   struct ctl_command *command)
> -{
> -    ctl_context_done(&sbctl_ctx->base, command);
> -}
> -
> -static void
> -run_prerequisites(struct ctl_command *commands, size_t n_commands,
> -                  struct ovsdb_idl *idl)
> -{
> -    ovsdb_idl_add_table(idl, &sbrec_table_sb_global);
> -
> -    for (struct ctl_command *c = commands; c < &commands[n_commands];
> c++) {
> -        if (c->syntax->prerequisites) {
> -            struct sbctl_context sbctl_ctx;
> -
> -            ds_init(&c->output);
> -            c->table = NULL;
> -
> -            sbctl_context_init(&sbctl_ctx, c, idl, NULL, NULL);
> -            (c->syntax->prerequisites)(&sbctl_ctx.base);
> -            if (sbctl_ctx.base.error) {
> -                ctl_fatal("%s", sbctl_ctx.base.error);
> -            }
> -            sbctl_context_done(&sbctl_ctx, c);
> -
> -            ovs_assert(!c->output.string);
> -            ovs_assert(!c->table);
> -        }
> -    }
> -}
> -
> -static bool
> -do_sbctl(const char *args, struct ctl_command *commands, size_t
> n_commands,
> -         struct ovsdb_idl *idl)
> -{
> -    struct ovsdb_idl_txn *txn;
> -    enum ovsdb_idl_txn_status status;
> -    struct ovsdb_symbol_table *symtab;
> -    struct sbctl_context sbctl_ctx;
> -    struct ctl_command *c;
> -    struct shash_node *node;
> -
> -    txn = the_idl_txn = ovsdb_idl_txn_create(idl);
> -    if (dry_run) {
> -        ovsdb_idl_txn_set_dry_run(txn);
> -    }
> -
> -    ovsdb_idl_txn_add_comment(txn, "ovs-sbctl: %s", args);
> -
> -    const struct sbrec_sb_global *sb = sbrec_sb_global_first(idl);
> -    if (!sb) {
> -        /* XXX add verification that table is empty */
> -        sbrec_sb_global_insert(txn);
> -    }
> -
> -    symtab = ovsdb_symbol_table_create();
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        ds_init(&c->output);
> -        c->table = NULL;
> -    }
> -    sbctl_context_init(&sbctl_ctx, NULL, idl, txn, symtab);
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        sbctl_context_init_command(&sbctl_ctx, c);
> -        if (c->syntax->run) {
> -            (c->syntax->run)(&sbctl_ctx.base);
> -        }
> -        if (sbctl_ctx.base.error) {
> -            ctl_fatal("%s", sbctl_ctx.base.error);
> -        }
> -        sbctl_context_done_command(&sbctl_ctx, c);
> -
> -        if (sbctl_ctx.base.try_again) {
> -            sbctl_context_done(&sbctl_ctx, NULL);
> -            goto try_again;
> -        }
> -    }
> -    sbctl_context_done(&sbctl_ctx, NULL);
> -
> -    SHASH_FOR_EACH (node, &symtab->sh) {
> -        struct ovsdb_symbol *symbol = node->data;
> -        if (!symbol->created) {
> -            ctl_fatal("row id \"%s\" is referenced but never created
> (e.g. "
> -                      "with \"-- --id=%s create ...\")",
> -                      node->name, node->name);
> -        }
> -        if (!symbol->strong_ref) {
> -            if (!symbol->weak_ref) {
> -                VLOG_WARN("row id \"%s\" was created but no reference to
> it "
> -                          "was inserted, so it will not actually appear
> in "
> -                          "the database", node->name);
> -            } else {
> -                VLOG_WARN("row id \"%s\" was created but only a weak "
> -                          "reference to it was inserted, so it will not "
> -                          "actually appear in the database", node->name);
> -            }
> -        }
> -    }
> -
> -    status = ovsdb_idl_txn_commit_block(txn);
> -    if (status == TXN_UNCHANGED || status == TXN_SUCCESS) {
> -        for (c = commands; c < &commands[n_commands]; c++) {
> -            if (c->syntax->postprocess) {
> -                sbctl_context_init(&sbctl_ctx, c, idl, txn, symtab);
> -                (c->syntax->postprocess)(&sbctl_ctx.base);
> -                if (sbctl_ctx.base.error) {
> -                    ctl_fatal("%s", sbctl_ctx.base.error);
> -                }
> -                sbctl_context_done(&sbctl_ctx, c);
> -            }
> -        }
> -    }
> -
> -    switch (status) {
> -    case TXN_UNCOMMITTED:
> -    case TXN_INCOMPLETE:
> -        OVS_NOT_REACHED();
> -
> -    case TXN_ABORTED:
> -        /* Should not happen--we never call ovsdb_idl_txn_abort(). */
> -        ctl_fatal("transaction aborted");
> -
> -    case TXN_UNCHANGED:
> -    case TXN_SUCCESS:
> -        break;
> -
> -    case TXN_TRY_AGAIN:
> -        goto try_again;
> -
> -    case TXN_ERROR:
> -        ctl_fatal("transaction error: %s", ovsdb_idl_txn_get_error(txn));
> -
> -    case TXN_NOT_LOCKED:
> -        /* Should not happen--we never call ovsdb_idl_set_lock(). */
> -        ctl_fatal("database not locked");
> -
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -
> -    ovsdb_symbol_table_destroy(symtab);
> -
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        struct ds *ds = &c->output;
> -
> -        if (c->table) {
> -            table_print(c->table, &table_style);
> -        } else if (oneline) {
> -            size_t j;
> -
> -            ds_chomp(ds, '\n');
> -            for (j = 0; j < ds->length; j++) {
> -                int ch = ds->string[j];
> -                switch (ch) {
> -                case '\n':
> -                    fputs("\\n", stdout);
> -                    break;
> -
> -                case '\\':
> -                    fputs("\\\\", stdout);
> -                    break;
> -
> -                default:
> -                    putchar(ch);
> -                }
> -            }
> -            putchar('\n');
> -        } else {
> -            fputs(ds_cstr(ds), stdout);
> -        }
> -        ds_destroy(&c->output);
> -        table_destroy(c->table);
> -        free(c->table);
> -
> -        shash_destroy_free_data(&c->options);
> -    }
> -    free(commands);
> -    ovsdb_idl_txn_destroy(txn);
> -    ovsdb_idl_destroy(idl);
> -
> -    return true;
> -
> -try_again:
> -    /* Our transaction needs to be rerun, or a prerequisite was not met.
> Free
> -     * resources and return so that the caller can try again. */
> -    ovsdb_idl_txn_abort(txn);
> -    ovsdb_idl_txn_destroy(txn);
> -    the_idl_txn = NULL;
> -
> -    ovsdb_symbol_table_destroy(symtab);
> -    for (c = commands; c < &commands[n_commands]; c++) {
> -        ds_destroy(&c->output);
> -        table_destroy(c->table);
> -        free(c->table);
> -    }
> -    return false;
> -}
> -
> -/* Frees the current transaction and the underlying IDL and then calls
> - * exit(status).
> - *
> - * Freeing the transaction and the IDL is not strictly necessary, but it
> makes
> - * for a clean memory leak report from valgrind in the normal case.  That
> makes
> - * it easier to notice real memory leaks. */
> -static void
> -sbctl_exit(int status)
> -{
> -    if (the_idl_txn) {
> -        ovsdb_idl_txn_abort(the_idl_txn);
> -        ovsdb_idl_txn_destroy(the_idl_txn);
> -    }
> -    ovsdb_idl_destroy(the_idl);
> -    exit(status);
> -}
> -
> -static const struct ctl_command_syntax sbctl_commands[] = {
> -    { "init", 0, 0, "", NULL, sbctl_init, NULL, "", RW },
> -
> -    /* Chassis commands. */
> -    {"chassis-add", 3, 3, "CHASSIS ENCAP-TYPE ENCAP-IP", pre_get_info,
> -     cmd_chassis_add, NULL, "--may-exist", RW},
> -    {"chassis-del", 1, 1, "CHASSIS", pre_get_info, cmd_chassis_del, NULL,
> -     "--if-exists", RW},
> -
> -    /* Port binding commands. */
> -    {"lsp-bind", 2, 2, "PORT CHASSIS", pre_get_info, cmd_lsp_bind, NULL,
> -     "--may-exist", RW},
> -    {"lsp-unbind", 1, 1, "PORT", pre_get_info, cmd_lsp_unbind, NULL,
> -     "--if-exists", RW},
> -
> -    /* Logical flow commands */
> -    {"lflow-list", 0, INT_MAX, "[DATAPATH] [LFLOW...]",
> -     pre_get_info, cmd_lflow_list, NULL,
> -     "--uuid,--ovs?,--stats", RO},
> -    {"dump-flows", 0, INT_MAX, "[DATAPATH] [LFLOW...]",
> -     pre_get_info, cmd_lflow_list, NULL,
> -     "--uuid,--ovs?,--stats", RO}, /* Friendly alias for lflow-list */
> -
> -    /* IP multicast commands. */
> -    {"ip-multicast-flush", 0, 1, "SWITCH",
> -     pre_get_info, sbctl_ip_mcast_flush, NULL, "", RW },
> -
> -    /* Connection commands. */
> -    {"get-connection", 0, 0, "", pre_connection, cmd_get_connection,
> NULL, "", RO},
> -    {"del-connection", 0, 0, "", pre_connection, cmd_del_connection,
> NULL, "", RW},
> -    {"set-connection", 1, INT_MAX, "TARGET...", pre_connection,
> cmd_set_connection,
> -     NULL, "--inactivity-probe=", RW},
> -
> -    /* SSL commands. */
> -    {"get-ssl", 0, 0, "", pre_cmd_get_ssl, cmd_get_ssl, NULL, "", RO},
> -    {"del-ssl", 0, 0, "", pre_cmd_del_ssl, cmd_del_ssl, NULL, "", RW},
> -    {"set-ssl", 3, 5,
> -        "PRIVATE-KEY CERTIFICATE CA-CERT [SSL-PROTOS [SSL-CIPHERS]]",
> -        pre_cmd_set_ssl, cmd_set_ssl, NULL, "--bootstrap", RW},
> -
> -    {NULL, 0, 0, NULL, NULL, NULL, NULL, NULL, RO},
> -};
> -
> -/* Registers sbctl and common db commands. */
> -static void
> -sbctl_cmd_init(void)
> -{
> -    ctl_init(&sbrec_idl_class, sbrec_table_classes, tables,
> -             cmd_show_tables, sbctl_exit);
> -    ctl_register_commands(sbctl_commands);
> -}
> diff --git a/ovn/utilities/ovn-trace.8.xml b/ovn/utilities/ovn-trace.8.xml
> deleted file mode 100644
> index 01e741119..000000000
> --- a/ovn/utilities/ovn-trace.8.xml
> +++ /dev/null
> @@ -1,485 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<manpage program="ovn-trace" section="8" title="ovn-trace">
> -  <h1>Name</h1>
> -  <p>ovn-trace -- Open Virtual Network logical network tracing utility</p>
> -
> -  <h1>Synopsis</h1>
> -  <p><code>ovn-trace</code> [<var>options</var>] <var>datapath</var>
> <var>microflow</var></p>
> -  <p><code>ovn-trace</code> [<var>options</var>] <code>--detach</code></p>
> -
> -  <h1>Description</h1>
> -  <p>
> -    This utility simulates packet forwarding within an OVN logical
> network.
> -    It can be used to run through ``what-if'' scenarios: if a packet
> -    originates at a logical port, what will happen to it and where will it
> -    ultimately end up?  Users already familiar with the Open vSwitch
> -    <code>ofproto/trace</code> command described in
> -    <code>ovs-vswitch</code>(8) will find <code>ovn-trace</code> to be a
> -    similar tool for logical networks.
> -  </p>
> -
> -  <p>
> -    <code>ovn-trace</code> works by reading the <code>Logical_Flow</code>
> and
> -    other tables from the OVN southbound database (see
> -    <code>ovn-sb</code>(5)).  It simulates a packet's path through logical
> -    networks by repeatedly looking it up in the logical flow table,
> following
> -    the entire tree of possibilities.
> -  </p>
> -
> -  <p>
> -    <code>ovn-trace</code> simulates only the OVN logical network.  It
> does
> -    not simulate the physical elements on which the logical network is
> -    layered.  This means that, for example, it is unimportant how VMs are
> -    distributed among hypervisors, or whether their hypervisors are
> -    functioning and reachable, so <code>ovn-trace</code> will yield the
> same
> -    results regardless.  There is one important exception:
> -    <code>ovn-northd</code>, the daemon that generates the logical flows
> that
> -    <code>ovn-trace</code> simulates, treats logical ports differently
> based
> -    on whether they are up or down.  Thus, if you see surprising results,
> -    ensure that the ports involved in a simulation are up.
> -  </p>
> -
> -  <p>
> -    The simplest way to use <code>ovn-trace</code> is to provide
> -    <var>datapath</var> and <var>microflow</var> arguments on the command
> -    line.  In this case, it simulates the behavior of a single packet and
> -    exits.  For an alternate usage model, see <code>Daemon Mode</code>
> below.
> -  </p>
> -
> -  <p>
> -    The <var>datapath</var> argument specifies the name of a logical
> -    datapath.  Acceptable names are the <code>name</code> from the
> northbound
> -    <code>Logical_Switch</code> or <code>Logical_Router</code> table, the
> -    UUID of a record from one of those tables, or the UUID of a record
> from
> -    the southbound <code>Datapath_Binding</code> table.
> -  </p>
> -
> -  <p>
> -    The <var>microflow</var> argument describes the packet whose
> forwarding
> -    is to be simulated, in the syntax of an OVN logical expression, as
> -    described in <code>ovn-sb</code>(5), to express constraints.  The
> parser
> -    understands prerequisites; for example, if the expression refers to
> -    <code>ip4.src</code>, there is no need to explicitly state
> -    <code>ip4</code> or <code>eth.type == 0x800</code>.
> -  </p>
> -
> -  <p>
> -    For reasonable L2 behavior, the microflow should include at least
> -    <code>inport</code> and <code>eth.dst</code>, plus
> <code>eth.src</code>
> -    if port security is enabled.  For example:
> -  </p>
> -  <pre>
> -    inport == "lp11" &amp;&amp; eth.src == 00:01:02:03:04:05 &amp;&amp;
> eth.dst == ff:ff:ff:ff:ff:ff
> -  </pre>
> -
> -  <p>
> -    For reasonable L3 behavior, <var>microflow</var> should also include
> -    <code>ip4.src</code> and <code>ip4.dst</code> (or <code>ip6.src</code>
> -    and <code>ip6.dst</code>) and <code>ip.ttl</code>.  For example:
> -  </p>
> -  <pre>
> -    inport == "lp111" &amp;&amp; eth.src == f0:00:00:00:01:11 &amp;&amp;
> eth.dst == 00:00:00:00:ff:11
> -    &amp;&amp; ip4.src == 192.168.11.1 &amp;&amp; ip4.dst == 192.168.22.2
> &amp;&amp; ip.ttl == 64
> -  </pre>
> -
> -  <p>Here's an ARP microflow example:</p>
> -  <pre>
> -    inport == "lp123"
> -    &amp;&amp; eth.dst == ff:ff:ff:ff:ff:ff &amp;&amp; eth.src ==
> f0:00:00:00:01:11
> -    &amp;&amp; arp.op == 1 &amp;&amp; arp.sha == f0:00:00:00:01:11
> &amp;&amp; arp.spa == 192.168.1.11
> -    &amp;&amp; arp.tha == ff:ff:ff:ff:ff:ff &amp;&amp; arp.tpa ==
> 192.168.2.22
> -  </pre>
> -
> -  <p>
> -    <code>ovn-trace</code> will reject erroneous microflow expressions,
> which
> -    beyond syntax errors fall into two categories.  First, they can be
> -    ambiguous.  For example, <code>tcp.src == 80</code> is ambiguous
> because
> -    it does not state IPv4 or IPv6 as the Ethernet type.  <code>ip4
> -    &amp;&amp; tcp.src > 1024</code> is also ambiguous because it does not
> -    constrain bits of <code>tcp.src</code> to particular values.  Second,
> -    they can be contradictory, e.g. <code>ip4 &amp;&amp; ip6</code>.
> -  </p>
> -
> -  <h1>Output</h1>
> -
> -  <p>
> -    <code>ovn-trace</code> supports the three different forms of output,
> each
> -    described in a separate section below.  Regardless of the selected
> output
> -    format, <code>ovn-trace</code> starts the output with a line that
> shows
> -    the microflow being traced in OpenFlow syntax.
> -  </p>
> -
> -  <h2>Detailed Output</h2>
> -
> -  <p>
> -    The detailed form of output is also the default form.  This form
> groups
> -    output into sections headed up by the ingress or egress pipeline being
> -    traversed.  Each pipeline lists each table that was visited (by
> number and
> -    name), the <code>ovn-northd</code> source file and line number of the
> code
> -    that added the flow, the match expression and priority of the logical
> flow
> -    that was matched, and the actions that were executed.
> -  </p>
> -
> -  <p>
> -    The execution of OVN logical actions naturally forms a ``control
> stack''
> -    that resembles that of a program in conventional programming languages
> -    such as C or Java.  Because the <code>next</code> action that calls
> into
> -    another logical flow table for a lookup is a recursive construct, OVN
> -    ``programs'' in practice tend to form deep control stacks that,
> displayed
> -    in the obvious way using additional indentation for each level,
> quickly
> -    use up the horizontal space on all but the widest displays.  To make
> -    detailed output more readable, without loss of generality,
> -    <code>ovn-trace</code> omits indentation for ``tail recursion,'' that
> is,
> -    when <code>next</code> is the last action in a logical flow, it does
> not
> -    indent details of the next table lookup more deeply.  Output still
> uses
> -    indentation when it is needed for clarity.
> -  </p>
> -
> -  <p>
> -    OVN ``programs'' traces also tend to encounter long strings of logical
> -    flows with match expression <code>1</code> (which matches every
> packet)
> -    and the single action <code>next;</code>.  These are uninteresting
> -    and merely clutter output, so <code>ovn-trace</code> omits them
> -    entirely even from detailed output.
> -  </p>
> -
> -  <p>
> -    The following excerpt from detailed <code>ovn-trace</code> output
> shows a
> -    section for a packet traversing the ingress pipeline of logical
> datapath
> -    <code>ls1</code> with ingress logical port <code>lp111</code>.  The
> -    packet matches a logical flow in table 0 (aka
> -    <code>ls_in_port_sec_l2</code>) with priority 50 and executes
> -    <code>next(1);</code> to pass to table 1.  Tables 1 through 11 are
> -    trivial and omitted.  In table 12 (aka <code>ls_in_l2_lkup</code>),
> the
> -    packet matches a flow with priority 50 based on its Ethernet
> destination
> -    address and the flow's actions output the packet to the
> -    <code>lrp11-attachement</code> logical port.
> -  </p>
> -
> -  <pre fixed="yes">
> -    ingress(dp="ls1", inport="lp111")
> -    ---------------------------------
> -    0. ls_in_port_sec_l2: inport == "lp111", priority 50
> -    next(1);
> -    12. ls_in_l2_lkup: eth.dst == 00:00:00:00:ff:11, priority 50
> -    outport = "lrp11-attachment";
> -    output;
> -  </pre>
> -
> -  <h2>Summary Output</h2>
> -
> -  <p>
> -    Summary output includes the logical pipelines visited by a packet and
> the
> -    logical actions executed on it.  Compared to the detailed output,
> -    however, it removes details of tables and logical flows traversed by a
> -    packet.  It uses a format closer to that of a programming language and
> -    does not attempt to avoid indentation.  The summary output equivalent
> to
> -    the above detailed output fragment is:
> -  </p>
> -
> -  <pre fixed="yes">
> -    ingress(dp="ls1", inport="lp111") {
> -    outport = "lrp11-attachment";
> -    output;
> -    ...
> -    };
> -  </pre>
> -
> -  <h2>Minimal Output</h2>
> -
> -  <p>
> -    Minimal output includes only actions that modify packet data (not
> -    including OVN registers or metadata such as <code>outport</code>) and
> -    <code>output</code> actions that actually deliver a packet to a
> logical
> -    port (excluding patch ports).  The operands of actions that modify
> packet
> -    data are displayed reduced to constants, e.g. <code>ip4.dst =
> -    reg0;</code> might be show as <code>ip4.dst = 192.168.0.1;</code> if
> that
> -    was the value actually loaded.  This yields output even simpler than
> the
> -    summary format.  (Users familiar with Open vSwitch may recognize this
> as
> -    similar in spirit to the datapath actions listed at the bottom of
> -    <code>ofproto/trace</code> output.)
> -  </p>
> -
> -  <p>
> -    The minimal output format reflects the externally seen behavior of the
> -    logical networks more than it does the implementation.  This makes
> this
> -    output format the most suitable for use in regression tests, because
> it
> -    is least likely to change when logical flow tables are rearranged
> without
> -    semantic change.
> -  </p>
> -
> -  <h1>Stateful Actions</h1>
> -
> -  <p>
> -    Some OVN logical actions use or update state that is not available in
> the
> -    southbound database.  <code>ovn-trace</code> handles these actions as
> -    described below:
> -  </p>
> -
> -  <dl>
> -    <dt><code>ct_next</code></dt>
> -    <dd>
> -      By default <code>ovn-trace</code> treats flows as ``tracked'' and
> -      ``established.''  See the description of the <code>--ct</code>
> option for
> -      a way to override this behavior.
> -    </dd>
> -
> -    <dt><code>ct_dnat</code> (without an argument)</dt>
> -    <dd>
> -      Forks the pipeline.  In one fork, advances to the next table as if
> -      <code>next;</code> were executed.  The packet is not changed, on the
> -      assumption that no NAT state was available.  In the other fork, the
> -      pipeline continues without change after the <code>ct_dnat</code>
> action.
> -    </dd>
> -
> -    <dt><code>ct_snat</code> (without an argument)</dt>
> -    <dd>
> -      This action distinguishes between gateway routers and distributed
> -      routers.  A gateway router is defined as a logical datapath that
> contains
> -      an <code>l3gateway</code> port; any other logical datapath is a
> -      distributed router.  On a gateway router, <code>ct_snat;</code> is
> -      treated as a no-op.  On a distributed router, it is treated the
> same way
> -      as <code>ct_dnat;</code>.
> -    </dd>
> -
> -    <dt><code>ct_dnat(<var>ip</var>)</code></dt>
> -    <dt><code>ct_snat(<var>ip</var>)</code></dt>
> -    <dd>
> -      Forks the pipeline.  In one fork, sets <code>ip4.dst</code> (or
> -      <code>ip4.src</code>) to <var>ip</var> and <code>ct.dnat</code> (or
> -      <code>ct.snat</code>) to 1 and advances to the next table as if
> -      <code>next;</code> were executed.  In the other fork, the pipeline
> -      continues without change after the <code>ct_dnat</code> (or
> -      <code>ct_snat</code>) action.
> -    </dd>
> -
> -    <dt><code>ct_lb;</code></dt>
> -
> <dt><code>ct_lb(<var>ip</var></code>[<code>:<var>port</var></code>]...<code>);</code></dt>
> -    <dd>
> -      Forks the pipeline. In one fork, sets <code>ip4.dst</code> (or
> -      <code>ip6.dst</code>) to one of the load-balancer addresses and the
> -      destination port to its associated port, if any, and sets
> -      <code>ct.dnat</code> to 1.  With one or more arguments, gives
> preference
> -      to the address specified on <code>--lb-dst</code>, if any; without
> -      arguments, uses the address and port specified on
> <code>--lb-dst</code>.
> -      In the other fork, the pipeline continues without change after the
> -      <code>ct_lb</code> action.
> -    </dd>
> -
> -    <dt><code>ct_commit</code></dt>
> -    <dt><code>put_arp</code></dt>
> -    <dt><code>put_nd</code></dt>
> -    <dd>
> -      These actions are treated as no-ops.
> -    </dd>
> -  </dl>
> -
> -  <h1>Daemon Mode</h1>
> -
> -  <p>
> -    If <code>ovn-trace</code> is invoked with the <code>--detach</code>
> option
> -    (see <code>Daemon Options</code>, below), it runs in the background
> as a
> -    daemon and accepts commands from <code>ovs-appctl</code> (or another
> -    JSON-RPC client) indefinitely.  The currently supported commands are
> -    described below.
> -  </p>
> -
> -  <p>
> -
> -  </p>
> -
> -  <dl>
> -    <dt><code>trace</code> [<var>options</var>] <var>datapath</var>
> <var>microflow</var></dt>
> -    <dd>
> -      Traces <var>microflow</var> through <var>datapath</var> and replies
> with
> -      the results of the trace.  Accepts the <var>options</var> described
> under
> -      <code>Trace Options</code> below.
> -    </dd>
> -
> -    <dt><code>exit</code></dt>
> -    <dd>Causes <code>ovn-trace</code> to gracefully terminate.</dd>
> -  </dl>
> -
> -  <h1>Options</h1>
> -
> -  <h2>Trace Options</h2>
> -
> -  <dl>
> -    <dt><code>--detailed</code></dt>
> -    <dt><code>--summary</code></dt>
> -    <dt><code>--minimal</code></dt>
> -    <dd>
> -      These options control the form and level of detail in
> -      <code>ovn-trace</code> output.  If more than one of these options is
> -      specified, all of the selected forms are output, in the order listed
> -      above, each headed by a banner line.  If none of these options is
> -      given, <code>--detailed</code> is the default.  See
> -      <code>Output</code>, above, for a description of each kind of
> output.
> -    </dd>
> -
> -    <dt><code>--all</code></dt>
> -    <dd>
> -      Selects all three forms of output.
> -    </dd>
> -
> -    <dt><code>--ovs</code>[<code>=</code><var>remote</var>]</dt>
> -    <dd>
> -      <p>
> -        Makes <code>ovn-trace</code> attempt to obtain and display the
> OpenFlow
> -        flows that correspond to each OVN logical flow.  To do so,
> -        <code>ovn-trace</code> connects to <var>remote</var> (by default,
> -        <code>unix:@RUNDIR@/br-int.mgmt</code>) over OpenFlow and
> retrieves the
> -        flows.  If <var>remote</var> is specified, it must be an active
> -        OpenFlow connection method described in <code>ovsdb</code>(7).
> -      </p>
> -
> -      <p>
> -        To make the best use of the output, it is important to understand
> the
> -        relationship between logical flows and OpenFlow flows.
> -        <code>ovn-architecture</code>(7), under <em>Architectural
> Physical Life
> -        Cycle of a Packet</em>, describes this relationship.  Keep in
> mind the
> -        following points:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          <code>ovn-trace</code> currently shows all the OpenFlow flows to
> -          which a logical flow corresponds, even though an actual packet
> -          ordinarily matches only one of these.
> -        </li>
> -
> -        <li>
> -          Some logical flows can map to the Open vSwitch ``conjunctive
> match''
> -          extension (see <code>ovs-fields</code>(7)).  Currently
> -          <code>ovn-trace</code> cannot display the flows with
> -          <code>conjunction</code> actions that effectively produce the
> -          <code>conj_id</code> match.
> -        </li>
> -
> -        <li>
> -          Some logical flows may not be represented in the OpenFlow
> tables on a
> -          given hypervisor, if they could not be used on that hypervisor.
> -        </li>
> -
> -        <li>
> -          Some OpenFlow flows do not correspond to logical flows, such as
> -          OpenFlow flows that map between physical and logical ports.
> These
> -          flows will never show up in a trace.
> -        </li>
> -
> -        <li>
> -          When <code>ovn-trace</code> omits uninteresting logical flows
> from
> -          output, it does not look up the corresponding OpenFlow flows.
> -        </li>
> -      </ul>
> -    </dd>
> -
> -    <dt><code>--ct=<var>flags</var></code></dt>
> -    <dd>
> -      <p>
> -        This option sets the <code>ct_state</code> flags that a
> -        <code>ct_next</code> logical action will report.  The
> <var>flags</var>
> -        must be a comma- or space-separated list of the following
> connection
> -        tracking flags:
> -      </p>
> -
> -      <ul>
> -        <li>
> -          <code>trk</code>: Include to indicate connection tracking has
> taken
> -          place.  (This bit is set automatically even if not listed in
> -          <var>flags</var>.
> -        </li>
> -        <li><code>new</code>: Include to indicate a new flow.</li>
> -        <li><code>est</code>: Include to indicate an established
> flow.</li>
> -        <li><code>rel</code>: Include to indicate a related flow.</li>
> -        <li><code>rpl</code>: Include to indicate a reply flow.</li>
> -        <li><code>inv</code>: Include to indicate a connection entry in a
> -        bad state.</li>
> -        <li><code>dnat</code>: Include to indicate a packet whose
> -        destination IP address has been changed.</li>
> -        <li><code>snat</code>: Include to indicate a packet whose source
> IP
> -        address has been changed.</li>
> -      </ul>
> -
> -      <p>
> -        The <code>ct_next</code> action is used to implement the OVN
> -        distributed firewall.  For testing, useful flag combinations
> include:
> -      </p>
> -
> -      <ul>
> -        <li><code>trk,new</code>: A packet in a flow in either direction
> -        through a firewall that has not yet been committed (with
> -        <code>ct_commit</code>).</li>
> -        <li><code>trk,est</code>: A packet in an established flow going
> out
> -        through a firewall.</li>
> -        <li><code>trk,rpl</code>: A packet coming in through a firewall in
> -        reply to an established flow.</li>
> -        <li><code>trk,inv</code>: An invalid packet in either
> direction.</li>
> -      </ul>
> -
> -      <p>
> -        A packet might pass through the connection tracker twice in one
> trip
> -        through OVN: once following egress from a VM as it passes outward
> -        through a firewall, and once preceding ingress to a second VM as
> it
> -        passes inward through a firewall.  Use multiple <code>--ct</code>
> -        options to specify the flags for multiple <code>ct_next</code>
> actions.
> -      </p>
> -
> -      <p>
> -        When <code>--ct</code> is unspecified, or when there are fewer
> -        <code>--ct</code> options than <code>ct_next</code> actions, the
> -        <var>flags</var> default to <code>trk,est</code>.
> -      </p>
> -    </dd>
> -
> -
> <dt><code>--lb-dst=</code><var>ip</var>[<code>:<var>port</var></code>]</dt>
> -    <dd>
> -      Sets the IP from VIP pool to use as destination of the packet.
> -      <code>--lb-dst</code> is not available in daemon mode.
> -    </dd>
> -
> -    <dt><code>--friendly-names</code></dt>
> -    <dt><code>--no-friendly-names</code></dt>
> -    <dd>
> -      When cloud management systems such as OpenStack are layered on top
> of
> -      OVN, they often use long, human-unfriendly names for ports and
> datapaths,
> -      for example, ones that include entire UUIDs.  They do usually
> include
> -      friendlier names, but the long, hard-to-read names are the ones that
> -      appear in matches and actions.  By default, or with
> -      <code>--friendly-names</code>, <code>ovn-trace</code> substitutes
> these
> -      friendlier names for the long names in its output.  Use
> -      <code>--no-friendly-names</code> to disable this behavior; this
> option
> -      might be useful, for example, if a program is going to parse
> -      <code>ovn-trace</code> output.
> -    </dd>
> -  </dl>
> -
> -  <h2>Daemon Options</h2>
> -  <xi:include href="lib/daemon.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -  <h2>Logging Options</h2>
> -  <xi:include href="lib/vlog.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -  <h2>PKI Options</h2>
> -  <p>
> -    PKI configuration is required to use SSL for the connection to the
> -    database (and the switch, if <code>--ovs</code> is specified).
> -  </p>
> -  <xi:include href="lib/ssl.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -  <h2>Other Options</h2>
> -
> -  <dl>
> -    <dt><code>--db</code> <var>database</var></dt>
> -    <dd>
> -      The OVSDB database remote to contact.  If the <env>OVN_SB_DB</env>
> -      environment variable is set, its value is used as the default.
> -      Otherwise, the default is <code>unix:@RUNDIR@/db.sock</code>, but
> this
> -      default is unlikely to be useful outside of single-machine OVN test
> -      environments.
> -    </dd>
> -  </dl>
> -
> -  <xi:include href="lib/common.xml" xmlns:xi="
> http://www.w3.org/2003/XInclude"/>
> -
> -</manpage>
> diff --git a/ovn/utilities/ovn-trace.c b/ovn/utilities/ovn-trace.c
> deleted file mode 100644
> index 044eb1cc2..000000000
> --- a/ovn/utilities/ovn-trace.c
> +++ /dev/null
> @@ -1,2373 +0,0 @@
> -/*
> - * Copyright (c) 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -
> -#include <getopt.h>
> -
> -#include "command-line.h"
> -#include "compiler.h"
> -#include "daemon.h"
> -#include "dirs.h"
> -#include "fatal-signal.h"
> -#include "flow.h"
> -#include "nx-match.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/json.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofp-flow.h"
> -#include "openvswitch/ofp-print.h"
> -#include "openvswitch/vconn.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/actions.h"
> -#include "ovn/expr.h"
> -#include "ovn/lex.h"
> -#include "ovn/logical-fields.h"
> -#include "ovn/lib/acl-log.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/ovn-sb-idl.h"
> -#include "ovn/lib/ovn-util.h"
> -#include "ovsdb-idl.h"
> -#include "openvswitch/poll-loop.h"
> -#include "stream-ssl.h"
> -#include "stream.h"
> -#include "unixctl.h"
> -#include "util.h"
> -#include "random.h"
> -
> -VLOG_DEFINE_THIS_MODULE(ovntrace);
> -
> -/* --db: The database server to contact. */
> -static const char *db;
> -
> -/* --unixctl-path: Path to use for unixctl server, for "monitor" and
> "snoop"
> -     commands. */
> -static char *unixctl_path;
> -
> -/* The southbound database. */
> -static struct ovsdb_idl *ovnsb_idl;
> -
> -/* --detailed: Show a detailed, table-by-table trace. */
> -static bool detailed;
> -
> -/* --summary: Show a trace that omits table information. */
> -static bool summary;
> -
> -/* --minimal: Show a trace with only minimal information. */
> -static bool minimal;
> -
> -/* --ovs: OVS instance to contact to get OpenFlow flows. */
> -static const char *ovs;
> -static struct vconn *vconn;
> -
> -/* --ct: Connection tracking state to use for ct_next() actions. */
> -static uint32_t *ct_states;
> -static size_t n_ct_states;
> -static size_t ct_state_idx;
> -
> -/* --lb-dst: load balancer destination info. */
> -static struct ovnact_ct_lb_dst lb_dst;
> -
> -/* --friendly-names, --no-friendly-names: Whether to substitute
> human-friendly
> - * port and datapath names for the awkward UUIDs typically used in the
> actual
> - * logical flows. */
> -static bool use_friendly_names = true;
> -
> -OVS_NO_RETURN static void usage(void);
> -static void parse_options(int argc, char *argv[]);
> -static char *trace(const char *datapath, const char *flow);
> -static void read_db(void);
> -static unixctl_cb_func ovntrace_exit;
> -static unixctl_cb_func ovntrace_trace;
> -
> -int
> -main(int argc, char *argv[])
> -{
> -    set_program_name(argv[0]);
> -    service_start(&argc, &argv);
> -    fatal_ignore_sigpipe();
> -    vlog_set_levels_from_string_assert("reconnect:warn");
> -
> -    /* Parse command line. */
> -    parse_options(argc, argv);
> -    argc -= optind;
> -    argv += optind;
> -
> -    if (get_detach()) {
> -        if (argc != 0) {
> -            ovs_fatal(0, "non-option arguments not supported with
> --detach "
> -                      "(use --help for help)");
> -        }
> -    } else {
> -        if (argc != 2) {
> -            ovs_fatal(0, "exactly two non-option arguments are required "
> -                      "(use --help for help)");
> -        }
> -    }
> -
> -    struct unixctl_server *server = NULL;
> -    bool exiting = false;
> -    if (get_detach()) {
> -        daemonize_start(false);
> -        int error = unixctl_server_create(unixctl_path, &server);
> -        if (error) {
> -            ovs_fatal(error, "failed to create unixctl server");
> -        }
> -        unixctl_command_register("exit", "", 0, 0, ovntrace_exit,
> &exiting);
> -        unixctl_command_register("trace", "[OPTIONS] DATAPATH MICROFLOW",
> -                                 2, INT_MAX, ovntrace_trace, NULL);
> -    }
> -    ovnsb_idl = ovsdb_idl_create(db, &sbrec_idl_class, true, false);
> -
> -    bool already_read = false;
> -    for (;;) {
> -        ovsdb_idl_run(ovnsb_idl);
> -        unixctl_server_run(server);
> -        if (!ovsdb_idl_is_alive(ovnsb_idl)) {
> -            int retval = ovsdb_idl_get_last_error(ovnsb_idl);
> -            ovs_fatal(0, "%s: database connection failed (%s)",
> -                      db, ovs_retval_to_string(retval));
> -        }
> -
> -        if (ovsdb_idl_has_ever_connected(ovnsb_idl)) {
> -            if (!already_read) {
> -                already_read = true;
> -                read_db();
> -            }
> -
> -            daemonize_complete();
> -            if (!get_detach()) {
> -                char *output = trace(argv[0], argv[1]);
> -                fputs(output, stdout);
> -                free(output);
> -                return 0;
> -            }
> -        }
> -
> -        if (exiting) {
> -            break;
> -        }
> -        ovsdb_idl_wait(ovnsb_idl);
> -        unixctl_server_wait(server);
> -        poll_block();
> -    }
> -}
> -
> -static char *
> -default_ovs(void)
> -{
> -    return xasprintf("unix:%s/br-int.mgmt", ovs_rundir());
> -}
> -
> -static void
> -parse_ct_option(const char *state_s_)
> -{
> -    uint32_t state;
> -    struct ds ds = DS_EMPTY_INITIALIZER;
> -
> -    if (!parse_ct_state(state_s_, CS_TRACKED, &state, &ds)) {
> -        ovs_fatal(0, "%s", ds_cstr(&ds));
> -    }
> -    if (!validate_ct_state(state, &ds)) {
> -        VLOG_WARN("%s", ds_cstr(&ds));
> -    }
> -    ds_destroy(&ds);
> -
> -    ct_states = xrealloc(ct_states, (n_ct_states + 1) * sizeof
> *ct_states);
> -    ct_states[n_ct_states++] = state;
> -}
> -
> -static void
> -parse_lb_option(const char *s)
> -{
> -    struct sockaddr_storage ss;
> -    if (!inet_parse_active(s, 0, &ss, false)) {
> -        ovs_fatal(0, "%s: bad address", s);
> -    }
> -
> -    lb_dst.family = ss.ss_family;
> -    struct in6_addr a = ss_get_address(&ss);
> -    if (ss.ss_family == AF_INET) {
> -        lb_dst.ipv4 = in6_addr_get_mapped_ipv4(&a);
> -    } else {
> -        lb_dst.ipv6 = a;
> -    }
> -    lb_dst.port = ss_get_port(&ss);
> -}
> -
> -static void
> -parse_options(int argc, char *argv[])
> -{
> -    enum {
> -        OPT_DB = UCHAR_MAX + 1,
> -        OPT_UNIXCTL,
> -        OPT_DETAILED,
> -        OPT_SUMMARY,
> -        OPT_MINIMAL,
> -        OPT_ALL,
> -        OPT_OVS,
> -        OPT_CT,
> -        OPT_FRIENDLY_NAMES,
> -        OPT_NO_FRIENDLY_NAMES,
> -        DAEMON_OPTION_ENUMS,
> -        SSL_OPTION_ENUMS,
> -        VLOG_OPTION_ENUMS,
> -        OPT_LB_DST
> -    };
> -    static const struct option long_options[] = {
> -        {"db", required_argument, NULL, OPT_DB},
> -        {"unixctl", required_argument, NULL, OPT_UNIXCTL},
> -        {"detailed", no_argument, NULL, OPT_DETAILED},
> -        {"summary", no_argument, NULL, OPT_SUMMARY},
> -        {"minimal", no_argument, NULL, OPT_MINIMAL},
> -        {"all", no_argument, NULL, OPT_ALL},
> -        {"ovs", optional_argument, NULL, OPT_OVS},
> -        {"ct", required_argument, NULL, OPT_CT},
> -        {"friendly-names", no_argument, NULL, OPT_FRIENDLY_NAMES},
> -        {"no-friendly-names", no_argument, NULL, OPT_NO_FRIENDLY_NAMES},
> -        {"help", no_argument, NULL, 'h'},
> -        {"version", no_argument, NULL, 'V'},
> -        {"lb-dst", required_argument, NULL, OPT_LB_DST},
> -        DAEMON_LONG_OPTIONS,
> -        VLOG_LONG_OPTIONS,
> -        STREAM_SSL_LONG_OPTIONS,
> -        {NULL, 0, NULL, 0},
> -    };
> -    char *short_options =
> ovs_cmdl_long_options_to_short_options(long_options);
> -
> -    for (;;) {
> -        int idx;
> -        int c;
> -
> -        c = getopt_long(argc, argv, short_options, long_options, &idx);
> -        if (c == -1) {
> -            break;
> -        }
> -
> -        switch (c) {
> -        case OPT_DB:
> -            db = optarg;
> -            break;
> -
> -        case OPT_UNIXCTL:
> -            unixctl_path = optarg;
> -            break;
> -
> -        case OPT_DETAILED:
> -            detailed = true;
> -            break;
> -
> -        case OPT_SUMMARY:
> -            summary = true;
> -            break;
> -
> -        case OPT_MINIMAL:
> -            minimal = true;
> -            break;
> -
> -        case OPT_ALL:
> -            detailed = summary = minimal = true;
> -            break;
> -
> -        case OPT_OVS:
> -            ovs = optarg ? optarg : default_ovs();
> -            break;
> -
> -        case OPT_CT:
> -            parse_ct_option(optarg);
> -            break;
> -
> -        case OPT_FRIENDLY_NAMES:
> -            use_friendly_names = true;
> -            break;
> -
> -        case OPT_NO_FRIENDLY_NAMES:
> -            use_friendly_names = false;
> -            break;
> -
> -        case OPT_LB_DST:
> -            parse_lb_option(optarg);
> -            break;
> -
> -        case 'h':
> -            usage();
> -
> -        case 'V':
> -            ovs_print_version(0, 0);
> -            printf("DB Schema %s\n", sbrec_get_db_version());
> -            exit(EXIT_SUCCESS);
> -
> -        DAEMON_OPTION_HANDLERS
> -        VLOG_OPTION_HANDLERS
> -        STREAM_SSL_OPTION_HANDLERS
> -
> -        case '?':
> -            exit(EXIT_FAILURE);
> -
> -        default:
> -            abort();
> -        }
> -    }
> -    free(short_options);
> -
> -    if (!db) {
> -        db = default_sb_db();
> -    }
> -
> -    if (!detailed && !summary && !minimal) {
> -        detailed = true;
> -    }
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN trace utility\n\
> -usage: %s [OPTIONS] DATAPATH MICROFLOW\n\
> -       %s [OPTIONS] --detach\n\
> -\n\
> -Output format options:\n\
> -  --detailed              table-by-table \"backtrace\" (default)\n\
> -  --summary               less detailed, more parseable\n\
> -  --minimal               minimum to explain externally visible
> behavior\n\
> -  --all                   provide all forms of output\n\
> -Output style options:\n\
> -  --no-friendly-names     do not substitute human friendly names for
> UUIDs\n",
> -           program_name, program_name, program_name);
> -    daemon_usage();
> -    vlog_usage();
> -    printf("\n\
> -Other options:\n\
> -  --db=DATABASE           connect to DATABASE\n\
> -                          (default: %s)\n\
> -  --ovs[=REMOTE]          obtain corresponding OpenFlow flows from
> REMOTE\n\
> -                          (default: %s)\n\
> -  --unixctl=SOCKET        set control socket name\n\
> -  -h, --help              display this help message\n\
> -  -V, --version           display version information\n",
> -           default_sb_db(), default_ovs());
> -    stream_usage("database", true, true, false);
> -    vconn_usage(true, false, false);
> -    exit(EXIT_SUCCESS);
> -}
> -
> -struct ovntrace_datapath {
> -    struct hmap_node sb_uuid_node;
> -    struct uuid sb_uuid;
> -    struct uuid nb_uuid;
> -    char *name;
> -    char *name2;
> -    char *friendly_name;
> -    uint32_t tunnel_key;
> -
> -    struct ovs_list mcgroups;   /* Contains "struct ovntrace_mcgroup"s. */
> -
> -    struct ovntrace_flow **flows;
> -    size_t n_flows, allocated_flows;
> -
> -    struct hmap mac_bindings;   /* Contains "struct
> ovntrace_mac_binding"s. */
> -
> -    bool has_local_l3gateway;
> -};
> -
> -struct ovntrace_port {
> -    struct ovntrace_datapath *dp;
> -    struct uuid uuid;
> -    char *name;
> -    char *name2;
> -    const char *friendly_name;
> -    char *type;
> -    uint16_t tunnel_key;
> -    struct ovntrace_port *peer; /* Patch ports only. */
> -    struct ovntrace_port *distributed_port; /* chassisredirect ports
> only. */
> -};
> -
> -struct ovntrace_mcgroup {
> -    struct ovs_list list_node;  /* In struct ovntrace_datapath's
> 'mcgroups'. */
> -
> -    struct ovntrace_datapath *dp;
> -    char *name;
> -
> -    uint16_t tunnel_key;
> -
> -    struct ovntrace_port **ports;
> -    size_t n_ports;
> -};
> -
> -struct ovntrace_flow {
> -    struct uuid uuid;
> -    enum ovnact_pipeline pipeline;
> -    int table_id;
> -    char *stage_name;
> -    char *source;
> -    int priority;
> -    char *match_s;
> -    struct expr *match;
> -    struct ovnact *ovnacts;
> -    size_t ovnacts_len;
> -};
> -
> -struct ovntrace_mac_binding {
> -    struct hmap_node node;
> -    uint16_t port_key;
> -    struct in6_addr ip;
> -    struct eth_addr mac;
> -};
> -
> -static inline uint32_t
> -hash_mac_binding(uint16_t port_key, const struct in6_addr *ip)
> -{
> -    return hash_bytes(ip, sizeof *ip, port_key);
> -}
> -
> -/* Every ovntrace_datapath, by southbound Datapath_Binding record UUID. */
> -static struct hmap datapaths;
> -
> -/* Every ovntrace_port, by name. */
> -static struct shash ports;
> -
> -/* Symbol table for expressions and actions. */
> -static struct shash symtab;
> -
> -/* Address sets. */
> -static struct shash address_sets;
> -
> -/* Port groups. */
> -static struct shash port_groups;
> -
> -/* DHCP options. */
> -static struct hmap dhcp_opts;   /* Contains "struct gen_opts_map"s. */
> -static struct hmap dhcpv6_opts; /* Contains "struct gen_opts_map"s. */
> -static struct hmap nd_ra_opts; /* Contains "struct gen_opts_map"s. */
> -
> -static struct ovntrace_datapath *
> -ovntrace_datapath_find_by_sb_uuid(const struct uuid *sb_uuid)
> -{
> -    struct ovntrace_datapath *dp;
> -    HMAP_FOR_EACH_WITH_HASH (dp, sb_uuid_node, uuid_hash(sb_uuid),
> -                             &datapaths) {
> -        if (uuid_equals(&dp->sb_uuid, sb_uuid)) {
> -            return dp;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static const struct ovntrace_datapath *
> -ovntrace_datapath_find_by_name(const char *name)
> -{
> -    struct ovntrace_datapath *dp;
> -    HMAP_FOR_EACH (dp, sb_uuid_node, &datapaths) {
> -        if (!strcmp(name, dp->name)
> -            || (dp->name2 && !strcmp(name, dp->name2))) {
> -            return dp;
> -        }
> -    }
> -
> -    struct ovntrace_datapath *match = NULL;
> -    HMAP_FOR_EACH (dp, sb_uuid_node, &datapaths) {
> -        if (uuid_is_partial_match(&dp->sb_uuid, name) >= 4 ||
> -            uuid_is_partial_match(&dp->nb_uuid, name) >= 4) {
> -            if (match) {
> -                VLOG_WARN("name \"%s\" matches multiple datapaths", name);
> -                return NULL;
> -            }
> -            match = dp;
> -        }
> -    }
> -    return match;
> -}
> -
> -static const struct ovntrace_port *
> -ovntrace_port_find_by_key(const struct ovntrace_datapath *dp,
> -                          uint16_t tunnel_key)
> -{
> -    const struct shash_node *node;
> -    SHASH_FOR_EACH (node, &ports) {
> -        const struct ovntrace_port *port = node->data;
> -        if (port->dp == dp && port->tunnel_key == tunnel_key) {
> -            return port;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static const char *
> -ovntrace_port_key_to_name(const struct ovntrace_datapath *dp,
> -                          uint16_t key)
> -{
> -    const struct ovntrace_port *port = ovntrace_port_find_by_key(dp, key);
> -    return (port ? port->friendly_name
> -            : !key ? ""
> -            : "(unnamed)");
> -}
> -
> -static const struct ovntrace_mcgroup *
> -ovntrace_mcgroup_find_by_key(const struct ovntrace_datapath *dp,
> -                             uint16_t tunnel_key)
> -{
> -    const struct ovntrace_mcgroup *mcgroup;
> -    LIST_FOR_EACH (mcgroup, list_node, &dp->mcgroups) {
> -        if (mcgroup->tunnel_key == tunnel_key) {
> -            return mcgroup;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static const struct ovntrace_mcgroup *
> -ovntrace_mcgroup_find_by_name(const struct ovntrace_datapath *dp,
> -                              const char *name)
> -{
> -    const struct ovntrace_mcgroup *mcgroup;
> -    LIST_FOR_EACH (mcgroup, list_node, &dp->mcgroups) {
> -        if (!strcmp(mcgroup->name, name)) {
> -            return mcgroup;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static const struct ovntrace_mac_binding *
> -ovntrace_mac_binding_find(const struct ovntrace_datapath *dp,
> -                          uint16_t port_key, const struct in6_addr *ip)
> -{
> -    const struct ovntrace_mac_binding *bind;
> -    HMAP_FOR_EACH_WITH_HASH (bind, node, hash_mac_binding(port_key, ip),
> -                             &dp->mac_bindings) {
> -        if (bind->port_key == port_key && ipv6_addr_equals(ip,
> &bind->ip)) {
> -            return bind;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -/* If 's' ends with a UUID, returns a copy of it with the UUID truncated
> to
> - * just the first 6 characters; otherwise, returns a copy of 's'. */
> -static char *
> -shorten_uuid(const char *s)
> -{
> -    size_t len = strlen(s);
> -    return (len >= UUID_LEN && uuid_is_partial_string(s + (len -
> UUID_LEN))
> -            ? xmemdup0(s, len - (UUID_LEN - 6))
> -            : xstrdup(s));
> -}
> -
> -static void
> -read_datapaths(void)
> -{
> -    hmap_init(&datapaths);
> -    const struct sbrec_datapath_binding *sbdb;
> -    SBREC_DATAPATH_BINDING_FOR_EACH (sbdb, ovnsb_idl) {
> -        struct ovntrace_datapath *dp = xzalloc(sizeof *dp);
> -        const struct smap *ids = &sbdb->external_ids;
> -
> -        dp->sb_uuid = sbdb->header_.uuid;
> -        if (!smap_get_uuid(ids, "logical-switch", &dp->nb_uuid) &&
> -            !smap_get_uuid(ids, "logical-router", &dp->nb_uuid)) {
> -            dp->nb_uuid = dp->sb_uuid;
> -        }
> -
> -        const char *name = smap_get(ids, "name");
> -        dp->name = (name
> -                    ? xstrdup(name)
> -                    : xasprintf(UUID_FMT, UUID_ARGS(&dp->nb_uuid)));
> -
> -        dp->name2 = nullable_xstrdup(smap_get(ids, "name2"));
> -        dp->friendly_name = (!use_friendly_names
> -                             ? xasprintf(UUID_FMT,
> UUID_ARGS(&dp->nb_uuid))
> -                             : shorten_uuid(dp->name2 ? dp->name2 :
> dp->name));
> -
> -        dp->tunnel_key = sbdb->tunnel_key;
> -
> -        ovs_list_init(&dp->mcgroups);
> -        hmap_init(&dp->mac_bindings);
> -
> -        hmap_insert(&datapaths, &dp->sb_uuid_node,
> uuid_hash(&dp->sb_uuid));
> -    }
> -}
> -
> -static void
> -read_ports(void)
> -{
> -    shash_init(&ports);
> -    const struct sbrec_port_binding *sbpb;
> -    SBREC_PORT_BINDING_FOR_EACH (sbpb, ovnsb_idl) {
> -        const char *port_name = sbpb->logical_port;
> -        struct ovntrace_datapath *dp
> -            =
> ovntrace_datapath_find_by_sb_uuid(&sbpb->datapath->header_.uuid);
> -        if (!dp) {
> -            VLOG_WARN("logical port %s missing datapath", port_name);
> -            continue;
> -        }
> -
> -        struct ovntrace_port *port = xzalloc(sizeof *port);
> -        if (!shash_add_once(&ports, port_name, port)) {
> -            VLOG_WARN("duplicate logical port name %s", port_name);
> -            free(port);
> -            continue;
> -        }
> -        port->dp = dp;
> -        port->uuid = sbpb->header_.uuid;
> -        port->name = xstrdup(port_name);
> -        port->type = xstrdup(sbpb->type);
> -        port->tunnel_key = sbpb->tunnel_key;
> -
> -        port->name2 = nullable_xstrdup(smap_get(&sbpb->external_ids,
> "name"));
> -        port->friendly_name = (!use_friendly_names ? xstrdup(port->name)
> -                               : shorten_uuid(port->name2
> -                                              ? port->name2 :
> port->name));
> -
> -        if (!strcmp(sbpb->type, "patch")) {
> -            const char *peer_name = smap_get(&sbpb->options, "peer");
> -            if (peer_name) {
> -                struct ovntrace_port *peer
> -                    = shash_find_data(&ports, peer_name);
> -                if (peer) {
> -                    port->peer = peer;
> -                    port->peer->peer = port;
> -                }
> -            }
> -        } else if (!strcmp(sbpb->type, "l3gateway")) {
> -            /* Treat all gateways as local for our purposes. */
> -            dp->has_local_l3gateway = true;
> -            const char *peer_name = smap_get(&sbpb->options, "peer");
> -            if (peer_name) {
> -                struct ovntrace_port *peer
> -                    = shash_find_data(&ports, peer_name);
> -                if (peer) {
> -                    port->peer = peer;
> -                    port->peer->peer = port;
> -                }
> -            }
> -        }
> -    }
> -
> -    SBREC_PORT_BINDING_FOR_EACH (sbpb, ovnsb_idl) {
> -        if (!strcmp(sbpb->type, "chassisredirect")) {
> -            struct ovntrace_port *port
> -                = shash_find_data(&ports, sbpb->logical_port);
> -            if (port) {
> -                const char *distributed_name = smap_get(&sbpb->options,
> -
>  "distributed-port");
> -                if (distributed_name) {
> -                    struct ovntrace_port *distributed_port
> -                        = shash_find_data(&ports, distributed_name);
> -                    if (distributed_port && distributed_port->dp ==
> port->dp) {
> -                        port->distributed_port = distributed_port;
> -                    }
> -                }
> -            }
> -        }
> -    }
> -}
> -
> -static int
> -compare_port(const void *a_, const void *b_)
> -{
> -    struct ovntrace_port *const *ap = a_;
> -    struct ovntrace_port *const *bp = b_;
> -    const struct ovntrace_port *a = *ap;
> -    const struct ovntrace_port *b = *bp;
> -
> -    return strcmp(a->name, b->name);
> -}
> -
> -static void
> -read_mcgroups(void)
> -{
> -    const struct sbrec_multicast_group *sbmg;
> -    SBREC_MULTICAST_GROUP_FOR_EACH (sbmg, ovnsb_idl) {
> -        struct ovntrace_datapath *dp
> -            =
> ovntrace_datapath_find_by_sb_uuid(&sbmg->datapath->header_.uuid);
> -        if (!dp) {
> -            VLOG_WARN("logical multicast group %s missing datapath",
> -                      sbmg->name);
> -            continue;
> -        }
> -
> -        struct ovntrace_mcgroup *mcgroup = xzalloc(sizeof *mcgroup);
> -        ovs_list_push_back(&dp->mcgroups, &mcgroup->list_node);
> -        mcgroup->dp = dp;
> -        mcgroup->tunnel_key = sbmg->tunnel_key;
> -        mcgroup->name = xstrdup(sbmg->name);
> -        mcgroup->ports = xmalloc(sbmg->n_ports * sizeof *mcgroup->ports);
> -        for (size_t i = 0; i < sbmg->n_ports; i++) {
> -            const char *port_name = sbmg->ports[i]->logical_port;
> -            struct ovntrace_port *p = shash_find_data(&ports, port_name);
> -            if (!p) {
> -                VLOG_WARN("missing port %s", port_name);
> -                continue;
> -            }
> -            if (!uuid_equals(&sbmg->ports[i]->datapath->header_.uuid,
> -                             &p->dp->sb_uuid)) {
> -                VLOG_WARN("multicast group %s in datapath %s contains "
> -                          "port %s outside that datapath",
> -                          mcgroup->name, mcgroup->dp->name, port_name);
> -                continue;
> -            }
> -            mcgroup->ports[mcgroup->n_ports++] = p;
> -        }
> -
> -        /* Sort the ports in alphabetical order to make output more
> -         * predictable. */
> -        qsort(mcgroup->ports, mcgroup->n_ports, sizeof *mcgroup->ports,
> -              compare_port);
> -    }
> -}
> -
> -static void
> -read_address_sets(void)
> -{
> -    shash_init(&address_sets);
> -
> -    const struct sbrec_address_set *sbas;
> -    SBREC_ADDRESS_SET_FOR_EACH (sbas, ovnsb_idl) {
> -        expr_const_sets_add(&address_sets, sbas->name,
> -                           (const char *const *) sbas->addresses,
> -                           sbas->n_addresses, true);
> -    }
> -}
> -
> -static void
> -read_port_groups(void)
> -{
> -    shash_init(&port_groups);
> -
> -    const struct sbrec_port_group *sbpg;
> -    SBREC_PORT_GROUP_FOR_EACH (sbpg, ovnsb_idl) {
> -        expr_const_sets_add(&port_groups, sbpg->name,
> -                           (const char *const *) sbpg->ports,
> -                           sbpg->n_ports, false);
> -    }
> -}
> -
> -static bool
> -ovntrace_is_chassis_resident(const void *aux OVS_UNUSED,
> -                             const char *port_name)
> -{
> -    if (port_name[0] == '\0') {
> -        return true;
> -    }
> -
> -    const struct ovntrace_port *port = shash_find_data(&ports, port_name);
> -    if (port) {
> -        /* Since ovntrace is not chassis specific, assume any port
> -         * that exists is resident. */
> -        return true;
> -    }
> -
> -    VLOG_WARN("%s: unknown logical port\n", port_name);
> -    return false;
> -}
> -
> -static int
> -compare_flow(const void *a_, const void *b_)
> -{
> -    struct ovntrace_flow *const *ap = a_;
> -    struct ovntrace_flow *const *bp = b_;
> -    const struct ovntrace_flow *a = *ap;
> -    const struct ovntrace_flow *b = *bp;
> -
> -    if (a->pipeline != b->pipeline) {
> -        /* Sort OVNACT_P_INGRESS before OVNACT_P_EGRESS. */
> -        return a->pipeline == OVNACT_P_EGRESS ? 1 : -1;
> -    } else if (a->table_id != b->table_id) {
> -        /* Sort in increasing order of table_id. */
> -        return a->table_id > b->table_id ? 1 : -1;
> -    } else if (a->priority != b->priority) {
> -        /* Sort in decreasing order of priority. */
> -        return a->priority > b->priority ? -1 : 1;
> -    } else {
> -        /* Otherwise who cares. */
> -        return 0;
> -    }
> -}
> -
> -static char *
> -ovntrace_make_names_friendly(const char *in)
> -{
> -    if (!use_friendly_names) {
> -        return xstrdup(in);
> -    }
> -
> -    struct ds out = DS_EMPTY_INITIALIZER;
> -    while (*in) {
> -        struct lex_token token;
> -        const char *start;
> -        const char *next;
> -
> -        next = lex_token_parse(&token, in, &start);
> -        if (token.type == LEX_T_STRING) {
> -            const struct ovntrace_port *port = shash_find_data(&ports,
> -                                                               token.s);
> -            if (port) {
> -                ds_put_buffer(&out, in, start - in);
> -                json_string_escape(port->friendly_name, &out);
> -            } else {
> -                ds_put_buffer(&out, in, next - in);
> -            }
> -        } else if (token.type != LEX_T_END) {
> -            ds_put_buffer(&out, in, next - in);
> -        } else {
> -            break;
> -        }
> -        lex_token_destroy(&token);
> -        in = next;
> -    }
> -    return ds_steal_cstr(&out);
> -}
> -
> -static void
> -read_flows(void)
> -{
> -    ovn_init_symtab(&symtab);
> -
> -    const struct sbrec_logical_flow *sblf;
> -    SBREC_LOGICAL_FLOW_FOR_EACH (sblf, ovnsb_idl) {
> -        const struct sbrec_datapath_binding *sbdb =
> sblf->logical_datapath;
> -        struct ovntrace_datapath *dp
> -            = ovntrace_datapath_find_by_sb_uuid(&sbdb->header_.uuid);
> -        if (!dp) {
> -            VLOG_WARN("logical flow missing datapath");
> -            continue;
> -        }
> -
> -        char *error;
> -        struct expr *match;
> -        match = expr_parse_string(sblf->match, &symtab, &address_sets,
> -                                  &port_groups, NULL, &error);
> -        if (error) {
> -            VLOG_WARN("%s: parsing expression failed (%s)",
> -                      sblf->match, error);
> -            free(error);
> -            continue;
> -        }
> -
> -        struct ovnact_parse_params pp = {
> -            .symtab = &symtab,
> -            .dhcp_opts = &dhcp_opts,
> -            .dhcpv6_opts = &dhcpv6_opts,
> -            .nd_ra_opts = &nd_ra_opts,
> -            .pipeline = (!strcmp(sblf->pipeline, "ingress")
> -                         ? OVNACT_P_INGRESS
> -                         : OVNACT_P_EGRESS),
> -            .n_tables = 24,
> -            .cur_ltable = sblf->table_id,
> -        };
> -        uint64_t stub[1024 / 8];
> -        struct ofpbuf ovnacts = OFPBUF_STUB_INITIALIZER(stub);
> -        struct expr *prereqs;
> -        error = ovnacts_parse_string(sblf->actions, &pp, &ovnacts,
> &prereqs);
> -        if (error) {
> -            VLOG_WARN("%s: parsing actions failed (%s)", sblf->actions,
> error);
> -            free(error);
> -            expr_destroy(match);
> -            continue;
> -        }
> -
> -        match = expr_combine(EXPR_T_AND, match, prereqs);
> -        match = expr_annotate(match, &symtab, &error);
> -        if (error) {
> -            VLOG_WARN("match annotation failed (%s)", error);
> -            free(error);
> -            expr_destroy(match);
> -            ovnacts_free(ovnacts.data, ovnacts.size);
> -            ofpbuf_uninit(&ovnacts);
> -            continue;
> -        }
> -        if (match) {
> -            match = expr_simplify(match, ovntrace_is_chassis_resident,
> NULL);
> -        }
> -
> -        struct ovntrace_flow *flow = xzalloc(sizeof *flow);
> -        flow->uuid = sblf->header_.uuid;
> -        flow->pipeline = (!strcmp(sblf->pipeline, "ingress")
> -                          ? OVNACT_P_INGRESS
> -                          : OVNACT_P_EGRESS);
> -        flow->table_id = sblf->table_id;
> -        flow->stage_name = nullable_xstrdup(smap_get(&sblf->external_ids,
> -                                                     "stage-name"));
> -        flow->source = nullable_xstrdup(smap_get(&sblf->external_ids,
> -                                                 "source"));
> -        flow->priority = sblf->priority;
> -        flow->match_s = ovntrace_make_names_friendly(sblf->match);
> -        flow->match = match;
> -        flow->ovnacts_len = ovnacts.size;
> -        flow->ovnacts = ofpbuf_steal_data(&ovnacts);
> -
> -        if (dp->n_flows >= dp->allocated_flows) {
> -            dp->flows = x2nrealloc(dp->flows, &dp->allocated_flows,
> -                                   sizeof *dp->flows);
> -        }
> -        dp->flows[dp->n_flows++] = flow;
> -    }
> -
> -    const struct ovntrace_datapath *dp;
> -    HMAP_FOR_EACH (dp, sb_uuid_node, &datapaths) {
> -        qsort(dp->flows, dp->n_flows, sizeof *dp->flows, compare_flow);
> -    }
> -}
> -
> -static void
> -read_gen_opts(void)
> -{
> -    hmap_init(&dhcp_opts);
> -    const struct sbrec_dhcp_options *sdo;
> -    SBREC_DHCP_OPTIONS_FOR_EACH (sdo, ovnsb_idl) {
> -        dhcp_opt_add(&dhcp_opts, sdo->name, sdo->code, sdo->type);
> -    }
> -
> -
> -    hmap_init(&dhcpv6_opts);
> -    const struct sbrec_dhcpv6_options *sdo6;
> -    SBREC_DHCPV6_OPTIONS_FOR_EACH(sdo6, ovnsb_idl) {
> -       dhcp_opt_add(&dhcpv6_opts, sdo6->name, sdo6->code, sdo6->type);
> -    }
> -
> -    hmap_init(&nd_ra_opts);
> -    nd_ra_opts_init(&nd_ra_opts);
> -}
> -
> -static void
> -read_mac_bindings(void)
> -{
> -    const struct sbrec_mac_binding *sbmb;
> -    SBREC_MAC_BINDING_FOR_EACH (sbmb, ovnsb_idl) {
> -        const struct ovntrace_port *port = shash_find_data(
> -            &ports, sbmb->logical_port);
> -        if (!port) {
> -            VLOG_WARN("missing port %s", sbmb->logical_port);
> -            continue;
> -        }
> -
> -        if (!uuid_equals(&port->dp->sb_uuid,
> &sbmb->datapath->header_.uuid)) {
> -            VLOG_WARN("port %s is in wrong datapath", sbmb->logical_port);
> -            continue;
> -        }
> -
> -        struct in6_addr ip6;
> -        ovs_be32 ip4;
> -        if (ip_parse(sbmb->ip, &ip4)) {
> -            ip6 = in6_addr_mapped_ipv4(ip4);
> -        } else if (!ipv6_parse(sbmb->ip, &ip6)) {
> -            VLOG_WARN("%s: bad IP address", sbmb->ip);
> -            continue;
> -        }
> -
> -        struct eth_addr mac;
> -        if (!eth_addr_from_string(sbmb->mac, &mac)) {
> -            VLOG_WARN("%s: bad Ethernet address", sbmb->mac);
> -            continue;
> -        }
> -
> -        struct ovntrace_mac_binding *binding = xmalloc(sizeof *binding);
> -        binding->port_key = port->tunnel_key;
> -        binding->ip = ip6;
> -        binding->mac = mac;
> -        hmap_insert(&port->dp->mac_bindings, &binding->node,
> -                    hash_mac_binding(binding->port_key, &ip6));
> -    }
> -}
> -
> -static void
> -read_db(void)
> -{
> -    read_datapaths();
> -    read_ports();
> -    read_mcgroups();
> -    read_address_sets();
> -    read_port_groups();
> -    read_gen_opts();
> -    read_flows();
> -    read_mac_bindings();
> -}
> -
> -static const struct ovntrace_port *
> -ovntrace_port_lookup_by_name(const char *name)
> -{
> -    const struct ovntrace_port *port = shash_find_data(&ports, name);
> -    if (port) {
> -        return port;
> -    }
> -
> -    const struct ovntrace_port *match = NULL;
> -
> -    struct shash_node *node;
> -    SHASH_FOR_EACH (node, &ports) {
> -        port = node->data;
> -
> -        if (port->name2 && !strcmp(port->name2, name)) {
> -            if (match) {
> -                VLOG_WARN("name \"%s\" matches multiple ports", name);
> -                return NULL;
> -            }
> -            match = port;
> -        }
> -    }
> -
> -    if (uuid_is_partial_string(name) >= 4) {
> -        SHASH_FOR_EACH (node, &ports) {
> -            port = node->data;
> -
> -            struct uuid name_uuid;
> -            if (uuid_is_partial_match(&port->uuid, name)
> -                || (uuid_from_string(&name_uuid, port->name)
> -                    && uuid_is_partial_match(&name_uuid, name))) {
> -                if (match && match != port) {
> -                    VLOG_WARN("name \"%s\" matches multiple ports", name);
> -                    return NULL;
> -                }
> -                match = port;
> -            }
> -        }
> -    }
> -
> -    return match;
> -}
> -
> -static bool
> -ovntrace_lookup_port(const void *dp_, const char *port_name,
> -                     unsigned int *portp)
> -{
> -    const struct ovntrace_datapath *dp = dp_;
> -
> -    if (port_name[0] == '\0') {
> -        *portp = 0;
> -        return true;
> -    }
> -
> -    const struct ovntrace_port *port =
> ovntrace_port_lookup_by_name(port_name);
> -    if (port) {
> -        if (port->dp == dp) {
> -            *portp = port->tunnel_key;
> -            return true;
> -        }
> -        /* The port is found but not in this datapath. It happens when
> port
> -         * group is used in match conditions. */
> -        return false;
> -    }
> -
> -    const struct ovntrace_mcgroup *mcgroup =
> ovntrace_mcgroup_find_by_name(dp, port_name);
> -    if (mcgroup) {
> -        *portp = mcgroup->tunnel_key;
> -        return true;
> -    }
> -
> -    VLOG_WARN("%s: unknown logical port", port_name);
> -    return false;
> -}
> -
> -static const struct ovntrace_flow *
> -ovntrace_flow_lookup(const struct ovntrace_datapath *dp,
> -                     const struct flow *uflow,
> -                     uint8_t table_id, enum ovnact_pipeline pipeline)
> -{
> -    for (size_t i = 0; i < dp->n_flows; i++) {
> -        const struct ovntrace_flow *flow = dp->flows[i];
> -        if (flow->pipeline == pipeline &&
> -            flow->table_id == table_id &&
> -            expr_evaluate(flow->match, uflow, ovntrace_lookup_port, dp)) {
> -            return flow;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -static char *
> -ovntrace_stage_name(const struct ovntrace_datapath *dp,
> -                    uint8_t table_id, enum ovnact_pipeline pipeline)
> -{
> -    for (size_t i = 0; i < dp->n_flows; i++) {
> -        const struct ovntrace_flow *flow = dp->flows[i];
> -        if (flow->pipeline == pipeline && flow->table_id == table_id) {
> -            return xstrdup(flow->stage_name);;
> -        }
> -    }
> -    return NULL;
> -}
> -
> -/* Type of a node within a trace. */
> -enum ovntrace_node_type {
> -    /* Nodes that may have children (nonterminal nodes). */
> -    OVNTRACE_NODE_OUTPUT,
> -    OVNTRACE_NODE_MODIFY,
> -    OVNTRACE_NODE_ACTION,
> -    OVNTRACE_NODE_ERROR,
> -
> -    /* Nodes that never have children (terminal nodes). */
> -    OVNTRACE_NODE_PIPELINE,
> -    OVNTRACE_NODE_TABLE,
> -    OVNTRACE_NODE_TRANSFORMATION
> -};
> -
> -static bool
> -ovntrace_node_type_is_terminal(enum ovntrace_node_type type)
> -{
> -    switch (type) {
> -    case OVNTRACE_NODE_OUTPUT:
> -    case OVNTRACE_NODE_MODIFY:
> -    case OVNTRACE_NODE_ACTION:
> -    case OVNTRACE_NODE_ERROR:
> -        return true;
> -
> -    case OVNTRACE_NODE_PIPELINE:
> -    case OVNTRACE_NODE_TABLE:
> -    case OVNTRACE_NODE_TRANSFORMATION:
> -        return false;
> -    }
> -
> -    OVS_NOT_REACHED();
> -}
> -
> -struct ovntrace_node {
> -    struct ovs_list node;       /* In parent. */
> -
> -    enum ovntrace_node_type type;
> -    char *name;
> -    bool always_indent;
> -    struct ovs_list subs;       /* List of children. */
> -};
> -
> -static void ovntrace_node_destroy(struct ovntrace_node *);
> -
> -static struct ovntrace_node * OVS_PRINTF_FORMAT(3, 4)
> -ovntrace_node_append(struct ovs_list *super, enum ovntrace_node_type type,
> -                     const char *format, ...)
> -{
> -    va_list args;
> -    va_start(args, format);
> -    char *s = xvasprintf(format, args);
> -    va_end(args);
> -
> -    struct ovntrace_node *node = xmalloc(sizeof *node);
> -    ovs_list_push_back(super, &node->node);
> -    node->type = type;
> -    node->name = s;
> -    node->always_indent = false;
> -    ovs_list_init(&node->subs);
> -
> -    return node;
> -}
> -
> -static void
> -ovntrace_node_clone(const struct ovs_list *old, struct ovs_list *new)
> -{
> -    const struct ovntrace_node *osub;
> -    LIST_FOR_EACH (osub, node, old) {
> -        struct ovntrace_node *nsub = ovntrace_node_append(new, osub->type,
> -                                                          "%s",
> osub->name);
> -        nsub->always_indent = osub->always_indent;
> -        ovntrace_node_clone(&osub->subs, &nsub->subs);
> -    }
> -}
> -
> -static void
> -ovntrace_node_list_destroy(struct ovs_list *list)
> -{
> -    if (list) {
> -        struct ovntrace_node *node, *next;
> -
> -        LIST_FOR_EACH_SAFE (node, next, node, list) {
> -            ovs_list_remove(&node->node);
> -            ovntrace_node_destroy(node);
> -        }
> -    }
> -}
> -
> -static void
> -ovntrace_node_destroy(struct ovntrace_node *node)
> -{
> -    if (node) {
> -        ovntrace_node_list_destroy(&node->subs);
> -        free(node->name);
> -        free(node);
> -    }
> -}
> -
> -static void
> -ovntrace_node_print_details(struct ds *output,
> -                            const struct ovs_list *nodes, int level)
> -{
> -    const struct ovntrace_node *sub;
> -    LIST_FOR_EACH (sub, node, nodes) {
> -        if (sub->type == OVNTRACE_NODE_MODIFY) {
> -            continue;
> -        }
> -
> -        bool more = (sub->node.next != nodes
> -                     || sub->always_indent
> -                     || ovntrace_node_type_is_terminal(sub->type));
> -        bool title = (sub->type == OVNTRACE_NODE_PIPELINE ||
> -                      sub->type == OVNTRACE_NODE_TRANSFORMATION);
> -        if (title) {
> -            ds_put_char(output, '\n');
> -        }
> -        ds_put_char_multiple(output, ' ', (level + more) * 4);
> -        ds_put_format(output, "%s\n", sub->name);
> -        if (title) {
> -            ds_put_char_multiple(output, ' ', (level + more) * 4);
> -            ds_put_char_multiple(output, '-', strlen(sub->name));
> -            ds_put_char(output, '\n');
> -        }
> -
> -        ovntrace_node_print_details(output, &sub->subs, level + more +
> more);
> -    }
> -}
> -
> -static void
> -ovntrace_node_prune_summary(struct ovs_list *nodes)
> -{
> -    struct ovntrace_node *sub, *next;
> -    LIST_FOR_EACH_SAFE (sub, next, node, nodes) {
> -        ovntrace_node_prune_summary(&sub->subs);
> -        if (sub->type == OVNTRACE_NODE_MODIFY ||
> -            sub->type == OVNTRACE_NODE_TABLE) {
> -            /* Replace 'sub' by its children, if any, */
> -            ovs_list_remove(&sub->node);
> -            ovs_list_splice(&next->node, sub->subs.next, &sub->subs);
> -            ovntrace_node_destroy(sub);
> -        }
> -    }
> -}
> -
> -static void
> -ovntrace_node_print_summary(struct ds *output, const struct ovs_list
> *nodes,
> -                            int level)
> -{
> -    const struct ovntrace_node *sub;
> -    LIST_FOR_EACH (sub, node, nodes) {
> -        if (sub->type == OVNTRACE_NODE_ACTION
> -            && !strncmp(sub->name, "next(", 5)) {
> -            continue;
> -        }
> -
> -        ds_put_char_multiple(output, ' ', level * 4);
> -        ds_put_cstr(output, sub->name);
> -        if (!ovs_list_is_empty(&sub->subs)) {
> -            ds_put_cstr(output, " {\n");
> -            ovntrace_node_print_summary(output, &sub->subs, level + 1);
> -            ds_put_char_multiple(output, ' ', level * 4);
> -            ds_put_char(output, '}');
> -        }
> -        if (sub->type != OVNTRACE_NODE_ACTION) {
> -            ds_put_char(output, ';');
> -        }
> -        ds_put_char(output, '\n');
> -    }
> -}
> -
> -static void
> -ovntrace_node_prune_hard(struct ovs_list *nodes)
> -{
> -    struct ovntrace_node *sub, *next;
> -    LIST_FOR_EACH_SAFE (sub, next, node, nodes) {
> -        ovntrace_node_prune_hard(&sub->subs);
> -        if (sub->type == OVNTRACE_NODE_ACTION ||
> -            sub->type == OVNTRACE_NODE_PIPELINE ||
> -            sub->type == OVNTRACE_NODE_TABLE ||
> -            sub->type == OVNTRACE_NODE_OUTPUT) {
> -            /* Replace 'sub' by its children, if any, */
> -            ovs_list_remove(&sub->node);
> -            ovs_list_splice(&next->node, sub->subs.next, &sub->subs);
> -            ovntrace_node_destroy(sub);
> -        }
> -    }
> -}
> -
> -static void
> -execute_load(const struct ovnact_load *load,
> -             const struct ovntrace_datapath *dp, struct flow *uflow,
> -             struct ovs_list *super OVS_UNUSED)
> -{
> -    const struct ovnact_encode_params ep = {
> -        .lookup_port = ovntrace_lookup_port,
> -        .aux = dp,
> -    };
> -    uint64_t stub[512 / 8];
> -    struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(stub);
> -
> -    ovnacts_encode(&load->ovnact, sizeof *load, &ep, &ofpacts);
> -
> -    struct ofpact *a;
> -    OFPACT_FOR_EACH (a, ofpacts.data, ofpacts.size) {
> -        struct ofpact_set_field *sf = ofpact_get_SET_FIELD(a);
> -
> -        if (!mf_is_register(sf->field->id)) {
> -            struct ds s = DS_EMPTY_INITIALIZER;
> -            ovnacts_format(&load->ovnact, OVNACT_LOAD_SIZE, &s);
> -            ds_chomp(&s, ';');
> -
> -            char *friendly = ovntrace_make_names_friendly(ds_cstr(&s));
> -            ovntrace_node_append(super, OVNTRACE_NODE_MODIFY, "%s",
> friendly);
> -            free(friendly);
> -
> -            ds_destroy(&s);
> -        }
> -
> -        if (mf_are_prereqs_ok(sf->field, uflow, NULL)) {
> -            mf_set_flow_value_masked(sf->field, sf->value,
> -                                     ofpact_set_field_mask(sf), uflow);
> -        }
> -    }
> -    ofpbuf_uninit(&ofpacts);
> -}
> -
> -static void
> -summarize_move(const struct mf_subfield *rsrc,
> -               const struct expr_field *dst, const struct mf_subfield
> *rdst,
> -               const struct flow *uflow, struct ovs_list *super
> OVS_UNUSED)
> -{
> -    if (!mf_is_register(rdst->field->id)) {
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        expr_field_format(dst, &s);
> -        ds_put_cstr(&s, " = ");
> -
> -        if (rsrc->ofs == 0 && rsrc->n_bits >= rsrc->field->n_bits) {
> -            union mf_value value;
> -            mf_get_value(rsrc->field, uflow, &value);
> -            mf_format(rsrc->field, &value, NULL, NULL, &s);
> -        } else {
> -            union mf_subvalue cst;
> -            mf_read_subfield(rsrc, uflow, &cst);
> -            ds_put_hex(&s, &cst, sizeof cst);
> -        }
> -
> -        ovntrace_node_append(super, OVNTRACE_NODE_MODIFY, "%s",
> ds_cstr(&s));
> -
> -        ds_destroy(&s);
> -    }
> -}
> -
> -static void
> -execute_move(const struct ovnact_move *move, struct flow *uflow,
> -             struct ovs_list *super)
> -{
> -    struct mf_subfield dst = expr_resolve_field(&move->lhs);
> -    struct mf_subfield src = expr_resolve_field(&move->rhs);
> -    summarize_move(&src, &move->lhs, &dst, uflow, super);
> -    mf_subfield_copy(&src, &dst, uflow, NULL);
> -}
> -
> -static void
> -execute_exchange(const struct ovnact_move *move, struct flow *uflow,
> -             struct ovs_list *super)
> -{
> -    struct mf_subfield a = expr_resolve_field(&move->lhs);
> -    struct mf_subfield b = expr_resolve_field(&move->rhs);
> -    summarize_move(&b, &move->lhs, &a, uflow, super);
> -    summarize_move(&a, &move->rhs, &b, uflow, super);
> -    mf_subfield_swap(&a, &b, uflow, NULL);
> -}
> -
> -static void
> -trace__(const struct ovntrace_datapath *dp, struct flow *uflow,
> -        uint8_t table_id, enum ovnact_pipeline pipeline,
> -        struct ovs_list *super);
> -
> -static void
> -trace_actions(const struct ovnact *ovnacts, size_t ovnacts_len,
> -              const struct ovntrace_datapath *dp, struct flow *uflow,
> -              uint8_t table_id, enum ovnact_pipeline pipeline,
> -              struct ovs_list *super);
> -static void
> -execute_output(const struct ovntrace_datapath *dp, struct flow *uflow,
> -               enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    uint16_t key = uflow->regs[MFF_LOG_OUTPORT - MFF_REG0];
> -    if (!key) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                             "*** output to null logical port");
> -        return;
> -    }
> -
> -    const struct ovntrace_port *port = ovntrace_port_find_by_key(dp, key);
> -    const struct ovntrace_mcgroup *mcgroup =
> ovntrace_mcgroup_find_by_key(dp,
> -
> key);
> -    const char *out_name = (port ? port->friendly_name
> -                            : mcgroup ? mcgroup->name
> -                            : "(unnamed)");
> -    if (!port && !mcgroup) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                             "*** unknown port or multicast group
> %"PRIu16,
> -                             key);
> -    }
> -
> -    if (pipeline == OVNACT_P_EGRESS) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_OUTPUT,
> -                             "/* output to \"%s\", type \"%s\" */",
> -                             out_name, port ? port->type : "");
> -        if (port && port->peer) {
> -            const struct ovntrace_port *peer = port->peer;
> -
> -            struct ovntrace_node *node = ovntrace_node_append(
> -                super, OVNTRACE_NODE_PIPELINE,
> -                "ingress(dp=\"%s\", inport=\"%s\")",
> -                peer->dp->friendly_name, peer->friendly_name);
> -
> -            struct flow new_uflow = *uflow;
> -            new_uflow.regs[MFF_LOG_INPORT - MFF_REG0] = peer->tunnel_key;
> -            new_uflow.regs[MFF_LOG_OUTPORT - MFF_REG0] = 0;
> -            trace__(peer->dp, &new_uflow, 0, OVNACT_P_INGRESS,
> &node->subs);
> -        } else {
> -            ovntrace_node_append(super, OVNTRACE_NODE_MODIFY,
> -                                 "output(\"%s\")", out_name);
> -
> -        }
> -        return;
> -    }
> -
> -    struct flow egress_uflow = *uflow;
> -    for (int i = 0; i < FLOW_N_REGS; i++) {
> -        if (i != MFF_LOG_INPORT - MFF_REG0 &&
> -            i != MFF_LOG_OUTPORT - MFF_REG0) {
> -            egress_uflow.regs[i] = 0;
> -        }
> -    }
> -
> -    uint16_t in_key = uflow->regs[MFF_LOG_INPORT - MFF_REG0];
> -    const char *inport_name = ovntrace_port_key_to_name(dp, in_key);
> -    uint32_t flags = uflow->regs[MFF_LOG_FLAGS - MFF_REG0];
> -    bool allow_loopback = (flags & MLF_ALLOW_LOOPBACK) != 0;
> -
> -    if (mcgroup) {
> -        struct ovntrace_node *mcnode = ovntrace_node_append(
> -            super, OVNTRACE_NODE_PIPELINE,
> -            "multicast(dp=\"%s\", mcgroup=\"%s\")",
> -            dp->friendly_name, mcgroup->name);
> -        for (size_t i = 0; i < mcgroup->n_ports; i++) {
> -            const struct ovntrace_port *p = mcgroup->ports[i];
> -
> -            struct ovntrace_node *node = ovntrace_node_append(
> -                &mcnode->subs, OVNTRACE_NODE_PIPELINE,
> -                "egress(dp=\"%s\", inport=\"%s\", outport=\"%s\")",
> -                dp->friendly_name, inport_name, p->friendly_name);
> -
> -            if (p->tunnel_key != in_key || allow_loopback) {
> -                node->always_indent = true;
> -
> -                egress_uflow.regs[MFF_LOG_OUTPORT - MFF_REG0] =
> p->tunnel_key;
> -                trace__(dp, &egress_uflow, 0, OVNACT_P_EGRESS,
> &node->subs);
> -            } else {
> -                ovntrace_node_append(&node->subs, OVNTRACE_NODE_OUTPUT,
> -                                     "/* omitting output because inport
> == outport && !flags.loopback */");
> -            }
> -        }
> -        return;
> -    }
> -
> -    if (port && !strcmp(port->type, "chassisredirect")) {
> -        if (port->distributed_port) {
> -            ovntrace_node_append(super, OVNTRACE_NODE_OUTPUT,
> -                                 "/* Replacing type \"%s\" outport \"%s\""
> -                                 " with distributed port \"%s\". */",
> -                                 port->type, port->friendly_name,
> -                                 port->distributed_port->friendly_name);
> -            port = port->distributed_port;
> -            out_name = port->friendly_name;
> -            egress_uflow.regs[MFF_LOG_OUTPORT - MFF_REG0] =
> port->tunnel_key;
> -        } else {
> -            ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                                 "*** output to type \"%s\" port \"%s\""
> -                                 " with no or invalid distributed port",
> -                                 port->type, out_name);
> -            return;
> -        }
> -    }
> -
> -    if ((port && port->tunnel_key != in_key) || allow_loopback) {
> -        struct ovntrace_node *node = ovntrace_node_append(
> -            super, OVNTRACE_NODE_PIPELINE,
> -            "egress(dp=\"%s\", inport=\"%s\", outport=\"%s\")",
> -            dp->friendly_name, inport_name, out_name);
> -
> -        trace__(dp, &egress_uflow, 0, OVNACT_P_EGRESS, &node->subs);
> -    } else {
> -        ovntrace_node_append(super, OVNTRACE_NODE_OUTPUT,
> -                             "/* omitting output because inport ==
> outport && !flags.loopback */");
> -    }
> -}
> -
> -static void
> -execute_clone(const struct ovnact_nest *on, const struct
> ovntrace_datapath *dp,
> -              const struct flow *uflow, uint8_t table_id,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow cloned_flow = *uflow;
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "clone");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &cloned_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_arp(const struct ovnact_nest *on, const struct ovntrace_datapath
> *dp,
> -            const struct flow *uflow, uint8_t table_id,
> -            enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow arp_flow = *uflow;
> -
> -    /* Zero fields that are no longer relevant. */
> -    arp_flow.nw_frag = 0;
> -    arp_flow.nw_tos = 0;
> -    arp_flow.nw_ttl = 0;
> -    arp_flow.tcp_flags = 0;
> -
> -    /* Update fields for ARP. */
> -    arp_flow.dl_type = htons(ETH_TYPE_ARP);
> -    arp_flow.nw_proto = ARP_OP_REQUEST;
> -    arp_flow.arp_sha = arp_flow.dl_src;
> -    arp_flow.arp_tha = eth_addr_zero;
> -    /* ARP SPA is already in arp_flow.nw_src. */
> -    /* ARP TPA is already in arp_flow.nw_dst. */
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "arp");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &arp_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_nd_na(const struct ovnact_nest *on, const struct
> ovntrace_datapath *dp,
> -              const struct flow *uflow, uint8_t table_id,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow na_flow = *uflow;
> -
> -    /* Update fields for NA. */
> -    na_flow.dl_src = uflow->dl_dst;
> -    na_flow.dl_dst = uflow->dl_src;
> -    na_flow.ipv6_dst = uflow->ipv6_src;
> -    na_flow.ipv6_src = uflow->nd_target;
> -    na_flow.tp_src = htons(136);
> -    na_flow.arp_sha = eth_addr_zero;
> -    na_flow.arp_tha = uflow->dl_dst;
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "nd_na");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &na_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_nd_ns(const struct ovnact_nest *on, const struct
> ovntrace_datapath *dp,
> -              const struct flow *uflow, uint8_t table_id,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow na_flow = *uflow;
> -
> -    /* Update fields for NA. */
> -    na_flow.dl_src = uflow->dl_src;
> -    na_flow.ipv6_src = uflow->ipv6_src;
> -    na_flow.ipv6_dst = uflow->ipv6_dst;
> -    struct in6_addr sn_addr;
> -    in6_addr_solicited_node(&sn_addr, &uflow->ipv6_dst);
> -    ipv6_multicast_to_ethernet(&na_flow.dl_dst, &sn_addr);
> -    na_flow.tp_src = htons(135);
> -    na_flow.arp_sha = eth_addr_zero;
> -    na_flow.arp_tha = uflow->dl_dst;
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "nd_ns");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &na_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_icmp4(const struct ovnact_nest *on,
> -              const struct ovntrace_datapath *dp,
> -              const struct flow *uflow, uint8_t table_id,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow icmp4_flow = *uflow;
> -
> -    /* Update fields for ICMP. */
> -    icmp4_flow.dl_dst = uflow->dl_dst;
> -    icmp4_flow.dl_src = uflow->dl_src;
> -    icmp4_flow.nw_dst = uflow->nw_dst;
> -    icmp4_flow.nw_src = uflow->nw_src;
> -    icmp4_flow.nw_proto = IPPROTO_ICMP;
> -    icmp4_flow.nw_ttl = 255;
> -    icmp4_flow.tp_src = htons(ICMP4_DST_UNREACH); /* icmp type */
> -    icmp4_flow.tp_dst = htons(1); /* icmp code */
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "icmp4");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &icmp4_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_icmp6(const struct ovnact_nest *on,
> -              const struct ovntrace_datapath *dp,
> -              const struct flow *uflow, uint8_t table_id,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow icmp6_flow = *uflow;
> -
> -    /* Update fields for ICMPv6. */
> -    icmp6_flow.dl_dst = uflow->dl_dst;
> -    icmp6_flow.dl_src = uflow->dl_src;
> -    icmp6_flow.ipv6_dst = uflow->ipv6_dst;
> -    icmp6_flow.ipv6_src = uflow->ipv6_src;
> -    icmp6_flow.nw_proto = IPPROTO_ICMPV6;
> -    icmp6_flow.nw_ttl = 255;
> -    icmp6_flow.tp_src = htons(ICMP6_DST_UNREACH); /* icmp type */
> -    icmp6_flow.tp_dst = htons(1); /* icmp code */
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "icmp6");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &icmp6_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_tcp_reset(const struct ovnact_nest *on,
> -                  const struct ovntrace_datapath *dp,
> -                  const struct flow *uflow, uint8_t table_id,
> -                  enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow tcp_flow = *uflow;
> -
> -    /* Update fields for TCP segment. */
> -    tcp_flow.dl_dst = uflow->dl_dst;
> -    tcp_flow.dl_src = uflow->dl_src;
> -    tcp_flow.nw_dst = uflow->nw_dst;
> -    tcp_flow.nw_src = uflow->nw_src;
> -    tcp_flow.nw_proto = IPPROTO_TCP;
> -    tcp_flow.nw_ttl = 255;
> -    tcp_flow.tp_src = uflow->tp_src;
> -    tcp_flow.tp_dst = uflow->tp_dst;
> -    tcp_flow.tcp_flags = htons(TCP_RST);
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "tcp_reset");
> -
> -    trace_actions(on->nested, on->nested_len, dp, &tcp_flow,
> -                  table_id, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_get_mac_bind(const struct ovnact_get_mac_bind *bind,
> -                     const struct ovntrace_datapath *dp,
> -                     struct flow *uflow, struct ovs_list *super)
> -{
> -    /* Get logical port number.*/
> -    struct mf_subfield port_sf = expr_resolve_field(&bind->port);
> -    ovs_assert(port_sf.n_bits == 32);
> -    uint32_t port_key = mf_get_subfield(&port_sf, uflow);
> -
> -    /* Get IP address. */
> -    struct mf_subfield ip_sf = expr_resolve_field(&bind->ip);
> -    ovs_assert(ip_sf.n_bits == 32 || ip_sf.n_bits == 128);
> -    union mf_subvalue ip_sv;
> -    mf_read_subfield(&ip_sf, uflow, &ip_sv);
> -    struct in6_addr ip = (ip_sf.n_bits == 32
> -                          ? in6_addr_mapped_ipv4(ip_sv.ipv4)
> -                          : ip_sv.ipv6);
> -
> -    const struct ovntrace_mac_binding *binding
> -        = ovntrace_mac_binding_find(dp, port_key, &ip);
> -
> -    uflow->dl_dst = binding ? binding->mac : eth_addr_zero;
> -    if (binding) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ACTION,
> -                             "/* MAC binding to "ETH_ADDR_FMT". */",
> -                             ETH_ADDR_ARGS(uflow->dl_dst));
> -    } else {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ACTION,
> -                             "/* No MAC binding. */");
> -    }
> -    ovntrace_node_append(super, OVNTRACE_NODE_MODIFY,
> -                         "eth.dst = "ETH_ADDR_FMT,
> -                         ETH_ADDR_ARGS(uflow->dl_dst));
> -}
> -
> -static void
> -execute_put_opts(const struct ovnact_put_opts *po,
> -                 const char *name, struct flow *uflow,
> -                 struct ovs_list *super)
> -{
> -    /* Format the put_dhcp_opts action. */
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    for (const struct ovnact_gen_option *o = po->options;
> -         o < &po->options[po->n_options]; o++) {
> -        if (o != po->options) {
> -            ds_put_cstr(&s, ", ");
> -        }
> -        ds_put_format(&s, "%s = ", o->option->name);
> -        expr_constant_set_format(&o->value, &s);
> -    }
> -    ovntrace_node_append(super, OVNTRACE_NODE_MODIFY, "%s(%s)",
> -                         name, ds_cstr(&s));
> -
> -    struct mf_subfield dst = expr_resolve_field(&po->dst);
> -    if (!mf_is_register(dst.field->id)) {
> -        /* Format assignment. */
> -        ds_clear(&s);
> -        expr_field_format(&po->dst, &s);
> -        ovntrace_node_append(super, OVNTRACE_NODE_MODIFY,
> -                             "%s = 1", ds_cstr(&s));
> -    }
> -    ds_destroy(&s);
> -
> -    struct mf_subfield sf = expr_resolve_field(&po->dst);
> -    union mf_subvalue sv = { .u8_val = 1 };
> -    mf_write_subfield_flow(&sf, &sv, uflow);
> -}
> -
> -static void
> -execute_put_dhcp_opts(const struct ovnact_put_opts *pdo,
> -                      const char *name, struct flow *uflow,
> -                      struct ovs_list *super)
> -{
> -    ovntrace_node_append(
> -        super, OVNTRACE_NODE_ERROR,
> -        "/* We assume that this packet is DHCPDISCOVER or DHCPREQUEST.
> */");
> -    execute_put_opts(pdo, name, uflow, super);
> -}
> -
> -static void
> -execute_put_nd_ra_opts(const struct ovnact_put_opts *pdo,
> -                       const char *name, struct flow *uflow,
> -                       struct ovs_list *super)
> -{
> -    execute_put_opts(pdo, name, uflow, super);
> -}
> -
> -static void
> -execute_next(const struct ovnact_next *next,
> -             const struct ovntrace_datapath *dp, struct flow *uflow,
> -             enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    if (pipeline != next->pipeline) {
> -        ovs_assert(next->pipeline == OVNACT_P_INGRESS);
> -
> -        uint16_t in_key = uflow->regs[MFF_LOG_INPORT - MFF_REG0];
> -        struct ovntrace_node *node = ovntrace_node_append(
> -            super, OVNTRACE_NODE_PIPELINE, "ingress(dp=\"%s\",
> inport=\"%s\")",
> -            dp->friendly_name, ovntrace_port_key_to_name(dp, in_key));
> -        super = &node->subs;
> -    }
> -    trace__(dp, uflow, next->ltable, next->pipeline, super);
> -}
> -
> -
> -static void
> -execute_dns_lookup(const struct ovnact_dns_lookup *dl, struct flow *uflow,
> -                   struct ovs_list *super)
> -{
> -    struct mf_subfield sf = expr_resolve_field(&dl->dst);
> -    union mf_subvalue sv = { .u8_val = 0 };
> -    mf_write_subfield_flow(&sf, &sv, uflow);
> -    ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                         "*** dns_lookup action not implemented");
> -}
> -
> -static void
> -execute_ct_next(const struct ovnact_ct_next *ct_next,
> -                const struct ovntrace_datapath *dp, struct flow *uflow,
> -                enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    /* Figure out ct_state. */
> -    uint32_t state;
> -    const char *comment;
> -    if (ct_state_idx < n_ct_states) {
> -        state = ct_states[ct_state_idx++];
> -        comment = "";
> -    } else {
> -        state = CS_ESTABLISHED | CS_TRACKED;
> -        comment = " /* default (use --ct to customize) */";
> -    }
> -
> -    /* Make a sub-node for attaching the next table. */
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    format_flags(&s, ct_state_to_string, state, '|');
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "ct_next(ct_state=%s%s)",
> -        ds_cstr(&s), comment);
> -    ds_destroy(&s);
> -
> -    /* Trace the actions in the next table. */
> -    struct flow ct_flow = *uflow;
> -    ct_flow.ct_state = state;
> -    trace__(dp, &ct_flow, ct_next->ltable, pipeline, &node->subs);
> -
> -    /* Upon return, we will trace the actions following the ct action in
> the
> -     * original table.  The pipeline forked, so we're using the original
> -     * flow, not ct_flow. */
> -}
> -
> -static void
> -execute_ct_nat(const struct ovnact_ct_nat *ct_nat,
> -               const struct ovntrace_datapath *dp, struct flow *uflow,
> -               enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    bool is_dst = ct_nat->ovnact.type == OVNACT_CT_DNAT;
> -    if (!is_dst && dp->has_local_l3gateway && !ct_nat->ip) {
> -        /* "ct_snat;" has no visible effect in a gateway router. */
> -        return;
> -    }
> -    const char *direction = is_dst ? "dst" : "src";
> -
> -    /* Make a sub-node for attaching the next table,
> -     * and figure out the changes if any. */
> -    struct flow ct_flow = *uflow;
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&s, "ct_%cnat", direction[0]);
> -    if (ct_nat->ip) {
> -        ds_put_format(&s, "(ip4.%s="IP_FMT")", direction,
> IP_ARGS(ct_nat->ip));
> -        ovs_be32 *ip = is_dst ? &ct_flow.nw_dst : &ct_flow.nw_src;
> -        *ip = ct_nat->ip;
> -
> -        uint8_t state = is_dst ? CS_DST_NAT : CS_SRC_NAT;
> -        ct_flow.ct_state |= state;
> -    } else {
> -        ds_put_format(&s, " /* assuming no un-%cnat entry, so no change
> */",
> -                      direction[0]);
> -    }
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "%s", ds_cstr(&s));
> -    ds_destroy(&s);
> -
> -    /* Trace the actions in the next table. */
> -    trace__(dp, &ct_flow, ct_nat->ltable, pipeline, &node->subs);
> -
> -    /* Upon return, we will trace the actions following the ct action in
> the
> -     * original table.  The pipeline forked, so we're using the original
> -     * flow, not ct_flow. */
> -}
> -
> -static void
> -execute_ct_lb(const struct ovnact_ct_lb *ct_lb,
> -              const struct ovntrace_datapath *dp, struct flow *uflow,
> -              enum ovnact_pipeline pipeline, struct ovs_list *super)
> -{
> -    struct flow ct_lb_flow = *uflow;
> -
> -    int family = (ct_lb_flow.dl_type == htons(ETH_TYPE_IP) ? AF_INET
> -                  : ct_lb_flow.dl_type == htons(ETH_TYPE_IPV6) ? AF_INET6
> -                  : AF_UNSPEC);
> -    if (family != AF_UNSPEC) {
> -        const struct ovnact_ct_lb_dst *dst = NULL;
> -        if (ct_lb->n_dsts) {
> -            /* For ct_lb with addresses, choose one of the addresses. */
> -            int n = 0;
> -            for (int i = 0; i < ct_lb->n_dsts; i++) {
> -                const struct ovnact_ct_lb_dst *d = &ct_lb->dsts[i];
> -                if (d->family != family) {
> -                    continue;
> -                }
> -
> -                /* Check for the destination specified by --lb-dst, if
> any. */
> -                if (lb_dst.family == family
> -                    && (family == AF_INET
> -                        ? d->ipv4 == lb_dst.ipv4
> -                        : ipv6_addr_equals(&d->ipv6, &lb_dst.ipv6))) {
> -                    lb_dst.family = AF_UNSPEC;
> -                    dst = d;
> -                    break;
> -                }
> -
> -                /* Select a random destination as a fallback. */
> -                if (!random_range(++n)) {
> -                    dst = d;
> -                }
> -            }
> -
> -            if (!dst) {
> -                ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                                     "*** no load balancing destination "
> -                                     "(use --lb-dst)");
> -            }
> -        } else if (lb_dst.family == family) {
> -            /* For ct_lb without addresses, use user-specified address. */
> -            dst = &lb_dst;
> -        }
> -
> -        if (dst) {
> -            if (family == AF_INET6) {
> -                ct_lb_flow.ipv6_dst = dst->ipv6;
> -            } else {
> -                ct_lb_flow.nw_dst = dst->ipv4;
> -            }
> -            if (dst->port) {
> -                ct_lb_flow.tp_dst = htons(dst->port);
> -            }
> -            ct_lb_flow.ct_state |= CS_DST_NAT;
> -        }
> -    }
> -
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TRANSFORMATION, "ct_lb");
> -    trace__(dp, &ct_lb_flow, ct_lb->ltable, pipeline, &node->subs);
> -}
> -
> -static void
> -execute_log(const struct ovnact_log *log, struct flow *uflow,
> -            struct ovs_list *super)
> -{
> -    char *packet_str = flow_to_string(uflow, NULL);
> -    ovntrace_node_append(super, OVNTRACE_NODE_TRANSFORMATION,
> -                    "LOG: ACL name=%s, verdict=%s, severity=%s,
> packet=\"%s\"",
> -                    log->name ? log->name : "<unnamed>",
> -                    log_verdict_to_string(log->verdict),
> -                    log_severity_to_string(log->severity),
> -                    packet_str);
> -    free(packet_str);
> -}
> -
> -static void
> -execute_ovnfield_load(const struct ovnact_load *load,
> -                      struct ovs_list *super)
> -{
> -    const struct ovn_field *f =
> ovn_field_from_name(load->dst.symbol->name);
> -    switch (f->id) {
> -    case OVN_ICMP4_FRAG_MTU: {
> -        ovntrace_node_append(super, OVNTRACE_NODE_MODIFY,
> -                             "icmp4.frag_mtu = %u",
> -                             ntohs(load->imm.value.be16_int));
> -        break;
> -    }
> -
> -    case OVN_FIELD_N_IDS:
> -    default:
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static void
> -trace_actions(const struct ovnact *ovnacts, size_t ovnacts_len,
> -              const struct ovntrace_datapath *dp, struct flow *uflow,
> -              uint8_t table_id, enum ovnact_pipeline pipeline,
> -              struct ovs_list *super)
> -{
> -    if (!ovnacts_len) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ACTION, "drop;");
> -        return;
> -    }
> -
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    const struct ovnact *a;
> -    OVNACT_FOR_EACH (a, ovnacts, ovnacts_len) {
> -        ds_clear(&s);
> -        ovnacts_format(a, sizeof *a * (ovnact_next(a) - a), &s);
> -        char *friendly = ovntrace_make_names_friendly(ds_cstr(&s));
> -        ovntrace_node_append(super, OVNTRACE_NODE_ACTION, "%s", friendly);
> -        free(friendly);
> -
> -        switch (a->type) {
> -        case OVNACT_OUTPUT:
> -            execute_output(dp, uflow, pipeline, super);
> -            break;
> -
> -        case OVNACT_NEXT:
> -            execute_next(ovnact_get_NEXT(a), dp, uflow, pipeline, super);
> -            break;
> -
> -        case OVNACT_LOAD:
> -            execute_load(ovnact_get_LOAD(a), dp, uflow, super);
> -            break;
> -
> -        case OVNACT_MOVE:
> -            execute_move(ovnact_get_MOVE(a), uflow, super);
> -            break;
> -
> -        case OVNACT_EXCHANGE:
> -            execute_exchange(ovnact_get_EXCHANGE(a), uflow, super);
> -            break;
> -
> -        case OVNACT_DEC_TTL:
> -            if (is_ip_any(uflow)) {
> -                if (uflow->nw_ttl) {
> -                    uflow->nw_ttl--;
> -                    ovntrace_node_append(super, OVNTRACE_NODE_MODIFY,
> -                                         "ip.ttl--");
> -                } else {
> -                    ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                                         "*** TTL underflow");
> -                }
> -            } else {
> -                ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                                     "*** TTL decrement of non-IP
> packet");
> -            }
> -            break;
> -
> -        case OVNACT_CT_NEXT:
> -            execute_ct_next(ovnact_get_CT_NEXT(a), dp, uflow, pipeline,
> super);
> -            break;
> -
> -        case OVNACT_CT_COMMIT:
> -            /* Nothing to do. */
> -            break;
> -
> -        case OVNACT_CT_DNAT:
> -            execute_ct_nat(ovnact_get_CT_DNAT(a), dp, uflow, pipeline,
> super);
> -            break;
> -
> -        case OVNACT_CT_SNAT:
> -            execute_ct_nat(ovnact_get_CT_SNAT(a), dp, uflow, pipeline,
> super);
> -            break;
> -
> -        case OVNACT_CT_LB:
> -            execute_ct_lb(ovnact_get_CT_LB(a), dp, uflow, pipeline,
> super);
> -            break;
> -
> -        case OVNACT_CT_CLEAR:
> -            flow_clear_conntrack(uflow);
> -            break;
> -
> -        case OVNACT_CLONE:
> -            execute_clone(ovnact_get_CLONE(a), dp, uflow, table_id,
> pipeline,
> -                          super);
> -            break;
> -
> -        case OVNACT_ARP:
> -            execute_arp(ovnact_get_ARP(a), dp, uflow, table_id, pipeline,
> -                        super);
> -            break;
> -
> -        case OVNACT_ND_NA:
> -        case OVNACT_ND_NA_ROUTER:
> -            execute_nd_na(ovnact_get_ND_NA(a), dp, uflow, table_id,
> pipeline,
> -                          super);
> -            break;
> -
> -        case OVNACT_ND_NS:
> -            execute_nd_ns(ovnact_get_ND_NS(a), dp, uflow, table_id,
> pipeline,
> -                          super);
> -            break;
> -
> -        case OVNACT_GET_ARP:
> -            execute_get_mac_bind(ovnact_get_GET_ARP(a), dp, uflow, super);
> -            break;
> -
> -        case OVNACT_GET_ND:
> -            execute_get_mac_bind(ovnact_get_GET_ND(a), dp, uflow, super);
> -            break;
> -
> -        case OVNACT_PUT_ARP:
> -        case OVNACT_PUT_ND:
> -            /* Nothing to do for tracing. */
> -            break;
> -
> -        case OVNACT_PUT_DHCPV4_OPTS:
> -            execute_put_dhcp_opts(ovnact_get_PUT_DHCPV4_OPTS(a),
> -                                  "put_dhcp_opts", uflow, super);
> -            break;
> -
> -        case OVNACT_PUT_DHCPV6_OPTS:
> -            execute_put_dhcp_opts(ovnact_get_PUT_DHCPV6_OPTS(a),
> -                                  "put_dhcpv6_opts", uflow, super);
> -            break;
> -
> -        case OVNACT_PUT_ND_RA_OPTS:
> -            execute_put_nd_ra_opts(ovnact_get_PUT_DHCPV6_OPTS(a),
> -                                   "put_nd_ra_opts", uflow, super);
> -            break;
> -
> -        case OVNACT_SET_QUEUE:
> -            /* The set_queue action is slippery from a logical
> perspective.  It
> -             * has no visible effect as long as the packet remains on the
> same
> -             * chassis: it can bounce from one logical datapath to another
> -             * retaining the queue and even end up at a VM on the same
> chassis.
> -             * Without taking the physical arrangement into account, we
> can't
> -             * do anything with this action other than just to note that
> it
> -             * happened.  If we ever add some physical knowledge to
> ovn-trace,
> -             * though, it would be easy enough to track the queue
> information
> -             * by adjusting uflow->skb_priority. */
> -            break;
> -
> -        case OVNACT_DNS_LOOKUP:
> -            execute_dns_lookup(ovnact_get_DNS_LOOKUP(a), uflow, super);
> -            break;
> -
> -        case OVNACT_LOG:
> -            execute_log(ovnact_get_LOG(a), uflow, super);
> -            break;
> -
> -        case OVNACT_SET_METER:
> -            /* Nothing to do. */
> -            break;
> -
> -        case OVNACT_ICMP4:
> -            execute_icmp4(ovnact_get_ICMP4(a), dp, uflow, table_id,
> pipeline,
> -                          super);
> -            break;
> -
> -        case OVNACT_ICMP4_ERROR:
> -            execute_icmp4(ovnact_get_ICMP4_ERROR(a), dp, uflow, table_id,
> -                          pipeline, super);
> -            break;
> -
> -        case OVNACT_ICMP6:
> -            execute_icmp6(ovnact_get_ICMP6(a), dp, uflow, table_id,
> pipeline,
> -                          super);
> -            break;
> -
> -        case OVNACT_IGMP:
> -            /* Nothing to do for tracing. */
> -            break;
> -
> -        case OVNACT_TCP_RESET:
> -            execute_tcp_reset(ovnact_get_TCP_RESET(a), dp, uflow,
> table_id,
> -                              pipeline, super);
> -            break;
> -
> -        case OVNACT_OVNFIELD_LOAD:
> -            execute_ovnfield_load(ovnact_get_OVNFIELD_LOAD(a), super);
> -            break;
> -
> -        case OVNACT_TRIGGER_EVENT:
> -            break;
> -
> -        case OVNACT_CHECK_PKT_LARGER:
> -            break;
> -        }
> -    }
> -    ds_destroy(&s);
> -}
> -
> -static bool
> -may_omit_stage(const struct ovntrace_flow *f, uint8_t table_id)
> -{
> -    return (f
> -            && f->match->type == EXPR_T_BOOLEAN && f->match->boolean
> -            && f->ovnacts_len == OVNACT_NEXT_SIZE
> -            && f->ovnacts->type == OVNACT_NEXT
> -            && ovnact_get_NEXT(f->ovnacts)->ltable == table_id + 1);
> -}
> -
> -static void
> -trace_openflow(const struct ovntrace_flow *f, struct ovs_list *super)
> -{
> -    struct ofputil_flow_stats_request fsr = {
> -        .cookie = htonll(f->uuid.parts[0]),
> -        .cookie_mask = OVS_BE64_MAX,
> -        .out_port = OFPP_ANY,
> -        .out_group = OFPG_ANY,
> -        .table_id = OFPTT_ALL,
> -    };
> -
> -    struct ofputil_flow_stats *fses;
> -    size_t n_fses;
> -    int error = vconn_dump_flows(vconn, &fsr, OFPUTIL_P_OF13_OXM,
> -                                 &fses, &n_fses);
> -    if (error) {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                             "*** error obtaining flow stats (%s)",
> -                             ovs_strerror(error));
> -        VLOG_WARN("%s: error obtaining flow stats (%s)",
> -                  ovs, ovs_strerror(error));
> -        return;
> -    }
> -
> -    if (n_fses) {
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        for (size_t i = 0; i < n_fses; i++) {
> -            ds_clear(&s);
> -            ofputil_flow_stats_format(&s, &fses[i], NULL, NULL, true);
> -            ovntrace_node_append(super, OVNTRACE_NODE_ACTION,
> -                                 "%s", ds_cstr(&s));
> -        }
> -        ds_destroy(&s);
> -    } else {
> -        ovntrace_node_append(super, OVNTRACE_NODE_ERROR,
> -                             "*** no OpenFlow flows");
> -    }
> -
> -    for (size_t i = 0; i < n_fses; i++) {
> -        free(CONST_CAST(struct ofpact *, fses[i].ofpacts));
> -    }
> -    free(fses);
> -}
> -
> -static void
> -trace__(const struct ovntrace_datapath *dp, struct flow *uflow,
> -        uint8_t table_id, enum ovnact_pipeline pipeline,
> -        struct ovs_list *super)
> -{
> -    const struct ovntrace_flow *f;
> -    for (;;) {
> -        f = ovntrace_flow_lookup(dp, uflow, table_id, pipeline);
> -        if (!may_omit_stage(f, table_id)) {
> -            break;
> -        }
> -        table_id++;
> -    }
> -
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    ds_put_format(&s, "%2d. ", table_id);
> -    if (f) {
> -        if (f->stage_name && f->source) {
> -            ds_put_format(&s, "%s (%s): ", f->stage_name, f->source);
> -        } else if (f->stage_name) {
> -            ds_put_format(&s, "%s: ", f->stage_name);
> -        } else if (f->source) {
> -            ds_put_format(&s, "(%s): ", f->source);
> -        }
> -        ds_put_format(&s, "%s, priority %d, uuid %08x",
> -                      f->match_s, f->priority, f->uuid.parts[0]);
> -    } else {
> -        char *stage_name = ovntrace_stage_name(dp, table_id, pipeline);
> -        ds_put_format(&s, "%s%sno match (implicit drop)",
> -                      stage_name ? stage_name : "",
> -                      stage_name ? ": " : "");
> -        free(stage_name);
> -    }
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        super, OVNTRACE_NODE_TABLE, "%s", ds_cstr(&s));
> -    ds_destroy(&s);
> -
> -    if (f) {
> -        if (vconn) {
> -            trace_openflow(f, &node->subs);
> -        }
> -        trace_actions(f->ovnacts, f->ovnacts_len, dp, uflow, table_id,
> -                      pipeline, &node->subs);
> -    }
> -}
> -
> -static char *
> -trace(const char *dp_s, const char *flow_s)
> -{
> -    const struct ovntrace_datapath *dp =
> ovntrace_datapath_find_by_name(dp_s);
> -    if (!dp) {
> -        return xasprintf("unknown datapath \"%s\"\n", dp_s);
> -    }
> -
> -    struct flow uflow;
> -    char *error = expr_parse_microflow(flow_s, &symtab, &address_sets,
> -                                       &port_groups, ovntrace_lookup_port,
> -                                       dp, &uflow);
> -    if (error) {
> -        char *s = xasprintf("error parsing flow: %s\n", error);
> -        free(error);
> -        return s;
> -    }
> -
> -    uint32_t in_key = uflow.regs[MFF_LOG_INPORT - MFF_REG0];
> -    if (!in_key) {
> -        VLOG_WARN("microflow does not specify ingress port");
> -    }
> -    const struct ovntrace_port *inport = ovntrace_port_find_by_key(dp,
> in_key);
> -    const char *inport_name = inport ? inport->friendly_name :
> "(unnamed)";
> -
> -    struct ds output = DS_EMPTY_INITIALIZER;
> -
> -    ds_put_cstr(&output, "# ");
> -    flow_format(&output, &uflow, NULL);
> -    ds_put_char(&output, '\n');
> -
> -    if (ovs) {
> -        int retval = vconn_open_block(ovs, 1 << OFP13_VERSION, 0, -1,
> &vconn);
> -        if (retval) {
> -            VLOG_WARN("%s: connection failed (%s)", ovs,
> ovs_strerror(retval));
> -        }
> -    }
> -
> -    struct ovs_list root = OVS_LIST_INITIALIZER(&root);
> -    struct ovntrace_node *node = ovntrace_node_append(
> -        &root, OVNTRACE_NODE_PIPELINE, "ingress(dp=\"%s\",
> inport=\"%s\")",
> -        dp->friendly_name, inport_name);
> -    trace__(dp, &uflow, 0, OVNACT_P_INGRESS, &node->subs);
> -
> -    bool multiple = (detailed + summary + minimal) > 1;
> -    if (detailed) {
> -        if (multiple) {
> -            ds_put_cstr(&output, "# Detailed trace.\n");
> -        }
> -        ovntrace_node_print_details(&output, &root, 0);
> -    }
> -
> -    if (summary) {
> -        if (multiple) {
> -            ds_put_cstr(&output, "# Summary trace.\n");
> -        }
> -        struct ovs_list clone = OVS_LIST_INITIALIZER(&clone);
> -        ovntrace_node_clone(&root, &clone);
> -        ovntrace_node_prune_summary(&clone);
> -        ovntrace_node_print_summary(&output, &clone, 0);
> -        ovntrace_node_list_destroy(&clone);
> -    }
> -
> -    if (minimal) {
> -        if (multiple) {
> -            ds_put_cstr(&output, "# Minimal trace.\n");
> -        }
> -        ovntrace_node_prune_hard(&root);
> -        ovntrace_node_print_summary(&output, &root, 0);
> -    }
> -
> -    ovntrace_node_list_destroy(&root);
> -
> -    vconn_close(vconn);
> -
> -    return ds_steal_cstr(&output);
> -}
> -
> -static void
> -ovntrace_exit(struct unixctl_conn *conn, int argc OVS_UNUSED,
> -              const char *argv[] OVS_UNUSED, void *exiting_)
> -{
> -    bool *exiting = exiting_;
> -    *exiting = true;
> -    unixctl_command_reply(conn, NULL);
> -}
> -
> -static void
> -ovntrace_trace(struct unixctl_conn *conn, int argc,
> -               const char *argv[], void *aux OVS_UNUSED)
> -{
> -    detailed = summary = minimal = false;
> -    while (argc > 1 && argv[1][0] == '-') {
> -        if (!strcmp(argv[1], "--detailed")) {
> -            detailed = true;
> -        } else if (!strcmp(argv[1], "--summary")) {
> -            summary = true;
> -        } else if (!strcmp(argv[1], "--minimal")) {
> -            minimal = true;
> -        } else if (!strcmp(argv[1], "--all")) {
> -            detailed = summary = minimal = true;
> -        } else {
> -            unixctl_command_reply_error(conn, "unknown option");
> -            return;
> -        }
> -        argc--;
> -        argv++;
> -    }
> -    if (!detailed && !summary && !minimal) {
> -        detailed = true;
> -    }
> -
> -    if (argc != 3) {
> -        unixctl_command_reply_error(
> -            conn, "exactly 2 non-option arguments are required");
> -        return;
> -    }
> -
> -    char *output = trace(argv[1], argv[2]);
> -    unixctl_command_reply(conn, output);
> -    free(output);
> -}
> diff --git a/ovn/utilities/ovndb-servers.ocf
> b/ovn/utilities/ovndb-servers.ocf
> deleted file mode 100755
> index cd4742668..000000000
> --- a/ovn/utilities/ovndb-servers.ocf
> +++ /dev/null
> @@ -1,642 +0,0 @@
> -#!/bin/bash
> -
> -: ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
> -. ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs
> -: ${OVN_CTL_DEFAULT="/usr/share/openvswitch/scripts/ovn-ctl"}
> -: ${NB_MASTER_PORT_DEFAULT="6641"}
> -: ${NB_MASTER_PROTO_DEFAULT="tcp"}
> -: ${SB_MASTER_PORT_DEFAULT="6642"}
> -: ${SB_MASTER_PROTO_DEFAULT="tcp"}
> -: ${MANAGE_NORTHD_DEFAULT="no"}
> -: ${INACTIVE_PROBE_DEFAULT="5000"}
> -: ${LISTEN_ON_MASTER_IP_ONLY_DEFAULT="yes"}
> -: ${NB_SSL_KEY_DEFAULT="/etc/openvswitch/ovnnb-privkey.pem"}
> -: ${NB_SSL_CERT_DEFAULT="/etc/openvswitch/ovnnb-cert.pem"}
> -: ${NB_SSL_CACERT_DEFAULT="/etc/openvswitch/cacert.pem"}
> -: ${SB_SSL_KEY_DEFAULT="/etc/openvswitch/ovnsb-privkey.pem"}
> -: ${SB_SSL_CERT_DEFAULT="/etc/openvswitch/ovnsb-cert.pem"}
> -: ${SB_SSL_CACERT_DEFAULT="/etc/openvswitch/cacert.pem"}
> -
> -CRM_MASTER="${HA_SBIN_DIR}/crm_master -l reboot"
> -CRM_ATTR_REPL_INFO="${HA_SBIN_DIR}/crm_attribute --type crm_config --name
> OVN_REPL_INFO -s ovn_ovsdb_master_server"
> -OVN_CTL=${OCF_RESKEY_ovn_ctl:-${OVN_CTL_DEFAULT}}
> -MASTER_IP=${OCF_RESKEY_master_ip}
> -NB_MASTER_PORT=${OCF_RESKEY_nb_master_port:-${NB_MASTER_PORT_DEFAULT}}
>
> -NB_MASTER_PROTO=${OCF_RESKEY_nb_master_protocol:-${NB_MASTER_PROTO_DEFAULT}}
> -SB_MASTER_PORT=${OCF_RESKEY_sb_master_port:-${SB_MASTER_PORT_DEFAULT}}
>
> -SB_MASTER_PROTO=${OCF_RESKEY_sb_master_protocol:-${SB_MASTER_PROTO_DEFAULT}}
> -MANAGE_NORTHD=${OCF_RESKEY_manage_northd:-${MANAGE_NORTHD_DEFAULT}}
>
> -INACTIVE_PROBE=${OCF_RESKEY_inactive_probe_interval:-${INACTIVE_PROBE_DEFAULT}}
> -NB_PRIVKEY=${OCF_RESKEY_ovn_nb_db_privkey:-${NB_SSL_KEY_DEFAULT}}
> -NB_CERT=${OCF_RESKEY_ovn_nb_db_cert:-${NB_SSL_CERT_DEFAULT}}
> -NB_CACERT=${OCF_RESKEY_ovn_nb_db_cacert:-${NB_SSL_CACERT_DEFAULT}}
> -SB_PRIVKEY=${OCF_RESKEY_ovn_sb_db_privkey:-${SB_SSL_KEY_DEFAULT}}
> -SB_CERT=${OCF_RESKEY_ovn_sb_db_cert:-${SB_SSL_CERT_DEFAULT}}
> -SB_CACERT=${OCF_RESKEY_ovn_sb_db_cacert:-${SB_SSL_CACERT_DEFAULT}}
> -
> -
> -# In order for pacemaker to work with LB, we can set
> LISTEN_ON_MASTER_IP_ONLY
> -# to false and pass LB vip IP while creating pcs resource.
>
> -LISTEN_ON_MASTER_IP_ONLY=${OCF_RESKEY_listen_on_master_ip_only:-${LISTEN_ON_MASTER_IP_ONLY_DEFAULT}}
> -
> -# Invalid IP address is an address that can never exist in the network, as
> -# mentioned in rfc-5737. The ovsdb servers connects to this IP address
> till
> -# a master is promoted and the IPAddr2 resource is started.
> -INVALID_IP_ADDRESS=192.0.2.254
> -
> -host_name=$(ocf_attribute_target)
> -if [ "x$host_name" = "x" ]; then
> -    # function ocf_attribute_target may not be available if the pacemaker
> -    # version is old. Fall back to ocf_local_nodename.
> -    host_name=$(ocf_local_nodename)
> -fi
> -: ${slave_score=5}
> -: ${master_score=10}
> -
> -ovsdb_server_metadata() {
> -    cat <<END
> -<?xml version="1.0"?>
> -<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
> -<resource-agent name="ovsdb-server">
> -  <version>1.0</version>
> -
> -  <longdesc lang="en">
> -    This resource manages ovsdb-server.
> -  </longdesc>
> -
> -  <shortdesc lang="en">
> -    Manages ovsdb-server.
> -  </shortdesc>
> -
> -  <parameters>
> -
> -  <parameter name="ovn_ctl" unique="1">
> -  <longdesc lang="en">
> -  Location to the ovn-ctl script file
> -  </longdesc>
> -  <shortdesc lang="en">ovn-ctl script</shortdesc>
> -  <content type="string" default="${OVN_CTL_DEFAULT}" />
> -  </parameter>
> -
> -  <parameter name="master_ip" unique="1">
> -  <longdesc lang="en">
> -  The IP address resource which will be available on the master ovsdb
> server
> -  </longdesc>
> -  <shortdesc lang="en">master ip address</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="nb_master_port" unique="1">
> -  <longdesc lang="en">
> -  The port which the master Northbound database server is listening
> -  </longdesc>
> -  <shortdesc lang="en">master Northbound database port</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="nb_master_protocol" unique="1">
> -  <longdesc lang="en">
> -  The protocol which the master Northbound database server used, 'tcp' or
> 'ssl'.
> -  </longdesc>
> -  <shortdesc lang="en">master Northbound database protocol</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="sb_master_port" unique="1">
> -  <longdesc lang="en">
> -  The port which the master Southbound database server is listening
> -  </longdesc>
> -  <shortdesc lang="en">master Southbound database port</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="sb_master_protocol" unique="1">
> -  <longdesc lang="en">
> -  The protocol which the master Southbound database server used, 'tcp' or
> 'ssl'.
> -  </longdesc>
> -  <shortdesc lang="en">master Southbound database protocol</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="manage_northd" unique="1">
> -  <longdesc lang="en">
> -  If set to yes, manages ovn-northd service. ovn-northd will be started in
> -  the master node.
> -  </longdesc>
> -  <shortdesc lang="en">manage ovn-northd service</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="inactive_probe_interval" unique="1">
> -  <longdesc lang="en">
> -  Inactive probe interval to set for ovsdb-server.
> -  </longdesc>
> -  <shortdesc lang="en">Set inactive probe interval</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="listen_on_master_ip_only" unique="1">
> -  <longdesc lang="en">
> -  If set to yes, the OVNDBs will listen on master IP. Otherwise, it will
> -  listen on 0.0.0.0. Set to yes when using pacemaker managed vip resource
> -  as MASTER_IP; set to no when using external LB VIP.
> -  </longdesc>
> -  <shortdesc lang="en">Listen on master IP or 0.0.0.0</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_nb_db_privkey" unique="1">
> -  <longdesc lang="en">
> -  OVN NB DB private key absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN NB DB private key file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_nb_db_cert" unique="1">
> -  <longdesc lang="en">
> -  OVN NB DB certificate absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN NB DB cert file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_nb_db_cacert" unique="1">
> -  <longdesc lang="en">
> -  OVN NB DB CA certificate absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN NB DB cacert file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_sb_db_privkey" unique="1">
> -  <longdesc lang="en">
> -  OVN SB DB private key absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN SB DB private key file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_sb_db_cert" unique="1">
> -  <longdesc lang="en">
> -  OVN SB DB certificate absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN SB DB cert file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  <parameter name="ovn_sb_db_cacert" unique="1">
> -  <longdesc lang="en">
> -  OVN SB DB CA certificate absolute path for ssl setup.
> -  </longdesc>
> -  <shortdesc lang="en">OVN SB DB cacert file</shortdesc>
> -  <content type="string" />
> -  </parameter>
> -
> -  </parameters>
> -
> -  <actions>
> -    <action name="notify"       timeout="20s" />
> -    <action name="start"        timeout="30s" />
> -    <action name="stop"         timeout="20s" />
> -    <action name="promote"      timeout="50s" />
> -    <action name="demote"       timeout="50s" />
> -    <action name="monitor"      timeout="20s"  depth="0" interval="10s"
> -     role="Master" />
> -    <action name="monitor"      timeout="20s"  depth="0" interval="30s"
> -     role="Slave"/>
> -    <action name="meta-data"    timeout="5s" />
> -    <action name="validate-all" timeout="20s" />
> -  </actions>
> -</resource-agent>
> -END
> -    exit $OCF_SUCCESS
> -}
> -
> -ovsdb_server_notify() {
> -    # requires the notify=true meta resource attribute
> -    local
> type_op="${OCF_RESKEY_CRM_meta_notify_type}-${OCF_RESKEY_CRM_meta_notify_operation}"
> -
> -    if [ "$type_op" != "post-promote" ]; then
> -        # We are only interested in specific events
> -        return $OCF_SUCCESS
> -    fi
> -
> -    ocf_log debug "ovndb_server: notified of event $type_op"
> -    if [ "x$(ovsdb_server_last_known_master)" = "x${host_name}" ]; then
> -        # Record ourselves so that the agent has a better chance of doing
> -        # the right thing at startup
> -        ocf_log debug "ovndb_server: $host_name is the master"
> -        ${CRM_ATTR_REPL_INFO} -v "$host_name"
> -        if [ "$MANAGE_NORTHD" = "yes" ]; then
> -            # Startup ovn-northd service
> -            ${OVN_CTL} --ovn-manage-ovsdb=no start_northd
> -        fi
> -
> -        # In order to over-ride inactivity_probe for LB use case, we need
> to
> -        # create connection entry to listen on 0.0.0.0 for master node.
> -        if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then
> -           LISTON_ON_IP="0.0.0.0"
> -        else
> -           LISTON_ON_IP=${MASTER_IP}
> -        fi
> -        conn=`ovn-nbctl get NB_global . connections`
> -        if [ "$conn" == "[]" ]
> -        then
> -            ovn-nbctl -- --id=@conn_uuid create Connection \
> -target="p${NB_MASTER_PROTO}\:${NB_MASTER_PORT}\:${LISTON_ON_IP}" \
> -inactivity_probe=$INACTIVE_PROBE -- set NB_Global . connections=@conn_uuid
> -        fi
> -
> -        conn=`ovn-sbctl get SB_global . connections`
> -        if [ "$conn" == "[]" ]
> -        then
> -            ovn-sbctl -- --id=@conn_uuid create Connection \
> -target="p${SB_MASTER_PROTO}\:${SB_MASTER_PORT}\:${LISTON_ON_IP}" \
> -inactivity_probe=$INACTIVE_PROBE -- set SB_Global . connections=@conn_uuid
> -        fi
> -
> -    else
> -        if [ "$MANAGE_NORTHD" = "yes" ]; then
> -            # Stop ovn-northd service. Set --ovn-manage-ovsdb=no so that
> -            # ovn-ctl doesn't stop ovsdb-servers.
> -            ${OVN_CTL} --ovn-manage-ovsdb=no stop_northd
> -        fi
> -        # Synchronize with the new master
> -        ocf_log debug "ovndb_server: Connecting to the new master
> ${OCF_RESKEY_CRM_meta_notify_promote_uname}"
> -        ${OVN_CTL} demote_ovnnb --db-nb-sync-from-addr=${MASTER_IP} \
> -                                --db-nb-sync-from-port=${NB_MASTER_PORT} \
> -                                --db-nb-sync-from-proto=${NB_MASTER_PROTO}
> -        ${OVN_CTL} demote_ovnsb --db-sb-sync-from-addr=${MASTER_IP} \
> -                                --db-sb-sync-from-port=${SB_MASTER_PORT} \
> -                                --db-sb-sync-from-proto=${SB_MASTER_PROTO}
> -    fi
> -}
> -
> -ovsdb_server_usage() {
> -    cat <<END
> -usage: $0 {start|stop|status|monitor|notify|validate-all|meta-data}
> -
> -Expects to have a fully populated OCF RA-compliant environment set.
> -END
> -    exit $1
> -}
> -
> -ovsdb_server_find_active_master() {
> -    # Operation sequence is Demote -> Stop -> Start -> Promote
> -    # At the point this is run, the only active masters will be
> -    # previous masters minus any that were scheduled to be demoted
> -
> -    for master in ${OCF_RESKEY_CRM_meta_notify_master_uname}; do
> -        found=0
> -        for old in ${OCF_RESKEY_CRM_meta_notify_demote_uname}; do
> -            if [ $master = $old ]; then
> -                found=1
> -            fi
> -        done
> -        if [ $found = 0 ]; then
> -            # Rely on master-max=1
> -            # Pacemaker will demote any additional ones it finds before
> starting new copies
> -            echo "$master"
> -            return
> -        fi
> -    done
> -
> -    local expected_master=$($CRM_ATTR_REPL_INFO --query  -q 2>/dev/null)
> -    case "x${OCF_RESKEY_CRM_meta_notify_start_uname}x" in
> -        *${expected_master}*) echo "${expected_master}";; # The previous
> master is expected to start
> -    esac
> -}
> -
> -ovsdb_server_last_known_master()
> -{
> -    if [ -z "$MASTER_HOST" ]; then
> -        MASTER_HOST="$(${CRM_ATTR_REPL_INFO} --query  -q  2>/dev/null)"
> -    fi
> -    echo "$MASTER_HOST"
> -}
> -
> -ovsdb_server_master_update() {
> -    case $1 in
> -        $OCF_SUCCESS)
> -        $CRM_MASTER -N $host_name -v ${slave_score};;
> -        $OCF_RUNNING_MASTER)
> -            $CRM_MASTER -N $host_name -v ${master_score};;
> -        #*) $CRM_MASTER -D;;
> -    esac
> -}
> -
> -ovsdb_server_monitor() {
> -    ovsdb_server_check_status $@
> -    rc=$?
> -
> -    ovsdb_server_master_update $rc
> -    return $rc
> -}
> -
> -ovsdb_server_check_status() {
> -    local sb_status=`${OVN_CTL} status_ovnsb`
> -    local nb_status=`${OVN_CTL} status_ovnnb`
> -
> -    if [[ $sb_status == "running/backup" && $nb_status ==
> "running/backup" ]]; then
> -        return $OCF_SUCCESS
> -    fi
> -
> -    check_northd="no"
> -    if [ "$MANAGE_NORTHD" == "yes" ] && [ "$1" != "ignore_northd" ]; then
> -        check_northd="yes"
> -    fi
> -
> -    if [[ $sb_status == "running/active" && $nb_status ==
> "running/active" ]]; then
> -        if [ "$check_northd" == "yes" ]; then
> -            # Verify if ovn-northd is running or not.
> -            ${OVN_CTL} status_northd
> -            if [ "$?" == "0" ] ; then
> -                return $OCF_RUNNING_MASTER
> -            fi
> -        else
> -            return $OCF_RUNNING_MASTER
> -        fi
> -    fi
> -
> -    # TODO: What about service running but not in either state above?
> -    # Eg. a transient state where one db is "active" and the other
> -    # "backup"
> -
> -    return $OCF_NOT_RUNNING
> -}
> -
> -ovsdb_server_start() {
> -    ovsdb_server_check_status
> -    local status=$?
> -    # If not in stopped state, return
> -    if [ $status -ne $OCF_NOT_RUNNING ]; then
> -        return $status
> -    fi
> -
> -    local present_master=$(ovsdb_server_find_active_master)
> -
> -    set ${OVN_CTL}
> -
> -    if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then
> -        set $@ --db-nb-port=${NB_MASTER_PORT}
> -        set $@ --db-sb-port=${SB_MASTER_PORT}
> -
> -    else
> -       set $@ --db-nb-addr=${MASTER_IP} --db-nb-port=${NB_MASTER_PORT}
> -       set $@ --db-sb-addr=${MASTER_IP} --db-sb-port=${SB_MASTER_PORT}
> -    fi
> -
> -    if [ "x${NB_MASTER_PROTO}" = xssl ]; then
> -            set $@ --ovn-nb-db-ssl-key=${NB_PRIVKEY}
> -            set $@ --ovn-nb-db-ssl-cert=${NB_CERT}
> -            set $@ --ovn-nb-db-ssl-ca-cert=${NB_CACERT}
> -    fi
> -    if [ "x${SB_MASTER_PROTO}" = xssl ]; then
> -            set $@ --ovn-sb-db-ssl-key=${SB_PRIVKEY}
> -            set $@ --ovn-sb-db-ssl-cert=${SB_CERT}
> -            set $@ --ovn-sb-db-ssl-ca-cert=${SB_CACERT}
> -    fi
> -    if [ "x${present_master}" = x ]; then
> -        # No master detected, or the previous master is not among the
> -        # set starting.
> -        #
> -        # Force all copies to come up as slaves by pointing them into
> -        # space and let pacemaker pick one to promote:
> -        #
> -        if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
> -            set $@ --db-nb-create-insecure-remote=yes
> -        fi
> -
> -        if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
> -            set $@ --db-sb-create-insecure-remote=yes
> -        fi
> -        set $@ --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
> --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
> -
> -    elif [ ${present_master} != ${host_name} ]; then
> -        if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xyes ]; then
> -            if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
> -                set $@ --db-nb-create-insecure-remote=yes
> -            fi
> -
> -            if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
> -                set $@ --db-sb-create-insecure-remote=yes
> -            fi
> -        fi
> -        # An existing master is active, connect to it
> -        set $@ --db-nb-sync-from-addr=${MASTER_IP}
> --db-sb-sync-from-addr=${MASTER_IP}
> -        set $@ --db-nb-sync-from-port=${NB_MASTER_PORT}
> -        set $@ --db-nb-sync-from-proto=${NB_MASTER_PROTO}
> -        set $@ --db-sb-sync-from-port=${SB_MASTER_PORT}
> -        set $@ --db-sb-sync-from-proto=${SB_MASTER_PROTO}
> -        if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then
> -            set $@ --db-sb-use-remote-in-db="no"
> -            set $@ --db-nb-use-remote-in-db="no"
> -        fi
> -    fi
> -
> -    $@ start_ovsdb
> -
> -    while [ 1 = 1 ]; do
> -        # It is important that we don't return until we're in a functional
> -        # state. When checking the status of the ovsdb-server's ignore
> northd.
> -        # It is possible that when the resource is restarted
> ovsdb-server's
> -        # can be started as masters and ovn-northd would not have been
> started.
> -        # ovn-northd will be started once a node is promoted to master and
> -        # 'manage_northd' is set to yes.
> -        ovsdb_server_monitor ignore_northd
> -        rc=$?
> -        case $rc in
> -            $OCF_SUCCESS)        return $rc;;
> -            $OCF_RUNNING_MASTER)
> -                # When a slave node is promoted as master, the action
> would be
> -                # STOP -> START -> PROMOTE.
> -                # When the start action is called, it is possible for the
> -                # ovsdb-server's to be started as active. This could
> happen
> -                # if the node owns the $MASTER_IP. At this point,
> pacemaker
> -                # has not promoted this node yet. Demote it and check for
> -                # status again.
> -                # Let pacemaker promote it in subsequent actions.
> -                # As per the OCF guidelines, only monitor action should
> return
> -                # OCF_RUNNING_MASTER.
> -                #
> http://www.linux-ha.org/doc/dev-guides/_literal_ocf_running_master_literal_8.html
> -                ${OVN_CTL} demote_ovnnb \
> -                --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
> -                ${OVN_CTL} demote_ovnsb \
> -                --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
> -                ;;
> -            $OCF_ERR_GENERIC)    return $rc;;
> -            # Otherwise loop, waiting for the service to start, until
> -            # the cluster times the operation out
> -        esac
> -        ocf_log warn "ovndb_servers: After starting ovsdb, status is $rc.
> Checking the status again"
> -    done
> -}
> -
> -ovsdb_server_stop() {
> -    if [ "$MANAGE_NORTHD" = "yes" ]; then
> -        # Stop ovn-northd service in case it was running. This is required
> -        # when the master is demoted. For other cases, it would be a
> no-op.
> -        # Set --ovn-manage-ovsdb=no so that ovn-ctl doesn't stop
> ovsdb-servers.
> -        ${OVN_CTL} --ovn-manage-ovsdb=no stop_northd
> -    fi
> -
> -    ovsdb_server_check_status ignore_northd
> -    case $? in
> -        $OCF_NOT_RUNNING)
> -          # Even if one server is down, check_status returns NOT_RUNNING.
> -          # So before returning call stop_ovsdb to be sure.
> -          ${OVN_CTL} stop_ovsdb
> -          return ${OCF_SUCCESS};;
> -    esac
> -
> -    ${OVN_CTL} stop_ovsdb
> -    ovsdb_server_master_update ${OCF_NOT_RUNNING}
> -
> -    while [ 1 = 1 ]; do
> -        # It is important that we don't return until we're stopped
> -        ovsdb_server_check_status ignore_northd
> -        rc=$?
> -        case $rc in
> -        $OCF_SUCCESS)
> -            # Loop, waiting for the service to stop, until the
> -            # cluster times the operation out
> -            ocf_log warn "ovndb_servers: Even after stopping, the servers
> seems to be running"
> -            ;;
> -        $OCF_NOT_RUNNING)
> -            return $OCF_SUCCESS
> -            ;;
> -        *)
> -            return $rc
> -            ;;
> -        esac
> -    done
> -
> -    return $OCF_ERR_GENERIC
> -}
> -
> -ovsdb_server_promote() {
> -    local state
> -
> -    ovsdb_server_check_status ignore_northd
> -    rc=$?
> -    case $rc in
> -        ${OCF_SUCCESS}) ;;
> -        ${OCF_RUNNING_MASTER}) ;;
> -        *)
> -            ovsdb_server_master_update $OCF_RUNNING_MASTER
> -            return ${rc}
> -            ;;
> -    esac
> -
> -    # Restart ovs so that new master can listen on tcp port
> -    if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then
> -        ${OVN_CTL} stop_ovsdb
> -        ovsdb_server_start
> -    fi
> -    ${OVN_CTL} promote_ovnnb
> -    ${OVN_CTL} promote_ovnsb
> -
> -    if [ "$MANAGE_NORTHD" = "yes" ]; then
> -        # Startup ovn-northd service
> -        ${OVN_CTL} --ovn-manage-ovsdb=no start_northd
> -    fi
> -
> -    ocf_log debug "ovndb_servers: Waiting for promotion $host_name as
> master to complete"
> -    ovsdb_server_check_status
> -    state=$?
> -    while [ "$state" != "$OCF_RUNNING_MASTER" ]; do
> -      sleep 1
> -      ovsdb_server_check_status
> -      state=$?
> -    done
> -    ocf_log debug "ovndb_servers: Promotion of $host_name as the master
> completed"
> -    # Record ourselves so that the agent has a better chance of doing
> -    # the right thing at startup
> -    ${CRM_ATTR_REPL_INFO} -v "$host_name"
> -    ovsdb_server_master_update $OCF_RUNNING_MASTER
> -    return $OCF_SUCCESS
> -}
> -
> -ovsdb_server_demote() {
> -    # While demoting, check the status of ovn_northd.
> -    # In case ovn_northd is not running, we should return OCF_NOT_RUNNING.
> -    ovsdb_server_check_status
> -    if [ $? = $OCF_NOT_RUNNING ]; then
> -        return $OCF_NOT_RUNNING
> -    fi
> -
> -    local present_master=$(ovsdb_server_find_active_master)
> -    local recorded_master=$($CRM_ATTR_REPL_INFO --query  -q 2>/dev/null)
> -
> -    ocf_log debug "ovndb_servers: Demoting $host_name, present master
> ${present_master}, recorded master ${recorded_master}"
> -    if [ "x${recorded_master}" = "x${host_name}" -a "x${present_master}"
> = x ]; then
> -        # We are the one and only master
> -        # This should be the "normal" case
> -        # The only way to be demoted is to call demote_ovn*
> -        #
> -        # The local database is only reset once we successfully
> -        # connect to the peer.  So specify one that doesn't exist.
> -        #
> -        # Eventually a new master will be promoted and we'll resync
> -        # using the logic in ovsdb_server_notify()
> -        ${OVN_CTL} demote_ovnnb
> --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
> -        ${OVN_CTL} demote_ovnsb
> --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
> -
> -    elif [ "x${present_master}" = "x${host_name}" ]; then
> -        # Safety check, should never be called
> -        #
> -        # Never allow sync'ing from ourselves, its a great way to
> -        # erase the local DB
> -        ${OVN_CTL} demote_ovnnb
> --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
> -        ${OVN_CTL} demote_ovnsb
> --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
> -
> -    elif [ "x${present_master}" != x ]; then
> -        # There are too many masters and we're an extra one that is
> -        # being demoted. Sync to the surviving one
> -        ${OVN_CTL} demote_ovnnb --db-nb-sync-from-addr=${MASTER_IP} \
> -                                --db-nb-sync-from-port=${NB_MASTER_PORT} \
> -                                --db-nb-sync-from-proto=${NB_MASTER_PROTO}
> -        ${OVN_CTL} demote_ovnsb --db-sb-sync-from-addr=${MASTER_IP} \
> -                                --db-sb-sync-from-port=${SB_MASTER_PORT} \
> -                                --db-sb-sync-from-proto=${SB_MASTER_PROTO}
> -
> -    else
> -        # For completeness, should never be called
> -        #
> -        # Something unexpected happened, perhaps CRM_ATTR_REPL_INFO is
> incorrect
> -        ${OVN_CTL} demote_ovnnb
> --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
> -        ${OVN_CTL} demote_ovnsb
> --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
> -    fi
> -
> -    if [ "$MANAGE_NORTHD" = "yes" ]; then
> -        # Stop ovn-northd service
> -        ${OVN_CTL} --ovn-manage-ovsdb=no stop_northd
> -    fi
> -    ovsdb_server_master_update $OCF_SUCCESS
> -    return $OCF_SUCCESS
> -}
> -
> -ovsdb_server_validate() {
> -    if [ ! -e ${OVN_CTL} ]; then
> -        return $OCF_ERR_INSTALLED
> -    fi
> -    return $OCF_SUCCESS
> -}
> -
> -
> -case $__OCF_ACTION in
> -start)          ovsdb_server_start;;
> -stop)           ovsdb_server_stop;;
> -promote)        ovsdb_server_promote;;
> -demote)         ovsdb_server_demote;;
> -notify)         ovsdb_server_notify;;
> -meta-data)      ovsdb_server_metadata;;
> -validate-all)   ovsdb_server_validate;;
> -status|monitor) ovsdb_server_monitor;;
> -usage|help)     ovsdb_server_usage $OCF_SUCCESS;;
> -*)              ovsdb_server_usage $OCF_ERR_UNIMPLEMENTED ;;
> -esac
> -
> -rc=$?
> -exit $rc
> diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> index ec85e14c4..6fdb4b5a5 100644
> --- a/ovsdb/ovsdb-tool.1.in
> +++ b/ovsdb/ovsdb-tool.1.in
> @@ -91,18 +91,17 @@ both its schema and data.)
>  .
>  .IP "\fBcreate\-cluster\fI db contents local"
>  Use this command to initialize the first server in a high-availability
> -cluster of 3 (or more) database servers, e.g. for an OVN northbound or
> -southbound database in an environment that cannot tolerate a single
> -point of failure.  It creates clustered database file \fIdb\fR and
> -configures the server to listen on \fIlocal\fR, which must take the
> -form \fIprotocol\fB:\fIip\fB:\fIport\fR, where \fIprotocol\fR is
> -\fBtcp\fR or \fBssl\fR, \fIip\fR is the server's IP (either an IPv4
> -address or an IPv6 address enclosed in square brackets), and
> -\fIport\fR is a TCP port number.  Only one address is specified, for
> -the first server in the cluster, ordinarily the one for the server
> -running \fBcreate\-cluster\fR.  The address is used for communication
> -within the cluster, not for communicating with OVSDB clients, and must
> -not use the same port used for the OVSDB protocol.
> +cluster of 3 (or more) database servers, e.g. for a database in an
> +environment that cannot tolerate a single point of failure.  It creates
> +clustered database file \fIdb\fR and configures the server to listen on
> +\fIlocal\fR, which must take the form \fIprotocol\fB:\fIip\fB:\fIport\fR,
> +where \fIprotocol\fR is \fBtcp\fR or \fBssl\fR, \fIip\fR is the server's
> +IP (either an IPv4 address or an IPv6 address enclosed in square
> +brackets), and \fIport\fR is a TCP port number.  Only one address is
> +specified, for the first server in the cluster, ordinarily the one for
> +the server running \fBcreate\-cluster\fR.  The address is used for
> +communication within the cluster, not for communicating with OVSDB
> +clients, and must not use the same port used for the OVSDB protocol.
>  .IP
>  The new database is initialized with \fIcontents\fR, which must name a
>  file that contains either an OVSDB schema in JSON format or a
> diff --git a/rhel/automake.mk b/rhel/automake.mk
> index 1c5bf153c..c75406e05 100644
> --- a/rhel/automake.mk
> +++ b/rhel/automake.mk
> @@ -23,8 +23,6 @@ EXTRA_DIST += \
>         rhel/openvswitch.spec.in \
>         rhel/openvswitch-fedora.spec \
>         rhel/openvswitch-fedora.spec.in \
> -       rhel/ovn-fedora.spec \
> -       rhel/ovn-fedora.spec.in \
>         rhel/usr_share_openvswitch_scripts_ovs-systemd-reload \
>         rhel/usr_share_openvswitch_scripts_sysconfig.template \
>         rhel/usr_share_openvswitch_scripts_systemd_sysconfig.template \
> @@ -34,12 +32,7 @@ EXTRA_DIST += \
>         rhel/usr_lib_systemd_system_ovsdb-server.service \
>         rhel/usr_lib_systemd_system_ovs-vswitchd.service.in \
>         rhel/usr_lib_systemd_system_ovs-delete-transient-ports.service \
> -       rhel/usr_lib_systemd_system_ovn-controller.service \
> -       rhel/usr_lib_systemd_system_ovn-controller-vtep.service \
> -       rhel/usr_lib_systemd_system_ovn-northd.service \
> -       rhel/usr_lib_systemd_system_openvswitch-ipsec.service \
> -       rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml \
> -       rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml
> +       rhel/usr_lib_systemd_system_openvswitch-ipsec.service
>
>  DISTCLEANFILES += rhel/usr_lib_systemd_system_ovs-vswitchd.service
>
> @@ -74,13 +67,6 @@ rpm-fedora: dist $(srcdir)/rhel/openvswitch-fedora.spec
>                   -D "_topdir ${RPMBUILD_TOP}" \
>                   -ba $(srcdir)/rhel/openvswitch-fedora.spec
>
> -rpm-fedora-ovn: dist $(srcdir)/rhel/ovn-fedora.spec
> -       ${MKDIR_P} ${RPMBUILD_TOP}/SOURCES
> -       cp ${DIST_ARCHIVES} ${RPMBUILD_TOP}/SOURCES
> -       rpmbuild ${RPMBUILD_OPT} \
> -                 -D "_topdir ${RPMBUILD_TOP}" \
> -                 -ba $(srcdir)/rhel/ovn-fedora.spec
> -
>  # Build kernel datapath RPM
>  rpm-fedora-kmod: dist $(srcdir)/rhel/openvswitch-kmod-fedora.spec
>         ${MKDIR_P} ${RPMBUILD_TOP}/SOURCES
> diff --git a/rhel/ovn-fedora.spec.in b/rhel/ovn-fedora.spec.in
> deleted file mode 100644
> index 2ecc629f2..000000000
> --- a/rhel/ovn-fedora.spec.in
> +++ /dev/null
> @@ -1,432 +0,0 @@
> -# Spec file for Open Virtual Network (OVN).
> -
> -# Copyright (C) 2018 Red Hat, Inc.
> -#
> -# Copying and distribution of this file, with or without modification,
> -# are permitted in any medium without royalty provided the copyright
> -# notice and this notice are preserved.  This file is offered as-is,
> -# without warranty of any kind.
> -#
> -# If tests have to be skipped while building, specify the '--without
> check'
> -# option. For example:
> -#     rpmbuild -bb --without check rhel/ovn-fedora.spec
> -#
> -
> -# If libcap-ng isn't available and there is no need for running OVS
> -# as regular user, specify the '--without libcapng'
> -%bcond_without libcapng
> -
> -# Enable Python 3 by specifying '--with build_python3'.
> -# This is enabled by default for versions of the distribution that
> -# have Python 3 by default (Fedora > 22).
> -%bcond_with build_python3
> -
> -# Enable PIE, bz#955181
> -%global _hardened_build 1
> -
> -# some distros (e.g: RHEL-7) don't define _rundir macro yet
> -# Fedora 15 onwards uses /run as _rundir
> -%if 0%{!?_rundir:1}
> -%define _rundir /run
> -%endif
> -
> -# define the python package prefix based on distribution version so that
> we can
> -# simultaneously support RHEL-based and later Fedora versions in this
> spec file.
> -%if 0%{?fedora} >= 25
> -%define _py2 python2
> -%endif
> -
> -%if 0%{?rhel} || 0%{?fedora} < 25
> -%define _py2 python
> -%endif
> -
> -Name: ovn
> -Summary: Open Virtual Network support
> -Group: System Environment/Daemons
> -URL: http://www.openvswitch.org/
> -Version: @VERSION@
> -Obsoletes: openvswitch-ovn-common <
> %{?epoch:%{epoch}:}%{version}-%{release}
> -Provides: openvswitch-ovn-common =
> %{?epoch:%{epoch}:}%{version}-%{release}
> -
> -# Nearly all of openvswitch is ASL 2.0.  The bugtool is LGPLv2+, and the
> -# lib/sflow*.[ch] files are SISSL
> -License: ASL 2.0 and LGPLv2+ and SISSL
> -Release: 1%{?dist}
> -Source: http://openvswitch.org/releases/openvswitch-%{version}.tar.gz
> -
> -BuildRequires: gcc gcc-c++
> -BuildRequires: autoconf automake libtool
> -BuildRequires: systemd-units openssl openssl-devel
> -BuildRequires: %{_py2}-devel
> -%if 0%{?fedora} > 22 || %{with build_python3}
> -BuildRequires: python3-devel
> -%endif
> -BuildRequires: desktop-file-utils
> -BuildRequires: groff graphviz
> -BuildRequires: checkpolicy, selinux-policy-devel
> -BuildRequires: /usr/bin/sphinx-build
> -# make check dependencies
> -BuildRequires: %{_py2}-twisted%{?rhel:-core} %{_py2}-zope-interface
> %{_py2}-six
> -BuildRequires: procps-ng
> -%if %{with libcapng}
> -BuildRequires: libcap-ng libcap-ng-devel
> -%endif
> -BuildRequires: unbound unbound-devel
> -
> -Requires: openssl hostname iproute module-init-tools openvswitch
> -
> -Requires(post): systemd-units
> -Requires(preun): systemd-units
> -Requires(postun): systemd-units
> -
> -# to skip running checks, pass --without check
> -%bcond_without check
> -
> -%description
> -OVN, the Open Virtual Network, is a system to support virtual network
> -abstraction.  OVN complements the existing capabilities of OVS to add
> -native support for virtual network abstractions, such as virtual L2 and L3
> -overlays and security groups.
> -
> -%package central
> -Summary: Open Virtual Network support
> -License: ASL 2.0
> -Requires: ovn
> -Requires: firewalld-filesystem
> -Obsoletes: openvswitch-ovn-central
> -Provides: openvswitch-ovn-central =
> %{?epoch:%{epoch}:}%{version}-%{release}
> -
> -%description central
> -OVN DB servers and ovn-northd running on a central node.
> -
> -%package host
> -Summary: Open Virtual Network support
> -License: ASL 2.0
> -Requires: ovn
> -Requires: firewalld-filesystem
> -Obsoletes: openvswitch-ovn-host
> -Provides: openvswitch-ovn-host = %{?epoch:%{epoch}:}%{version}-%{release}
> -
> -%description host
> -OVN controller running on each host.
> -
> -%package vtep
> -Summary: Open Virtual Network support
> -License: ASL 2.0
> -Requires: ovn
> -Obsoletes: openvswitch-ovn-vtep
> -Provides: openvswitch-ovn-vtep = %{?epoch:%{epoch}:}%{version}-%{release}
> -
> -%description vtep
> -OVN vtep controller
> -
> -%package docker
> -Summary: Open Virtual Network support
> -License: ASL 2.0
> -Requires: ovn %{_py2}-openvswitch
> -Obsoletes: openvswitch-ovn-docker
> -Provides: openvswitch-ovn-docker =
> %{?epoch:%{epoch}:}%{version}-%{release}
> -
> -%description docker
> -Docker network plugins for OVN.
> -
> -%prep
> -%setup -n openvswitch-%{version}
> -
> -%build
> -%configure \
> -%if %{with libcapng}
> -        --enable-libcapng \
> -%else
> -        --disable-libcapng \
> -%endif
> -        --enable-ssl \
> -        --with-pkidir=%{_sharedstatedir}/openvswitch/pki \
> -%if 0%{?fedora} > 22 || %{with build_python3}
> -        PYTHON3=%{__python3} \
> -        PYTHON=%{__python2}
> -%else
> -        PYTHON=%{__python}
> -%endif
> -
> -make %{?_smp_mflags}
> -
> -%install
> -rm -rf $RPM_BUILD_ROOT
> -make install DESTDIR=$RPM_BUILD_ROOT
> -
> -for service in ovn-controller ovn-controller-vtep ovn-northd; do
> -        install -p -D -m 0644 \
> -                        rhel/usr_lib_systemd_system_${service}.service \
> -                        $RPM_BUILD_ROOT%{_unitdir}/${service}.service
> -done
> -
> -rm -rf $RPM_BUILD_ROOT/%{_datadir}/openvswitch/python/
> -
> -install -d -m 0755 $RPM_BUILD_ROOT/%{_sharedstatedir}/openvswitch
> -
> -install -d $RPM_BUILD_ROOT%{_prefix}/lib/firewalld/services/
> -install -p -m 0644
> rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml \
> -
> $RPM_BUILD_ROOT%{_prefix}/lib/firewalld/services/ovn-central-firewall-service.xml
> -install -p -m 0644
> rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml \
> -
> $RPM_BUILD_ROOT%{_prefix}/lib/firewalld/services/ovn-host-firewall-service.xml
> -
> -install -d -m 0755 $RPM_BUILD_ROOT%{_prefix}/lib/ocf/resource.d/ovn
> -ln -s %{_datadir}/openvswitch/scripts/ovndb-servers.ocf \
> -      $RPM_BUILD_ROOT%{_prefix}/lib/ocf/resource.d/ovn/ovndb-servers
> -
> -# remove OVS unpackages files
> -rm -f $RPM_BUILD_ROOT%{_bindir}/ovs*
> -rm -f $RPM_BUILD_ROOT%{_bindir}/vtep-ctl
> -rm -f $RPM_BUILD_ROOT%{_sbindir}/ovs*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man1/ovs*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man5/ovs*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man5/vtep*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man7/ovs*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man8/ovs*
> -rm -f $RPM_BUILD_ROOT%{_mandir}/man8/vtep*
> -rm -f $RPM_BUILD_ROOT%{_datadir}/openvswitch/ovs*
> -rm -f $RPM_BUILD_ROOT%{_datadir}/openvswitch/vswitch.ovsschema
> -rm -f $RPM_BUILD_ROOT%{_datadir}/openvswitch/vtep.ovsschema
> -rm -f $RPM_BUILD_ROOT%{_datadir}/openvswitch/scripts/ovs*
> -rm -rf $RPM_BUILD_ROOT%{_datadir}/openvswitch/bugtool-plugins
> -rm -f $RPM_BUILD_ROOT%{_includedir}/openvswitch/*
> -rm -f $RPM_BUILD_ROOT%{_includedir}/openflow/*
> -rm -f $RPM_BUILD_ROOT%{_libdir}/*.a
> -rm -f $RPM_BUILD_ROOT%{_libdir}/*.la
> -rm -f $RPM_BUILD_ROOT%{_libdir}/pkgconfig/*.pc
> -rm -f $RPM_BUILD_ROOT%{_includedir}/openvswitch/*
> -rm -f $RPM_BUILD_ROOT%{_includedir}/openflow/*
> -rm -f $RPM_BUILD_ROOT%{_includedir}/ovn/*
> -rm -f
> $RPM_BUILD_ROOT%{_sysconfdir}/bash_completion.d/ovs-appctl-bashcomp.bash
> -rm -f
> $RPM_BUILD_ROOT%{_sysconfdir}/bash_completion.d/ovs-vsctl-bashcomp.bash
> -rm -rf $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/openvswitch
> -
> -%check
> -%if %{with check}
> -    touch resolv.conf
> -    export OVS_RESOLV_CONF=$(pwd)/resolv.conf
> -    if make check TESTSUITEFLAGS='%{_smp_mflags}' RECHECK=yes; then :;
> -    else
> -        cat tests/testsuite.log
> -        exit 1
> -    fi
> -%endif
> -
> -%clean
> -rm -rf $RPM_BUILD_ROOT
> -
> -%pre central
> -if [ $1 -eq 1 ] ; then
> -    # Package install.
> -    /bin/systemctl status ovn-northd.service >/dev/null
> -    ovn_status=$?
> -    rpm -ql openvswitch-ovn-central > /dev/null
> -    if [[ "$?" = "0" && "$ovn_status" = "0" ]]; then
> -        # ovn-northd service is running which means old
> openvswitch-ovn-central
> -        # is already installed and it will be cleaned up. So start
> ovn-northd
> -        # service when posttrans central is called.
> -        touch %{_localstatedir}/lib/rpm-state/ovn-northd
> -    fi
> -fi
> -
> -%pre host
> -if [ $1 -eq 1 ] ; then
> -    # Package install.
> -    /bin/systemctl status ovn-controller.service >/dev/null
> -    ovn_status=$?
> -    rpm -ql openvswitch-ovn-host > /dev/null
> -    if [[ "$?" = "0" && "$ovn_status" = "0" ]]; then
> -        # ovn-controller service is running which means old
> -        # openvswitch-ovn-host is installed and it will be cleaned up. So
> -        # start ovn-controller service when posttrans host is called.
> -        touch %{_localstatedir}/lib/rpm-state/ovn-controller
> -    fi
> -fi
> -
> -%pre vtep
> -if [ $1 -eq 1 ] ; then
> -    # Package install.
> -    /bin/systemctl status ovn-controller-vtep.service >/dev/null
> -    ovn_status=$?
> -    rpm -ql openvswitch-ovn-vtep > /dev/null
> -    if [[ "$?" = "0" && "$ovn_status" = "0" ]]; then
> -        # ovn-controller-vtep service is running which means old
> -        # openvswitch-ovn-vtep is installed and it will be cleaned up. So
> -        # start ovn-controller-vtep service when posttrans host is called.
> -        touch %{_localstatedir}/lib/rpm-state/ovn-controller-vtep
> -    fi
> -fi
> -
> -%preun central
> -%if 0%{?systemd_preun:1}
> -    %systemd_preun ovn-northd.service
> -%else
> -    if [ $1 -eq 0 ] ; then
> -        # Package removal, not upgrade
> -        /bin/systemctl --no-reload disable ovn-northd.service >/dev/null
> 2>&1 || :
> -        /bin/systemctl stop ovn-northd.service >/dev/null 2>&1 || :
> -    fi
> -%endif
> -
> -%preun host
> -%if 0%{?systemd_preun:1}
> -    %systemd_preun ovn-controller.service
> -%else
> -    if [ $1 -eq 0 ] ; then
> -        # Package removal, not upgrade
> -        /bin/systemctl --no-reload disable ovn-controller.service
> >/dev/null 2>&1 || :
> -        /bin/systemctl stop ovn-controller.service >/dev/null 2>&1 || :
> -    fi
> -%endif
> -
> -%preun vtep
> -%if 0%{?systemd_preun:1}
> -    %systemd_preun ovn-controller-vtep.service
> -%else
> -    if [ $1 -eq 0 ] ; then
> -        # Package removal, not upgrade
> -        /bin/systemctl --no-reload disable ovn-controller-vtep.service
> >/dev/null 2>&1 || :
> -        /bin/systemctl stop ovn-controller-vtep.service >/dev/null 2>&1
> || :
> -    fi
> -%endif
> -
> -%post central
> -%if 0%{?systemd_post:1}
> -    %systemd_post ovn-northd.service
> -%else
> -    # Package install, not upgrade
> -    if [ $1 -eq 1 ]; then
> -        /bin/systemctl daemon-reload >dev/null || :
> -    fi
> -%endif
> -
> -%post host
> -%if 0%{?systemd_post:1}
> -    %systemd_post ovn-controller.service
> -%else
> -    # Package install, not upgrade
> -    if [ $1 -eq 1 ]; then
> -        /bin/systemctl daemon-reload >dev/null || :
> -    fi
> -%endif
> -
> -%post vtep
> -%if 0%{?systemd_post:1}
> -    %systemd_post ovn-controller-vtep.service
> -%else
> -    # Package install, not upgrade
> -    if [ $1 -eq 1 ]; then
> -        /bin/systemctl daemon-reload >dev/null || :
> -    fi
> -%endif
> -
> -%postun
> -
> -%postun central
> -%if 0%{?systemd_postun_with_restart:1}
> -    %systemd_postun_with_restart ovn-northd.service
> -%else
> -    /bin/systemctl daemon-reload >/dev/null 2>&1 || :
> -    if [ "$1" -ge "1" ] ; then
> -    # Package upgrade, not uninstall
> -        /bin/systemctl try-restart ovn-northd.service >/dev/null 2>&1 || :
> -    fi
> -%endif
> -
> -%postun host
> -%if 0%{?systemd_postun_with_restart:1}
> -    %systemd_postun_with_restart ovn-controller.service
> -%else
> -    /bin/systemctl daemon-reload >/dev/null 2>&1 || :
> -    if [ "$1" -ge "1" ] ; then
> -        # Package upgrade, not uninstall
> -        /bin/systemctl try-restart ovn-controller.service >/dev/null 2>&1
> || :
> -    fi
> -%endif
> -
> -%postun vtep
> -%if 0%{?systemd_postun_with_restart:1}
> -    %systemd_postun_with_restart ovn-controller-vtep.service
> -%else
> -    /bin/systemctl daemon-reload >/dev/null 2>&1 || :
> -    if [ "$1" -ge "1" ] ; then
> -        # Package upgrade, not uninstall
> -        /bin/systemctl try-restart ovn-controller-vtep.service >/dev/null
> 2>&1 || :
> -    fi
> -%endif
> -
> -%posttrans central
> -if [ $1 -eq 1 ]; then
> -    # Package install, not upgrade
> -    if [ -e %{_localstatedir}/lib/rpm-state/ovn-northd ]; then
> -        rm %{_localstatedir}/lib/rpm-state/ovn-northd
> -        /bin/systemctl start ovn-northd.service >/dev/null 2>&1 || :
> -    fi
> -fi
> -
> -
> -%posttrans host
> -if [ $1 -eq 1 ]; then
> -    # Package install, not upgrade
> -    if [ -e %{_localstatedir}/lib/rpm-state/ovn-controller ]; then
> -        rm %{_localstatedir}/lib/rpm-state/ovn-controller
> -        /bin/systemctl start ovn-controller.service >/dev/null 2>&1 || :
> -    fi
> -fi
> -
> -%posttrans vtep
> -if [ $1 -eq 1 ]; then
> -    # Package install, not upgrade
> -    if [ -e %{_localstatedir}/lib/rpm-state/ovn-controller-vtep ]; then
> -        rm %{_localstatedir}/lib/rpm-state/ovn-controller-vtep
> -        /bin/systemctl start ovn-controller-vtep.service >/dev/null 2>&1
> || :
> -    fi
> -fi
> -
> -%files
> -%{_bindir}/ovn-nbctl
> -%{_bindir}/ovn-sbctl
> -%{_bindir}/ovn-trace
> -%{_bindir}/ovn-detrace
> -%{_datadir}/openvswitch/scripts/ovn-ctl
> -%{_datadir}/openvswitch/scripts/ovndb-servers.ocf
> -%{_datadir}/openvswitch/scripts/ovn-bugtool-nbctl-show
> -%{_datadir}/openvswitch/scripts/ovn-bugtool-sbctl-lflow-list
> -%{_datadir}/openvswitch/scripts/ovn-bugtool-sbctl-show
> -%{_mandir}/man8/ovn-ctl.8*
> -%{_mandir}/man8/ovn-nbctl.8*
> -%{_mandir}/man8/ovn-trace.8*
> -%{_mandir}/man1/ovn-detrace.1*
> -%{_mandir}/man7/ovn-architecture.7*
> -%{_mandir}/man8/ovn-sbctl.8*
> -%{_mandir}/man5/ovn-nb.5*
> -%{_mandir}/man5/ovn-sb.5*
> -%{_prefix}/lib/ocf/resource.d/ovn/ovndb-servers
> -
> -%files docker
> -%{_bindir}/ovn-docker-overlay-driver
> -%{_bindir}/ovn-docker-underlay-driver
> -
> -%files central
> -%{_bindir}/ovn-northd
> -%{_mandir}/man8/ovn-northd.8*
> -%config %{_datadir}/openvswitch/ovn-nb.ovsschema
> -%config %{_datadir}/openvswitch/ovn-sb.ovsschema
> -%{_unitdir}/ovn-northd.service
> -%{_prefix}/lib/firewalld/services/ovn-central-firewall-service.xml
> -
> -%files host
> -%{_bindir}/ovn-controller
> -%{_mandir}/man8/ovn-controller.8*
> -%{_unitdir}/ovn-controller.service
> -%{_prefix}/lib/firewalld/services/ovn-host-firewall-service.xml
> -
> -%files vtep
> -%{_bindir}/ovn-controller-vtep
> -%{_mandir}/man8/ovn-controller-vtep.8*
> -%{_unitdir}/ovn-controller-vtep.service
> -
> -%changelog
> -* Thu Dec 20 2018 Numan Siddique <nusiddiq at redhat.com>
> -- OVS/OVN split.
> diff --git
> a/rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml
> b/rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml
> deleted file mode 100644
> index a005f325c..000000000
> --- a/rhel/usr_lib_firewalld_services_ovn-central-firewall-service.xml
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<service>
> -  <short>ovn-central-firewall-service</short>
> -  <description>Firewall service for ovn central</description>
> -  <port protocol="tcp" port="6641"/>
> -  <port protocol="tcp" port="6642"/>
> -</service>
> diff --git a/rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml
> b/rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml
> deleted file mode 100644
> index f606890c3..000000000
> --- a/rhel/usr_lib_firewalld_services_ovn-host-firewall-service.xml
> +++ /dev/null
> @@ -1,6 +0,0 @@
> -<?xml version="1.0" encoding="utf-8"?>
> -<service>
> -  <short>ovn-host-firewall-service</short>
> -  <description>Firewall service for ovn host</description>
> -  <port protocol="udp" port="6081"/>
> -</service>
> diff --git a/rhel/usr_lib_systemd_system_ovn-controller-vtep.service
> b/rhel/usr_lib_systemd_system_ovn-controller-vtep.service
> deleted file mode 100644
> index b1e239f57..000000000
> --- a/rhel/usr_lib_systemd_system_ovn-controller-vtep.service
> +++ /dev/null
> @@ -1,50 +0,0 @@
> -# See ovn-controller-vtep(8) for details about ovn-controller-vtep.
> -#
> -# You may override the following variables to customize
> ovn-controller-vtep
> -# behavior:
> -#
> -#   OVN_DB - Set this variable to the location of the ovsdb server that is
> -#            serving the OVN_Southbound database.  See the manpage for
> -#            ovn-controller-vtep for more details on the format for the db
> -#            location.
> -#
> -#   VTEP_DB - Set this variable to the location of the ovsdb server that
> is
> -#             serving the hardware_vtep database.  See the manpage for
> -#             ovn-controller-vtep for more details on the format for the
> db
> -#             location.
> -#
> -# To override these variables, you may create a configuration file
> -# in the /etc/systemd/system/ovn-controller-vtep.d/ directory.  For
> example,
> -# you could place the following contents in
> -# /etc/systemd/system/ovn-controller-vtep.d/local.conf:
> -#
> -#   [System]
> -#   Environment="OVN_DB=unix:/usr/local/var/run/openvswitch/db.sock"
> "VTEP_DB=unix:/usr/local/var/run/openvswitch/vtep.sock"
> -#
> -# Alternatively, you may specify environment variables in the file
> /etc/sysconfig/ovn-controller-vtep:
> -#
> -#   OVN_DB="unix:/usr/local/var/run/openvswitch/db.sock"
> -#   VTEP_DB="unix:/usr/local/var/run/openvswitch/vtep.sock"
> -
> -[Unit]
> -Description=OVN VTEP gateway controller daemon
> -After=syslog.target
> -Requires=openvswitch.service
> -After=openvswitch.service
> -
> -[Service]
> -Type=forking
> -PIDFile=/var/run/openvswitch/ovn-controller-vtep.pid
> -Restart=on-failure
> -Environment=OVN_DB=unix:%t/openvswitch/ovnsb_db.sock
> -Environment=VTEP_DB=unix:%t/openvswitch/db.sock
> -EnvironmentFile=-/etc/sysconfig/ovn-controller-vtep
> -EnvironmentFile=/run/openvswitch.useropts
> -ExecStart=/usr/share/openvswitch/scripts/ovn-ctl \
> -          --db-sb-sock=${OVN_DB} --db-sock=${VTEP_DB} \
> -          --ovn-user=${OVS_USER_ID} \
> -          start_controller_vtep
> -ExecStop=/usr/share/openvswitch/scripts/ovn-ctl stop_controller_vtep
> -
> -[Install]
> -WantedBy=multi-user.target
> diff --git a/rhel/usr_lib_systemd_system_ovn-controller.service
> b/rhel/usr_lib_systemd_system_ovn-controller.service
> deleted file mode 100644
> index 335cd5a52..000000000
> --- a/rhel/usr_lib_systemd_system_ovn-controller.service
> +++ /dev/null
> @@ -1,34 +0,0 @@
> -# See ovn-controller(8) for details about ovn-controller.
> -#
> -# To customize the ovn-controller service, you may create a configuration
> file
> -# in the /etc/systemd/system/ovn-controller.d/ directory.  For example,
> to specify
> -# additional options to be passed to the "ovn-ctl start_controller"
> command, you
> -# could place the following contents in
> -# /etc/systemd/system/ovn-controller.d/local.conf:
> -#
> -#   [System]
> -#   Environment="OVN_CONTROLLER_OPTS=--ovn-controller-log=-vconsole:emer
> -vsyslog:err -vfile:info"
> -#
> -# Alternatively, you may specify environment variables in the file
> /etc/sysconfig/ovn-controller:
> -#
> -#   OVN_CONTROLLER_OPTS="--ovn-controller-log=-vconsole:emer -vsyslog:err
> -vfile:info"
> -
> -[Unit]
> -Description=OVN controller daemon
> -After=syslog.target
> -Requires=openvswitch.service
> -After=openvswitch.service
> -
> -[Service]
> -Type=forking
> -PIDFile=/var/run/openvswitch/ovn-controller.pid
> -Restart=on-failure
> -EnvironmentFile=-/etc/sysconfig/ovn-controller
> -EnvironmentFile=/run/openvswitch.useropts
> -ExecStart=/usr/share/openvswitch/scripts/ovn-ctl --no-monitor \
> -          --ovn-user=${OVS_USER_ID} \
> -          start_controller $OVN_CONTROLLER_OPTS
> -ExecStop=/usr/share/openvswitch/scripts/ovn-ctl stop_controller
> -
> -[Install]
> -WantedBy=multi-user.target
> diff --git a/rhel/usr_lib_systemd_system_ovn-northd.service
> b/rhel/usr_lib_systemd_system_ovn-northd.service
> deleted file mode 100644
> index ea8c191e3..000000000
> --- a/rhel/usr_lib_systemd_system_ovn-northd.service
> +++ /dev/null
> @@ -1,35 +0,0 @@
> -# See ovn-northd(8) for details about ovn-northd.
> -#
> -# To customize the ovn-northd service, you may create a configuration file
> -# in the /etc/systemd/system/ovn-northd.d/ directory.  For example, to
> specify
> -# additional options to be passed to the "ovn-ctl start_northd" command,
> you
> -# could place the following contents in
> -# /etc/systemd/system/ovn-northd.d/local.conf:
> -#
> -#   [System]
> -#
>  Environment="OVN_NORTHD_OPTS=--db-nb-sock=/usr/local/var/run/openvswitch/ovnnb_db.sock
> --db-sb-sock=/usr/local/var/run/openvswitch/ovnsb_db.sock"
> -#
> -# Alternatively, you may specify environment variables in the file
> /etc/sysconfig/ovn-northd:
> -#
> -#
>  OVN_NORTHD_OPTS="--db-nb-sock=/usr/local/var/run/openvswitch/ovnnb_db.sock
> --db-sb-sock=/usr/local/var/run/openvswitch/ovnsb_db.sock"
> -
> -[Unit]
> -Description=OVN northd management daemon
> -After=syslog.target
> -Requires=openvswitch.service
> -After=openvswitch.service
> -
> -[Service]
> -Type=oneshot
> -RemainAfterExit=yes
> -Environment=OVS_RUNDIR=%t/openvswitch OVS_DBDIR=/var/lib/openvswitch
> -EnvironmentFile=-/etc/sysconfig/ovn-northd
> -EnvironmentFile=/run/openvswitch.useropts
> -ExecStartPre=-/usr/bin/chown -R ${OVS_USER_ID} ${OVS_DBDIR}
> -ExecStart=/usr/share/openvswitch/scripts/ovn-ctl \
> -          --ovs-user=${OVS_USER_ID} --ovn-user=${OVS_USER_ID} \
> -          start_northd $OVN_NORTHD_OPTS
> -ExecStop=/usr/share/openvswitch/scripts/ovn-ctl stop_northd
> -
> -[Install]
> -WantedBy=multi-user.target
> diff --git a/tests/atlocal.in b/tests/atlocal.in
> index 2e565d788..556f8681c 100644
> --- a/tests/atlocal.in
> +++ b/tests/atlocal.in
> @@ -217,10 +217,6 @@ unset HTTPS_PROXY
>  unset FTP_PROXY
>  unset NO_PROXY
>
> -# Avoid OVN environment variables leaking in from external environment.
> -unset OVN_NB_DB
> -unset OVN_SB_DB
> -
>  # Prevent logging to syslog during tests.
>  OVS_SYSLOG_METHOD=null
>  export OVS_SYSLOG_METHOD
> diff --git a/tests/automake.mk b/tests/automake.mk
> index d6ab51732..45313aa74 100644
> --- a/tests/automake.mk
> +++ b/tests/automake.mk
> @@ -7,14 +7,12 @@ EXTRA_DIST += \
>         $(SYSTEM_AFXDP_TESTSUITE_AT) \
>         $(SYSTEM_OFFLOADS_TESTSUITE_AT) \
>         $(SYSTEM_DPDK_TESTSUITE_AT) \
> -       $(OVSDB_CLUSTER_TESTSUITE_AT) \
>         $(TESTSUITE) \
>         $(SYSTEM_KMOD_TESTSUITE) \
>         $(SYSTEM_USERSPACE_TESTSUITE) \
>         $(SYSTEM_AFXDP_TESTSUITE) \
>         $(SYSTEM_OFFLOADS_TESTSUITE) \
>         $(SYSTEM_DPDK_TESTSUITE) \
> -       $(OVSDB_CLUSTER_TESTSUITE) \
>         tests/atlocal.in \
>         $(srcdir)/package.m4 \
>         $(srcdir)/tests/testsuite \
> @@ -23,8 +21,7 @@ EXTRA_DIST += \
>  COMMON_MACROS_AT = \
>         tests/ovsdb-macros.at \
>         tests/ovs-macros.at \
> -       tests/ofproto-macros.at \
> -       tests/ovn-macros.at
> +       tests/ofproto-macros.at
>
>  TESTSUITE_AT = \
>         tests/testsuite.at \
> @@ -104,16 +101,9 @@ TESTSUITE_AT = \
>         tests/vlog.at \
>         tests/vtep-ctl.at \
>         tests/auto-attach.at \
> -       tests/ovn.at \
> -       tests/ovn-northd.at \
> -       tests/ovn-nbctl.at \
> -       tests/ovn-sbctl.at \
> -       tests/ovn-controller.at \
> -       tests/ovn-controller-vtep.at \
>         tests/mcast-snooping.at \
>         tests/packet-type-aware.at \
> -       tests/nsh.at \
> -       tests/ovn-performance.at
> +       tests/nsh.at
>
>  EXTRA_DIST += $(FUZZ_REGRESSION_TESTS)
>  FUZZ_REGRESSION_TESTS = \
> @@ -146,11 +136,6 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/
> automake.mk
>             echo "TEST_FUZZ_REGRESSION([$$basename])"; \
>         done > $@.tmp && mv $@.tmp $@
>
> -OVSDB_CLUSTER_TESTSUITE_AT = \
> -       tests/ovsdb-cluster-testsuite.at \
> -       tests/ovsdb-execution.at \
> -       tests/ovsdb-cluster.at
> -
>  SYSTEM_KMOD_TESTSUITE_AT = \
>         tests/system-common-macros.at \
>         tests/system-kmod-testsuite.at \
> @@ -158,7 +143,6 @@ SYSTEM_KMOD_TESTSUITE_AT = \
>
>  SYSTEM_USERSPACE_TESTSUITE_AT = \
>         tests/system-userspace-testsuite.at \
> -       tests/system-ovn.at \
>         tests/system-userspace-macros.at \
>         tests/system-userspace-packet-type-aware.at
>
> @@ -169,7 +153,6 @@ SYSTEM_AFXDP_TESTSUITE_AT = \
>
>  SYSTEM_TESTSUITE_AT = \
>         tests/system-common-macros.at \
> -       tests/system-ovn.at \
>         tests/system-layer3-tunnels.at \
>         tests/system-traffic.at \
>         tests/system-interface.at
> @@ -194,11 +177,9 @@ SYSTEM_USERSPACE_TESTSUITE =
> $(srcdir)/tests/system-userspace-testsuite
>  SYSTEM_AFXDP_TESTSUITE = $(srcdir)/tests/system-afxdp-testsuite
>  SYSTEM_OFFLOADS_TESTSUITE = $(srcdir)/tests/system-offloads-testsuite
>  SYSTEM_DPDK_TESTSUITE = $(srcdir)/tests/system-dpdk-testsuite
> -OVSDB_CLUSTER_TESTSUITE = $(srcdir)/tests/ovsdb-cluster-testsuite
>  DISTCLEANFILES += tests/atconfig tests/atlocal
>
> -AUTOTEST_PATH =
> utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):ovn/controller-vtep:ovn/northd:ovn/utilities:ovn/controller
> -
> +AUTOTEST_PATH =
> utilities:vswitchd:ovsdb:vtep:tests:$(PTHREAD_WIN32_DIR_DLL):$(SSL_DIR):
>  check-local:
>         set $(SHELL) '$(TESTSUITE)' -C tests
> AUTOTEST_PATH=$(AUTOTEST_PATH); \
>         "$$@" $(TESTSUITEFLAGS) || (test X'$(RECHECK)' = Xyes && "$$@"
> --recheck)
> @@ -238,10 +219,6 @@ check-lcov: all $(check_DATA) clean-lcov
>  # valgrind support
>
>  valgrind_wrappers = \
> -       tests/valgrind/ovn-controller \
> -       tests/valgrind/ovn-nbctl \
> -       tests/valgrind/ovn-northd \
> -       tests/valgrind/ovn-sbctl \
>         tests/valgrind/ovs-appctl \
>         tests/valgrind/ovs-ofctl \
>         tests/valgrind/ovs-vsctl \
> @@ -277,12 +254,6 @@ check-valgrind: all $(valgrind_wrappers) $(check_DATA)
>         @echo
> '----------------------------------------------------------------------'
>         @echo 'Valgrind output can be found in
> tests/testsuite.dir/*/valgrind.*'
>         @echo
> '----------------------------------------------------------------------'
> -check-ovsdb-cluster-valgrind: all $(valgrind_wrappers) $(check_DATA)
> -       $(SHELL) '$(OVSDB_CLUSTER_TESTSUITE)' -C tests CHECK_VALGRIND=true
> VALGRIND='$(VALGRIND)' AUTOTEST_PATH='tests/valgrind:$(AUTOTEST_PATH)' -d
> $(TESTSUITEFLAGS) -j1
> -       @echo
> -       @echo
> '----------------------------------------------------------------------'
> -       @echo 'Valgrind output can be found in
> tests/ovsdb-cluster-testsuite.dir/*/valgrind.*'
> -       @echo
> '----------------------------------------------------------------------'
>  check-kernel-valgrind: all $(valgrind_wrappers) $(check_DATA)
>         $(SHELL) '$(SYSTEM_KMOD_TESTSUITE)' -C tests
> VALGRIND='$(VALGRIND)' AUTOTEST_PATH='tests/valgrind:$(AUTOTEST_PATH)' -d
> $(TESTSUITEFLAGS) -j1
>         @echo
> @@ -340,10 +311,6 @@ check-dpdk: all
>  clean-local:
>         test ! -f '$(TESTSUITE)' || $(SHELL) '$(TESTSUITE)' -C tests
> --clean
>
> -# Run OVSDB cluster tests.
> -check-ovsdb-cluster: all
> -       set $(SHELL) '$(OVSDB_CLUSTER_TESTSUITE)' -C tests
> AUTOTEST_PATH='$(AUTOTEST_PATH)'; \
> -       "$$@" $(TESTSUITEFLAGS) -j1 || (test X'$(RECHECK)' = Xyes && "$$@"
> --recheck)
>
>  AUTOTEST = $(AUTOM4TE) --language=autotest
>
> @@ -378,10 +345,6 @@ $(SYSTEM_DPDK_TESTSUITE): package.m4
> $(SYSTEM_TESTSUITE_AT) $(SYSTEM_DPDK_TESTSU
>         $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
>         $(AM_V_at)mv $@.tmp $@
>
> -$(OVSDB_CLUSTER_TESTSUITE): package.m4 $(OVSDB_CLUSTER_TESTSUITE_AT)
> $(COMMON_MACROS_AT)
> -       $(AM_V_GEN)$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
> -       $(AM_V_at)mv $@.tmp $@
> -
>  # The `:;' works around a Bash 3.2 bug when the output is not writeable.
>  $(srcdir)/package.m4: $(top_srcdir)/configure.ac
>         $(AM_V_GEN):;{ \
> @@ -446,7 +409,6 @@ tests_ovstest_SOURCES = \
>         tests/test-netflow.c \
>         tests/test-odp.c \
>         tests/test-ofpbuf.c \
> -       tests/test-ovn.c \
>         tests/test-packets.c \
>         tests/test-random.c \
>         tests/test-rcu.c \
> @@ -474,7 +436,7 @@ tests_ovstest_SOURCES += \
>         tests/test-netlink-conntrack.c
>  endif
>
> -tests_ovstest_LDADD = lib/libopenvswitch.la ovn/lib/libovn.la
> +tests_ovstest_LDADD = lib/libopenvswitch.la
>
>  noinst_PROGRAMS += tests/test-stream
>  tests_test_stream_SOURCES = tests/test-stream.c
> diff --git a/tests/ofproto-macros.at b/tests/ofproto-macros.at
> index db0cd5108..04d4ed7e2 100644
> --- a/tests/ofproto-macros.at
> +++ b/tests/ofproto-macros.at
> @@ -214,7 +214,7 @@ check_logs () {
>
>      # We most notably ignore 'Broken pipe' warnings.  These often and
>      # intermittently appear in ovsdb-server.log, because *ctl commands
> -    # (e.g. ovs-vsctl, ovn-nbctl) exit right after committing a change to
> the
> +    # (e.g. ovs-vsctl) exit right after committing a change to the
>      # database.  However, in reaction, some daemon may immediately update
> the
>      # database, and this later update may cause database sending update
> back to
>      # *ctl command if *ctl has not exited yet.  If *ctl command exits
> before
> diff --git a/tests/oss-fuzz/automake.mk b/tests/oss-fuzz/automake.mk
> index 5bf7d0d7c..2b116e7a5 100644
> --- a/tests/oss-fuzz/automake.mk
> +++ b/tests/oss-fuzz/automake.mk
> @@ -2,7 +2,6 @@ OSS_FUZZ_TARGETS = \
>         tests/oss-fuzz/flow_extract_target \
>         tests/oss-fuzz/json_parser_target \
>         tests/oss-fuzz/ofp_print_target \
> -       tests/oss-fuzz/expr_parse_target \
>         tests/oss-fuzz/odp_target \
>         tests/oss-fuzz/miniflow_target \
>         tests/oss-fuzz/ofctl_parse_target
> @@ -27,13 +26,6 @@ tests_oss_fuzz_ofp_print_target_SOURCES = \
>  tests_oss_fuzz_ofp_print_target_LDADD = lib/libopenvswitch.la
>  tests_oss_fuzz_ofp_print_target_LDFLAGS = $(LIB_FUZZING_ENGINE) -lc++
>
> -tests_oss_fuzz_expr_parse_target_SOURCES = \
> -        tests/oss-fuzz/expr_parse_target.c \
> -        tests/oss-fuzz/fuzzer.h
> -tests_oss_fuzz_expr_parse_target_LDADD = lib/libopenvswitch.la \
> -                                         ovn/lib/libovn.la
> -tests_oss_fuzz_expr_parse_target_LDFLAGS = $(LIB_FUZZING_ENGINE) -lc++
> -
>  tests_oss_fuzz_odp_target_SOURCES = \
>          tests/oss-fuzz/odp_target.c \
>          tests/oss-fuzz/fuzzer.h
> @@ -56,11 +48,9 @@ EXTRA_DIST += \
>         tests/oss-fuzz/config/flow_extract_target.options \
>         tests/oss-fuzz/config/json_parser_target.options \
>         tests/oss-fuzz/config/ofp_print_target.options \
> -       tests/oss-fuzz/config/expr_parse_target.options \
>         tests/oss-fuzz/config/odp_target.options \
>         tests/oss-fuzz/config/miniflow_target.options \
>          tests/oss-fuzz/config/ofctl_parse_target.options \
>         tests/oss-fuzz/config/ovs.dict \
> -       tests/oss-fuzz/config/expr.dict \
>         tests/oss-fuzz/config/odp.dict \
>         tests/oss-fuzz/config/ofp-flow.dict
> diff --git a/tests/oss-fuzz/config/expr.dict
> b/tests/oss-fuzz/config/expr.dict
> deleted file mode 100644
> index 03741ad7d..000000000
> --- a/tests/oss-fuzz/config/expr.dict
> +++ /dev/null
> @@ -1,120 +0,0 @@
> -" = "
> -" = ("
> -" = dns_lookup();"
> -" { "
> -" };"
> -"!"
> -"!="
> -"$"
> -"&&"
> -"("
> -"()"
> -")"
> -"),commit,table=,zone=NXM_NX_REG)"
> -");"
> -", "
> -", meter=\"\""
> -","
> -",bucket=bucket_id=,weight:100,actions=ct(nat(dst="
> -"--"
> -".."
> -"/"
> -"/%"
> -"0"
> -":"
> -"<"
> -"<->"
> -"<="
> -"="
> -"=="
> -"=r"
> -">"
> -">="
> -"["
> -"\x00"
> -"\x28"
> -"]"
> -"]:"
> -"allow"
> -"arp"
> -"bool"
> -"bswap "
> -"bswap %0"
> -"cc"
> -"clone"
> -"ct_clear"
> -"ct_clear;"
> -"ct_commit"
> -"ct_commit("
> -"ct_dnat"
> -"ct_label"
> -"ct_label="
> -"ct_lb"
> -"ct_mark"
> -"ct_mark="
> -"ct_mark=%#x"
> -"ct_next"
> -"ct_next;"
> -"ct_snat"
> -"decimal"
> -"dhcpv6_stateful"
> -"dhcpv6_stateless"
> -"dns_lookup"
> -"drop"
> -"drop;"
> -"egress"
> -"error("
> -"get_arp"
> -"get_nd"
> -"hexadecimal"
> -"icmp4"
> -"icmp6"
> -"ingress"
> -"ip.ttl"
> -"ip.ttl--;"
> -"ipv4"
> -"ipv6"
> -"log"
> -"log("
> -"mac"
> -"meter"
> -"name"
> -"name=\"\", "
> -"nd_na"
> -"nd_na_router"
> -"nd_ns"
> -"next"
> -"next();"
> -"next(pipeline=, table=);"
> -"next;"
> -"output"
> -"output;"
> -"pipeline"
> -"put_arp"
> -"put_dhcp_opts"
> -"put_dhcpv6_opts"
> -"put_nd"
> -"put_nd_ra_opts"
> -"reject"
> -"set_meter"
> -"set_meter();"
> -"set_meter(, );"
> -"set_meter(,);"
> -"set_queue"
> -"set_queue();"
> -"severity"
> -"severity="
> -"slaac"
> -"static_routes"
> -"str"
> -"table"
> -"tcp_reset"
> -"type=select,selection_method=dp_hash"
> -"uint16"
> -"uint32"
> -"uint8"
> -"verdict"
> -"verdict=, "
> -"{"
> -"||"
> -"}"
> diff --git a/tests/oss-fuzz/config/expr_parse_target.options
> b/tests/oss-fuzz/config/expr_parse_target.options
> deleted file mode 100644
> index fc254e84f..000000000
> --- a/tests/oss-fuzz/config/expr_parse_target.options
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -[libfuzzer]
> -dict = expr.dict
> -close_fd_mask = 3
> diff --git a/tests/oss-fuzz/expr_parse_target.c
> b/tests/oss-fuzz/expr_parse_target.c
> deleted file mode 100644
> index 170186d9d..000000000
> --- a/tests/oss-fuzz/expr_parse_target.c
> +++ /dev/null
> @@ -1,464 +0,0 @@
> -#include <config.h>
> -#include "fuzzer.h"
> -#include <errno.h>
> -#include <getopt.h>
> -#include <sys/wait.h>
> -
> -#include "command-line.h"
> -#include "dp-packet.h"
> -#include "fatal-signal.h"
> -#include "flow.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/actions.h"
> -#include "ovn/expr.h"
> -#include "ovn/lex.h"
> -#include "ovn/logical-fields.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/extend-table.h"
> -#include "openvswitch/shash.h"
> -#include "simap.h"
> -#include "util.h"
> -
> -static void
> -compare_token(const struct lex_token *a, const struct lex_token *b)
> -{
> -    if (a->type != b->type) {
> -        fprintf(stderr, "type differs: %d -> %d\n", a->type, b->type);
> -        return;
> -    }
> -
> -    if (!((a->s && b->s && !strcmp(a->s, b->s))
> -          || (!a->s && !b->s))) {
> -        fprintf(stderr, "string differs: %s -> %s\n",
> -                a->s ? a->s : "(null)",
> -                b->s ? b->s : "(null)");
> -        return;
> -    }
> -
> -    if (a->type == LEX_T_INTEGER || a->type == LEX_T_MASKED_INTEGER) {
> -        if (memcmp(&a->value, &b->value, sizeof a->value)) {
> -            fprintf(stderr, "value differs\n");
> -            return;
> -        }
> -
> -        if (a->type == LEX_T_MASKED_INTEGER
> -            && memcmp(&a->mask, &b->mask, sizeof a->mask)) {
> -            fprintf(stderr, "mask differs\n");
> -            return;
> -        }
> -
> -        if (a->format != b->format
> -            && !(a->format == LEX_F_HEXADECIMAL
> -                 && b->format == LEX_F_DECIMAL
> -                 && a->value.integer == 0)) {
> -            fprintf(stderr, "format differs: %d -> %d\n",
> -                    a->format, b->format);
> -        }
> -    }
> -}
> -
> -static void
> -test_lex(const char *input)
> -{
> -    struct ds output;
> -
> -    ds_init(&output);
> -    struct lexer lexer;
> -
> -    lexer_init(&lexer, input);
> -    ds_clear(&output);
> -    while (lexer_get(&lexer) != LEX_T_END) {
> -        size_t len = output.length;
> -        lex_token_format(&lexer.token, &output);
> -
> -        /* Check that the formatted version can really be parsed back
> -         * losslessly. */
> -        if (lexer.token.type != LEX_T_ERROR) {
> -            const char *s = ds_cstr(&output) + len;
> -            struct lexer l2;
> -
> -            lexer_init(&l2, s);
> -            lexer_get(&l2);
> -            compare_token(&lexer.token, &l2.token);
> -            lexer_destroy(&l2);
> -        }
> -        ds_put_char(&output, ' ');
> -    }
> -    lexer_destroy(&lexer);
> -
> -    ds_chomp(&output, ' ');
> -    puts(ds_cstr(&output));
> -    ds_destroy(&output);
> -}
> -
> -static void
> -create_symtab(struct shash *symtab)
> -{
> -    ovn_init_symtab(symtab);
> -
> -    /* For negative testing. */
> -    expr_symtab_add_field(symtab, "bad_prereq", MFF_XREG0, "xyzzy",
> false);
> -    expr_symtab_add_field(symtab, "self_recurse", MFF_XREG0,
> -                          "self_recurse != 0", false);
> -    expr_symtab_add_field(symtab, "mutual_recurse_1", MFF_XREG0,
> -                          "mutual_recurse_2 != 0", false);
> -    expr_symtab_add_field(symtab, "mutual_recurse_2", MFF_XREG0,
> -                          "mutual_recurse_1 != 0", false);
> -    expr_symtab_add_string(symtab, "big_string", MFF_XREG0, NULL);
> -}
> -
> -static void
> -create_gen_opts(struct hmap *dhcp_opts, struct hmap *dhcpv6_opts,
> -                struct hmap *nd_ra_opts)
> -{
> -    hmap_init(dhcp_opts);
> -    dhcp_opt_add(dhcp_opts, "offerip", 0, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "netmask", 1, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "router",  3, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "dns_server", 6, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "log_server", 7, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "lpr_server",  9, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "domain_name", 15, "str");
> -    dhcp_opt_add(dhcp_opts, "swap_server", 16, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "policy_filter", 21, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "router_solicitation",  32, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "nis_server", 41, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "ntp_server", 42, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "server_id",  54, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "tftp_server", 66, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "classless_static_route", 121,
> "static_routes");
> -    dhcp_opt_add(dhcp_opts, "ip_forward_enable",  19, "bool");
> -    dhcp_opt_add(dhcp_opts, "router_discovery", 31, "bool");
> -    dhcp_opt_add(dhcp_opts, "ethernet_encap", 36, "bool");
> -    dhcp_opt_add(dhcp_opts, "default_ttl",  23, "uint8");
> -    dhcp_opt_add(dhcp_opts, "tcp_ttl", 37, "uint8");
> -    dhcp_opt_add(dhcp_opts, "mtu", 26, "uint16");
> -    dhcp_opt_add(dhcp_opts, "lease_time",  51, "uint32");
> -    dhcp_opt_add(dhcp_opts, "wpad", 252, "str");
> -
> -    /* DHCPv6 options. */
> -    hmap_init(dhcpv6_opts);
> -    dhcp_opt_add(dhcpv6_opts, "server_id",  2, "mac");
> -    dhcp_opt_add(dhcpv6_opts, "ia_addr",  5, "ipv6");
> -    dhcp_opt_add(dhcpv6_opts, "dns_server",  23, "ipv6");
> -    dhcp_opt_add(dhcpv6_opts, "domain_search",  24, "str");
> -
> -    /* IPv6 ND RA options. */
> -    hmap_init(nd_ra_opts);
> -    nd_ra_opts_init(nd_ra_opts);
> -}
> -
> -static void
> -create_addr_sets(struct shash *addr_sets)
> -{
> -    shash_init(addr_sets);
> -
> -    static const char *const addrs1[] = {
> -        "10.0.0.1", "10.0.0.2", "10.0.0.3",
> -    };
> -    static const char *const addrs2[] = {
> -        "::1", "::2", "::3",
> -    };
> -    static const char *const addrs3[] = {
> -        "00:00:00:00:00:01", "00:00:00:00:00:02", "00:00:00:00:00:03",
> -    };
> -    static const char *const addrs4[] = { NULL };
> -
> -    expr_const_sets_add(addr_sets, "set1", addrs1, 3, true);
> -    expr_const_sets_add(addr_sets, "set2", addrs2, 3, true);
> -    expr_const_sets_add(addr_sets, "set3", addrs3, 3, true);
> -    expr_const_sets_add(addr_sets, "set4", addrs4, 0, true);
> -}
> -
> -static void
> -create_port_groups(struct shash *port_groups)
> -{
> -    shash_init(port_groups);
> -
> -    static const char *const pg1[] = {
> -        "lsp1", "lsp2", "lsp3",
> -    };
> -    static const char *const pg2[] = { NULL };
> -
> -    expr_const_sets_add(port_groups, "pg1", pg1, 3, false);
> -    expr_const_sets_add(port_groups, "pg_empty", pg2, 0, false);
> -}
> -
> -static bool
> -lookup_port_cb(const void *ports_, const char *port_name, unsigned int
> *portp)
> -{
> -    const struct simap *ports = ports_;
> -    const struct simap_node *node = simap_find(ports, port_name);
> -    if (!node) {
> -        return false;
> -    }
> -    *portp = node->data;
> -    return true;
> -}
> -
> -static bool
> -is_chassis_resident_cb(const void *ports_, const char *port_name)
> -{
> -    const struct simap *ports = ports_;
> -    const struct simap_node *node = simap_find(ports, port_name);
> -    if (node) {
> -        return true;
> -    }
> -    return false;
> -}
> -
> -static void
> -test_parse_actions(const char *input)
> -{
> -    struct shash symtab;
> -    struct hmap dhcp_opts;
> -    struct hmap dhcpv6_opts;
> -    struct hmap nd_ra_opts;
> -    struct simap ports;
> -
> -    create_symtab(&symtab);
> -    create_gen_opts(&dhcp_opts, &dhcpv6_opts, &nd_ra_opts);
> -
> -    /* Initialize group ids. */
> -    struct ovn_extend_table group_table;
> -    ovn_extend_table_init(&group_table);
> -
> -    /* Initialize meter ids for QoS. */
> -    struct ovn_extend_table meter_table;
> -    ovn_extend_table_init(&meter_table);
> -
> -    simap_init(&ports);
> -    simap_put(&ports, "eth0", 5);
> -    simap_put(&ports, "eth1", 6);
> -    simap_put(&ports, "LOCAL", ofp_to_u16(OFPP_LOCAL));
> -
> -    struct ofpbuf ovnacts;
> -    struct expr *prereqs;
> -    char *error;
> -
> -    puts(input);
> -
> -    ofpbuf_init(&ovnacts, 0);
> -
> -    const struct ovnact_parse_params pp = {
> -        .symtab = &symtab,
> -        .dhcp_opts = &dhcp_opts,
> -        .dhcpv6_opts = &dhcpv6_opts,
> -        .nd_ra_opts = &nd_ra_opts,
> -        .n_tables = 24,
> -        .cur_ltable = 10,
> -    };
> -    error = ovnacts_parse_string(input, &pp, &ovnacts, &prereqs);
> -    if (!error) {
> -        /* Convert the parsed representation back to a string and print
> it,
> -         * if it's different from the input. */
> -        struct ds ovnacts_s = DS_EMPTY_INITIALIZER;
> -        ovnacts_format(ovnacts.data, ovnacts.size, &ovnacts_s);
> -        if (strcmp(input, ds_cstr(&ovnacts_s))) {
> -            printf("    formats as %s\n", ds_cstr(&ovnacts_s));
> -        }
> -
> -        /* Encode the actions into OpenFlow and print. */
> -        const struct ovnact_encode_params ep = {
> -            .lookup_port = lookup_port_cb,
> -            .aux = &ports,
> -            .is_switch = true,
> -            .group_table = &group_table,
> -            .meter_table = &meter_table,
> -
> -            .pipeline = OVNACT_P_INGRESS,
> -            .ingress_ptable = 8,
> -            .egress_ptable = 40,
> -            .output_ptable = 64,
> -            .mac_bind_ptable = 65,
> -        };
> -        struct ofpbuf ofpacts;
> -        ofpbuf_init(&ofpacts, 0);
> -        ovnacts_encode(ovnacts.data, ovnacts.size, &ep, &ofpacts);
> -        struct ds ofpacts_s = DS_EMPTY_INITIALIZER;
> -        struct ofpact_format_params fp = { .s = &ofpacts_s };
> -        ofpacts_format(ofpacts.data, ofpacts.size, &fp);
> -        printf("    encodes as %s\n", ds_cstr(&ofpacts_s));
> -        ds_destroy(&ofpacts_s);
> -        ofpbuf_uninit(&ofpacts);
> -
> -        /* Print prerequisites if any. */
> -        if (prereqs) {
> -            struct ds prereqs_s = DS_EMPTY_INITIALIZER;
> -            expr_format(prereqs, &prereqs_s);
> -            printf("    has prereqs %s\n", ds_cstr(&prereqs_s));
> -            ds_destroy(&prereqs_s);
> -        }
> -
> -        /* Now re-parse and re-format the string to verify that it's
> -         * round-trippable. */
> -        struct ofpbuf ovnacts2;
> -        struct expr *prereqs2;
> -        ofpbuf_init(&ovnacts2, 0);
> -        error = ovnacts_parse_string(ds_cstr(&ovnacts_s), &pp, &ovnacts2,
> -                                     &prereqs2);
> -        if (!error) {
> -            struct ds ovnacts2_s = DS_EMPTY_INITIALIZER;
> -            ovnacts_format(ovnacts2.data, ovnacts2.size, &ovnacts2_s);
> -            if (strcmp(ds_cstr(&ovnacts_s), ds_cstr(&ovnacts2_s))) {
> -                printf("    bad reformat: %s\n", ds_cstr(&ovnacts2_s));
> -            }
> -            ds_destroy(&ovnacts2_s);
> -        } else {
> -            printf("    reparse error: %s\n", error);
> -            free(error);
> -        }
> -        expr_destroy(prereqs2);
> -
> -        ovnacts_free(ovnacts2.data, ovnacts2.size);
> -        ofpbuf_uninit(&ovnacts2);
> -        ds_destroy(&ovnacts_s);
> -    } else {
> -        printf("    %s\n", error);
> -        free(error);
> -    }
> -
> -    expr_destroy(prereqs);
> -    ovnacts_free(ovnacts.data, ovnacts.size);
> -    ofpbuf_uninit(&ovnacts);
> -
> -    simap_destroy(&ports);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -    dhcp_opts_destroy(&dhcp_opts);
> -    dhcp_opts_destroy(&dhcpv6_opts);
> -    nd_ra_opts_destroy(&nd_ra_opts);
> -    ovn_extend_table_destroy(&group_table);
> -    ovn_extend_table_destroy(&meter_table);
> -}
> -
> -static void
> -test_parse_expr(const char *input)
> -{
> -    struct shash symtab;
> -    struct shash addr_sets;
> -    struct shash port_groups;
> -    struct simap ports;
> -    struct expr *expr;
> -    char *error;
> -
> -    create_symtab(&symtab);
> -    create_addr_sets(&addr_sets);
> -    create_port_groups(&port_groups);
> -
> -    simap_init(&ports);
> -    simap_put(&ports, "eth0", 5);
> -    simap_put(&ports, "eth1", 6);
> -    simap_put(&ports, "LOCAL", ofp_to_u16(OFPP_LOCAL));
> -    simap_put(&ports, "lsp1", 0x11);
> -    simap_put(&ports, "lsp2", 0x12);
> -    simap_put(&ports, "lsp3", 0x13);
> -
> -    expr = expr_parse_string(input, &symtab, &addr_sets,
> -                             &port_groups, NULL, &error);
> -    if (!error) {
> -        expr = expr_annotate(expr, &symtab, &error);
> -    }
> -    if (!error) {
> -        expr = expr_simplify(expr, is_chassis_resident_cb, &ports);
> -        expr = expr_normalize(expr);
> -        ovs_assert(expr_is_normalized(expr));
> -    }
> -    if (!error) {
> -        struct hmap matches;
> -
> -        expr_to_matches(expr, lookup_port_cb, &ports, &matches);
> -        expr_matches_print(&matches, stdout);
> -        expr_matches_destroy(&matches);
> -    } else {
> -        puts(error);
> -        free(error);
> -    }
> -    expr_destroy(expr);
> -    simap_destroy(&ports);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -    expr_const_sets_destroy(&addr_sets);
> -    shash_destroy(&addr_sets);
> -    expr_const_sets_destroy(&port_groups);
> -    shash_destroy(&port_groups);
> -}
> -
> -static bool
> -lookup_atoi_cb(const void *aux OVS_UNUSED, const char *port_name,
> -               unsigned int *portp)
> -{
> -    *portp = atoi(port_name);
> -    return true;
> -}
> -
> -static void
> -test_expr_to_packets(const char *input)
> -{
> -    struct shash symtab;
> -    create_symtab(&symtab);
> -
> -    struct flow uflow;
> -    char *error = expr_parse_microflow(input, &symtab, NULL, NULL,
> -                                       lookup_atoi_cb, NULL, &uflow);
> -    if (error) {
> -        puts(error);
> -        free(error);
> -        expr_symtab_destroy(&symtab);
> -        shash_destroy(&symtab);
> -        return;
> -    }
> -
> -    uint64_t packet_stub[128 / 8];
> -    struct dp_packet packet;
> -    dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -    flow_compose(&packet, &uflow, NULL, 64);
> -
> -    struct ds output = DS_EMPTY_INITIALIZER;
> -    const uint8_t *buf = dp_packet_data(&packet);
> -    for (int i = 0; i < dp_packet_size(&packet); i++) {
> -        uint8_t val = buf[i];
> -        ds_put_format(&output, "%02"PRIx8, val);
> -    }
> -    puts(ds_cstr(&output));
> -    ds_destroy(&output);
> -    dp_packet_uninit(&packet);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -int
> -LLVMFuzzerTestOneInput(const uint8_t *input_, size_t size)
> -{
> -    /* Bail out if we cannot construct at least a 1 char string. */
> -    const char *input = (const char *) input_;
> -    if (size < 2 || input[size - 1] != '\0' || strchr(input, '\n') ||
> -        strlen(input) != size - 1) {
> -        return 0;
> -    }
> -
> -    /* Disable logging to avoid write to disk. */
> -    static bool isInit = false;
> -    if (!isInit) {
> -        vlog_set_verbosity("off");
> -        isInit = true;
> -    }
> -
> -    /* Parse, annotate, simplify, normalize expr and convert to flows. */
> -    test_parse_expr(input);
> -
> -    /* Parse actions. */
> -    test_parse_actions(input);
> -
> -    /* Test OVN lexer. */
> -    test_lex(input);
> -
> -    /* Expr to packets. */
> -    test_expr_to_packets(input);
> -
> -    return 0;
> -}
> diff --git a/tests/ovn-controller-vtep.at b/tests/ovn-controller-vtep.at
> deleted file mode 100644
> index a3fe8cb88..000000000
> --- a/tests/ovn-controller-vtep.at
> +++ /dev/null
> @@ -1,467 +0,0 @@
> -AT_BANNER([ovn_controller_vtep])
> -
> -# OVN_CONTROLLER_VTEP_START
> -#
> -# Starts the test with a setup with vtep device.  Each test case must
> first
> -# call this macro.
> -#
> -# Uses vtep-ovs to simulate the vtep switch 'br-vtep' with two physical
> ports
> -# 'p0', 'p1'.
> -#
> -# Configures ovn-nb with a logical switch 'br-test'.
> -#
> -#
> -m4_define([OVN_CONTROLLER_VTEP_START],
> -  [
> -   AT_KEYWORDS([ovn])
> -   # this will cause skip when 'make check' using Windows setup.
> -   AT_SKIP_IF([test $HAVE_PYTHON = no])
> -
> -   dnl Create databases (ovn-nb, ovn-sb, vtep).
> -   AT_CHECK([ovsdb-tool create vswitchd.db
> $abs_top_srcdir/vswitchd/vswitch.ovsschema])
> -   for daemon in ovn-nb ovn-sb vtep; do
> -      AT_CHECK([ovsdb-tool create $daemon.db
> $abs_top_srcdir/${daemon%%-*}/${daemon}.ovsschema])
> -   done
> -
> -   dnl Start ovsdb-server.
> -   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file
> --remote=punix:$OVS_RUNDIR/db.sock vswitchd.db vtep.db], [0], [], [stderr])
> -   AT_CHECK([ovsdb-server --detach --no-chdir
> --pidfile=ovsdb-nb-server.pid --log-file=ovsdb-nb-server.log
> --remote=punix:$OVS_RUNDIR/ovnnb_db.sock ovn-nb.db], [0], [], [stderr])
> -   AT_CHECK([ovsdb-server --detach --no-chdir
> --pidfile=ovsdb-sb-server.pid --log-file=ovsdb-sb-server.log
> --remote=punix:$OVS_RUNDIR/ovnsb_db.sock ovn-sb.db ovn-sb.db], [0], [],
> [stderr])
> -   on_exit "kill `cat ovsdb-server.pid` `cat ovsdb-nb-server.pid` `cat
> ovsdb-sb-server.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d
> -/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> -   AT_CAPTURE_FILE([ovsdb-server.log])
> -
> -   dnl Start ovs-vswitchd.
> -   AT_CHECK([ovs-vswitchd --enable-dummy=system --disable-system --detach
> --no-chdir --pidfile --log-file -vvconn -vofproto_dpif], [0], [], [stderr])
> -   AT_CAPTURE_FILE([ovs-vswitchd.log])
> -   on_exit "kill `cat ovs-vswitchd.pid`"
> -   AT_CHECK([[sed < stderr '
> -/ovs_numa|INFO|Discovered /d
> -/vlog|INFO|opened log file/d
> -/vswitchd|INFO|ovs-vswitchd (Open vSwitch)/d
> -/reconnect|INFO|/d
> -/ofproto|INFO|using datapath ID/d
> -/netlink_socket|INFO|netlink: could not enable listening to all nsid/d
> -/ofproto|INFO|datapath ID changed to fedcba9876543210/d']])
> -   AT_CHECK([ovs-vsctl -- add-br br-vtep \
> -              -- set bridge br-vtep datapath-type=dummy
> other-config:datapath-id=fedcba9876543210
> other-config:hwaddr=aa:55:aa:55:00:00
> protocols=[[OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15]]
> fail-mode=secure \
> -              -- add-port br-vtep p0 -- set Interface p0 type=dummy
> ofport_request=1 \
> -              -- add-port br-vtep p1 -- set Interface p1 type=dummy
> ofport_request=2])
> -
> -   dnl Start ovs-vtep.
> -   AT_CHECK([vtep-ctl add-ps br-vtep -- set Physical_Switch br-vtep
> tunnel_ips=1.2.3.4])
> -   AT_CHECK([ovs-vtep --log-file=ovs-vtep.log --pidfile=ovs-vtep.pid
> --detach --no-chdir br-vtep \], [0], [], [stderr])
> -   on_exit "kill `cat ovs-vtep.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d']])
> -   # waits until ovs-vtep starts up.
> -   OVS_WAIT_UNTIL([test -n "`vtep-ctl show | grep Physical_Port`"])
> -
> -   dnl Start ovn-northd.
> -   AT_CHECK([ovn-nbctl ls-add br-test])
> -   AT_CHECK([ovn-northd --detach --no-chdir --pidfile --log-file], [0],
> [], [stderr])
> -   on_exit "kill `cat ovn-northd.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d']])
> -   AT_CAPTURE_FILE([ovn-northd.log])
> -
> -   dnl Start ovn-controllger-vtep.
> -   AT_CHECK([ovn-controller-vtep --detach --no-chdir --pidfile --log-file
> --vtep-db=unix:$OVS_RUNDIR/db.sock
> --ovnsb-db=unix:$OVS_RUNDIR/ovnsb_db.sock], [0], [], [stderr])
> -   AT_CAPTURE_FILE([ovn-controller-vtep.log])
> -   on_exit "kill `cat ovn-controller-vtep.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d
> -/reconnect|INFO|/d']])
> -])
> -
> -# OVN_CONTROLLER_VTEP_STOP
> -#
> -# So many exits... Yeah, we started a lot daemons~
> -#
> -m4_define([OVN_CONTROLLER_VTEP_STOP],
> -  [AT_CHECK([check_logs "$1"])
> -   OVS_APP_EXIT_AND_WAIT([ovs-vtep])
> -   OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -   OVS_APP_EXIT_AND_WAIT([ovn-controller-vtep])
> -   OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -   OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])])
> -
> -# Adds logical port for a vtep gateway chassis in ovn-nb database.
> -#
> -# $1: logical switch name in ovn-nb database
> -# $2: logical port name
> -# $3: physical vtep gateway name
> -# $4: logical switch name on vtep gateway chassis
> -m4_define([OVN_NB_ADD_VTEP_PORT], [
> -AT_CHECK([ovn-nbctl lsp-add $1 $2])
> -
> -AT_CHECK([ovn-nbctl lsp-set-type $2 vtep])
> -AT_CHECK([ovn-nbctl lsp-set-options $2 vtep-physical-switch=$3
> vtep-logical-switch=$4])
> -])
> -
> -##############################################
> -
> -# tests chassis related updates.
> -AT_SETUP([ovn-controller-vtep - chassis])
> -OVN_CONTROLLER_VTEP_START
> -
> -# verifies the initial ovn-sb db configuration.
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl show | grep Chassis`"])
> -AT_CHECK([ovn-sbctl show], [0], [dnl
> -Chassis br-vtep
> -    Encap vxlan
> -        ip: "1.2.3.4"
> -        options: {csum="false"}
> -])
> -
> -# deletes the chassis via ovn-sbctl and check that it is readded back
> -# with the log.
> -AT_CHECK([ovn-sbctl chassis-del br-vtep])
> -OVS_WAIT_UNTIL([test -n "`grep WARN ovn-controller-vtep.log`"])
> -AT_CHECK([sed -n 's/^.*\(|WARN|.*\)$/\1/p' ovn-controller-vtep.log], [0],
> [dnl
> -|WARN|Chassis for VTEP physical switch (br-vtep) disappears, maybe
> deleted by ovn-sbctl, adding it back
> -])
> -
> -# changes the tunnel_ip on physical switch, watches the update of
> chassis's
> -# encap.
> -AT_CHECK([vtep-ctl set Physical_Switch br-vtep tunnel_ips=1.2.3.5])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl show | grep 1\.2\.3\.5`"])
> -AT_CHECK([ovn-sbctl --columns=ip list Encap | cut -d ':' -f2 | tr -d '
> '], [0], [dnl
> -"1.2.3.5"
> -])
> -
> -# adds vlan_bindings to physical ports.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0 --
> bind-ls br-vtep p0 200 lswitch0 -- bind-ls br-vtep p1 300 lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Chassis | grep -- lswitch0`"])
> -AT_CHECK([ovn-sbctl --columns=vtep_logical_switches list Chassis | cut -d
> ':' -f2 | tr -d ' ' ], [0], [dnl
> -[[lswitch0]]
> -])
> -
> -# adds another logical switch and new vlan_bindings.
> -AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep p0 300 lswitch1])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Chassis | grep -- lswitch1`"])
> -AT_CHECK([ovn-sbctl --columns=vtep_logical_switches list Chassis | cut -d
> ':' -f2 | tr -d ' '], [0], [dnl
> -[[lswitch0,lswitch1]]
> -])
> -
> -# unbinds one port from lswitch0, nothing should change.
> -AT_CHECK([vtep-ctl unbind-ls br-vtep p0 200])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl --columns=vlan_bindings list
> physical_port p0 | grep -- '200='`"])
> -AT_CHECK([ovn-sbctl --columns=vtep_logical_switches list Chassis | cut -d
> ':' -f2 | tr -d ' ' ], [0], [dnl
> -[[lswitch0,lswitch1]]
> -])
> -
> -# unbinds all ports from lswitch0.
> -AT_CHECK([vtep-ctl unbind-ls br-vtep p0 100 -- unbind-ls br-vtep p1 300])
> -OVS_WAIT_UNTIL([test -z "`ovn-sbctl list Chassis | grep --
> br-vtep_lswitch0`"])
> -AT_CHECK([ovn-sbctl --columns=vtep_logical_switches list Chassis | cut -d
> ':' -f2 | tr -d ' ' ], [0], [dnl
> -[[lswitch1]]
> -])
> -
> -# unbinds all ports from lswitch1.
> -AT_CHECK([vtep-ctl unbind-ls br-vtep p0 300])
> -OVS_WAIT_UNTIL([test -z "`ovn-sbctl list Chassis | grep --
> br-vtep_lswitch1`"])
> -AT_CHECK([ovn-sbctl --columns=vtep_logical_switches list Chassis | cut -d
> ':' -f2 | tr -d ' '], [0], [dnl
> -[[]]
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP([/Chassis for VTEP physical switch (br-vtep)
> disappears/d])
> -AT_CLEANUP
> -
> -
> -# Tests binding updates.
> -AT_SETUP([ovn-controller-vtep - binding 1])
> -OVN_CONTROLLER_VTEP_START
> -
> -# adds logical switch 'lswitch0' and vlan_bindings.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0 --
> bind-ls br-vtep p1 300 lswitch0])
> -# adds logical switch port in ovn-nb database, and sets the type and
> options.
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0])
> -ovn-sbctl --timeout=10 wait-until Port_Binding br-vtep_lswitch0
> chassis!='[[]]'
> -# should see one binding, associated to chassis of 'br-vtep'.
> -chassis_uuid=$(ovn-sbctl --columns=_uuid list Chassis br-vtep | cut -d
> ':' -f2 | tr -d ' ')
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=chassis list Port_Binding
> br-vtep_lswitch0 | cut -d ':' -f2 | tr -d ' '], [0], [dnl
> -${chassis_uuid}
> -])
> -
> -# adds another logical switch 'lswitch1' and vlan_bindings.
> -AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep p0 200 lswitch1])
> -# adds logical switch port in ovn-nb database for lswitch1.
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch1], [br-vtep], [lswitch1])
> -ovn-sbctl --timeout=10 wait-until Port_Binding br-vtep_lswitch1
> chassis!='[[]]'
> -# This is allowed, but not recommended, to have two vlan_bindings (to
> different vtep logical switches)
> -# from one vtep gateway physical port in one ovn-nb logical swithch.
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=chassis list Port_Binding | cut -d
> ':' -f2 | tr -d ' ' | sort], [0], [dnl
> -
> -${chassis_uuid}
> -${chassis_uuid}
> -])
> -
> -# adds another logical switch port in ovn-nb database for lswitch0.
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0_dup], [br-vtep],
> [lswitch0])
> -ovn-sbctl --timeout=10 wait-until Port_Binding br-vtep_lswitch0_dup
> chassis!='[[]]'
> -# it is not allowed to have more than one ovn-nb logical port for the same
> -# vtep logical switch on a vtep gateway chassis, so should still see only
> -# two port_binding entries bound.
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=chassis list Port_Binding | cut -d
> ':' -f2 | tr -d ' ' | sort | sort -d], [0], [dnl
> -
> -
> -[[]]
> -${chassis_uuid}
> -${chassis_uuid}
> -])
> -# confirms the warning log.
> -AT_CHECK([sed -n 's/^.*\(|WARN|.*\)$/\1/p' ovn-controller-vtep.log | sed
> 's/([[-_0-9a-z]][[-_0-9a-z]]*)/()/g' | uniq], [0], [dnl
> -|WARN|logical switch (), on vtep gateway chassis () has already been
> associated with logical port (), ignore logical port ()
> -])
> -
> -# deletes physical ports from vtep.
> -AT_CHECK([ovs-vsctl del-port p0 -- del-port p1])
> -OVS_WAIT_UNTIL([test -z "`ovn-sbctl list Chassis | grep --
> br-vtep_lswitch`"])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl list physical_port p0`"])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl list physical_port p1`"])
> -# should see empty chassis column in both binding entries.
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=chassis list Port_Binding | cut -d
> ':' -f2 | tr -d ' ' | sort], [0], [dnl
> -
> -
> -[[]]
> -[[]]
> -[[]]
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP([/has already been associated with logical
> port/d])
> -AT_CLEANUP
> -
> -
> -# Tests corner case: Binding the vtep logical switch from two different
> -# datapath.
> -AT_SETUP([ovn-controller-vtep - binding 2])
> -OVN_CONTROLLER_VTEP_START
> -
> -# adds logical switch 'lswitch0' and vlan_bindings.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0])
> -# adds logical switch port in ovn-nb database, and sets the type and
> options.
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0])
> -ovn-sbctl --timeout=10 wait-until Port_Binding br-vtep_lswitch0
> chassis!='[[]]'
> -
> -# adds another lswitch 'br-void' in ovn-nb database.
> -AT_CHECK([ovn-nbctl ls-add br-void])
> -# adds another vtep pswitch 'br-vtep-void' in vtep database.
> -AT_CHECK([vtep-ctl add-ps br-vtep-void -- add-port br-vtep-void p0-void
> -- bind-ls br-vtep-void p0-void 100 lswitch0])
> -# adds a conflicting logical port (both br-vtep_lswitch0 and
> br-vtep-void_lswitch0
> -# are bound to the same logical switch, but they are on different
> datapath).
> -OVN_NB_ADD_VTEP_PORT([br-void], [br-vtep-void_lswitch0], [br-vtep-void],
> [lswitch0])
> -ovn-sbctl --timeout=10 wait-until Port_Binding br-vtep_lswitch0
> -OVS_WAIT_UNTIL([test -n "`grep WARN ovn-controller-vtep.log`"])
> -# confirms the warning log.
> -AT_CHECK([sed -n 's/^.*\(|WARN|.*\)$/\1/p' ovn-controller-vtep.log | sed
> 's/([[-_0-9a-z]][[-_0-9a-z]]*)/()/g;s/(with tunnel key
> [[0-9]][[0-9]]*)/()/g' | uniq], [0], [dnl
> -|WARN|logical switch (), on vtep gateway chassis () has already been
> associated with logical datapath (), ignore logical port () which belongs
> to logical datapath ()
> -])
> -
> -# then deletes 'br-void' and 'br-vtep-void', should see 'br-vtep_lswitch0'
> -# bound correctly.
> -AT_CHECK([ovn-nbctl ls-del br-void])
> -# adds another vtep pswitch 'br-vtep-void' in vtep database.
> -AT_CHECK([vtep-ctl del-ps br-vtep-void])
> -OVS_WAIT_UNTIL([test -z "`ovn-sbctl list Port_Binding | grep
> br-vtep-void_lswitch0`"])
> -chassis_uuid=$(ovn-sbctl --columns=_uuid list Chassis br-vtep | cut -d
> ':' -f2 | tr -d ' ')
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=chassis list Port_Binding
> br-vtep_lswitch0 | cut -d ':' -f2 | tr -d ' '], [0], [dnl
> -${chassis_uuid}
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP([/has already been associated with logical
> datapath/d])
> -AT_CLEANUP
> -
> -
> -# Tests vtep module vtep logical switch tunnel key update.
> -AT_SETUP([ovn-controller-vtep - vtep-lswitch])
> -OVN_CONTROLLER_VTEP_START
> -
> -# creates the logical switch in vtep and adds the corresponding logical
> -# port to 'br-test'.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0])
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding  | grep --
> br-vtep_lswitch0`"])
> -
> -# retrieves the expected tunnel key.
> -datapath_uuid=$(ovn-sbctl --columns=datapath list Port_Binding
> br-vtep_lswitch0 | cut -d ':' -f2 | tr -d ' ')
> -tunnel_key=$(ovn-sbctl --columns=tunnel_key list Datapath_Binding
> ${datapath_uuid} | cut -d ':' -f2 | tr -d ' ')
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl --columns=tunnel_key list
> Logical_Switch | grep 0`"])
> -# checks the vtep logical switch tunnel key configuration.
> -AT_CHECK_UNQUOTED([vtep-ctl --columns=tunnel_key list Logical_Switch |
> cut -d ':' -f2 | tr -d ' '], [0], [dnl
> -${tunnel_key}
> -])
> -
> -# creates a second physical switch in vtep database, and binds its p0
> vlan-100
> -# to the same logical switch 'lswitch0'.
> -AT_CHECK([vtep-ctl add-ps br-vtep-void -- add-port br-vtep-void p0 --
> bind-ls br-vtep-void p0 100 lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl --columns=name list Chassis  | grep
> -- br-vtep-void`"])
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep-void_lswitch0], [br-vtep-void],
> [lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding  | grep --
> br-vtep-void_lswitch0`"])
> -
> -# checks the vtep logical switch tunnel key configuration.
> -AT_CHECK_UNQUOTED([vtep-ctl --columns=tunnel_key list Logical_Switch |
> cut -d ':' -f2 | tr -d ' '], [0], [dnl
> -${tunnel_key}
> -])
> -
> -# now, deletes br-vtep-void.
> -AT_CHECK([vtep-ctl del-ps br-vtep-void])
> -OVS_WAIT_UNTIL([test -z "`ovn-sbctl --columns=name list Chassis  | grep
> -- br-vtep-void`"])
> -# checks the vtep logical switch tunnel key configuration.
> -AT_CHECK_UNQUOTED([vtep-ctl --columns=tunnel_key list Logical_Switch |
> cut -d ':' -f2 | tr -d ' '], [0], [dnl
> -${tunnel_key}
> -])
> -
> -# changes the ovn-nb logical port type so that it is no longer
> -# vtep port.
> -AT_CHECK([ovn-nbctl lsp-set-type br-vtep_lswitch0 ""])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl --columns=tunnel_key list
> Logical_Switch | grep 1`"])
> -# now should see the tunnel key reset.
> -AT_CHECK([vtep-ctl --columns=tunnel_key list Logical_Switch | cut -d ':'
> -f2 | tr -d ' '], [0], [dnl
> -0
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP
> -AT_CLEANUP
> -
> -
> -# Tests vtep module 'Ucast_Macs_Remote's.
> -AT_SETUP([ovn-controller-vtep - vtep-macs 1])
> -OVN_CONTROLLER_VTEP_START
> -
> -# creates a simple logical network with the vtep device and a fake hv
> chassis
> -# 'ch0'.
> -AT_CHECK([ovn-nbctl lsp-add br-test vif0])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-nbctl --timeout=10 --wait=sb sync])
> -AT_CHECK([ovn-sbctl chassis-add ch0 vxlan 1.2.3.5])
> -AT_CHECK([ovn-sbctl lsp-bind vif0 ch0])
> -
> -# creates the logical switch in vtep and adds the corresponding logical
> -# port to 'br-test'.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0])
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding  | grep
> br-vtep_lswitch0`"])
> -
> -# adds another lswitch 'br-void' in ovn-nb database.
> -AT_CHECK([ovn-nbctl ls-add br-void])
> -# adds fake hv chassis 'ch1'.
> -AT_CHECK([ovn-nbctl lsp-add br-void vif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif1 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-nbctl --timeout=10 --wait=sb sync])
> -AT_CHECK([ovn-sbctl chassis-add ch1 vxlan 1.2.3.6])
> -AT_CHECK([ovn-sbctl lsp-bind vif1 ch1])
> -
> -# checks Ucast_Macs_Remote creation.
> -OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Remote | grep _uuid`"])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' '], [0], [dnl
> -"f0:ab:cd:ef:01:02"
> -])
> -
> -# checks physical locator creation.
> -OVS_WAIT_UNTIL([test -n "`vtep-ctl list Physical_Locator | grep _uuid`"])
> -AT_CHECK([vtep-ctl --columns=dst_ip list Physical_Locator | cut -d ':'
> -f2 | tr -d ' ' | grep -v 1.2.3.4 | sed '/^$/d'], [0], [dnl
> -"1.2.3.5"
> -])
> -
> -# checks tunnel creation by ovs-vtep.
> -OVS_WAIT_UNTIL([test -n "`ovs-vsctl list Interface bfd1.2.3.5`"])
> -AT_CHECK([ovs-vsctl --columns=options list Interface bfd1.2.3.5 | cut -d
> ':' -f2 | tr -d ' '], [0], [dnl
> -{remote_ip="1.2.3.5"}
> -])
> -
> -# adds another mac to logical switch port.
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0 f0:ab:cd:ef:01:02
> f0:ab:cd:ef:01:03])
> -OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Remote | grep 03`"])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' ' | sort], [0], [dnl
> -
> -"f0:ab:cd:ef:01:02"
> -"f0:ab:cd:ef:01:03"
> -])
> -
> -# removes one mac to logical switch port.
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0 f0:ab:cd:ef:01:03])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl --columns=MAC list Ucast_Macs_Remote |
> grep 02`"])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' ' | sort], [0], [dnl
> -"f0:ab:cd:ef:01:03"
> -])
> -
> -# migrates mac to logical switch port vif1 on 'br-void'.
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif1 f0:ab:cd:ef:01:03])
> -OVS_WAIT_UNTIL([test -z "`vtep-ctl --columns=MAC list Ucast_Macs_Remote |
> grep 03`"])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' ' | sort], [0], [dnl
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP
> -AT_CLEANUP
> -
> -
> -# Tests vtep module 'Ucast_Macs_Remote's (corner cases).
> -AT_SETUP([ovn-controller-vtep - vtep-macs 2])
> -OVN_CONTROLLER_VTEP_START
> -
> -# creates a simple logical network with the vtep device and a fake hv
> chassis
> -# 'ch0'.
> -AT_CHECK([ovn-nbctl lsp-add br-test vif0])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-nbctl --timeout=10 --wait=sb sync])
> -AT_CHECK([ovn-sbctl chassis-add ch0 vxlan 1.2.3.5])
> -AT_CHECK([ovn-sbctl lsp-bind vif0 ch0])
> -
> -# creates another vif in the same logical switch with duplicate mac.
> -AT_CHECK([ovn-nbctl lsp-add br-test vif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif1 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-nbctl --timeout=10 --wait=sb sync])
> -AT_CHECK([ovn-sbctl lsp-bind vif1 ch0])
> -
> -# creates the logical switch in vtep and adds the corresponding logical
> -# port to 'br-test'.
> -AT_CHECK([vtep-ctl add-ls lswitch0 -- bind-ls br-vtep p0 100 lswitch0])
> -OVN_NB_ADD_VTEP_PORT([br-test], [br-vtep_lswitch0], [br-vtep], [lswitch0])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding  | grep
> br-vtep_lswitch0`"])
> -
> -# checks Ucast_Macs_Remote creation.  Should still only be one entry,
> since duplicate
> -# mac in the same logical switch is not allowed.
> -OVS_WAIT_UNTIL([test -n "`vtep-ctl list Ucast_Macs_Remote | grep _uuid`"])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' '], [0], [dnl
> -"f0:ab:cd:ef:01:02"
> -])
> -# confirms the warning log.
> -OVS_WAIT_UNTIL([test -n "`grep WARN ovn-controller-vtep.log`"])
> -AT_CHECK([sed -n 's/^.*\(|WARN|.*\)$/\1/p' ovn-controller-vtep.log | sed
> 's/([[-_:0-9a-z]][[-_:0-9a-z]]*)/()/g' | uniq], [0], [dnl
> -|WARN|MAC address () has already been known to be on logical port () in
> the same logical datapath, so just ignore this logical port ()
> -])
> -
> -# deletes vif1.
> -AT_CHECK([ovn-nbctl lsp-del vif1])
> -
> -# adds another lswitch 'br-void' in ovn-nb database.
> -AT_CHECK([ovn-nbctl ls-add br-void])
> -# adds fake hv chassis 'ch1' and vif1 with same mac address as vif0.
> -AT_CHECK([ovn-nbctl lsp-add br-void vif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif1 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-nbctl --timeout=10 --wait=sb sync])
> -AT_CHECK([ovn-sbctl chassis-add ch1 vxlan 1.2.3.6])
> -AT_CHECK([ovn-sbctl lsp-bind vif1 ch1])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding | grep vif1`"])
> -
> -# creates another logical switch in vtep and adds the corresponding
> logical
> -# port to 'br-void'.
> -AT_CHECK([vtep-ctl add-ls lswitch1 -- bind-ls br-vtep p0 200 lswitch1])
> -OVN_NB_ADD_VTEP_PORT([br-void], [br-void_lswitch1], [br-vtep], [lswitch1])
> -OVS_WAIT_UNTIL([test -n "`ovn-sbctl list Port_Binding  | grep
> br-void_lswitch1`"])
> -
> -# checks Ucast_Macs_Remote creation.  Should see two entries since it is
> allowed
> -# to have duplicate macs in different logical switches.
> -OVS_WAIT_UNTIL([test `vtep-ctl --columns=MAC list Ucast_Macs_Remote |
> grep 02 | wc -l` -gt 1])
> -AT_CHECK([vtep-ctl --columns=MAC list Ucast_Macs_Remote | cut -d ':' -f2-
> | tr -d ' ' | sort], [0], [dnl
> -
> -"f0:ab:cd:ef:01:02"
> -"f0:ab:cd:ef:01:02"
> -])
> -
> -OVN_CONTROLLER_VTEP_STOP([/has already been known to be on logical
> port/d])
> -AT_CLEANUP
> diff --git a/tests/ovn-controller.at b/tests/ovn-controller.at
> deleted file mode 100644
> index 343c2abed..000000000
> --- a/tests/ovn-controller.at
> +++ /dev/null
> @@ -1,294 +0,0 @@
> -AT_BANNER([ovn-controller])
> -
> -AT_SETUP([ovn-controller - ovn-bridge-mappings])
> -AT_KEYWORDS([ovn])
> -ovn_init_db ovn-sb
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0 \
> -    -- add-br br-eth1 \
> -    -- add-br br-eth2
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -# Waits until the OVS database contains exactly the specified patch ports.
> -# Each argument should be of the form BRIDGE PORT PEER.
> -check_patches () {
> -    # Generate code to check that the set of patch ports is exactly as
> -    # specified.
> -    echo 'ovs-vsctl -f csv -d bare --no-headings --columns=name find
> Interface type=patch | sort' > query
> -    for patch
> -    do
> -        echo $patch
> -    done | cut -d' ' -f 2 | sort > expout
> -
> -    # Generate code to verify that the configuration of each patch
> -    # port is correct.
> -    for patch
> -    do
> -        set $patch; bridge=$1 port=$2 peer=$3
> -        echo >>query "ovs-vsctl iface-to-br $port -- get Interface $port
> type options"
> -        echo >>expout "$bridge
> -patch
> -{peer=$peer}"
> -    done
> -
> -    # Run the query until we get the expected result (or until a timeout).
> -    #
> -    # (We use sed to drop all "s from output because ovs-vsctl quotes some
> -    # of the port names but not others.)
> -    AT_CAPTURE_FILE([query])
> -    AT_CAPTURE_FILE([expout])
> -    AT_CAPTURE_FILE([stdout])
> -    OVS_WAIT_UNTIL([. ./query | sed 's/"//g' > stdout #"
> -                    diff -u stdout expout >/dev/null])
> -}
> -
> -# Make sure that the configured bridge mappings in the Open_vSwitch db
> -# is mirrored into the Chassis record in the OVN_Southbound db.
> -check_bridge_mappings () {
> -    local_mappings=$1
> -    sysid=$(ovs-vsctl get Open_vSwitch . external_ids:system-id)
> -    OVS_WAIT_UNTIL([test x"${local_mappings}" = x$(ovn-sbctl get Chassis
> ${sysid} external_ids:ovn-bridge-mappings | sed -e 's/\"//g')])
> -}
> -
> -# Initially there should be no patch ports.
> -check_patches
> -
> -# Configure two ovn-bridge mappings, but no patch ports should be created
> yet
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0,physnet2:br-eth1])
> -check_bridge_mappings "physnet1:br-eth0,physnet2:br-eth1"
> -check_patches
> -
> -# Create a localnet port, but we should still have no patch ports, as they
> -# won't be created until there's a localnet port on a logical switch with
> -# another logical port bound to this chassis.
> -ovn-sbctl \
> -    -- --id=@dp101 create Datapath_Binding tunnel_key=101 \
> -    -- create Port_Binding datapath=@dp101 logical_port=localnet1
> tunnel_key=1 \
> -        type=localnet options:network_name=physnet1
> -check_patches
> -
> -# Create a localnet port on a logical switch with a port bound to this
> chassis.
> -# Now we should get some patch ports created.
> -ovn-sbctl \
> -    -- --id=@dp102 create Datapath_Binding tunnel_key=102 \
> -    -- create Port_Binding datapath=@dp102 logical_port=localnet2
> tunnel_key=1 \
> -        type=localnet options:network_name=physnet1 \
> -    -- create Port_Binding datapath=@dp102 logical_port=localvif2
> tunnel_key=2
> -ovs-vsctl add-port br-int localvif2 -- set Interface localvif2
> external_ids:iface-id=localvif2
> -check_patches \
> -    'br-int  patch-br-int-to-localnet2 patch-localnet2-to-br-int' \
> -    'br-eth0 patch-localnet2-to-br-int patch-br-int-to-localnet2'
> -
> -# Add logical patch ports to connect new logical datapath.
> -#
> -# OVN no longer uses OVS patch ports to implement logical patch ports, so
> -# the set of OVS patch ports doesn't change.
> -AT_CHECK([ovn-sbctl \
> -    -- --id=@dp1 create Datapath_Binding tunnel_key=1 \
> -    -- --id=@dp2 create Datapath_Binding tunnel_key=2 \
> -    -- create Port_Binding datapath=@dp1 logical_port=foo tunnel_key=1
> type=patch options:peer=bar \
> -    -- create Port_Binding datapath=@dp2 logical_port=bar tunnel_key=2
> type=patch options:peer=foo \
> -    -- create Port_Binding datapath=@dp1 logical_port=dp1vif tunnel_key=3
> \
> -| uuidfilt], [0], [<0>
> -<1>
> -<2>
> -<3>
> -<4>
> -])
> -ovs-vsctl add-port br-int dp1vif -- set Interface dp1vif
> external_ids:iface-id=dp1vif
> -check_patches \
> -    'br-int  patch-br-int-to-localnet2 patch-localnet2-to-br-int' \
> -    'br-eth0 patch-localnet2-to-br-int patch-br-int-to-localnet2'
> -
> -# Delete the mapping and the ovn-bridge-mapping patch ports should go
> away.
> -AT_CHECK([ovs-vsctl remove Open_vSwitch . external-ids
> ovn-bridge-mappings])
> -check_bridge_mappings
> -check_patches
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP_SBOX([hv])
> -OVN_CLEANUP_VSWITCH([main])
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -AT_CLEANUP
> -
> -# Checks that ovn-controller populates datapath-type and iface-types
> -# correctly in the Chassis external-ids column.
> -AT_SETUP([ovn-controller - Chassis external_ids])
> -AT_KEYWORDS([ovn])
> -ovn_init_db ovn-sb
> -
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0 \
> -    -- add-br br-eth1 \
> -    -- add-br br-eth2
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -sysid=$(ovs-vsctl get Open_vSwitch . external_ids:system-id)
> -
> -# Make sure that the datapath_type set in the Bridge table
> -# is mirrored into the Chassis record in the OVN_Southbound db.
> -check_datapath_type () {
> -    datapath_type=$1
> -    chassis_datapath_type=$(ovn-sbctl get Chassis ${sysid}
> external_ids:datapath-type | sed -e 's/"//g') #"
> -    test "${datapath_type}" = "${chassis_datapath_type}"
> -}
> -
> -OVS_WAIT_UNTIL([check_datapath_type ""])
> -
> -ovs-vsctl set Bridge br-int datapath-type=foo
> -OVS_WAIT_UNTIL([check_datapath_type foo])
> -
> -# Change "ovn-bridge-mappings" value. It should not change the
> "datapath-type".
> -ovs-vsctl set Open_vSwitch . external_ids:ovn-bridge-mappings=foo-mapping
> -check_datapath_type foo
> -
> -ovs-vsctl set Bridge br-int datapath-type=bar
> -OVS_WAIT_UNTIL([check_datapath_type bar])
> -
> -ovs-vsctl set Bridge br-int datapath-type=\"\"
> -OVS_WAIT_UNTIL([check_datapath_type ""])
> -
> -# Set the datapath_type in external_ids:ovn-bridge-datapath-type.
> -ovs-vsctl set Open_vSwitch . external_ids:ovn-bridge-datapath-type=foo
> -OVS_WAIT_UNTIL([check_datapath_type foo])
> -
> -# Change the br-int's datapath type to bar.
> -# It should be reset to foo since ovn-bridge-datapath-type is configured.
> -ovs-vsctl set Bridge br-int datapath-type=bar
> -OVS_WAIT_UNTIL([test foo=`ovs-vsctl get Bridge br-int datapath-type`])
> -OVS_WAIT_UNTIL([check_datapath_type foo])
> -
> -ovs-vsctl set Open_vSwitch . external_ids:ovn-bridge-datapath-type=foobar
> -OVS_WAIT_UNTIL([test foobar=`ovs-vsctl get Bridge br-int datapath-type`])
> -OVS_WAIT_UNTIL([check_datapath_type foobar])
> -
> -expected_iface_types=$(ovs-vsctl get Open_vSwitch . iface_types | tr -d
> '[[]] ""')
> -echo "expected_iface_types = ${expected_iface_types}"
> -chassis_iface_types=$(ovn-sbctl get Chassis ${sysid}
> external_ids:iface-types | sed -e 's/\"//g')
> -echo "chassis_iface_types = ${chassis_iface_types}"
> -AT_CHECK([test "${expected_iface_types}" = "${chassis_iface_types}"])
> -
> -# Change the value of external_ids:iface-types using ovn-sbctl.
> -# ovn-controller should again set it back to proper one.
> -ovn-sbctl set Chassis ${sysid} external_ids:iface-types="foo"
> -OVS_WAIT_UNTIL([
> -    chassis_iface_types=$(ovn-sbctl get Chassis ${sysid}
> external_ids:iface-types | sed -e 's/\"//g')
> -    echo "chassis_iface_types = ${chassis_iface_types}"
> -    test "${expected_iface_types}" = "${chassis_iface_types}"
> -])
> -
> -# Change the value of external_ids:system-id and make sure it's mirrored
> -# in the Chassis record in the OVN_Southbound database.
> -sysid=${sysid}-foo
> -ovs-vsctl set Open_vSwitch . external-ids:system-id="${sysid}"
> -OVS_WAIT_UNTIL([
> -    chassis_id=$(ovn-sbctl get Chassis "${sysid}" name)
> -    test "${sysid}" = "${chassis_id}"
> -])
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP_SBOX([hv])
> -OVN_CLEANUP_VSWITCH([main])
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -AT_CLEANUP
> -
> -# Checks that ovn-controller correctly maintains the mapping from the
> Encap
> -# table in the Southbound database to OVS in the face of changes on both
> sides
> -AT_SETUP([ovn-controller - change Encap properties])
> -AT_KEYWORDS([ovn])
> -ovn_init_db ovn-sb
> -
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0 \
> -    -- add-br br-eth1 \
> -    -- add-br br-eth2
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -check_tunnel_property () {
> -    test "`ovs-vsctl get interface ovn-fakech-0 $1`" = "$2"
> -}
> -
> -# Start off with a remote chassis supporting STT
> -ovn-sbctl chassis-add fakechassis stt 192.168.0.2
> -OVS_WAIT_UNTIL([check_tunnel_property type stt])
> -
> -# See if we switch to Geneve as the first choice when it is available
> -# With multi-VTEP support we support tunnels with different IPs to the
> -# same chassis, and hence use the IP to annotate the tunnel (along with
> -# the chassis-id in ovn-chassis-id); if we supply a different IP here
> -# we won't be able to co-relate this to the tunnel port that was created
> -# in the previous step and, as a result, will end up creating another
> tunnel,
> -# ie. we can't just lookup using "ovn-fakech-0". So, need to use the same
> IP
> -# as above, i.e 192.168.0.2, here.
> -encap_uuid=$(ovn-sbctl add chassis fakechassis encaps @encap --
> --id=@encap create encap type=geneve ip="192.168.0.2")
> -OVS_WAIT_UNTIL([check_tunnel_property type geneve])
> -
> -# Check that changes within an encap row are propagated
> -ovn-sbctl set encap ${encap_uuid} ip=192.168.0.2
> -OVS_WAIT_UNTIL([check_tunnel_property options:remote_ip
> "\"192.168.0.2\""])
> -
> -# Change the type on the OVS side and check than OVN fixes it
> -ovs-vsctl set interface ovn-fakech-0 type=vxlan
> -OVS_WAIT_UNTIL([check_tunnel_property type geneve])
> -
> -# Delete the port entirely and it should be resurrected
> -ovs-vsctl del-port ovn-fakech-0
> -OVS_WAIT_UNTIL([check_tunnel_property type geneve])
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP_SBOX([hv])
> -OVN_CLEANUP_VSWITCH([main])
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -AT_CLEANUP
> -
> -# Check ovn-controller connection status to Southbound database
> -AT_SETUP([ovn-controller - check sbdb connection])
> -AT_KEYWORDS([ovn])
> -ovn_init_db ovn-sb
> -
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0 \
> -    -- add-br br-eth1 \
> -    -- add-br br-eth2
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -check_sbdb_connection () {
> -    test "$(ovs-appctl -t ovn-controller connection-status)" = "$1"
> -}
> -
> -OVS_WAIT_UNTIL([check_sbdb_connection connected])
> -
> -ovs-vsctl set open . external_ids:ovn-remote=tcp:192.168.0.10:6642
> -OVS_WAIT_UNTIL([check_sbdb_connection 'not connected'])
> -
> -# reset the remote for clean-up
> -ovs-vsctl set open .
> external_ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock
> -# Gracefully terminate daemons
> -OVN_CLEANUP_SBOX([hv])
> -OVN_CLEANUP_VSWITCH([main])
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -AT_CLEANUP
> diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at
> deleted file mode 100644
> index 7dba42c99..000000000
> --- a/tests/ovn-macros.at
> +++ /dev/null
> @@ -1,180 +0,0 @@
> -# OVN_CLEANUP_VSWITCH(sim)
> -#
> -# Gracefully terminate vswitch daemons in the
> -# specified sandbox.
> -m4_define([OVN_CLEANUP_VSWITCH],[
> -    as $1
> -    OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
> -    OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -])
> -
> -# OVN_CLEANUP_SBOX(sbox)
> -#
> -# Gracefully terminate OVN daemons in the specified
> -# sandbox instance. The sandbox name "vtep" is treated
> -# as a special case, and is assumed to have ovn-controller-vtep
> -# and ovs-vtep daemons running instead of ovn-controller.
> -m4_define([OVN_CLEANUP_SBOX],[
> -    as $1
> -    if test "$1" = "vtep"; then
> -        OVS_APP_EXIT_AND_WAIT([ovn-controller-vtep])
> -        OVS_APP_EXIT_AND_WAIT([ovs-vtep])
> -    else
> -        OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -    fi
> -    OVN_CLEANUP_VSWITCH([$1])
> -])
> -
> -# OVN_CLEANUP(sim [, sim ...])
> -#
> -# Gracefully terminate all OVN daemons, including those in the
> -# specified sandbox instances.
> -m4_define([OVN_CLEANUP],[
> -    m4_foreach([sbox], [$@], [
> -        OVN_CLEANUP_SBOX([sbox])
> -    ])
> -    as ovn-sb
> -    OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -    as ovn-nb
> -    OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -    as northd
> -    OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -    as northd-backup
> -    OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -    OVN_CLEANUP_VSWITCH([main])
> -])
> -
> -m4_divert_push([PREPARE_TESTS])
> -
> -# ovn_init_db DATABASE
> -#
> -# Creates and initializes the given DATABASE (one of "ovn-sb" or
> "ovn-nb"),
> -# starts its ovsdb-server instance, and sets the appropriate environment
> -# variable (OVN_SB_DB or OVN_NB_DB) so that ovn-sbctl or ovn-nbctl uses
> the
> -# database by default.
> -#
> -# Usually invoked from ovn_start.
> -ovn_init_db () {
> -    echo "creating $1 database"
> -    local d=$ovs_base/$1
> -    mkdir "$d" || return 1
> -    : > "$d"/.$1.db.~lock~
> -    as $1 ovsdb-tool create "$d"/$1.db "$abs_top_srcdir"/ovn/$1.ovsschema
> -    as $1 start_daemon ovsdb-server --remote=punix:"$d"/$1.sock "$d"/$1.db
> -    local var=`echo $1_db | tr a-z- A-Z_`
> -    AS_VAR_SET([$var], [unix:$ovs_base/$1/$1.sock]); export $var
> -}
> -
> -# ovn_start
> -#
> -# Creates and initializes ovn-sb and ovn-nb databases and starts their
> -# ovsdb-server instance, sets appropriate environment variables so that
> -# ovn-sbctl and ovn-nbctl use them by default, and starts ovn-northd
> running
> -# against them.
> -ovn_start () {
> -    ovn_init_db ovn-sb; ovn-sbctl init
> -    ovn_init_db ovn-nb; ovn-nbctl init
> -
> -    echo "starting ovn-northd"
> -    mkdir "$ovs_base"/northd
> -    as northd start_daemon ovn-northd -v \
> -               --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock \
> -               --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -
> -    echo "starting backup ovn-northd"
> -    mkdir "$ovs_base"/northd-backup
> -    as northd-backup start_daemon ovn-northd -v \
> -               --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock \
> -               --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -}
> -
> -# Interconnection networks.
> -#
> -# When multiple sandboxed Open vSwitch instances exist, one will
> inevitably
> -# want to connect them together.  These commands allow for that.
> Conceptually,
> -# an interconnection network is a switch for which these functions make
> it easy
> -# to plug into other switches in other sandboxed Open vSwitch instances.
> -# Interconnection networks are implemented as bridges in a switch named
> "main",
> -# so to use interconnection networks please avoid working with that switch
> -# directly.
> -
> -# net_add NETWORK
> -#
> -# Creates a new interconnection network named NETWORK.
> -net_add () {
> -    test -d "$ovs_base"/main || sim_add main || return 1
> -    as main ovs-vsctl add-br "$1"
> -}
> -
> -# net_attach NETWORK BRIDGE
> -#
> -# Adds a new port to BRIDGE in the default sandbox (as set with as()) and
> plugs
> -# it into the NETWORK interconnection network.  NETWORK must already have
> been
> -# created by a previous invocation of net_add.  The default sandbox must
> not be
> -# "main".
> -net_attach () {
> -    local net=$1 bridge=$2
> -
> -    local port=${sandbox}_$bridge
> -    as main ovs-vsctl \
> -        -- add-port $net $port \
> -        -- set Interface $port
> options:pstream="punix:$ovs_base/main/$port.sock"
> options:rxq_pcap="$ovs_base/main/$port-rx.pcap"
> options:tx_pcap="$ovs_base/main/$port-tx.pcap" \
> -        || return 1
> -
> -    ovs-vsctl \
> -        -- set Interface $bridge
> options:tx_pcap="$ovs_base/$sandbox/$bridge-tx.pcap"
> options:rxq_pcap="$ovs_base/$sandbox/$bridge-rx.pcap" \
> -        -- add-port $bridge ${bridge}_$net \
> -        -- set Interface ${bridge}_$net
> options:stream="unix:$ovs_base/main/$port.sock"
> options:rxq_pcap="$ovs_base/$sandbox/${bridge}_$net-rx.pcap"
> options:tx_pcap="$ovs_base/$sandbox/${bridge}_$net-tx.pcap" \
> -        || return 1
> -}
> -
> -# ovn_attach NETWORK BRIDGE IP [MASKLEN]
> -#
> -# First, this command attaches BRIDGE to interconnection network NETWORK,
> just
> -# like "net_attach NETWORK BRIDGE".  Second, it configures (simulated) IP
> -# address IP (with network mask length MASKLEN, which defaults to 24) on
> -# BRIDGE.  Finally, it configures the Open vSwitch database to work with
> OVN
> -# and starts ovn-controller.
> -ovn_attach() {
> -    local net=$1 bridge=$2 ip=$3 masklen=${4-24}
> -    net_attach $net $bridge || return 1
> -
> -    mac=`ovs-vsctl get Interface $bridge mac_in_use | sed s/\"//g`
> -    arp_table="$arp_table $sandbox,$bridge,$ip,$mac"
> -    ovs-appctl netdev-dummy/ip4addr $bridge $ip/$masklen >/dev/null ||
> return 1
> -    ovs-appctl ovs/route/add $ip/$masklen $bridge >/dev/null || return 1
> -    ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=$sandbox \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve,vxlan \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=$ip \
> -        -- add-br br-int \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true \
> -        || return 1
> -    start_daemon ovn-controller || return 1
> -}
> -
> -# OVN_POPULATE_ARP
> -#
> -# This pre-populates the ARP tables of all of the OVN instances that have
> been
> -# started with ovn_attach().  That means that packets sent from one
> hypervisor
> -# to another never get dropped or delayed by ARP resolution, which makes
> -# testing easier.
> -ovn_populate_arp__() {
> -    for e1 in $arp_table; do
> -        set `echo $e1 | sed 's/,/ /g'`; sb1=$1 br1=$2 ip=$3 mac=$4
> -        for e2 in $arp_table; do
> -            set `echo $e2 | sed 's/,/ /g'`; sb2=$1 br2=$2
> -            if test $sb1,$br1 != $sb2,$br2; then
> -                as $sb2 ovs-appctl tnl/neigh/set $br2 $ip $mac || return 1
> -            fi
> -        done
> -    done
> -}
> -m4_divert_pop([PREPARE_TESTS])
> -
> -m4_define([OVN_POPULATE_ARP], [AT_CHECK(ovn_populate_arp__, [0],
> [ignore])])
> diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
> deleted file mode 100644
> index d99d3af9b..000000000
> --- a/tests/ovn-nbctl.at
> +++ /dev/null
> @@ -1,1660 +0,0 @@
> -AT_BANNER([ovn-nbctl])
> -
> -OVS_START_SHELL_HELPERS
> -# OVN_NBCTL_TEST_START
> -m4_define([OVN_NBCTL_TEST_START],
> -  [AT_KEYWORDS([ovn])
> -   AT_CAPTURE_FILE([ovsdb-server.log])
> -   ovn_nbctl_test_start $1])
> -ovn_nbctl_test_start() {
> -   dnl Create ovn-nb database.
> -   AT_CHECK([ovsdb-tool create ovn-nb.db
> $abs_top_srcdir/ovn/ovn-nb.ovsschema])
> -
> -   dnl Start ovsdb-server.
> -   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file
> --remote=punix:$OVS_RUNDIR/ovnnb_db.sock ovn-nb.db], [0], [], [stderr])
> -   on_exit "kill `cat ovsdb-server.pid`"
> -   AS_CASE([$1],
> -     [daemon],
> -       [export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach --no-chdir
> --log-file -vsocket_util:off)
> -        on_exit "kill `cat ovn-nbctl.pid`"],
> -     [direct], [],
> -     [*], [AT_FAIL_IF(:)])
> -   AT_CHECK([ovn-nbctl init])
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d
> -/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> -}
> -
> -# OVN_NBCTL_TEST_STOP
> -m4_define([OVN_NBCTL_TEST_STOP], [ovn_nbctl_test_stop])
> -ovn_nbctl_test_stop() {
> -   AT_CHECK([check_logs "$1"])
> -   OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -}
> -OVS_END_SHELL_HELPERS
> -
> -# OVN_NBCTL_TEST(NAME, TITLE, COMMANDS)
> -m4_define([OVN_NBCTL_TEST],
> -   [OVS_START_SHELL_HELPERS
> -    $1() {
> -      $3
> -    }
> -    OVS_END_SHELL_HELPERS
> -
> -    AT_SETUP([ovn-nbctl - $2 - direct])
> -    OVN_NBCTL_TEST_START direct
> -    $1
> -    OVN_NBCTL_TEST_STOP
> -    AT_CLEANUP
> -
> -    AT_SETUP([ovn-nbctl - $2 - daemon])
> -    OVN_NBCTL_TEST_START daemon
> -    $1
> -    OVN_NBCTL_TEST_STOP
> -    AT_CLEANUP])
> -
> -OVN_NBCTL_TEST([ovn_nbctl_basic_switch], [basic switch commands], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)
> -])
> -
> -AT_CHECK([ovn-nbctl ls-add ls1])
> -AT_CHECK([ovn-nbctl ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)
> -<1> (ls1)
> -])
> -
> -AT_CHECK([ovn-nbctl ls-del ls0])
> -AT_CHECK([ovn-nbctl ls-list | uuidfilt], [0], [dnl
> -<0> (ls1)
> -])
> -
> -AT_CHECK([ovn-nbctl show ls0])
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl show ls0 | uuidfilt], [0],
> -  [switch <0> (ls0)
> -])
> -AT_CHECK([ovn-nbctl ls-add ls0], [1], [],
> -  [ovn-nbctl: ls0: a switch with this name already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist ls-add ls0])
> -AT_CHECK([ovn-nbctl show ls0 | uuidfilt], [0],
> -  [switch <0> (ls0)
> -])
> -AT_CHECK([ovn-nbctl --add-duplicate ls-add ls0])
> -AT_CHECK([ovn-nbctl --may-exist --add-duplicate ls-add ls0], [1], [],
> -  [ovn-nbctl: --may-exist and --add-duplicate may not be used together
> -])
> -AT_CHECK([ovn-nbctl ls-del ls0], [1], [],
> -  [ovn-nbctl: Multiple logical switches named 'ls0'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl ls-del ls2], [1], [],
> -  [ovn-nbctl: ls2: switch name not found
> -])
> -AT_CHECK([ovn-nbctl --if-exists ls-del ls2])
> -
> -AT_CHECK([ovn-nbctl ls-add])
> -AT_CHECK([ovn-nbctl ls-add])
> -AT_CHECK([ovn-nbctl --add-duplicate ls-add], [1], [],
> -  [ovn-nbctl: --add-duplicate requires specifying a name
> -])
> -AT_CHECK([ovn-nbctl --may-exist ls-add], [1], [],
> -  [ovn-nbctl: --may-exist requires specifying a name
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_basic_lsp], [basic logical switch port
> commands], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0], [1], [],
> -  [ovn-nbctl: lp0: a port with this name already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls0 lp0])
> -AT_CHECK([ovn-nbctl lsp-list ls0 | uuidfilt], [0], [dnl
> -<0> (lp0)
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp1])
> -AT_CHECK([ovn-nbctl lsp-list ls0 | uuidfilt], [0], [dnl
> -<0> (lp0)
> -<1> (lp1)
> -])
> -
> -AT_CHECK([ovn-nbctl ls-add ls1])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp1], [1], [],
> -  [ovn-nbctl: lp1: a port with this name already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls1 lp1], [1], [],
> -  [ovn-nbctl: lp1: port already exists but in switch ls0
> -])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls0 lp1 lp0 5], [1], [],
> -  [ovn-nbctl: lp1: port already exists but has no parent
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-del lp1])
> -AT_CHECK([ovn-nbctl lsp-list ls0 | uuidfilt], [0], [dnl
> -<0> (lp0)
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp2 lp3 5])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls0 lp2 lp4 5], [1], [],
> -  [ovn-nbctl: lp2: port already exists with different parent lp3
> -])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls0 lp2 lp3 10], [1], [],
> -  [ovn-nbctl: lp2: port already exists with different tag_request 5
> -])
> -AT_CHECK([ovn-nbctl clear Logical_Switch_Port lp2 tag_request])
> -AT_CHECK([ovn-nbctl --may-exist lsp-add ls0 lp2 lp3 5], [1], [],
> -  [ovn-nbctl: lp2: port already exists but has no tag_request
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lsp_get_ls], [lsp get ls], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -
> -AT_CHECK([ovn-nbctl lsp-get-ls lp0 | uuidfilt], [0], [dnl
> -<0> (ls0)
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-get-ls lp1], [1], [],
> -  [ovn-nbctl: lp1: port name not found
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lport_addresses], [lport addresses], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -AT_CHECK([ovn-nbctl lsp-get-addresses lp0], [0], [dnl
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-addresses lp0 00:11:22:33:44:55 unknown])
> -AT_CHECK([ovn-nbctl lsp-get-addresses lp0], [0], [dnl
> -00:11:22:33:44:55
> -unknown
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-addresses lp0])
> -AT_CHECK([ovn-nbctl lsp-get-addresses lp0], [0], [dnl
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_port_security], [port security], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -AT_CHECK([ovn-nbctl lsp-get-addresses lp0], [0], [dnl
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-port-security lp0 aa:bb:cc:dd:ee:ff
> 00:11:22:33:44:55])
> -AT_CHECK([ovn-nbctl lsp-get-port-security lp0], [0], [dnl
> -00:11:22:33:44:55
> -aa:bb:cc:dd:ee:ff
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-port-security lp0])
> -AT_CHECK([ovn-nbctl lsp-get-port-security lp0], [0], [dnl
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_acls], [ACLs], [
> -ovn_nbctl_test_acl() {
> -   AT_CHECK([ovn-nbctl $2 --log acl-add $1 from-lport 600 udp drop])
> -   AT_CHECK([ovn-nbctl $2 --log --name=test --severity=info acl-add $1
> to-lport 500 udp drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 from-lport 400 tcp drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 to-lport 300 tcp drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 from-lport 200 ip drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 to-lport 100 ip drop])
> -   dnl Add duplicated ACL
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 to-lport 100 ip drop], [1], [],
> [stderr])
> -   AT_CHECK([grep 'already existed' stderr], [0], [ignore])
> -   AT_CHECK([ovn-nbctl $2 --may-exist acl-add $1 to-lport 100 ip drop])
> -
> -   AT_CHECK([ovn-nbctl $2 acl-list $1], [0], [dnl
> -from-lport   600 (udp) drop log()
> -from-lport   400 (tcp) drop
> -from-lport   200 (ip) drop
> -  to-lport   500 (udp) drop log(name=test,severity=info)
> -  to-lport   300 (tcp) drop
> -  to-lport   100 (ip) drop
> -])
> -
> -   dnl Delete in one direction.
> -   AT_CHECK([ovn-nbctl $2 acl-del $1 to-lport])
> -   AT_CHECK([ovn-nbctl $2 acl-list $1], [0], [dnl
> -from-lport   600 (udp) drop log()
> -from-lport   400 (tcp) drop
> -from-lport   200 (ip) drop
> -])
> -
> -   dnl Delete all ACLs.
> -   AT_CHECK([ovn-nbctl $2 acl-del $1])
> -   AT_CHECK([ovn-nbctl $2 acl-list $1], [0], [dnl
> -])
> -
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 from-lport 600 udp drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 from-lport 400 tcp drop])
> -   AT_CHECK([ovn-nbctl $2 acl-add $1 from-lport 200 ip drop])
> -
> -   dnl Delete a single flow.
> -   AT_CHECK([ovn-nbctl $2 acl-del $1 from-lport 400 tcp])
> -   AT_CHECK([ovn-nbctl $2 acl-list $1], [0], [dnl
> -from-lport   600 (udp) drop
> -from-lport   200 (ip) drop
> -])
> -}
> -
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -ovn_nbctl_test_acl ls0
> -AT_CHECK([ovn-nbctl ls-add ls1])
> -ovn_nbctl_test_acl ls1 --type=switch
> -AT_CHECK([ovn-nbctl create port_group name=pg0], [0], [ignore])
> -ovn_nbctl_test_acl pg0 --type=port-group
> -
> -dnl Test when port group doesn't exist
> -AT_CHECK([ovn-nbctl --type=port-group acl-add pg1 to-lport 100 ip drop],
> [1], [], [dnl
> -ovn-nbctl: pg1: port group name not found
> -])
> -
> -dnl Test when same name exists in logical switches and portgroups
> -AT_CHECK([ovn-nbctl create port_group name=ls0], [0], [ignore])
> -AT_CHECK([ovn-nbctl acl-add ls0 to-lport 100 ip drop], [1], [], [stderr])
> -AT_CHECK([grep 'exists in both' stderr], [0], [ignore])
> -AT_CHECK([ovn-nbctl --type=port-group acl-add ls0 to-lport 100 ip drop],
> [0], [ignore])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_qos], [QoS], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 tcp dscp=63])
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 500 udp rate=100 burst=1000])
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 400 tcp dscp=0 rate=300
> burst=3000])
> -AT_CHECK([ovn-nbctl qos-add ls0 to-lport 300 tcp dscp=48])
> -AT_CHECK([ovn-nbctl qos-add ls0 to-lport 200 ip rate=101])
> -AT_CHECK([ovn-nbctl qos-add ls0 to-lport 100 ip4 dscp=13 rate=301
> burst=30000])
> -
> -dnl Add duplicated qos
> -AT_CHECK([ovn-nbctl qos-add ls0 to-lport 100 ip4 dscp=11 rate=302
> burst=30002], [1], [], [stderr])
> -AT_CHECK([grep 'already existed' stderr], [0], [ignore])
> -AT_CHECK([ovn-nbctl --may-exist qos-add ls0 to-lport 100 ip4 dscp=11
> rate=302 burst=30002])
> -
> -AT_CHECK([ovn-nbctl qos-list ls0], [0], [dnl
> -from-lport   600 (tcp) dscp=63
> -from-lport   500 (udp) rate=100 burst=1000
> -from-lport   400 (tcp) rate=300 burst=3000 dscp=0
> -  to-lport   300 (tcp) dscp=48
> -  to-lport   200 (ip) rate=101
> -  to-lport   100 (ip4) rate=301 burst=30000 dscp=13
> -])
> -
> -dnl Delete in one direction.
> -AT_CHECK([ovn-nbctl qos-del ls0 to-lport])
> -AT_CHECK([ovn-nbctl qos-list ls0], [0], [dnl
> -from-lport   600 (tcp) dscp=63
> -from-lport   500 (udp) rate=100 burst=1000
> -from-lport   400 (tcp) rate=300 burst=3000 dscp=0
> -])
> -
> -dnl Delete all qos_rules.
> -AT_CHECK([ovn-nbctl qos-del ls0])
> -AT_CHECK([ovn-nbctl qos-list ls0], [0], [dnl
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip rate=1000101])
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 400 tcp dscp=44])
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 200 ip burst=1000102 rate=301
> dscp=19])
> -
> -dnl Delete a single flow.
> -AT_CHECK([ovn-nbctl qos-del ls0 from-lport 400 tcp])
> -AT_CHECK([ovn-nbctl qos-list ls0], [0], [dnl
> -from-lport   600 (ip) rate=1000101
> -from-lport   200 (ip) rate=301 burst=1000102 dscp=19
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip rate=100010111111],
> [1], [],
> -[ovn-nbctl: 100010111111: rate must be in the range 1...4294967295
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip burst=100010111112
> rate=100010], [1], [],
> -[ovn-nbctl: 100010111112: burst must be in the range 1...4294967295
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip dscp=-1], [1], [],
> -[ovn-nbctl: -1: dscp must be in the range 0...63
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip dscpa=-1], [1], [],
> -[ovn-nbctl: dscpa=-1: supported arguments are "dscp=", "rate=", and
> "burst="
> -])
> -
> -AT_CHECK([ovn-nbctl qos-add ls0 from-lport 600 ip burst=123], [1], [],
> -[ovn-nbctl: Either "rate" and/or "dscp" must be specified
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_meters], [meters], [
> -AT_CHECK([ovn-nbctl meter-add meter1 drop 10 kbps])
> -AT_CHECK([ovn-nbctl meter-add meter2 drop 3 kbps 2])
> -AT_CHECK([ovn-nbctl meter-add meter3 drop 100 kbps 200])
> -AT_CHECK([ovn-nbctl meter-add meter4 drop 10 pktps 30])
> -
> -dnl Add duplicate meter name
> -AT_CHECK([ovn-nbctl meter-add meter1 drop 10 kbps], [1], [], [stderr])
> -AT_CHECK([grep 'already exists' stderr], [0], [ignore])
> -
> -dnl Add reserved meter name
> -AT_CHECK([ovn-nbctl meter-add __meter1 drop 10 kbps], [1], [], [stderr])
> -AT_CHECK([grep 'reserved' stderr], [0], [ignore])
> -
> -dnl Add meter with invalid rates
> -AT_CHECK([ovn-nbctl meter-add meter5 drop 100010111111 kbps], [1], [],
> -[ovn-nbctl: rate must be in the range 1...4294967295
> -])
> -
> -dnl Add meter with invalid rates
> -AT_CHECK([ovn-nbctl meter-add meter5 drop 100010111111 foo], [1], [],
> -[ovn-nbctl: rate must be in the range 1...4294967295
> -])
> -
> -AT_CHECK([ovn-nbctl meter-add meter5 drop 0 kbps], [1], [],
> -[ovn-nbctl: rate must be in the range 1...4294967295
> -])
> -
> -dnl Add meter with invalid burst
> -AT_CHECK([ovn-nbctl meter-add meter5 drop 10 100010111111 kbps], [1], [],
> -[ovn-nbctl: unit must be "kbps" or "pktps"
> -])
> -
> -AT_CHECK([ovn-nbctl meter-list], [0], [dnl
> -meter1: bands:
> -  drop: 10 kbps
> -meter2: bands:
> -  drop: 3 kbps, 2 kb burst
> -meter3: bands:
> -  drop: 100 kbps, 200 kb burst
> -meter4: bands:
> -  drop: 10 pktps, 30 packet burst
> -])
> -
> -dnl Delete a single meter.
> -AT_CHECK([ovn-nbctl meter-del meter2])
> -AT_CHECK([ovn-nbctl meter-list], [0], [dnl
> -meter1: bands:
> -  drop: 10 kbps
> -meter3: bands:
> -  drop: 100 kbps, 200 kb burst
> -meter4: bands:
> -  drop: 10 pktps, 30 packet burst
> -])
> -
> -dnl Delete all meters.
> -AT_CHECK([ovn-nbctl meter-del])
> -AT_CHECK([ovn-nbctl meter-list], [0], [dnl
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_nats], [NATs], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snatt 30.0.0.2 192.168.1.2], [1], [],
> -[ovn-nbctl: snatt: type must be one of "dnat", "snat" and "dnat_and_snat".
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2a 192.168.1.2], [1], [],
> -[ovn-nbctl: 30.0.0.2a: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0 192.168.1.2], [1], [],
> -[ovn-nbctl: 30.0.0: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2/24 192.168.1.2], [1],
> [],
> -[ovn-nbctl: 30.0.0.2/24: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2:80 192.168.1.2], [1],
> [],
> -[ovn-nbctl: 30.0.0.2:80: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1.2a], [1], [],
> -[ovn-nbctl: 192.168.1.2a: should be an IPv4 address or network.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1], [1], [],
> -[ovn-nbctl: 192.168.1: should be an IPv4 address or network.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1.2:80], [1],
> [],
> -[ovn-nbctl: 192.168.1.2:80: should be an IPv4 address or network.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1.2/a], [1], [],
> -[ovn-nbctl: 192.168.1.2/a: should be an IPv4 address or network.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.2 192.168.1.2a], [1], [],
> -[ovn-nbctl: 192.168.1.2a: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.2 192.168.1], [1], [],
> -[ovn-nbctl: 192.168.1: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.2 192.168.1.2:80], [1],
> [],
> -[ovn-nbctl: 192.168.1.2:80: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.2 192.168.1.2/24], [1],
> [],
> -[ovn-nbctl: 192.168.1.2/24: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.2 192.168.1.2/24],
> [1], [],
> -[ovn-nbctl: 192.168.1.2/24: should be an IPv4 address.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.2 192.168.1.2
> lp0], [1], [],
> -[ovn-nbctl: lr-nat-add with logical_port must also specify external_mac.
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.2 192.168.1.2 lp0
> 00:00:00:01:02:03], [1], [],
> -[ovn-nbctl: logical_port and external_mac are only valid when type is
> "dnat_and_snat".
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1.2 lp0
> 00:00:00:01:02:03], [1], [],
> -[ovn-nbctl: logical_port and external_mac are only valid when type is
> "dnat_and_snat".
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.2 192.168.1.2 lp0
> 00:00:00:01:02:03], [1], [],
> -[ovn-nbctl: lp0: port name not found
> -])
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.2 192.168.1.2 lp0
> 00:00:00:01:02], [1], [],
> -[ovn-nbctl: invalid mac address 00:00:00:01:02.
> -])
> -
> -dnl Add snat and dnat
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.1 192.168.1.0/24])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.1 192.168.1.2])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.1 192.168.1.2])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.2 192.168.1.3 lp0
> 00:00:00:01:02:03])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [dnl
> -TYPE             EXTERNAL_IP        LOGICAL_IP            EXTERNAL_MAC
>      LOGICAL_PORT
> -dnat             30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.2           192.168.1.3
>  00:00:00:01:02:03    lp0
> -snat             30.0.0.1           192.168.1.0/24
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.1 192.168.1.0/24], [1],
> [],
> -[ovn-nbctl: 30.0.0.1, 192.168.1.0/24: a NAT with this external_ip and
> logical_ip already exists
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.1 192.168.1.10/24], [1],
> [],
> -[ovn-nbctl: 30.0.0.1, 192.168.1.0/24: a NAT with this external_ip and
> logical_ip already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-nat-add lr0 snat 30.0.0.1
> 192.168.1.0/24])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 30.0.0.2 192.168.1.0/24], [1],
> [],
> -[ovn-nbctl: a NAT with this type (snat) and logical_ip (192.168.1.0/24)
> already exists
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.1 192.168.1.2], [1], [],
> -[ovn-nbctl: 30.0.0.1, 192.168.1.2: a NAT with this external_ip and
> logical_ip already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-nat-add lr0 dnat 30.0.0.1 192.168.1.2])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 30.0.0.1 192.168.1.3], [1], [],
> -[ovn-nbctl: a NAT with this type (dnat) and external_ip (30.0.0.1)
> already exists
> -])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.1 192.168.1.2],
> [1], [],
> -[ovn-nbctl: 30.0.0.1, 192.168.1.2: a NAT with this external_ip and
> logical_ip already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-nat-add lr0 dnat_and_snat 30.0.0.1
> 192.168.1.2])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 30.0.0.1 192.168.1.3],
> [1], [],
> -[ovn-nbctl: a NAT with this type (dnat_and_snat) and external_ip
> (30.0.0.1) already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-nat-add lr0 dnat_and_snat 30.0.0.2
> 192.168.1.3 lp0 00:00:00:04:05:06])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [dnl
> -TYPE             EXTERNAL_IP        LOGICAL_IP            EXTERNAL_MAC
>      LOGICAL_PORT
> -dnat             30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.2           192.168.1.3
>  00:00:00:04:05:06    lp0
> -snat             30.0.0.1           192.168.1.0/24
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-nat-add lr0 dnat_and_snat 30.0.0.2
> 192.168.1.3])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [dnl
> -TYPE             EXTERNAL_IP        LOGICAL_IP            EXTERNAL_MAC
>      LOGICAL_PORT
> -dnat             30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.2           192.168.1.3
> -snat             30.0.0.1           192.168.1.0/24
> -])
> -
> -dnl Deletes the NATs
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 dnat_and_snat 30.0.0.3], [1], [],
> -[ovn-nbctl: no matching NAT with the type (dnat_and_snat) and external_ip
> (30.0.0.3)
> -])
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 dnat 30.0.0.2], [1], [],
> -[ovn-nbctl: no matching NAT with the type (dnat) and external_ip
> (30.0.0.2)
> -])
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 snat 192.168.10.0/24], [1], [],
> -[ovn-nbctl: no matching NAT with the type (snat) and logical_ip (
> 192.168.10.0/24)
> -])
> -AT_CHECK([ovn-nbctl --if-exists lr-nat-del lr0 snat 192.168.10.0/24])
> -
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 dnat_and_snat 30.0.0.1])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [dnl
> -TYPE             EXTERNAL_IP        LOGICAL_IP            EXTERNAL_MAC
>      LOGICAL_PORT
> -dnat             30.0.0.1           192.168.1.2
> -dnat_and_snat    30.0.0.2           192.168.1.3
> -snat             30.0.0.1           192.168.1.0/24
> -])
> -
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 dnat])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [dnl
> -TYPE             EXTERNAL_IP        LOGICAL_IP            EXTERNAL_MAC
>      LOGICAL_PORT
> -dnat_and_snat    30.0.0.2           192.168.1.3
> -snat             30.0.0.1           192.168.1.0/24
> -])
> -
> -AT_CHECK([ovn-nbctl lr-nat-del lr0])
> -AT_CHECK([ovn-nbctl lr-nat-list lr0], [0], [])
> -AT_CHECK([ovn-nbctl lr-nat-del lr0])
> -AT_CHECK([ovn-nbctl lr-nat-del lr0 dnat])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lbs], [LBs], [
> -dnl Add two LBs.
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10:80a
> 192.168.10.10:80,192.168.10.20:80 tcp], [1], [],
> -[ovn-nbctl: 30.0.0.10:80a: should be an IP address (or an IP address and
> a port number with : as a separator).
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10:a80
> 192.168.10.10:80,192.168.10.20:80 tcp], [1], [],
> -[ovn-nbctl: 30.0.0.10:a80: should be an IP address (or an IP address and
> a port number with : as a separator).
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10:80
> 192.168.10.10:80,192.168.10.20 tcp], [1], [],
> -[ovn-nbctl: 192.168.10.20: should be an IP address and a port number
> with : as a separator.
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.1a
> 192.168.10.10:80,192.168.10.20:80], [1], [],
> -[ovn-nbctl: 30.0.0.1a: should be an IP address (or an IP address and a
> port number with : as a separator).
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0 192.168.10.10:80,
> 192.168.10.20:80], [1], [],
> -[ovn-nbctl: 30.0.0: should be an IP address (or an IP address and a port
> number with : as a separator).
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.10,
> 192.168.10.20:80], [1], [],
> -[ovn-nbctl: 192.168.10.20:80: should be an IP address.
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.10:a80],
> [1], [],
> -[ovn-nbctl: 192.168.10.10:a80: should be an IP address.
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10 192.168.10.10:],
> [1], [],
> -[ovn-nbctl: 192.168.10.10:: should be an IP address.
> -])
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 30.0.0.10
> 192.168.10.1a], [1], [],
> -[ovn-nbctl: 192.168.10.1a: should be an IP address.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10: 192.168.10.10:80,
> 192.168.10.20:80 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 192.168.10.10 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 192.168.10.10:900 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -dnl Add ips to lb
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10:80 ,,,192.168.10.10:80,,,,,])
> -AT_CHECK([ovn-nbctl lb-add lb1 30.0.0.10:80 ,,,192.168.10.10:80,,,,
> 192.168.10.20:80,,,,])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80
> -<1>    lb1                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -])
> -AT_CHECK([ovn-nbctl lb-del lb0])
> -AT_CHECK([ovn-nbctl lb-del lb1])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:80])
> -AT_CHECK([ovn-nbctl lb-add lb1 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:80 tcp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -])
> -
> -dnl Update the VIP of the lb1.
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:8080])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:8080
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:8080 udp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:8080
> -])
> -
> -dnl Config lb1 with another VIP.
> -AT_CHECK([ovn-nbctl lb-add lb1 30.0.0.20:80 192.168.10.10:80 udp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:8080
> -                                                            udp
> 30.0.0.20:80    192.168.10.10:80
> -])
> -
> -AT_CHECK([ovn-nbctl lb-del lb1 30.0.0.20:80])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:8080
> -])
> -
> -dnl Add LBs whose vip is just an IP address.
> -AT_CHECK([ovn-nbctl lb-add lb2 30.0.0.30 192.168.10.10])
> -AT_CHECK([ovn-nbctl lb-add lb3 30.0.0.30 192.168.10.10])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:8080
> -<2>    lb2                 tcp/udp    30.0.0.30       192.168.10.10
> -<3>    lb3                 tcp/udp    30.0.0.30       192.168.10.10
> -])
> -AT_CHECK([ovn-nbctl lb-del lb2 30.0.0.30])
> -AT_CHECK([ovn-nbctl lb-del lb3 30.0.0.30])
> -
> -AT_CHECK([ovn-nbctl lb-add lb2 30.0.0.10:8080 192.168.10.10:80,
> 192.168.10.20:80 tcp])
> -AT_CHECK([ovn-nbctl --add-duplicate lb-add lb2 30.0.0.10:8080
> 192.168.10.10:80,192.168.10.20:80 tcp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP               IPs
> -<0>    lb0                 tcp        30.0.0.10:80      192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80      192.168.10.10:80,
> 192.168.10.20:8080
> -<2>    lb2                 tcp        30.0.0.10:8080    192.168.10.10:80,
> 192.168.10.20:80
> -<3>    lb2                 tcp        30.0.0.10:8080    192.168.10.10:80,
> 192.168.10.20:80
> -])
> -
> -dnl If there are multiple load balancers with the same name, use a UUID
> to update/delete.
> -AT_CHECK([ovn-nbctl lb-add lb2 30.0.0.10:8080 192.168.10.10:80,
> 192.168.10.20:80 tcp], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-del lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 30.0.0.10:80
> 192.168.10.10:8080,192.168.10.20:8080 udp])
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 30.0.0.10:8080
> 192.168.10.10:8080,192.168.10.20:8080 udp])
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 30.0.0.10:9090
> 192.168.10.10:8080,192.168.10.20:8080 udp])
> -AT_CHECK([ovn-nbctl lb-del lb0 30.0.0.10:80])
> -AT_CHECK([ovn-nbctl lb-del lb1])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP               IPs
> -<0>    lb2                 tcp        30.0.0.10:8080    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb2                 tcp        30.0.0.10:8080    192.168.10.10:80,
> 192.168.10.20:80
> -])
> -
> -dnl Add load balancer to logical switch.
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:80])
> -AT_CHECK([ovn-nbctl lb-add lb1 30.0.0.10:80 192.168.10.10:80,
> 192.168.10.20:80 udp])
> -AT_CHECK([ovn-nbctl lb-add lb3 30.0.0.10 192.168.10.10,192.168.10.20])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl --may-exist ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb3])
> -
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<2>    lb3                 tcp/udp    30.0.0.10
>  192.168.10.10,192.168.10.20
> -])
> -
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb3                 tcp/udp    30.0.0.10
>  192.168.10.10,192.168.10.20
> -])
> -
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb3])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])
> -AT_CHECK([ovn-nbctl --if-exists ls-lb-del ls0 lb1])
> -
> -dnl Remove all load balancers from logical switch.
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb3])
> -AT_CHECK([ovn-nbctl ls-lb-del ls0])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])
> -
> -dnl Add load balancer to logical router.
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl --may-exist lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb3])
> -
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb0                 tcp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<2>    lb3                 tcp/udp    30.0.0.10
>  192.168.10.10,192.168.10.20
> -])
> -
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP             IPs
> -<0>    lb1                 udp        30.0.0.10:80    192.168.10.10:80,
> 192.168.10.20:80
> -<1>    lb3                 tcp/udp    30.0.0.10
>  192.168.10.10,192.168.10.20
> -])
> -
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb3])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
> -AT_CHECK([ovn-nbctl --if-exists lr-lb-del lr0 lb1])
> -
> -dnl Remove all load balancers from logical router.
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb3])
> -AT_CHECK([ovn-nbctl lr-lb-del lr0])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
> -
> -dnl Remove load balancers after adding them to a logical router/switch.
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl lb-del lb0])
> -AT_CHECK([ovn-nbctl lb-del lb1])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lbs_ipv6], [LBs IPv6], [
> -dnl A bunch of commands that should fail
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 [[ae0f::10]]:80a
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp], [1], [],
> -[ovn-nbctl: [[ae0f::10]]:80a: should be an IP address (or an IP address
> and a port number with : as a separator).
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 [[ae0f::10]]:a80
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp], [1], [],
> -[ovn-nbctl: [[ae0f::10]]:a80: should be an IP address (or an IP address
> and a port number with : as a separator).
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 [[ae0f::10]]:80
> [[fd0f::10]]:80,fd0f::20 tcp], [1], [],
> -[ovn-nbctl: fd0f::20: should be an IP address and a port number with : as
> a separator.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10fff
> [[fd0f::10]]:80,fd0f::20 tcp], [1], [],
> -[ovn-nbctl: ae0f::10fff: should be an IP address (or an IP address and a
> port number with : as a separator).
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10
> [[fd0f::10]]:80,[[fd0f::20]]:80], [1], [],
> -[ovn-nbctl: [[fd0f::10]]:80: should be an IP address.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10
> fd0f::10,[[fd0f::20]]:80], [1], [],
> -[ovn-nbctl: [[fd0f::20]]:80: should be an IP address.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10
> [[fd0f::10]]:a80], [1], [],
> -[ovn-nbctl: [[fd0f::10]]:a80: should be an IP address.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 [[fd0f::10]]:],
> [1], [],
> -[ovn-nbctl: [[fd0f::10]]:: should be an IP address.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 ae0f::10 fd0f::1001a],
> [1], [],
> -[ovn-nbctl: fd0f::1001a: should be an IP address.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl -vsocket_util:off lb-add lb0 [[ae0f::10]]:
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 fd0f::10 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 [[fd0f::10]]:900 tcp], [1], [],
> -[ovn-nbctl: Protocol is unnecessary when no port of vip is given.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 192.168.10.10], [1], [],
> -[ovn-nbctl: 192.168.10.10: IP address family is different from VIP
> ae0f::10.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 192.168.10.10], [1], [],
> -[ovn-nbctl: 192.168.10.10: IP address family is different from VIP
> ae0f::10.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 [[ae0f::10]]:80 192.168.10.10:80], [1],
> [],
> -[ovn-nbctl: 192.168.10.10:80: IP address family is different from VIP
> [[ae0f::10]]:80.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10 ae0f::10], [1], [],
> -[ovn-nbctl: ae0f::10: IP address family is different from VIP 30.0.0.10.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 30.0.0.10:80 [[ae0f::10]]:80], [1], [],
> -[ovn-nbctl: [[ae0f::10]]:80: IP address family is different from VIP
> 30.0.0.10:80.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f::10 fd0f::10])
> -AT_CHECK([ovn-nbctl lb-add lb0 ae0f:0000:0000:0000:0000:0000:0000:0010
> fd0f::20],
> -[1], [], [ovn-nbctl: lb0: a load balancer with this vip (ae0f::10)
> already exists
> -])
> -
> -AT_CHECK([ovn-nbctl lb-del lb0])
> -
> -dnl Add ips to lb
> -AT_CHECK([ovn-nbctl lb-add lb0 [[ae0f::10]]:80 ,,,[[fd0f::10]]:80,,,,,])
> -AT_CHECK([ovn-nbctl lb-add lb1 [[ae0f::10]]:80
> ,,,[[fd0f::10]]:80,,,,[[fd0f::20]]:80,,,,])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80    [[fd0f::10]]:80
> -<1>    lb1                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -])
> -AT_CHECK([ovn-nbctl lb-del lb0])
> -AT_CHECK([ovn-nbctl lb-del lb1])
> -
> -
> -AT_CHECK([ovn-nbctl lb-add lb0 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80])
> -AT_CHECK([ovn-nbctl lb-add lb1 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -])
> -
> -dnl Update the VIP of the lb1.
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080 udp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -])
> -
> -dnl Config lb1 with another VIP.
> -AT_CHECK([ovn-nbctl lb-add lb1 [[ae0f::20]]:80 [[fd0f::10]]:80 udp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -                                                            udp
> [[ae0f::20]]:80    [[fd0f::10]]:80
> -])
> -
> -AT_CHECK([ovn-nbctl lb-del lb1 [[ae0f::20]]:80])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -])
> -
> -dnl Add LBs whose vip is just an IP address.
> -AT_CHECK([ovn-nbctl lb-add lb2 ae0f::30 fd0f::10])
> -AT_CHECK([ovn-nbctl lb-add lb3 ae0f::30 fd0f::10])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -<2>    lb2                 tcp/udp    ae0f::30         fd0f::10
> -<3>    lb3                 tcp/udp    ae0f::30         fd0f::10
> -])
> -AT_CHECK([ovn-nbctl lb-del lb2 ae0f::30])
> -AT_CHECK([ovn-nbctl lb-del lb3 ae0f::30])
> -
> -AT_CHECK([ovn-nbctl lb-add lb2 [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp])
> -AT_CHECK([ovn-nbctl --add-duplicate lb-add lb2 [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP                IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:8080
> -<2>    lb2                 tcp        [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<3>    lb2                 tcp        [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -])
> -
> -dnl If there are multiple load balancers with the same name, use a UUID
> to update/delete.
> -AT_CHECK([ovn-nbctl lb-add lb2 [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80 tcp], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl lb-del lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 [[ae0f::10]]:80
> [[fd0f::10]]:8080,[[fd0f::20]]:8080 udp])
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 [[ae0f::10]]:8080
> [[fd0f::10]]:8080,[[fd0f::20]]:8080 udp])
> -AT_CHECK([ovn-nbctl --may-exist lb-add lb1 [[ae0f::10]]:9090
> [[fd0f::10]]:8080,[[fd0f::20]]:8080 udp])
> -AT_CHECK([ovn-nbctl lb-del lb0 [[ae0f::10]]:80])
> -AT_CHECK([ovn-nbctl lb-del lb1])
> -AT_CHECK([ovn-nbctl lb-list | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP                IPs
> -<0>    lb2                 tcp        [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb2                 tcp        [[ae0f::10]]:8080
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -])
> -
> -dnl Add load balancer to logical switch.
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lb-add lb0 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80])
> -AT_CHECK([ovn-nbctl lb-add lb1 [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80 udp])
> -AT_CHECK([ovn-nbctl lb-add lb3 ae0f::10 fd0f::10,fd0f::20])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl --may-exist ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb3])
> -
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<2>    lb3                 tcp/udp    ae0f::10         fd0f::10,fd0f::20
> -])
> -
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb3                 tcp/udp    ae0f::10         fd0f::10,fd0f::20
> -])
> -
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-del ls0 lb3])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])
> -AT_CHECK([ovn-nbctl --if-exists ls-lb-del ls0 lb1])
> -
> -dnl Remove all load balancers from logical switch.
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb0])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
> -AT_CHECK([ovn-nbctl ls-lb-add ls0 lb3])
> -AT_CHECK([ovn-nbctl ls-lb-del ls0])
> -AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])
> -
> -dnl Add load balancer to logical router.
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl --may-exist lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb2], [1], [],
> -[ovn-nbctl: Multiple load balancers named 'lb2'.  Use a UUID.
> -])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb3])
> -
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb0                 tcp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<2>    lb3                 tcp/udp    ae0f::10         fd0f::10,fd0f::20
> -])
> -
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [dnl
> -UUID                                    LB                  PROTO
> VIP              IPs
> -<0>    lb1                 udp        [[ae0f::10]]:80
> [[fd0f::10]]:80,[[fd0f::20]]:80
> -<1>    lb3                 tcp/udp    ae0f::10         fd0f::10,fd0f::20
> -])
> -
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-del lr0 lb3])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
> -AT_CHECK([ovn-nbctl --if-exists lr-lb-del lr0 lb1])
> -
> -dnl Remove all load balancers from logical router.
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb3])
> -AT_CHECK([ovn-nbctl lr-lb-del lr0])
> -AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_basic_lr], [basic logical router commands], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lr-list | uuidfilt], [0], [dnl
> -<0> (lr0)
> -])
> -
> -AT_CHECK([ovn-nbctl lr-add lr1])
> -AT_CHECK([ovn-nbctl lr-list | uuidfilt], [0], [dnl
> -<0> (lr0)
> -<1> (lr1)
> -])
> -
> -AT_CHECK([ovn-nbctl lr-del lr0])
> -AT_CHECK([ovn-nbctl lr-list | uuidfilt], [0], [dnl
> -<0> (lr1)
> -])
> -
> -AT_CHECK([ovn-nbctl show lr0])
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl show lr0 | uuidfilt], [0],
> -  [router <0> (lr0)
> -])
> -AT_CHECK([ovn-nbctl lr-add lr0], [1], [],
> -  [ovn-nbctl: lr0: a router with this name already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-add lr0])
> -AT_CHECK([ovn-nbctl show lr0 | uuidfilt], [0],
> -  [router <0> (lr0)
> -])
> -AT_CHECK([ovn-nbctl --add-duplicate lr-add lr0])
> -AT_CHECK([ovn-nbctl --may-exist --add-duplicate lr-add lr0], [1], [],
> -  [ovn-nbctl: --may-exist and --add-duplicate may not be used together
> -])
> -AT_CHECK([ovn-nbctl lr-del lr0], [1], [],
> -  [ovn-nbctl: Multiple logical routers named 'lr0'.  Use a UUID.
> -])
> -
> -AT_CHECK([ovn-nbctl lr-del lr2], [1], [],
> -  [ovn-nbctl: lr2: router name not found
> -])
> -AT_CHECK([ovn-nbctl --if-exists lr-del lr2])
> -
> -AT_CHECK([ovn-nbctl lr-add])
> -AT_CHECK([ovn-nbctl lr-add])
> -AT_CHECK([ovn-nbctl --add-duplicate lr-add], [1], [],
> -  [ovn-nbctl: --add-duplicate requires specifying a name
> -])
> -AT_CHECK([ovn-nbctl --may-exist lr-add], [1], [],
> -  [ovn-nbctl: --may-exist requires specifying a name
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_basic_lrp], [basic logical router port
> commands], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02 192.168.1.1/24],
> [1], [],
> -  [ovn-nbctl: lrp0: invalid mac address 00:00:00:01:02
> -])
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02:03:04 192.168.1.1/24],
> [1], [],
> -  [ovn-nbctl: lrp0: invalid mac address 00:00:00:01:02:03:04
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02:03 192.168.1.1/24])
> -
> -AT_CHECK([ovn-nbctl show lr0 | uuidfilt], [0], [dnl
> -router <0> (lr0)
> -    port lrp0
> -        mac: "00:00:00:01:02:03"
> -        networks: [["192.168.1.1/24"]]
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02:03 192.168.1.1/24],
> [1], [],
> -  [ovn-nbctl: lrp0: a port with this name already exists
> -])
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp0 00:00:00:01:02:03
> 192.168.1.1/24])
> -AT_CHECK([ovn-nbctl lrp-list lr0 | uuidfilt], [0], [dnl
> -<0> (lrp0)
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp1 00:00:00:01:02:03 192.168.1.1/24
> peer=lrp1-peer])
> -AT_CHECK([ovn-nbctl lrp-list lr0 | uuidfilt], [0], [dnl
> -<0> (lrp0)
> -<1> (lrp1)
> -])
> -
> -AT_CHECK([ovn-nbctl lr-add lr1])
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp1 00:00:00:01:02:03 192.168.1.1/24],
> [1], [],
> -  [ovn-nbctl: lrp1: a port with this name already exists
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr1 lrp1 00:00:00:01:02:03
> 192.168.1.1/24], [1], [],
> -  [ovn-nbctl: lrp1: port already exists but in router lr0
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:04:05:06
> 192.168.1.1/24], [1], [],
> -  [ovn-nbctl: lrp1: port already exists with mac 00:00:00:01:02:03
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 192.168.1.1/24], [1], [],
> -  [ovn-nbctl: lrp1: port already exists with mismatching peer
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 10.0.0.1/24 peer=lrp1-peer], [1], [],
> -  [ovn-nbctl: lrp1: port already exists with different network
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 192.168.1.1/24 peer=lrp1-peer])
> -
> -AT_CHECK([ovn-nbctl lrp-del lrp1])
> -AT_CHECK([ovn-nbctl lrp-list lr0 | uuidfilt], [0], [dnl
> -<0> (lrp0)
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 192.168.1.1/24 10.0.0.1/24 peer=lrp1-peer])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 192.168.1.1/24 172.16.0.1/24 peer=lrp1-peer], [1], [],
> -  [ovn-nbctl: lrp1: port already exists with different network
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lrp-add lr0 lrp1 00:00:00:01:02:03
> 10.0.0.1/24 192.168.1.1/24 peer=lrp1-peer])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lrp_gw_chassi], [logical router port gateway
> chassis], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02:03 192.168.1.1/24])
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [])
> -
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lp0 chassis1], [1], [],
> -[ovn-nbctl: lp0: port name not found
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lp0], [1], [],
> -[ovn-nbctl: lp0: port name not found
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-del-gateway-chassis lp0 chassis1], [1], [],
> -[ovn-nbctl: lp0: port name not found
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-del-gateway-chassis lrp0 chassis1], [1], [],
> -[ovn-nbctl: chassis chassis1 is not added to logical port lrp0
> -])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis1])
> -
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis1     0
> -])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis1 10])
> -
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis1    10
> -])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis1 20])
> -
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis1    20
> -])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis2 5])
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis1    20
> -lrp0-chassis2     5
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-del-gateway-chassis lrp0 chassis1])
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis2     5
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-del-gateway-chassis lrp0 chassis2])
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0])
> -
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis1 1])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis2 10])
> -AT_CHECK([ovn-nbctl lrp-set-gateway-chassis lrp0 chassis3 5])
> -AT_CHECK([ovn-nbctl lrp-get-gateway-chassis lrp0], [0], [dnl
> -lrp0-chassis2    10
> -lrp0-chassis3     5
> -lrp0-chassis1     1
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lrp_enable], [logical router port enable and
> disable], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -AT_CHECK([ovn-nbctl lrp-add lr0 lrp0 00:00:00:01:02:03 192.168.1.1/24])
> -AT_CHECK([ovn-nbctl lrp-get-enabled lrp0], [0], [enabled
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-set-enabled lrp0 disabled])
> -AT_CHECK([ovn-nbctl lrp-get-enabled lrp0], [0], [disabled
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-set-enabled lrp0 enabled])
> -AT_CHECK([ovn-nbctl lrp-get-enabled lrp0], [0], [enabled
> -])
> -
> -AT_CHECK([ovn-nbctl lrp-set-enabled lrp0 xyzzy], [1], [],
> -  [ovn-nbctl: xyzzy: state must be "enabled" or "disabled"
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_routes], [routes], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -
> -dnl Check IPv4 routes
> -AT_CHECK([ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.1.0/24 11.0.1.1 lp0])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.2])
> -
> -dnl Add overlapping route with 10.0.0.1/24
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1], [1], [],
> -  [ovn-nbctl: duplicate prefix: 10.0.0.0/24
> -])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111a/24 11.0.0.1], [1], [],
> -  [ovn-nbctl: bad prefix argument: 10.0.0.111a/24
> -])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24a 11.0.0.1], [1], [],
> -  [ovn-nbctl: bad prefix argument: 10.0.0.111/24a
> -])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1a], [1], [],
> -  [ovn-nbctl: bad next hop argument: 11.0.0.1a
> -])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1/24], [1], [],
> -  [ovn-nbctl: bad IPv4 nexthop argument: 11.0.0.1/24
> -])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64
> 2001:0db8:0:f103::1/64], [1], [],
> -  [ovn-nbctl: bad IPv6 nexthop argument: 2001:0db8:0:f103::1/64
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24 11.0.0.1])
> -AT_CHECK([ovn-nbctl --policy=src-ip lr-route-add lr0 9.16.1.0/24
> 11.0.0.1])
> -
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv4 Routes
> -              10.0.0.0/24                  11.0.0.1 dst-ip
> -              10.0.1.0/24                  11.0.1.1 dst-ip lp0
> -              9.16.1.0/24                  11.0.0.1 src-ip
> -                0.0.0.0/0               192.168.0.1 dst-ip
> -])
> -
> -AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24 11.0.0.1
> lp1])
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv4 Routes
> -              10.0.0.0/24                  11.0.0.1 dst-ip lp1
> -              10.0.1.0/24                  11.0.1.1 dst-ip lp0
> -              9.16.1.0/24                  11.0.0.1 src-ip
> -                0.0.0.0/0               192.168.0.1 dst-ip
> -])
> -
> -dnl Delete non-existent prefix
> -AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.2.1/24], [1], [],
> -  [ovn-nbctl: no matching prefix: 10.0.2.0/24
> -])
> -AT_CHECK([ovn-nbctl --if-exists lr-route-del lr0 10.0.2.1/24])
> -
> -AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.1.1/24])
> -AT_CHECK([ovn-nbctl lr-route-del lr0 9.16.1.0/24])
> -
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv4 Routes
> -              10.0.0.0/24                  11.0.0.1 dst-ip lp1
> -                0.0.0.0/0               192.168.0.1 dst-ip
> -])
> -
> -AT_CHECK([ovn-nbctl lr-route-del lr0])
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -])
> -
> -dnl Check IPv6 routes
> -AT_CHECK([ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0
> 2001:0db8:0:f101::1])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1
> lp0])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64
> 2001:0db8:0:f103::1])
> -
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv6 Routes
> -            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
> -          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
> -                     ::/0        2001:db8:0:f101::1 dst-ip
> -])
> -
> -AT_CHECK([ovn-nbctl lr-route-del lr0 2001:0db8:0::/64])
> -
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv6 Routes
> -          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
> -                     ::/0        2001:db8:0:f101::1 dst-ip
> -])
> -
> -AT_CHECK([ovn-nbctl lr-route-del lr0])
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -])
> -
> -dnl Check IPv4 and IPv6 routes
> -AT_CHECK([ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.1])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0
> 2001:0db8:0:f101::1])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1
> lp0])
> -AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64
> 2001:0db8:0:f103::1])
> -
> -AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> -IPv4 Routes
> -              10.0.0.0/24                  11.0.0.1 dst-ip
> -              10.0.1.0/24                  11.0.1.1 dst-ip lp0
> -                0.0.0.0/0               192.168.0.1 dst-ip
> -
> -IPv6 Routes
> -            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
> -          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
> -                     ::/0        2001:db8:0:f101::1 dst-ip
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_policies], [policies], [
> -AT_CHECK([ovn-nbctl lr-add lr0])
> -
> -dnl Add policies with allow and drop actions
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 100 "ip4.src == 1.1.1.0/24" drop])
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 100 "ip4.src == 1.1.2.0/24" allow])
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 101 "ip4.src == 2.1.1.0/24" allow])
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 101 "ip4.src == 2.1.2.0/24" drop])
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 101 "ip6.src == 2002::/64" drop])
> -
> -dnl Add duplicated policy
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 100 "ip4.src == 1.1.1.0/24" drop],
> [1], [],
> -  [ovn-nbctl: Same routing policy already existed on the logical router
> lr0.
> -])
> -
> -dnl Add duplicated policy
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 103 "ip4.src == 1.1.1.0/24" deny],
> [1], [],
> -  [ovn-nbctl: deny: action must be one of "allow", "drop", and "reroute"
> -])
> -
> -dnl Delete by priority and match string
> -AT_CHECK([ovn-nbctl lr-policy-del lr0 100 "ip4.src == 1.1.1.0/24"])
> -AT_CHECK([ovn-nbctl lr-policy-list lr0], [0], [dnl
> -Routing Policies
> -       101                              ip4.src == 2.1.1.0/24
>  allow
> -       101                              ip4.src == 2.1.2.0/24
> drop
> -       101                               ip6.src == 2002::/64
> drop
> -       100                              ip4.src == 1.1.2.0/24
>  allow
> -])
> -
> -dnl Delete all policies for given priority
> -AT_CHECK([ovn-nbctl lr-policy-del lr0 101])
> -AT_CHECK([ovn-nbctl lr-policy-list lr0], [0], [dnl
> -Routing Policies
> -       100                              ip4.src == 1.1.2.0/24
>  allow
> -])
> -
> -dnl Add policy with reroute action
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 102 "ip4.src == 3.1.2.0/24"
> reroute 3.3.3.3])
> -
> -dnl Add policy with invalid reroute ip
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 103 "ip4.src == 3.1.2.0/24"
> reroute 3.3.3.x], [1], [],
> -  [ovn-nbctl: bad next hop argument: 3.3.3.x
> -])
> -
> -dnl Add policy with reroute action
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 104 "ip6.src == 2001::/64" reroute
> 2002::5])
> -
> -dnl Add policy with invalid reroute ip
> -AT_CHECK([ovn-nbctl lr-policy-add lr0 105 "ip6.src == 2001::/64" reroute
> 2002::x], [1], [],
> -  [ovn-nbctl: bad next hop argument: 2002::x
> -])
> -
> -])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_lsp_types], [lsp types], [
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 lp0])
> -
> -dnl switchport type defaults to empty
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -
> -])
> -
> -dnl The following are the valid entries for
> -dnl switchport type
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 l2gateway])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -l2gateway
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 router])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -router
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 localnet])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -localnet
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 localport])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -localport
> -])
> -
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 vtep])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -vtep
> -])
> -
> -dnl All of these are valid southbound port types but
> -dnl should be rejected for northbound logical switch
> -dnl ports.
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 l3gateway], [1], [], [dnl
> -ovn-nbctl: Logical switch port type 'l3gateway' is unrecognized. Not
> setting type.
> -])
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 patch], [1], [], [dnl
> -ovn-nbctl: Logical switch port type 'patch' is unrecognized. Not setting
> type.
> -])
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 chassisredirect], [1], [], [dnl
> -ovn-nbctl: Logical switch port type 'chassisredirect' is unrecognized.
> Not setting type.
> -])
> -
> -dnl switch port type should still be "vtep" since previous
> -dnl commands failed.
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -vtep
> -])
> -
> -dnl Attempt a nonsense type
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 eggs], [1], [], [dnl
> -ovn-nbctl: Logical switch port type 'eggs' is unrecognized. Not setting
> type.
> -])
> -
> -dnl Empty string should work too
> -AT_CHECK([ovn-nbctl lsp-set-type lp0 ""])
> -AT_CHECK([ovn-nbctl lsp-get-type lp0], [0], [dnl
> -
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_connection], [connection], [
> -AT_CHECK([ovn-nbctl --inactivity-probe=30000 set-connection
> ptcp:6641:127.0.0.1 punix:$OVS_RUNDIR/ovnnb_db.sock])
> -AT_CHECK([ovn-nbctl list connection | grep inactivity_probe], [0], [dnl
> -inactivity_probe    : 30000
> -inactivity_probe    : 30000
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_dry_run_mode], [dry run mode], [
> -dnl Check that dry run has no permanent effect.
> -AT_CHECK([ovn-nbctl --dry-run ls-add ls0 -- ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)
> -])
> -AT_CHECK([ovn-nbctl ls-list | uuidfilt], [0], [dnl
> -])
> -
> -dnl Check that dry-run mode is not sticky.
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_oneline_output], [oneline output], [
> -AT_CHECK([ovn-nbctl ls-add ls0 -- ls-add ls1])
> -
> -dnl Expect one line for one command.
> -AT_CHECK([ovn-nbctl --oneline ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)\n<1> (ls1)
> -])
> -
> -dnl Expect lines for two commands.
> -AT_CHECK([ovn-nbctl --oneline ls-list -- ls-list | uuidfilt], [0], [dnl
> -<0> (ls0)\n<1> (ls1)
> -<0> (ls0)\n<1> (ls1)
> -])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_error_paths], [commands parser error paths], [
> -dnl FIXME: Duplicate options are allowed when passed with global options.
> -dnl        For example: ovn-nbctl --if-exists --if-exists list
> Logical_Switch
> -
> -dnl Duplicate option
> -AT_CHECK([ovn-nbctl -- --if-exists --if-exists list Logical_Switch], [1],
> [], [stderr])
> -AT_CHECK([grep 'option specified multiple times' stderr], [0], [ignore])
> -
> -dnl Missing command
> -AT_CHECK([ovn-nbctl], [1], [], [stderr])
> -AT_CHECK([grep 'missing command name' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl --if-exists], [1], [], [stderr])
> -AT_CHECK([grep 'missing command name' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl --], [1], [], [stderr])
> -AT_CHECK([grep 'missing command name' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl -- --if-exists], [1], [], [stderr])
> -AT_CHECK([grep 'missing command name' stderr], [0], [ignore])
> -
> -dnl Unknown command
> -AT_CHECK([ovn-nbctl foo], [1], [], [stderr])
> -AT_CHECK([grep 'unknown command' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl -- foo], [1], [], [stderr])
> -AT_CHECK([grep 'unknown command' stderr], [0], [ignore])
> -
> -dnl Unknown option
> -AT_CHECK([ovn-nbctl --foo list Logical_Switch], [1], [], [stderr])
> -AT_CHECK([grep 'unrecognized option' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl -- --foo list Logical_Switch], [1], [], [stderr])
> -AT_CHECK([grep 'command has no .* option' stderr], [0], [ignore])
> -
> -dnl Missing option argument
> -AT_CHECK([ovn-nbctl --columns], [1], [], [stderr])
> -AT_CHECK([grep 'option .* requires an argument' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl -- --columns list Logical_Switch], [1], [], [stderr])
> -AT_CHECK([grep 'missing argument to .* option' stderr], [0], [ignore])
> -
> -dnl Unexpected option argument
> -AT_CHECK([ovn-nbctl --if-exists=foo list Logical_Switch], [1], [],
> [stderr])
> -AT_CHECK([egrep 'option .* doesn'\''t allow an argument|option .*
> requires an argument' stderr], [0], [ignore])
> -
> -AT_CHECK([ovn-nbctl -- --if-exists=foo list Logical_Switch], [1], [],
> [stderr])
> -AT_CHECK([grep 'option on .* does not accept an argument' stderr], [0],
> [ignore])
> -
> -dnl Not enough arguments
> -AT_CHECK([ovn-nbctl list], [1], [], [stderr])
> -AT_CHECK([grep 'command requires at least .* arguments' stderr], [0],
> [ignore])
> -
> -AT_CHECK([ovn-nbctl -- list], [1], [], [stderr])
> -AT_CHECK([grep 'command requires at least .* arguments' stderr], [0],
> [ignore])
> -
> -dnl Too many arguments
> -AT_CHECK([ovn-nbctl show foo bar], [1], [], [stderr])
> -AT_CHECK([grep 'command takes at most .* arguments' stderr], [0],
> [ignore])
> -
> -AT_CHECK([ovn-nbctl -- show foo bar], [1], [], [stderr])
> -AT_CHECK([grep 'command takes at most .* arguments' stderr], [0],
> [ignore])
> -
> -AT_CHECK([ovn-nbctl show foo --bar], [1], [], [stderr])
> -AT_CHECK([grep 'command takes at most .* arguments' stderr], [0],
> [ignore])
> -
> -AT_CHECK([ovn-nbctl -- show foo --bar], [1], [], [stderr])
> -AT_CHECK([grep 'command takes at most .* arguments' stderr], [0],
> [ignore])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_port_groups], [port groups], [
> -dnl Check that port group can be looked up by name
> -AT_CHECK([ovn-nbctl create Port_Group name=pg0], [0], [ignore])
> -AT_CHECK([ovn-nbctl get Port_Group pg0 name], [0], [dnl
> -pg0
> -])])
> -
> -OVN_NBCTL_TEST([ovn_nbctl_extra_newlines], [extra newlines], [
> -dnl This test addresses a specific issue seen when running ovn-nbctl in
> -dnl daemon mode. All we have to do is ensure that each time we list
> database
> -dnl information, there is not an extra newline at the beginning of the
> output.
> -AT_CHECK([ovn-nbctl ls-add sw1], [0], [ignore])
> -AT_CHECK([ovn-nbctl --columns=name list logical_switch sw1], [0], [dnl
> -name                : sw1
> -])
> -AT_CHECK([ovn-nbctl --columns=name list logical_switch sw1], [0], [dnl
> -name                : sw1
> -])])
> -
> -OVN_NBCTL_TEST([ovn_nbctl_table_formatting], [table formatting], [
> -dnl This test addresses a specific issue seen when running ovn-nbctl in
> -dnl daemon mode. We need to ensure that table formatting options are
> honored
> -dnl when listing database information.
> -AT_CHECK([ovn-nbctl ls-add sw1], [0], [ignore])
> -AT_CHECK([ovn-nbctl --bare --columns=name list logical_switch sw1], [0],
> [dnl
> -sw1
> -])])
> -dnl ---------------------------------------------------------------------
> -
> -OVN_NBCTL_TEST([ovn_nbctl_port_group_commands], [port group commands], [
> -AT_CHECK([ovn-nbctl pg-add pg1], [0], [ignore])
> -AT_CHECK([ovn-nbctl --bare --columns=name list port_group pg1], [0],
> -[pg1
> -])
> -
> -AT_CHECK([ovn-nbctl pg-del pg1], [0], [ignore])
> -AT_CHECK([ovn-nbctl list port_group], [0], [])
> -
> -AT_CHECK([ovn-nbctl ls-add sw1], [0], [ignore])
> -AT_CHECK([ovn-nbctl lsp-add sw1 sw1-p1], [0], [ignore])
> -SW1P1=$(ovn-nbctl --bare --columns=_uuid list logical_switch_port sw1-p1)
> -AT_CHECK([ovn-nbctl lsp-add sw1 sw1-p2], [0], [ignore])
> -SW1P2=$(ovn-nbctl --bare --columns=_uuid list logical_switch_port sw1-p2)
> -
> -AT_CHECK([ovn-nbctl pg-add pg1 sw1-p1], [0], [ignore])
> -AT_CHECK([ovn-nbctl --bare --columns=name list port_group pg1], [0],[dnl
> -pg1
> -])
> -AT_CHECK_UNQUOTED([ovn-nbctl --bare --columns=ports list port_group pg1],
> [0], [dnl
> -$SW1P1
> -])
> -
> -AT_CHECK([ovn-nbctl pg-set-ports pg1 sw1-p2], [0], [ignore])
> -AT_CHECK_UNQUOTED([ovn-nbctl --bare --columns=ports list port_group pg1],
> [0], [dnl
> -$SW1P2
> -])
> -
> -AT_CHECK([ovn-nbctl pg-del pg1], [0], [ignore])
> -AT_CHECK([ovn-nbctl list port_group], [0], [])
> -])
> -
> -AT_SETUP([ovn-nbctl - daemon retry connection])
> -OVN_NBCTL_TEST_START daemon
> -AT_CHECK([kill `cat ovsdb-server.pid`])
> -AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --log-file
> --remote=punix:$OVS_RUNDIR/ovnnb_db.sock ovn-nb.db], [0], [], [stderr])
> -AT_CHECK([ovn-nbctl show], [0], [ignore])
> -OVN_NBCTL_TEST_STOP /Terminated/d
> -AT_CLEANUP
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> deleted file mode 100644
> index 62e58fd0e..000000000
> --- a/tests/ovn-northd.at
> +++ /dev/null
> @@ -1,900 +0,0 @@
> -AT_BANNER([OVN northd])
> -AT_SETUP([ovn -- check   from NBDB to SBDB])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> -ovn-sbctl chassis-add gw2 geneve 1.2.4.8
> -
> -# Connect alice to R1 as distributed router gateway port on hv2
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24
> -
> -ovn-nbctl --wait=sb \
> -    --id=@gc0 create Gateway_Chassis name=alice_gw1 \
> -                                     chassis_name=gw1 \
> -                                     priority=20 -- \
> -    --id=@gc1 create Gateway_Chassis name=alice_gw2 \
> -                                     chassis_name=gw2 \
> -                                     priority=10 -- \
> -    set Logical_Router_Port alice 'gateway_chassis=[@gc0, at gc1]'
> -
> -nb_gwc1_uuid=`ovn-nbctl --bare --columns _uuid find Gateway_Chassis
> name="alice_gw1"`
> -
> -# With the new ha_chassis_group table added, there should be no rows in
> -# gateway_chassis table in SB DB.
> -AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0
> -])
> -
> -# There should be one ha_chassis_group with the name "alice"
> -ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="alice"`
> -
> -AT_CHECK([test $ha_chassi_grp_name = alice])
> -
> -ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group
> name=alice`
> -
> -AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \
> -logical_port="cr-alice" | grep $ha_chgrp_uuid | wc -l], [0], [1
> -])
> -
> -# There should be one ha_chassis_group with the name "alice"
> -ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="alice"`
> -
> -AT_CHECK([test $ha_chassi_grp_name = alice])
> -
> -ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group
> name=alice`
> -
> -AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \
> -logical_port="cr-alice" | grep $ha_chgrp_uuid | wc -l], [0], [1
> -])
> -
> -ha_ch=`ovn-sbctl --bare --columns ha_chassis  find ha_chassis_group`
> -# Trim the spaces.
> -ha_ch=`echo $ha_ch | sed 's/ //g'`
> -
> -ha_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list="$ha_ch_list $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list=`echo $ha_ch_list | sed 's/ //g'`
> -
> -AT_CHECK([test "$ha_ch_list" = "$ha_ch"])
> -
> -# Delete chassis - gw2 in SB DB.
> -# ovn-northd should not recreate ha_chassis rows
> -# repeatedly when gw2 is deleted.
> -ovn-sbctl chassis-del gw2
> -
> -ha_ch_list_1=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list_1="$ha_ch_list_1 $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list_1=`echo $ha_ch_list_1 | sed 's/ //g'`
> -
> -ha_ch_list_2=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list_2="$ha_ch_list_2 $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list_2=`echo $ha_ch_list_2 | sed 's/ //g'`
> -
> -AT_CHECK([test "$ha_ch_list_1" = "$ha_ch_list_2"])
> -
> -# Add back the gw2 chassis
> -ovn-sbctl chassis-add gw2 geneve 1.2.4.8
> -
> -# delete the 2nd Gateway_Chassis on NBDB for alice port
> -gw_ch=`ovn-sbctl --bare --columns gateway_chassis find port_binding \
> -logical_port="cr-alice"`
> -AT_CHECK([test "$gw_ch" = ""])
> -
> -ha_ch=`ovn-sbctl --bare --columns ha_chassis  find ha_chassis_group`
> -ha_ch=`echo $ha_ch | sed 's/ //g'`
> -# Trim the spaces.
> -echo "ha ch in grp = $ha_ch"
> -
> -ha_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list="$ha_ch_list $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list=`echo $ha_ch_list | sed 's/ //g'`
> -
> -AT_CHECK([test "$ha_ch_list" = "$ha_ch"])
> -
> -# delete the 2nd Gateway_Chassis on NBDB for alice port
> -ovn-nbctl --wait=sb set Logical_Router_Port alice
> gateway_chassis=${nb_gwc1_uuid}
> -
> -# There should be only 1 row in ha_chassis SB DB table.
> -AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0],
> [1
> -])
> -
> -AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0
> -])
> -
> -# There should be only 1 row in ha_chassis SB DB table.
> -AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0],
> [1
> -])
> -
> -# delete all the gateway_chassis on NBDB for alice port
> -
> -ovn-nbctl --wait=sb clear Logical_Router_Port alice gateway_chassis
> -
> -# expect that the ha_chassis doesn't exist anymore
> -AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0
> -])
> -
> -AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0
> -])
> -
> -AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0
> -])
> -
> -# expect that the ha_chassis doesn't exist anymore
> -AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0
> -])
> -AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check Gateway_Chassis propagation from NBDB to SBDB
> backwards compatibility])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> -ovn-sbctl chassis-add gw2 geneve 1.2.4.8
> -
> -ovn-nbctl --wait=sb lrp-add R1 bob 00:00:02:01:02:03 172.16.1.1/24 \
> -    -- set Logical_Router_Port bob options:redirect-chassis="gw1"
> -
> -
> -# It should be converted to ha_chassis_group entries in SBDB, and
> -# still redirect-chassis is kept for backwards compatibility
> -
> -AT_CHECK([ovn-sbctl list gateway_chassis | wc -l], [0], [0
> -])
> -
> -AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis | wc -l], [0],
> [1
> -])
> -
> -AT_CHECK([ovn-sbctl --bare --columns _uuid list ha_chassis_group | wc
> -l], [0], [1
> -])
> -
> -# There should be one ha_chassis_group with the name "bob_gw1"
> -ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="bob_gw1"`
> -
> -AT_CHECK([test $ha_chassi_grp_name = bob_gw1])
> -
> -ha_chgrp_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group
> name=bob_gw1`
> -
> -AT_CHECK([ovn-sbctl --bare --columns ha_chassis_group find port_binding \
> -logical_port="cr-bob" | grep $ha_chgrp_uuid | wc -l], [0], [1
> -])
> -
> -ovn-nbctl --wait=sb remove Logical_Router_Port bob options
> redirect-chassis
> -
> -# expect that the ha_chassis/ha_chassis_group doesn't exist anymore
> -
> -AT_CHECK([ovn-sbctl find Gateway_Chassis name=bob_gw1], [0], [])
> -AT_CHECK([ovn-sbctl list ha_chassis | wc -l], [0], [0
> -])
> -
> -AT_CHECK([ovn-sbctl list ha_chassis_group | wc -l], [0], [0
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check up state of VIF LSP])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add S1
> -ovn-nbctl --wait=sb lsp-add S1 S1-vm1
> -AT_CHECK([test x`ovn-nbctl lsp-get-up S1-vm1` = xdown])
> -
> -ovn-sbctl chassis-add hv1 geneve 127.0.0.1
> -ovn-sbctl lsp-bind S1-vm1 hv1
> -AT_CHECK([test x`ovn-nbctl lsp-get-up S1-vm1` = xup])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check up state of router LSP linked to a distributed LR])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lrp-add R1 R1-S1 02:ac:10:01:00:01 172.16.1.1/24
> -
> -ovn-nbctl ls-add S1
> -ovn-nbctl lsp-add S1 S1-R1
> -ovn-nbctl lsp-set-type S1-R1 router
> -ovn-nbctl lsp-set-addresses S1-R1 02:ac:10:01:00:01
> -ovn-nbctl --wait=sb lsp-set-options S1-R1 router-port=R1-S1
> -AT_CHECK([test x`ovn-nbctl lsp-get-up S1-R1` = xup])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check up state of router LSP linked to a gateway LR])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> -
> -ovn-nbctl create Logical_Router name=R1 options:chassis=gw1
> -ovn-nbctl lrp-add R1 R1-S1 02:ac:10:01:00:01 172.16.1.1/24
> -
> -ovn-nbctl ls-add S1
> -ovn-nbctl lsp-add S1 S1-R1
> -ovn-nbctl lsp-set-type S1-R1 router
> -ovn-nbctl lsp-set-addresses S1-R1 02:ac:10:01:00:01
> -ovn-nbctl --wait=sb lsp-set-options S1-R1 router-port=R1-S1
> -
> -ovn-sbctl lsp-bind S1-R1 gw1
> -AT_CHECK([test x`ovn-nbctl lsp-get-up S1-R1` = xup])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check up state of router LSP linked to an LRP with set
> Gateway Chassis])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lrp-add R1 R1-S1 02:ac:10:01:00:01 172.16.1.1/24
> -ovn-nbctl lrp-set-gateway-chassis R1-S1 gw1
> -
> -ovn-nbctl ls-add S1
> -ovn-nbctl lsp-add S1 S1-R1
> -ovn-nbctl lsp-set-type S1-R1 router
> -ovn-nbctl lsp-set-addresses S1-R1 router
> -ovn-nbctl --wait=sb lsp-set-options S1-R1 router-port=R1-S1
> -AT_CHECK([test x`ovn-nbctl lsp-get-up S1-R1` = xup])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check IPv6 RA config propagation to SBDB])
> -ovn_start
> -
> -ovn-nbctl lr-add ro
> -ovn-nbctl lrp-add ro ro-sw 00:00:00:00:00:01 aef0:0:0:0:0:0:0:1/64
> -ovn-nbctl ls-add sw
> -ovn-nbctl lsp-add sw sw-ro
> -ovn-nbctl lsp-set-type sw-ro router
> -ovn-nbctl lsp-set-options sw-ro router-port=ro-sw
> -ovn-nbctl lsp-set-addresses sw-ro 00:00:00:00:00:01
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:send_periodic=true
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:address_mode=slaac
> -ovn-nbctl --wait=sb set Logical_Router_Port ro-sw ipv6_ra_configs:mtu=1280
> -
> -uuid=$(ovn-sbctl --columns=_uuid --bare find Port_Binding
> logical_port=ro-sw)
> -
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_send_periodic],
> -[0], ["true"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_address_mode],
> -[0], [slaac
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[0], ["600"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[0], ["200"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid} options:ipv6_ra_mtu],
> -[0], ["1280"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid} options:ipv6_ra_src_eth],
> -[0], ["00:00:00:00:00:01"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid} options:ipv6_ra_src_addr],
> -[0], ["fe80::200:ff:fe00:1"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid} options:ipv6_ra_prefixes],
> -[0], ["aef0::/64"
> -])
> -
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:max_interval=300
> -ovn-nbctl --wait=sb set Logical_Router_Port ro-sw
> ipv6_ra_configs:min_interval=600
> -
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[0], ["300"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[0], ["225"
> -])
> -
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:max_interval=300
> -ovn-nbctl --wait=sb set Logical_Router_Port ro-sw
> ipv6_ra_configs:min_interval=250
> -
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[0], ["300"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[0], ["225"
> -])
> -
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:max_interval=0
> -ovn-nbctl --wait=sb set Logical_Router_Port ro-sw
> ipv6_ra_configs:min_interval=0
> -
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[0], ["4"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[0], ["3"
> -])
> -
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:max_interval=3600
> -ovn-nbctl --wait=sb set Logical_Router_Port ro-sw
> ipv6_ra_configs:min_interval=2400
> -
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[0], ["1800"
> -])
> -AT_CHECK([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[0], ["1350"
> -])
> -
> -ovn-nbctl --wait=sb set Logical_Router_port ro-sw
> ipv6_ra_configs:send_periodic=false
> -
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_send_periodic],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_send_periodic" in Port_Binding
> record "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_max_interval],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_max_interval" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_min_interval],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_min_interval" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_mtu],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_mtu" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_address_mode],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_address_mode" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_src_eth],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_src_eth" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_src_addr],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_src_addr" in Port_Binding record
> "${uuid}" column options
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get Port_Binding ${uuid}
> options:ipv6_ra_prefixes],
> -[1], [], [ovn-sbctl: no key "ipv6_ra_prefixes" in Port_Binding record
> "${uuid}" column options
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- test unixctl])
> -ovn_init_db ovn-sb; ovn-sbctl init
> -ovn_init_db ovn-nb; ovn-nbctl init
> -
> -# test unixctl option
> -mkdir "$ovs_base"/northd
> -as northd start_daemon ovn-northd
> --unixctl="$ovs_base"/northd/ovn-northd.ctl
> --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock
> --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -ovn-nbctl ls-add sw
> -ovn-nbctl --wait=sb lsp-add sw p1
> -# northd created with unixctl option successfully created port_binding
> entry
> -AT_CHECK([ovn-sbctl --bare --columns datapath find port_binding
> logical_port="p1" | wc -l], [0], [1
> -])
> -AT_CHECK([ovn-nbctl --wait=sb lsp-del p1])
> -
> -# ovs-appctl exit with unixctl option
> -OVS_APP_EXIT_AND_WAIT_BY_TARGET(["$ovs_base"/northd/ovn-northd.ctl],
> ["$ovs_base"/northd/ovn-northd.pid])
> -
> -# Check no port_binding entry for new port as ovn-northd is not running
> -ovn-nbctl lsp-add sw p2
> -ovn-nbctl --timeout=10 --wait=sb sync
> -AT_CHECK([ovn-sbctl --bare --columns datapath find port_binding
> logical_port="p2" | wc -l], [0], [0
> -])
> -
> -# test default unixctl path
> -as northd start_daemon ovn-northd
> --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock
> --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -ovn-nbctl --wait=sb lsp-add sw p3
> -# northd created with default unixctl path successfully created
> port_binding entry
> -AT_CHECK([ovn-sbctl --bare --columns datapath find port_binding
> logical_port="p3" | wc -l], [0], [1
> -])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- check HA_Chassis_Group propagation from NBDB to SBDB])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl --wait=sb ha-chassis-group-add hagrp1
> -
> -# ovn-northd should not create HA chassis group and HA chassis rows
> -# unless the HA chassis group in OVN NB DB is associated to
> -# a logical router port or logical port of type external.
> -AT_CHECK([ovn-sbctl --bare --columns name find ha_chassis_group
> name="hagrp1" \
> -| wc -l], [0], [0
> -])
> -
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 30
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch2 20
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch3 10
> -
> -# There should be no HA_Chassis rows in SB DB.
> -AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \
> -| grep -v '-' | wc -l ], [0], [0
> -])
> -
> -# Add chassis ch1.
> -ovn-sbctl chassis-add ch1 geneve 127.0.0.2
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl list chassis | grep ch1 | wc -l`])
> -
> -# There should be no HA_Chassis rows
> -AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | awk '{print $3}' \
> -| grep -v '-' | wc -l ], [0], [0
> -])
> -
> -# Create a logical router port and attach ha chassis group.
> -ovn-nbctl lr-add lr0
> -ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24
> -
> -hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group
> name=hagrp1`
> -ovn-nbctl set logical_router_port lr0-public ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Make sure that ovn-northd doesn't recreate the ha_chassis
> -# records if the chassis record is missing in SB DB.
> -
> -ha_ch_list_1=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list_1="$ha_ch_list_1 $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list_1=`echo $ha_ch_list_1 | sed 's/ //g'`
> -
> -ha_ch_list_2=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list_2="$ha_ch_list_2 $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list_2=`echo $ha_ch_list_2 | sed 's/ //g'`
> -
> -AT_CHECK([test "$ha_ch_list_1" = "$ha_ch_list_2"])
> -
> -# 2 HA chassis should be created with 'chassis' column empty because
> -# we have not added hv1 and hv2 chassis to the SB DB.
> -AT_CHECK([test 2 = `ovn-sbctl list ha_chassis | grep chassis | awk
> '{print $3}' \
> -| grep -v '-' | wc -l`])
> -
> -# We should have 1 ha chassis with 'chassis' column set for hv1
> -AT_CHECK([test 1 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | awk '{print $3}' \
> -| grep '-' | wc -l`])
> -
> -# Create another logical router port and associate to the same
> ha_chasis_group
> -ovn-nbctl lr-add lr1
> -ovn-nbctl lrp-add lr1 lr1-public 00:00:20:20:12:14 182.168.0.100/24
> -
> -ovn-nbctl set logical_router_port lr1-public ha_chassis_group=$hagrp1_uuid
> -
> -# We should still have 1 HA chassis group and 3 HA chassis in SB DB.
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Change the priority of ch1 - ha chassis in NB DB. It should get
> -# reflected in SB DB.
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 100
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns priority find \
> -ha_chassis | grep 100 | wc -l`])
> -
> -# Delete ch1 HA chassis in NB DB.
> -ovn-nbctl --wait=sb ha-chassis-group-remove-chassis hagrp1 ch1
> -
> -OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Add back the ha chassis
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 40
> -OVS_WAIT_UNTIL([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Delete lr0-public. We should still have 1 HA chassis group and
> -# 3 HA chassis in SB DB.
> -ovn-nbctl --wait=sb lrp-del lr0-public
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Delete lr1-public. There should be no HA chassis group in SB DB.
> -ovn-nbctl --wait=sb lrp-del lr1-public
> -
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`])
> -
> -# Add lr0-public again
> -ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24
> -ovn-nbctl <http://172.168.0.100/24-ovn-nbctl> set logical_router_port
> lr0-public ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Create a Gateway chassis. ovn-northd should ignore this.
> -ovn-nbctl lrp-set-gateway-chassis lr0-public ch-1 20
> -
> -# There should be only 1 HA chassis group in SB DB with the
> -# name hagrp1.
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group | wc -l`])
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Now delete HA chassis group. ovn-northd should create HA chassis group
> -# with the Gateway chassis name
> -ovn-nbctl clear logical_router_port lr0-public ha_chassis_group
> -ovn-nbctl ha-chassis-group-del hagrp1
> -
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="lr0-public" | wc -l`])
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns _uuid \
> -find ha_chassis | wc -l`])
> -
> -ovn-nbctl lrp-set-gateway-chassis lr0-public ch2 10
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="lr0-public" | wc -l`])
> -
> -ovn-sbctl --bare --columns _uuid find ha_chassis
> -OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Test if 'ref_chassis' column is properly set or not in
> -# SB DB ha_chassis_group.
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 sw0-p1
> -
> -ovn-sbctl chassis-add ch2 geneve 127.0.0.3
> -ovn-sbctl chassis-add ch3 geneve 127.0.0.4
> -ovn-sbctl chassis-add comp1 geneve 127.0.0.5
> -ovn-sbctl chassis-add comp2 geneve 127.0.0.6
> -
> -ovn-nbctl lrp-add lr0 lr0-sw0 00:00:20:20:12:14 10.0.0.1/24
> -ovn-nbctl lsp-add sw0 sw0-lr0
> -ovn-nbctl lsp-set-type sw0-lr0 router
> -ovn-nbctl lsp-set-addresses sw0-lr0 router
> -ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> -
> -ovn-sbctl lsp-bind sw0-p1 comp1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p1` = xup])
> -
> -comp1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp1"`
> -comp2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp2"`
> -ch2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="comp1"`
> -
> -echo "comp1_ch_uuid = $comp1_ch_uuid"
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp1_ch_uuid" = "$ref_ch_list"])
> -
> -# unbind sw0-p1
> -ovn-sbctl lsp-unbind sw0-p1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p1` = xdown])
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "" = "$ref_ch_list"])
> -
> -# Bind sw0-p1 in comp2
> -ovn-sbctl lsp-bind sw0-p1 comp2
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp2_ch_uuid" = "$ref_ch_list"])
> -
> -ovn-nbctl ls-add sw1
> -ovn-nbctl lsp-add sw1 sw1-p1
> -ovn-nbctl lr-add lr1
> -ovn-nbctl lrp-add lr1 lr1-sw1 00:00:20:20:12:15 20.0.0.1/24
> -ovn-nbctl lsp-add sw1 sw1-lr1
> -ovn-nbctl lsp-set-type sw1-lr1 router
> -ovn-nbctl lsp-set-addresses sw1-lr1 router
> -ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> -
> -# Bind sw1-p1 in comp1.
> -ovn-sbctl lsp-bind sw1-p1 comp1
> -# Wait until sw1-p1 is up
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw1-p1` = xup])
> -
> -# sw1-p1 is not connected to lr0. So comp1 should not be in 'ref_chassis'
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp2_ch_uuid" = "$ref_ch_list"])
> -
> -# Now attach sw0 to lr1
> -ovn-nbctl lrp-add lr1 lr1-sw0 00:00:20:20:12:16 10.0.0.10/24
> -ovn-nbctl lsp-add sw0 sw0-lr1
> -ovn-nbctl lsp-set-type sw0-lr1 router
> -ovn-nbctl lsp-set-addresses sw0-lr1 router
> -ovn-nbctl lsp-set-options sw0-lr1 router-port=lr1-sw0
> -
> -# Both comp1 and comp2 should be in 'ref_chassis' as sw1 is indirectly
> -# connected to lr0
> -exp_ref_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list chassis | sort`
> -do
> -    if test $i = $comp1_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    elif test $i = $comp2_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    fi
> -done
> -
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# Unind sw1-p1. comp2 should not be in the ref_chassis.
> -ovn-sbctl lsp-unbind sw1-p1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw1-p1` = xdown])
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp2_ch_uuid" = "$ref_ch_list"])
> -
> -# Create sw2 and attach it to lr2
> -ovn-nbctl ls-add sw2
> -ovn-nbctl lsp-add sw2 sw2-p1
> -ovn-nbctl lr-add lr2
> -ovn-nbctl lrp-add lr2 lr2-sw2 00:00:20:20:12:17 30.0.0.1/24
> -ovn-nbctl lsp-add sw2 sw2-lr2
> -ovn-nbctl lsp-set-type sw2-lr2 router
> -ovn-nbctl lsp-set-addresses sw2-lr2 router
> -ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2
> -
> -# Bind sw2-p1 to comp1
> -ovn-sbctl lsp-bind sw2-p1 comp1
> -# Wait until sw2-p1 is up
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw2-p1` = xup])
> -
> -# sw2-p1 is not connected to lr0. So comp1 should not be in 'ref_chassis'
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp2_ch_uuid" = "$ref_ch_list"])
> -
> -# Now attach sw1 to lr2. With this sw2-p1 is indirectly connected to lr0.
> -ovn-nbctl lrp-add lr2 lr2-sw1 00:00:20:20:12:18 20.0.0.10/24
> -ovn-nbctl lsp-add sw1 sw1-lr2
> -ovn-nbctl lsp-set-type sw1-lr2 router
> -ovn-nbctl lsp-set-addresses sw1-lr2 router
> -ovn-nbctl lsp-set-options sw1-lr2 router-port=lr2-sw1
> -
> -# sw2-p1 is indirectly connected to lr0. So comp1 (and comp2) should be in
> -# 'ref_chassis'
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# Create sw0-p2 and bind it to comp1
> -ovn-nbctl lsp-add sw0 sw0-p2
> -ovn-sbctl lsp-bind sw0-p2 comp1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p2` = xup])
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# unbind sw0-p2
> -ovn-sbctl lsp-unbind sw0-p2
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0-p2` = xdown])
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# Delete lr1-sw0. comp1 should be deleted from ref_chassis as there is no
> link
> -# from sw1 and sw2 to lr0.
> -ovn-nbctl lrp-del lr1-sw0
> -
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$comp2_ch_uuid" = "$ref_ch_list"])
> -
> -# Set redirect-chassis option to lr0-public. It should be ignored.
> -ovn-nbctl set logical_router_port lr0-public options:redirect-chassis=ch1
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group | wc -l`])
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="lr0-public" | wc -l`])
> -
> -ovn-sbctl --bare --columns _uuid find ha_chassis
> -OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Delete the gateway chassis. HA chassis group should be created in SB DB
> -# for the redirect-chassis option.
> -ovn-nbctl clear logical_router_port lr0-public gateway_chassis
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group | wc -l`])
> -
> -ovn-sbctl list ha_chassis_group
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="lr0-public_ch1" | wc -l`])
> -
> -ovn-sbctl --bare --columns _uuid find ha_chassis
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl list ha_chassis | grep chassis |
> -grep -v chassis-name | wc -l`])
> -
> -# Clear the redirect-chassis option.
> -ovn-nbctl clear logical_router_port lr0-public options
> -
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group |  wc -l`])
> -AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`])
> -
> -# Delete old sw0.
> -ovn-nbctl ls-del sw0
> -
> -# Create external logical ports and associate ha_chassis_group
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 sw0-pext1
> -ovn-nbctl lsp-add sw0 sw0-pext2
> -ovn-nbctl lsp-add sw0 sw0-p1
> -
> -ovn-nbctl lsp-set-addresses sw0-pext1 "00:00:00:00:00:03 10.0.0.3"
> -ovn-nbctl lsp-set-addresses sw0-pext2 "00:00:00:00:00:03 10.0.0.4"
> -ovn-nbctl lsp-set-addresses sw0-p1 "00:00:00:00:00:03 10.0.0.5"
> -
> -ovn-nbctl --wait=sb ha-chassis-group-add hagrp1
> -
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch1 30
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch2 20
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 ch3 10
> -
> -# ovn-northd should not create HA chassis group and HA chassis rows
> -# unless the HA chassis group in OVN NB DB is associated to
> -# a logical router port or logical port of type external.
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group |  wc -l`])
> -AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`])
> -
> -hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group \
> -name=hagrp1`
> -
> -# The type of the lsp - sw0-pext1 is still not set to external.
> -# So ha_chassis_group should be ignored.
> -ovn-nbctl set logical_switch_port sw0-pext1 ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | grep chassis | wc -l`])
> -
> -# Set the type of sw0-pext1 to external
> -ovn-nbctl lsp-set-type sw0-pext1 external
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -sb_hagrp1_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group \
> -name=hagrp1`
> -
> -AT_CHECK([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \
> -ha_chassis_group find port_binding logical_port=sw0-pext1`])
> -
> -# Set the type of sw0-pext2 to external and associate ha_chassis_group
> -ovn-nbctl lsp-set-type sw0-pext2 external
> -ovn-nbctl set logical_switch_port sw0-pext2 ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis |
> -grep -v chassis-name | wc -l`])
> -AT_CHECK([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \
> -ha_chassis_group find port_binding logical_port=sw0-pext1`])
> -
> -OVS_WAIT_UNTIL([test "$sb_hagrp1_uuid" = `ovn-sbctl --bare --columns \
> -ha_chassis_group find port_binding logical_port=sw0-pext2`])
> -
> -# sw0-p1 is a normal port. So ha_chassis_group should not be set
> -# in port_binding.
> -ovn-nbctl --wait=sb set logical_switch_port sw0-p1 \
> -ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find
> port_binding \
> -logical_port=sw0-p1) = x], [0], [])
> -
> -# Clear ha_chassis_group for sw0-pext1
> -ovn-nbctl --wait=sb clear logical_switch_port sw0-pext1 ha_chassis_group
> -
> -OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find
> port_binding \
> -logical_port=sw0-pext1) = x], [0], [])
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="hagrp1" | wc -l`])
> -
> -AT_CHECK([test 3 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -# Clear ha_chassis_group for sw0-pext2
> -ovn-nbctl --wait=sb clear logical_switch_port sw0-pext2 ha_chassis_group
> -
> -OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find
> port_binding \
> -logical_port=sw0-pext2) = x], [0], [])
> -
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group |  wc -l`])
> -AT_CHECK([test 0 = `ovn-sbctl list ha_chassis | wc -l`])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -AT_CLEANUP
> diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at
> deleted file mode 100644
> index a8a15f8fe..000000000
> --- a/tests/ovn-performance.at
> +++ /dev/null
> @@ -1,424 +0,0 @@
> -#
> -# Tests targeting performance of OVN components.
> -#
> -
> -m4_divert_push([PREPARE_TESTS])
> -
> -# vec_cmp VALUE_VEC OP-VALUE_VEC
> -#
> -# Compares each value from VALUE_VEC to the operator-value pair from the
> -# OP-VALUE_VEC.
> -#
> -# VALUE_VEC must be a list of values separated by a character from $IFS.
> -# OP-VALUE_VEC must be a list of operator-value expressions separated by a
> -# character from $IFS.  Operator-value expressions cannot contain any
> characters
> -# from $IFS like spaces. '=' is treated as an equality operator ('==') for
> -# conciseness.
> -#
> -# Returns the result of each comparison as a list of boolean values (0 or
> 1)
> -# separated by a new-line character.
> -vec_cmp() {
> -    local a b i j
> -
> -    i=0
> -    for a in $1; do
> -        j=0
> -        for b in $2; do
> -            if test $i -eq $j; then
> -                # Replace assignment '=' with equality comparison '=='
> -                case "$b" in
> -                =[[0-9]]*) b="=$b" ;;
> -                esac
> -
> -                echo $(($a $b))
> -                break
> -            fi
> -            j=$((j + 1))
> -        done
> -        i=$((i + 1))
> -    done
> -}
> -
> -# vec_sub VEC_A VEC_B
> -#
> -# Subtracts two vectors:
> -#
> -#     VEC_A = [a1, a2, ...]
> -#     VEC_B = [b1, b2, ...]
> -#     OUT = [(a1 - b1), (a2 - b2), ...]
> -#
> -# VEC_A and VEC_B must be lists of values separated by a character from
> $IFS.
> -vec_sub() {
> -    local a b i j
> -
> -    i=0
> -    for a in $1; do
> -        j=0
> -        for b in $2; do
> -            if test $i -eq $j; then
> -                echo $((a - b))
> -                break
> -            fi
> -            j=$((j + 1))
> -        done
> -        i=$((i + 1))
> -    done
> -}
> -
> -# vec_fold VEC OP
> -#
> -# Reduces a vector to a single value by applying the binary operator OP
> (i.e.,
> -# one that requires two arguments) cumulatively to all vector elements
> from left
> -# to right:
> -#
> -#     VEC = [e1, e2, e3 ...]
> -#     OUT = (...((e1 OP e2) OP e3) OP ...)
> -#
> -# VEC must be a list of values separated by a character from $IFS.
> -vec_fold() {
> -    local first op prod
> -
> -    first=1
> -    op=$2
> -    for a in $1; do
> -        if test $first -eq 1; then
> -            prod=$a
> -            first=0
> -        else
> -            prod=$((prod $op a))
> -        fi
> -    done
> -    echo $prod
> -}
> -
> -# read_counters SANDBOXES TARGET COUNTER
> -#
> -# Prints out the coverage COUNTER for the TARGET in each of the SANDBOXES.
> -#
> -# SANDBOXES must be a list of strings separated by a character from $IFS.
> -read_counters() {
> -    local sims="$1" target="$2" counter="$3"
> -
> -    for sim in $sims; do
> -        as $sim ovs-appctl -t "$target" coverage/read-counter "$counter"
> || return 1
> -    done
> -}
> -
> -# counter_delta_ SANDBOXES TARGET COUNTER COMMAND
> -#
> -# Runs the COMMAND and reports the COUNTER change registered during the
> command
> -# run for the given TARGET in each of the SANDBOXES.
> -counter_delta_() {
> -    local sims="$1" target="$2" counter="$3" cmd="$4"
> -    local before after
> -
> -    before=$(read_counters "$sims" "$target" "$counter") || return 1
> -    eval "$cmd" >/dev/null || return 1
> -    after=$(read_counters "$sims" "$target" "$counter") || return 1
> -
> -    vec_sub "$after" "$before"
> -}
> -
> -# counter_delta SANDBOXES TARGET COUNTER COMMAND
> -#
> -# Same as counter_delta_ but also prints the COUNTER values together with
> the
> -# COMMAND to standard error.
> -counter_delta() {
> -    local cmd="$4"
> -    local v
> -
> -    v=$(counter_delta_ "$@") || return 1
> -
> -    # Dump the counters and the command for troubleshooting
> -    echo "$v" | tr '\n' '\t' >&2
> -    echo "$cmd" >&2
> -
> -    echo "$v"
> -}
> -
> -# vec_cmp_counter_delta SANDBOXES TARGET COUNTER CONDS COMMAND
> -#
> -# Check if COUNTER change in the TARGET app in each of the SANDBOXES after
> -# running the COMMAND meets the conditions listed as operator-value pairs
> in
> -# CONDS.
> -vec_cmp_counter_delta() {
> -    local v
> -
> -    v=$(counter_delta "$1" "$2" "$3" "$5") || return 1
> -    v=$(vec_cmp "$v" "$4") || return 1
> -    v=$(vec_fold "$v" "&&") || return 1
> -
> -    echo "$v"
> -}
> -
> -# cmp_counter_delta SANDBOXES TARGET COUNTER COND COMMAND
> -#
> -# Check if COUNTER change in the TARGET app in each of the SANDBOXES after
> -# running the COMMAND meets the COND condition given as a operator-value
> pair.
> -cmp_counter_delta() {
> -    local conds=""
> -
> -    # Use the same condition for each sandbox
> -    for _ in $1; do
> -        conds="$conds $4"
> -    done
> -
> -    vec_cmp_counter_delta "$1" "$2" "$3" "$conds" "$5"
> -}
> -
> -m4_divert_pop([PREPARE_TESTS])
> -
> -# CHECK_COUNTER_DELTA_IS_ZERO SANDBOXES TARGET COUNTER COMMAND
> -#
> -# Runs the COMMAND and checks if the COUNTER value for the TARGET in all
> of
> -# the SANDBOXES did not change.
> -m4_define([CHECK_COUNTER_DELTA_IS_ZERO],[
> -    rv=$(cmp_counter_delta "$1" "$2" "$3" "=0" "$4")
> -    rc=$?
> -    AT_CHECK([test $rc -eq 0 -a $rv -eq 1])
> -])
> -
> -# CHECK_COUNTER_DELTA_IS_NOT_ZERO SANDBOXES TARGET COUNTER COMMAND
> -#
> -# Runs the COMMAND and checks if the COUNTER value for the TARGET in
> -# all of the SANDBOXES has changed.
> -m4_define([CHECK_COUNTER_DELTA_IS_NOT_ZERO],[
> -    rv=$(cmp_counter_delta "$1" "$2" "$3" ">0" "$4")
> -    rc=$?
> -    AT_CHECK([test $rc -eq 0 -a $rv -eq 1])
> -])
> -
> -# CHECK_COUNTER_DELTA_COND SANDBOXES TARGET COUNTER CONDS COMMAND
> -#
> -# Runs the COMMAND and checks if the COUNTER value for the TARGET in all
> of the
> -# SANDBOXES satisfies the conditions listed in CONDS.
> -m4_define([CHECK_COUNTER_DELTA_COND],[
> -    rv=$(vec_cmp_counter_delta "$1" "$2" "$3" "$4" "$5")
> -    rc=$?
> -    AT_CHECK([test $rc -eq 0 -a $rv -eq 1])
> -])
> -
> -# OVN_CONTROLLER_EXPECT_HIT SANDBOXES COUNTER COMMAND
> -#
> -# Checks if the COUNTER value has changed for any of the ovn-controller
> -# processes in the SANDBOXES when the COMMAND was run.
> -m4_define([OVN_CONTROLLER_EXPECT_HIT],[
> -    CHECK_COUNTER_DELTA_IS_NOT_ZERO([$1], [ovn-controller], [$2], [$3])
> -])
> -
> -# OVN_CONTROLLER_EXPECT_NO_HIT SANDBOXES COUNTER COMMAND
> -#
> -# Checks if the COUNTER value has not changed for any of the
> ovn-controller
> -# processes in the SANDBOXES when the COMMAND was run.
> -m4_define([OVN_CONTROLLER_EXPECT_NO_HIT],[
> -    CHECK_COUNTER_DELTA_IS_ZERO([$1], [ovn-controller], [$2], [$3])
> -])
> -
> -# OVN_CONTROLLER_EXPECT_HIT_COND SANDBOXES COUNTER CONDS COMMAND
> -#
> -# Checks if the change of the COUNTER value, when the COMMAND was run, of
> the
> -# ovn-controller process in each of the SANDBOXES meets the conditions in
> -# CONDS. CONDS must be a list of operator-value pairs, for example "[>0
> =0]",
> -# following the same order as SANDBOXES.
> -m4_define([OVN_CONTROLLER_EXPECT_HIT_COND],[
> -    CHECK_COUNTER_DELTA_COND([$1], [ovn-controller], [$2], [$3], [$4])
> -])
> -
> -AT_SETUP([ovn -- ovn-controller incremental processing])
> -# Check which operations the trigger full logical flow processing.
> -#
> -# Create and destroy logical routers, switches, ports, address sets and
> ACLs
> -# while counting calls to lflow_run() in ovn-controller.
> -
> -ovn_start
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -done
> -
> -# Add router lr1
> -OVN_CONTROLLER_EXPECT_HIT(
> -    [hv1 hv2], [lflow_run],
> -    [ovn-nbctl --wait=hv lr-add lr1]
> -)
> -
> -for i in 1 2; do
> -    ls=ls$i
> -    lsp=$ls-lr1
> -    lrp=lr1-$ls
> -
> -    # Add switch $ls
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv ls-add $ls]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv add Logical_Switch $ls other_config
> subnet=10.0.$i.0/24]
> -    )
> -
> -    # Add router port to $ls
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lrp-add lr1 $lrp 02:00:00:00:0$i:01
> 10.0.$i.1/24]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-add $ls $lsp]
> -    )
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-set-type $lsp router]
> -    )
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-set-options $lsp router-port=$lrp]
> -    )
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-set-addresses $lsp router]
> -    )
> -done
> -
> -get_lsp_uuid () {
> -    ovn-nbctl lsp-list ls${1#lp} | grep $1 | awk '{ print $1 }'
> -}
> -
> -pg_ports=
> -
> -for i in 1 2; do
> -    j=$((i%2 + 1))
> -    as=as$i
> -    ls=ls$i
> -    lp=lp$i
> -    vif=vif$i
> -
> -    # Add port $lp
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-add $ls $lp]
> -    )
> -
> -    pg_ports="$pg_port `get_lsp_uuid $lp`"
> -
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv lsp-set-addresses $lp "dynamic"]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl wait-until Logical_Switch_Port $lp
> dynamic_addresses!=[[]] &&
> -         ovn-nbctl --wait=hv sync]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl get Logical_Switch_Port $lp dynamic_addresses &&
> -         ovn-nbctl --wait=hv sync]
> -    )
> -
> -    # Add address set $as
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv create Address_Set name="$as"]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv add Address_Set "$as" addresses "10.0.$i.10"]
> -    )
> -
> -    # Add ACLs for port $lp
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv acl-add $ls to-lport 1001 'outport ==
> \"$lp\" && ip4.src == \$$as' allow]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv acl-add $ls to-lport 1000 'outport ==
> \"$lp\"' drop]
> -    )
> -
> -    # Bind port $lp and wait for it to come up
> -    OVN_CONTROLLER_EXPECT_HIT_COND(
> -        [hv$i hv$j], [lflow_run], [>0 =0],
> -        [as hv$i ovs-vsctl add-port br-int $vif -- set Interface $vif
> external-ids:iface-id=$lp &&
> -         ovn-nbctl wait-until Logical_Switch_Port $lp 'up=true' &&
> -         ovn-nbctl --wait=hv sync]
> -    )
> -done
> -
> -for i in 1 2; do
> -    j=$((i%2 + 1))
> -    as=as$i
> -    ls=ls$i
> -    lp=lp$i
> -
> -    # Delete ACLs for port $lp
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv acl-del $ls to-lport 1001 'outport ==
> \"$lp\" && ip4.src == \$$as']
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv acl-del $ls to-lport 1000 'outport ==
> \"$lp\"']
> -    )
> -
> -    # Delete address set $as
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv remove Address_Set "$as" addresses
> "10.0.$i.10"]
> -    )
> -    OVN_CONTROLLER_EXPECT_NO_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv destroy Address_Set "$as"]
> -    )
> -done
> -
> -OVN_CONTROLLER_EXPECT_NO_HIT(
> -    [hv1 hv2], [lflow_run],
> -    [ovn-nbctl --wait=hv create Port_Group name=pg1 ports=\"$pg_ports\"]
> -)
> -
> -# Add ACLs for port group pg1
> -OVN_CONTROLLER_EXPECT_NO_HIT(
> -    [hv1 hv2], [lflow_run],
> -    [ovn-nbctl --wait=hv acl-add pg1 to-lport 1001 'outport == @pg1 &&
> ip4.src == $pg1_ip4' allow]
> -)
> -
> -for i in 1 2; do
> -    j=$((i%2 + 1))
> -    lp=lp$i
> -
> -    # Delete port $lp
> -    OVN_CONTROLLER_EXPECT_HIT_COND(
> -        [hv$i hv$j], [lflow_run], [>0 =0],
> -        [ovn-nbctl --wait=hv lsp-del $lp]
> -    )
> -done
> -
> -# Delete port group pg1
> -OVN_CONTROLLER_EXPECT_NO_HIT(
> -    [hv1 hv2], [lflow_run],
> -    [ovn-nbctl --wait=hv destroy Port_Group pg1]
> -)
> -
> -for i in 1 2; do
> -    ls=ls$i
> -
> -    # Delete switch $ls
> -    OVN_CONTROLLER_EXPECT_HIT(
> -        [hv1 hv2], [lflow_run],
> -        [ovn-nbctl --wait=hv ls-del $ls]
> -    )
> -done
> -
> -# Delete router lr1
> -OVN_CONTROLLER_EXPECT_HIT(
> -    [hv1 hv2], [lflow_run],
> -    [ovn-nbctl --wait=hv lr-del lr1]
> -)
> -
> -OVN_CLEANUP([hv1], [hv2])
> -
> -AT_CLEANUP
> diff --git a/tests/ovn-sbctl.at b/tests/ovn-sbctl.at
> deleted file mode 100644
> index 650e357da..000000000
> --- a/tests/ovn-sbctl.at
> +++ /dev/null
> @@ -1,150 +0,0 @@
> -AT_BANNER([ovn-sbctl])
> -
> -# OVN_SBCTL_TEST_START
> -m4_define([OVN_SBCTL_TEST_START],
> -  [dnl Create databases (ovn-nb, ovn-sb).
> -   AT_KEYWORDS([ovn])
> -   for daemon in ovn-nb ovn-sb; do
> -      AT_CHECK([ovsdb-tool create $daemon.db
> $abs_top_srcdir/${daemon%%-*}/${daemon}.ovsschema])
> -   done
> -
> -   dnl Start ovsdb-servers.
> -   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile=ovnnb_db.pid
> --unixctl=$OVS_RUNDIR/ovnnb_db.ctl --log-file=ovsdb_nb.log
> --remote=punix:$OVS_RUNDIR/ovnnb_db.sock ovn-nb.db ], [0], [], [stderr])
> -   AT_CHECK([ovsdb-server --detach --no-chdir --pidfile=ovnsb_db.pid
> --unixctl=$OVS_RUNDIR/ovnsb_db.ctl --log-file=ovsdb_sb.log
> --remote=punix:$OVS_RUNDIR/ovnsb_db.sock ovn-sb.db], [0], [], [stderr])
> -   on_exit "kill `cat ovnnb_db.pid` `cat ovnsb_db.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d
> -/ovsdb_server|INFO|ovsdb-server (Open vSwitch)/d']])
> -   AT_CAPTURE_FILE([ovsdb-server.log])
> -
> -   dnl Start ovn-northd.
> -   AT_CHECK([ovn-northd --detach --no-chdir --pidfile --log-file
> --ovnnb-db=unix:$OVS_RUNDIR/ovnnb_db.sock
> --ovnsb-db=unix:$OVS_RUNDIR/ovnsb_db.sock], [0], [], [stderr])
> -   on_exit "kill `cat ovn-northd.pid`"
> -   AT_CHECK([[sed < stderr '
> -/vlog|INFO|opened log file/d']])
> -   AT_CAPTURE_FILE([ovn-northd.log])
> -])
> -
> -# OVN_SBCTL_TEST_STOP
> -m4_define([OVN_SBCTL_TEST_STOP],
> -  [AT_CHECK([check_logs "$1"])
> -   OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -   OVS_APP_EXIT_AND_WAIT_BY_TARGET([$OVS_RUNDIR/ovnnb_db.ctl],
> [$OVS_RUNDIR/ovnnb_db.pid])
> -   OVS_APP_EXIT_AND_WAIT_BY_TARGET([$OVS_RUNDIR/ovnsb_db.ctl],
> [$OVS_RUNDIR/ovnsb_db.pid])])
> -
> -dnl ---------------------------------------------------------------------
> -
> -AT_SETUP([ovn-sbctl - chassis commands])
> -OVN_SBCTL_TEST_START
> -ovn_init_db ovn-sb
> -
> -AT_CHECK([ovn-sbctl chassis-add ch0 geneve 1.2.3.4])
> -AT_CHECK([ovn-sbctl -f csv -d bare --no-headings --columns ip,type list
> encap | sort],
> -         [0], [dnl
> -1.2.3.4,geneve
> -])
> -
> -AT_CHECK([ovn-sbctl chassis-add ch1 stt,geneve,vxlan 1.2.3.5])
> -AT_CHECK([ovn-sbctl -f csv -d bare --no-headings --columns ip,type list
> encap | sort],
> -         [0], [dnl
> -1.2.3.4,geneve
> -1.2.3.5,geneve
> -1.2.3.5,stt
> -1.2.3.5,vxlan
> -])
> -
> -AT_CHECK([ovn-sbctl chassis-del ch0])
> -AT_CHECK([ovn-sbctl -f csv -d bare --no-headings --columns ip,type list
> encap | sort],
> -         [0], [dnl
> -1.2.3.5,geneve
> -1.2.3.5,stt
> -1.2.3.5,vxlan
> -])
> -
> -OVN_SBCTL_TEST_STOP
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -AT_CLEANUP
> -
> -dnl ---------------------------------------------------------------------
> -
> -AT_SETUP([ovn-sbctl])
> -OVN_SBCTL_TEST_START
> -
> -AT_CHECK([ovn-nbctl ls-add br-test])
> -AT_CHECK([ovn-nbctl lsp-add br-test vif0])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif0 f0:ab:cd:ef:01:02])
> -AT_CHECK([ovn-sbctl chassis-add ch0 stt 1.2.3.5])
> -AT_CHECK([ovn-nbctl --wait=sb sync])
> -AT_CHECK([ovn-sbctl lsp-bind vif0 ch0])
> -
> -AT_CHECK([ovn-sbctl show], [0], [dnl
> -Chassis ch0
> -    Encap stt
> -        ip: "1.2.3.5"
> -        options: {csum="true"}
> -    Port_Binding vif0
> -])
> -
> -# adds another 'vif1'
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add br-test vif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses vif1 f0:ab:cd:ef:01:03])
> -AT_CHECK([ovn-sbctl lsp-bind vif1 ch0])
> -
> -AT_CHECK([ovn-sbctl show | sed 's/vif[[0-9]]/vif/'], [0], [dnl
> -Chassis ch0
> -    Encap stt
> -        ip: "1.2.3.5"
> -        options: {csum="true"}
> -    Port_Binding vif
> -    Port_Binding vif
> -])
> -
> -# deletes 'vif1'
> -AT_CHECK([ovn-nbctl lsp-del vif1])
> -AT_CHECK([ovn-nbctl --wait=sb sync])
> -
> -AT_CHECK([ovn-sbctl show], [0], [dnl
> -Chassis ch0
> -    Encap stt
> -        ip: "1.2.3.5"
> -        options: {csum="true"}
> -    Port_Binding vif0
> -])
> -
> -uuid=$(ovn-sbctl --columns=_uuid list Chassis ch0 | cut -d ':' -f2 | tr
> -d ' ')
> -AT_CHECK_UNQUOTED([ovn-sbctl --columns=logical_port,mac,chassis list
> Port_Binding], [0], [dnl
> -logical_port        : vif0
> -mac                 : [["f0:ab:cd:ef:01:02"]]
> -chassis             : ${uuid}
> -])
> -
> -# test the passing down of logical port type and options.
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add br-test vtep0])
> -AT_CHECK([ovn-nbctl lsp-set-type vtep0 vtep])
> -AT_CHECK([ovn-nbctl lsp-set-options vtep0 vtep_physical_switch=p0
> vtep_logical_switch=l0])
> -
> -AT_CHECK([ovn-sbctl --timeout=10 wait-until Port_Binding vtep0
> options!={}])
> -AT_CHECK([ovn-sbctl --columns=logical_port,mac,type,options list
> Port_Binding vtep0], [0], [dnl
> -logical_port        : vtep0
> -mac                 : [[]]
> -type                : vtep
> -options             : {vtep_logical_switch=l0, vtep_physical_switch=p0}
> -])
> -
> -OVN_SBCTL_TEST_STOP
> -AT_CLEANUP
> -
> -dnl ---------------------------------------------------------------------
> -
> -AT_SETUP([ovn-sbctl - connection])
> -OVN_SBCTL_TEST_START
> -
> -AT_CHECK([ovn-sbctl --inactivity-probe=30000 set-connection
> ptcp:6641:127.0.0.1 punix:$OVS_RUNDIR/ovnsb_db.sock])
> -AT_CHECK([ovn-sbctl list connection | grep inactivity_probe], [0], [dnl
> -inactivity_probe    : 30000
> -inactivity_probe    : 30000
> -])
> -
> -OVN_SBCTL_TEST_STOP
> -AT_CLEANUP
> diff --git a/tests/ovn.at b/tests/ovn.at
> deleted file mode 100644
> index cb380d275..000000000
> --- a/tests/ovn.at
> +++ /dev/null
> @@ -1,14702 +0,0 @@
> -# OVN_CHECK_PACKETS([PCAP], [EXPECTED])
> -#
> -# This compares packets read from PCAP, in pcap format, to those read
> -# from EXPECTED, which is a text file containing packets as hex
> -# strings, one per line.  If PCAP contains fewer packets than
> -# EXPECTED, it waits up to 10 seconds for more packets to appear.
> -#
> -# The implementation is an m4 macro that is mostly implemented in
> -# terms of a shell function.  This reduces the size of the generated
> -# testsuite file since the shell function is only emitted once even
> -# when this macro is invoked many times.
> -m4_divert_text([PREPARE_TESTS],
> -  [ovn_check_packets__ () {
> -     echo
> -     echo "checking packets in $1 against $2:"
> -     rcv_pcap=$1
> -     rcv_text=`echo "$rcv_pcap.packets" | sed 's/\.pcap//'`
> -     exp_text=$2
> -     exp_n=`wc -l < "$exp_text"`
> -     OVS_WAIT_UNTIL(
> -       [$PYTHON "$top_srcdir/utilities/ovs-pcap.in" $rcv_pcap > $rcv_text
> -        rcv_n=`wc -l < "$rcv_text"`
> -        echo "rcv_n=$rcv_n exp_n=$exp_n"
> -        test $rcv_n -ge $exp_n])
> -     sort $exp_text > expout
> -   }
> -])
> -m4_define([OVN_CHECK_PACKETS],
> -  [ovn_check_packets__ "$1" "$2"
> -   AT_CHECK([sort $rcv_text], [0], [expout])])
> -
> -AT_BANNER([OVN components])
> -
> -AT_SETUP([ovn -- lexer])
> -dnl For lines without =>, input and expected output are identical.
> -dnl For lines with =>, input precedes => and expected output follows =>.
> -AT_DATA([test-cases.txt], [dnl
> -foo bar baz quuxquuxquux _abcd_ a.b.c.d a123_.456
> -"abc\u0020def" => "abc def"
> -" => error("Input ends inside quoted string.")dnl "
> -
> -$foo $bar $baz $quuxquuxquux $_abcd_ $a.b.c.d $a123_.456
> -$1 => error("`$' must be followed by a valid identifier.") 1
> -
> -a/*b*/c => a c
> -a//b c => a
> -a/**/b => a b
> -a/*/b => a error("`/*' without matching `*/'.")
> -a/*/**/b => a b
> -a/b => a error("`/' is only valid as part of `//' or `/*'.") b
> -
> -0 1 12345 18446744073709551615
> -18446744073709551616 => error("Decimal constants must be less than
> 2**64.")
> -9999999999999999999999 => error("Decimal constants must be less than
> 2**64.")
> -01 => error("Decimal constants must not have leading zeros.")
> -
> -0/0
> -0/1
> -1/0 => error("Value contains unmasked 1-bits.")
> -1/1
> -128/384
> -1/3
> -1/ => error("Integer constant expected.")
> -
> -1/0x123 => error("Value and mask have incompatible formats.")
> -
> -0x1234
> -0x01234 => 0x1234
> -0x0 => 0
> -0x000 => 0
> -0xfedcba9876543210
> -0XFEDCBA9876543210 => 0xfedcba9876543210
> -0xfedcba9876543210fedcba9876543210
> -0x0000fedcba9876543210fedcba9876543210 =>
> 0xfedcba9876543210fedcba9876543210
> -0x => error("Hex digits expected following 0x.")
> -0X => error("Hex digits expected following 0X.")
> -0x0/0x0 => 0/0
> -0x0/0x1 => 0/0x1
> -0x1/0x0 => error("Value contains unmasked 1-bits.")
> -0xffff/0x1ffff
> -0x. => error("Invalid syntax in hexadecimal constant.")
> -
> -192.168.128.1 1.2.3.4 255.255.255.255 0.0.0.0
> -256.1.2.3 => error("Invalid numeric constant.")
> -192.168.0.0/16
> -192.168.0.0/255.255.0.0 => 192.168.0.0/16
> -192.168.0.0/255.255.255.0 => 192.168.0.0/24
> -192.168.0.0/255.255.0.255
> -192.168.0.0/255.0.0.0 => error("Value contains unmasked 1-bits.")
> -192.168.0.0/32
> -192.168.0.0/255.255.255.255 => 192.168.0.0/32
> -1.2.3.4:5 => 1.2.3.4 : 5
> -
> -::
> -::1
> -ff00::1234 => ff00::1234
> -2001:db8:85a3::8a2e:370:7334
> -2001:db8:85a3:0:0:8a2e:370:7334 => 2001:db8:85a3::8a2e:370:7334
> -2001:0db8:85a3:0000:0000:8a2e:0370:7334 => 2001:db8:85a3::8a2e:370:7334
> -::ffff:192.0.2.128
> -::ffff:c000:0280 => ::ffff:192.0.2.128
> -::1/::1
> -::1/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff => ::1/128
> -::1/128
> -ff00::/8
> -ff00::/ff00:: => ff00::/8
> -
> -01:23:45:67:ab:cd
> -01:23:45:67:AB:CD => 01:23:45:67:ab:cd
> -fe:dc:ba:98:76:54
> -FE:DC:ba:98:76:54 => fe:dc:ba:98:76:54
> -01:00:00:00:00:00/01:00:00:00:00:00
> -ff:ff:ff:ff:ff:ff/ff:ff:ff:ff:ff:ff
> -fe:ff:ff:ff:ff:ff/ff:ff:ff:ff:ff:ff
> -ff:ff:ff:ff:ff:ff/fe:ff:ff:ff:ff:ff => error("Value contains unmasked
> 1-bits.")
> -fe:x => error("Invalid numeric constant.")
> -00:01:02:03:04:x => error("Invalid numeric constant.")
> -
> -# Test that operators are tokenized as expected, even without white space.
> -(){}[[]]==!=<<=>>=!&&||..,;=<->--: => ( ) { } [[ ]] == != < <= > >= ! &&
> || .. , ; = <-> -- :
> -& => error("`&' is only valid as part of `&&'.")
> -| => error("`|' is only valid as part of `||'.")
> -- => error("`-' is only valid as part of `--'.")
> -
> -^ => error("Invalid character `^' in input.")
> -])
> -AT_CAPTURE_FILE([input.txt])
> -sed 's/ =>.*//' test-cases.txt > input.txt
> -sed 's/.* => //' test-cases.txt > expout
> -AT_CHECK([ovstest test-ovn lex < input.txt], [0], [expout])
> -AT_CLEANUP
> -
> -dnl The OVN expression parser needs to know what fields overlap with one
> -dnl another.  This test therefore verifies that all the smaller registers
> -dnl are defined as terms of subfields of the larger ones.
> -dnl
> -dnl When we add or remove registers this test needs to be updated, of
> course.
> -AT_SETUP([ovn -- registers])
> -AT_CHECK([ovstest test-ovn dump-symtab | grep reg | sort], [0],
> -[[reg0 = xxreg0[96..127]
> -reg1 = xxreg0[64..95]
> -reg2 = xxreg0[32..63]
> -reg3 = xxreg0[0..31]
> -reg4 = xxreg1[96..127]
> -reg5 = xxreg1[64..95]
> -reg6 = xxreg1[32..63]
> -reg7 = xxreg1[0..31]
> -reg8 = xreg4[32..63]
> -reg9 = xreg4[0..31]
> -xreg0 = xxreg0[64..127]
> -xreg1 = xxreg0[0..63]
> -xreg2 = xxreg1[64..127]
> -xreg3 = xxreg1[0..63]
> -xreg4 = OXM_OF_PKT_REG4
> -xxreg0 = NXM_NX_XXREG0
> -xxreg1 = NXM_NX_XXREG1
> -]])
> -AT_CLEANUP
> -
> -dnl Check that the OVN conntrack field definitions are correct.
> -AT_SETUP([ovn -- conntrack fields])
> -AT_CHECK([ovstest test-ovn dump-symtab | grep ^ct | sort], [0],
> -[[ct.dnat = ct_state[7]
> -ct.est = ct_state[1]
> -ct.inv = ct_state[4]
> -ct.new = ct_state[0]
> -ct.rel = ct_state[2]
> -ct.rpl = ct_state[3]
> -ct.snat = ct_state[6]
> -ct.trk = ct_state[5]
> -ct_label = NXM_NX_CT_LABEL
> -ct_label.blocked = ct_label[0]
> -ct_mark = NXM_NX_CT_MARK
> -ct_state = NXM_NX_CT_STATE
> -]])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- composition])
> -AT_CHECK([ovstest test-ovn composition 2], [0], [ignore])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- expression parser])
> -dnl For lines without =>, input and expected output are identical.
> -dnl For lines with =>, input precedes => and expected output follows =>.
> -AT_DATA([test-cases.txt], [[
> -eth.type == 0x800
> -eth.type==0x800 => eth.type == 0x800
> -eth.type[0..15] == 0x800 => eth.type == 0x800
> -
> -vlan.present
> -vlan.present == 1 => vlan.present
> -!(vlan.present == 0) => vlan.present
> -!(vlan.present != 1) => vlan.present
> -!vlan.present
> -vlan.present == 0 => !vlan.present
> -vlan.present != 1 => !vlan.present
> -!(vlan.present == 1) => !vlan.present
> -!(vlan.present != 0) => !vlan.present
> -
> -eth.dst[0]
> -eth.dst[0] == 1 => eth.dst[0]
> -eth.dst[0] != 0 => eth.dst[0]
> -!(eth.dst[0] == 0) => eth.dst[0]
> -!(eth.dst[0] != 1) => eth.dst[0]
> -
> -!eth.dst[0]
> -eth.dst[0] == 0 => !eth.dst[0]
> -eth.dst[0] != 1 => !eth.dst[0]
> -!(eth.dst[0] == 1) => !eth.dst[0]
> -!(eth.dst[0] != 0) => !eth.dst[0]
> -
> -vlan.tci[12..15] == 0x3
> -vlan.tci == 0x3000/0xf000 => vlan.tci[12..15] == 0x3
> -vlan.tci[12..15] != 0x3
> -vlan.tci != 0x3000/0xf000 => vlan.tci[12..15] != 0x3
> -
> -!vlan.pcp => vlan.pcp == 0
> -!(vlan.pcp) => vlan.pcp == 0
> -vlan.pcp == 0x4
> -vlan.pcp != 0x4
> -vlan.pcp > 0x4
> -vlan.pcp >= 0x4
> -vlan.pcp < 0x4
> -vlan.pcp <= 0x4
> -!(vlan.pcp != 0x4) => vlan.pcp == 0x4
> -!(vlan.pcp == 0x4) => vlan.pcp != 0x4
> -!(vlan.pcp <= 0x4) => vlan.pcp > 0x4
> -!(vlan.pcp < 0x4) => vlan.pcp >= 0x4
> -!(vlan.pcp >= 0x4) => vlan.pcp < 0x4
> -!(vlan.pcp > 0x4) => vlan.pcp <= 0x4
> -0x4 == vlan.pcp => vlan.pcp == 0x4
> -0x4 != vlan.pcp => vlan.pcp != 0x4
> -0x4 < vlan.pcp => vlan.pcp > 0x4
> -0x4 <= vlan.pcp => vlan.pcp >= 0x4
> -0x4 > vlan.pcp => vlan.pcp < 0x4
> -0x4 >= vlan.pcp => vlan.pcp <= 0x4
> -!(0x4 != vlan.pcp) => vlan.pcp == 0x4
> -!(0x4 == vlan.pcp) => vlan.pcp != 0x4
> -!(0x4 >= vlan.pcp) => vlan.pcp > 0x4
> -!(0x4 > vlan.pcp) => vlan.pcp >= 0x4
> -!(0x4 <= vlan.pcp) => vlan.pcp < 0x4
> -!(0x4 < vlan.pcp) => vlan.pcp <= 0x4
> -
> -1 < vlan.pcp < 4 => vlan.pcp > 0x1 && vlan.pcp < 0x4
> -1 <= vlan.pcp <= 4 => vlan.pcp >= 0x1 && vlan.pcp <= 0x4
> -1 < vlan.pcp <= 4 => vlan.pcp > 0x1 && vlan.pcp <= 0x4
> -1 <= vlan.pcp < 4 => vlan.pcp >= 0x1 && vlan.pcp < 0x4
> -1 <= vlan.pcp <= 4 => vlan.pcp >= 0x1 && vlan.pcp <= 0x4
> -4 > vlan.pcp > 1 => vlan.pcp < 0x4 && vlan.pcp > 0x1
> -4 >= vlan.pcp > 1 => vlan.pcp <= 0x4 && vlan.pcp > 0x1
> -4 > vlan.pcp >= 1 => vlan.pcp < 0x4 && vlan.pcp >= 0x1
> -4 >= vlan.pcp >= 1 => vlan.pcp <= 0x4 && vlan.pcp >= 0x1
> -!(1 < vlan.pcp < 4) => vlan.pcp <= 0x1 || vlan.pcp >= 0x4
> -!(1 <= vlan.pcp <= 4) => vlan.pcp < 0x1 || vlan.pcp > 0x4
> -!(1 < vlan.pcp <= 4) => vlan.pcp <= 0x1 || vlan.pcp > 0x4
> -!(1 <= vlan.pcp < 4) => vlan.pcp < 0x1 || vlan.pcp >= 0x4
> -!(1 <= vlan.pcp <= 4) => vlan.pcp < 0x1 || vlan.pcp > 0x4
> -!(4 > vlan.pcp > 1) => vlan.pcp >= 0x4 || vlan.pcp <= 0x1
> -!(4 >= vlan.pcp > 1) => vlan.pcp > 0x4 || vlan.pcp <= 0x1
> -!(4 > vlan.pcp >= 1) => vlan.pcp >= 0x4 || vlan.pcp < 0x1
> -!(4 >= vlan.pcp >= 1) => vlan.pcp > 0x4 || vlan.pcp < 0x1
> -
> -vlan.pcp == {1, 2, 3, 4} => vlan.pcp == 0x1 || vlan.pcp == 0x2 ||
> vlan.pcp == 0x3 || vlan.pcp == 0x4
> -vlan.pcp == 1 || ((vlan.pcp == 2 || vlan.pcp == 3) || vlan.pcp == 4) =>
> vlan.pcp == 0x1 || vlan.pcp == 0x2 || vlan.pcp == 0x3 || vlan.pcp == 0x4
> -
> -vlan.pcp != {1, 2, 3, 4} => vlan.pcp != 0x1 && vlan.pcp != 0x2 &&
> vlan.pcp != 0x3 && vlan.pcp != 0x4
> -vlan.pcp == 1 && ((vlan.pcp == 2 && vlan.pcp == 3) && vlan.pcp == 4) =>
> vlan.pcp == 0x1 && vlan.pcp == 0x2 && vlan.pcp == 0x3 && vlan.pcp == 0x4
> -
> -vlan.pcp == 1 && !((vlan.pcp == 2 && vlan.pcp == 3) && vlan.pcp == 4) =>
> vlan.pcp == 0x1 && (vlan.pcp != 0x2 || vlan.pcp != 0x3 || vlan.pcp != 0x4)
> -vlan.pcp == 1 && (!(vlan.pcp == 2 && vlan.pcp == 3) && vlan.pcp == 4) =>
> vlan.pcp == 0x1 && (vlan.pcp != 0x2 || vlan.pcp != 0x3) && vlan.pcp == 0x4
> -vlan.pcp == 1 && !(!(vlan.pcp == 2 && vlan.pcp == 3) && vlan.pcp == 4) =>
> vlan.pcp == 0x1 && ((vlan.pcp == 0x2 && vlan.pcp == 0x3) || vlan.pcp != 0x4)
> -
> -ip4.src == {10.0.0.0/8, 192.168.0.0/16, 172.16.20.0/24, 8.8.8.8} =>
> ip4.src[24..31] == 0xa || ip4.src[16..31] == 0xc0a8 || ip4.src[8..31] ==
> 0xac1014 || ip4.src == 0x8080808
> -ip6.src == ::1 => ip6.src == 0x1
> -
> -ip4.src == 1.2.3.4 => ip4.src == 0x1020304
> -ip4.src == ::1.2.3.4/::ffff:ffff => ip4.src == 0x1020304
> -ip6.src == ::1 => ip6.src == 0x1
> -
> -1
> -0
> -!1 => 0
> -!0 => 1
> -
> -inport == "eth0"
> -!(inport != "eth0") => inport == "eth0"
> -
> -(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((0)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
> => 0
> -
> -ip4.src == "eth0" => Integer field ip4.src is not compatible with string
> constant.
> -inport == 1 => String field inport is not compatible with integer
> constant.
> -ip4.src = 1.2.3.4 => Syntax error at `=' expecting relational operator.
> -
> -ip4.src > {1, 2, 3} => Only == and != operators may be used with value
> sets.
> -eth.type > 0x800 => Only == and != operators may be used with nominal
> field eth.type.
> -vlan.present > 0 => Only == and != operators may be used with Boolean
> field vlan.present.
> -
> -inport != "eth0" => Nominal field inport may only be tested for equality
> (taking enclosing `!' operators into account).
> -!(inport == "eth0") => Nominal field inport may only be tested for
> equality (taking enclosing `!' operators into account).
> -eth.type != 0x800 => Nominal field eth.type may only be tested for
> equality (taking enclosing `!' operators into account).
> -!(eth.type == 0x800) => Nominal field eth.type may only be tested for
> equality (taking enclosing `!' operators into account).
> -inport = "eth0" => Syntax error at `=' expecting relational operator.
> -
> -123 == 123 => Syntax error at `123' expecting field name.
> -
> -$name => Syntax error at `$name' expecting address set name.
> - at name => Syntax error at `@name' expecting port group name.
> -
> -123 == xyzzy => Syntax error at `xyzzy' expecting field name.
> -xyzzy == 1 => Syntax error at `xyzzy' expecting field name.
> -
> -inport[1] == 1 => Cannot select subfield of string field inport.
> -
> -eth.type[] == 1 => Syntax error at `@:>@' expecting small integer.
> -eth.type[::1] == 1 => Syntax error at `::1' expecting small integer.
> -eth.type[18446744073709551615] == 1 => Syntax error at
> `18446744073709551615' expecting small integer.
> -
> -eth.type[5!] => Syntax error at `!' expecting `@:>@'.
> -
> -eth.type[5..1] => Invalid bit range 5 to 1.
> -
> -eth.type[12..16] => Cannot select bits 12 to 16 of 16-bit field eth.type.
> -
> -eth.type[10] == 1 => Cannot select subfield of nominal field eth.type.
> -
> -eth.type => Explicit `!= 0' is required for inequality test of multibit
> field against 0.
> -
> -!(!(vlan.pcp)) => Explicit `!= 0' is required for inequality test of
> multibit field against 0.
> -
> -123 => Syntax error at end of input expecting relational operator.
> -
> -123 x => Syntax error at `x' expecting relational operator.
> -
> -{1, "eth0"} => Syntax error at `"eth0"' expecting integer.
> -
> -eth.type == xyzzy => Syntax error at `xyzzy' expecting constant.
> -
> -(1 x) => Syntax error at `x' expecting `)'.
> -
> -!0x800 != eth.type => Missing parentheses around operand of !.
> -
> -eth.type == 0x800 || eth.type == 0x86dd && ip.proto == 17 => && and ||
> must be parenthesized when used together.
> -
> -eth.dst == {} => Syntax error at `}' expecting constant.
> -
> -eth.src > 00:00:00:00:11:11/00:00:00:00:ff:ff => Only == and != operators
> may be used with masked constants.  Consider using subfields instead (e.g.
> eth.src[0..15] > 0x1111 in place of eth.src >
> 00:00:00:00:11:11/00:00:00:00:ff:ff).
> -
> -ip4.src == ::1 => 128-bit constant is not compatible with 32-bit field
> ip4.src.
> -
> -1 == eth.type == 2 => Range expressions must have the form `x < field <
> y' or `x > field > y', with each `<' optionally replaced by `<=' or `>' by
> `>=').
> -
> -eth.dst[40] x => Syntax error at `x' expecting end of input.
> -
> -ip4.src == {1.2.3.4, $set1, $unknownset} => Syntax error at `$unknownset'
> expecting address set name.
> -eth.src == {$set3, badmac, 00:00:00:00:00:01} => Syntax error at `badmac'
> expecting constant.
> -
> -((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
> => Parentheses nested too deeply.
> -
> -ct_label > $set4 => Only == and != operators may be used to compare a
> field against an empty value set.
> -]])
> -sed 's/ =>.*//' test-cases.txt > input.txt
> -sed 's/.* => //' test-cases.txt > expout
> -AT_CHECK([ovstest test-ovn parse-expr < input.txt], [0], [expout])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- expression annotation])
> -dnl Input precedes =>, expected output follows =>.
> -dnl Empty lines and lines starting with # are ignored.
> -AT_DATA([test-cases.txt], [[
> -ip4.src == 1.2.3.4 => ip4.src == 0x1020304 && eth.type == 0x800
> -ip4.src != 1.2.3.4 => ip4.src != 0x1020304 && eth.type == 0x800
> -ip.proto == 123 => ip.proto == 0x7b && (eth.type == 0x800 || eth.type ==
> 0x86dd)
> -ip.proto == {123, 234} => (ip.proto == 0x7b || ip.proto == 0xea) &&
> (eth.type == 0x800 || eth.type == 0x86dd)
> -ip4.src == 1.2.3.4 && ip4.dst == 5.6.7.8 => ip4.src == 0x1020304 &&
> eth.type == 0x800 && ip4.dst == 0x5060708 && eth.type == 0x800
> -
> -# Nested expressions over a single symbol should be annotated with
> symbol's
> -# prerequisites only once, at the top level.
> -tcp.dst == 1 || (tcp.dst >= 2 && tcp.dst <= 3) => (tcp.dst == 0x1 ||
> (tcp.dst >= 0x2 && tcp.dst <= 0x3)) && ip.proto == 0x6 && (eth.type ==
> 0x800 || eth.type == 0x86dd)
> -
> -ip => eth.type == 0x800 || eth.type == 0x86dd
> -ip == 1 => eth.type == 0x800 || eth.type == 0x86dd
> -ip[0] == 1 => eth.type == 0x800 || eth.type == 0x86dd
> -ip > 0 => Only == and != operators may be used with nominal field ip.
> -!ip => Nominal predicate ip may only be tested positively, e.g. `ip' or
> `ip == 1' but not `!ip' or `ip == 0'.
> -ip == 0 => Nominal predicate ip may only be tested positively, e.g. `ip'
> or `ip == 1' but not `!ip' or `ip == 0'.
> -
> -vlan.present => vlan.tci[12]
> -!vlan.present => !vlan.tci[12]
> -
> -!vlan.pcp => vlan.tci[13..15] == 0 && vlan.tci[12]
> -vlan.pcp == 1 && vlan.vid == 2 => vlan.tci[13..15] == 0x1 && vlan.tci[12]
> && vlan.tci[0..11] == 0x2 && vlan.tci[12]
> -!reg0 && !reg1 && !reg2 && !reg3 => xxreg0[96..127] == 0 &&
> xxreg0[64..95] == 0 && xxreg0[32..63] == 0 && xxreg0[0..31] == 0
> -
> -ip.first_frag => ip.frag[0] && (eth.type == 0x800 || eth.type == 0x86dd)
> && (!ip.frag[1] || (eth.type != 0x800 && eth.type != 0x86dd))
> -!ip.first_frag => !ip.frag[0] || (eth.type != 0x800 && eth.type !=
> 0x86dd) || (ip.frag[1] && (eth.type == 0x800 || eth.type == 0x86dd))
> -ip.later_frag => ip.frag[1] && (eth.type == 0x800 || eth.type == 0x86dd)
> -
> -bad_prereq != 0 => Error parsing expression `xyzzy' encountered as
> prerequisite or predicate of initial expression: Syntax error at `xyzzy'
> expecting field name.
> -self_recurse != 0 => Error parsing expression `self_recurse != 0'
> encountered as prerequisite or predicate of initial expression: Recursive
> expansion of symbol `self_recurse'.
> -mutual_recurse_1 != 0 => Error parsing expression `mutual_recurse_2 != 0'
> encountered as prerequisite or predicate of initial expression: Error
> parsing expression `mutual_recurse_1 != 0' encountered as prerequisite or
> predicate of initial expression: Recursive expansion of symbol
> `mutual_recurse_1'.
> -mutual_recurse_2 != 0 => Error parsing expression `mutual_recurse_1 != 0'
> encountered as prerequisite or predicate of initial expression: Error
> parsing expression `mutual_recurse_2 != 0' encountered as prerequisite or
> predicate of initial expression: Recursive expansion of symbol
> `mutual_recurse_2'.
> -]])
> -sed 's/ =>.*//' test-cases.txt > input.txt
> -sed 's/.* => //' test-cases.txt > expout
> -AT_CHECK([ovstest test-ovn annotate-expr < input.txt], [0], [expout])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 1-term expression conversion])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=convert 1], [0],
> -  [Tested converting all 1-terminal expressions with 2 numeric vars (each
> 3 bits) in terms of operators == != < <= > >= and 2 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2-term expression conversion])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=convert 2], [0],
> -  [Tested converting 578 expressions of 2 terminals with 2 numeric vars
> (each 3 bits) in terms of operators == != < <= > >= and 2 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 3-term expression conversion])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=convert --bits=2 3],
> [0],
> -  [Tested converting 67410 expressions of 3 terminals with 2 numeric vars
> (each 2 bits) in terms of operators == != < <= > >= and 2 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 3-term numeric expression simplification])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=simplify --nvars=2
> --svars=0 3], [0],
> -  [Tested simplifying 490770 expressions of 3 terminals with 2 numeric
> vars (each 3 bits) in terms of operators == != < <= > >=.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term string expression simplification])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=simplify --nvars=0
> --svars=4 4], [0],
> -  [Tested simplifying 21978 expressions of 4 terminals with 4 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 3-term mixed expression simplification])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=simplify --nvars=1
> --svars=1 3], [0],
> -  [Tested simplifying 127890 expressions of 3 terminals with 1 numeric
> vars (each 3 bits) in terms of operators == != < <= > >= and 1 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- simplification special cases])
> -simplify() {
> -    echo "$1" | ovstest test-ovn simplify-expr
> -}
> -AT_CHECK([simplify 'eth.dst == 0/0'], [0], [1
> -])
> -AT_CHECK([simplify 'eth.dst != 0/0'], [0], [0
> -])
> -AT_CHECK([simplify 'tcp.dst >= 0'], [0],
> -    [ip.proto == 0x6 && (eth.type == 0x800 || eth.type == 0x86dd)
> -])
> -AT_CHECK([simplify 'tcp.dst <= 65535'], [0],
> -    [ip.proto == 0x6 && (eth.type == 0x800 || eth.type == 0x86dd)
> -])
> -AT_CHECK([simplify 'tcp.dst > 0'], [0],
> -    [[(tcp.dst[0] || tcp.dst[1] || tcp.dst[2] || tcp.dst[3] || tcp.dst[4]
> || tcp.dst[5] || tcp.dst[6] || tcp.dst[7] || tcp.dst[8] || tcp.dst[9] ||
> tcp.dst[10] || tcp.dst[11] || tcp.dst[12] || tcp.dst[13] || tcp.dst[14] ||
> tcp.dst[15]) && ip.proto == 0x6 && (eth.type == 0x800 || eth.type == 0x86dd)
> -]])
> -AT_CHECK([simplify 'tcp.dst < 65535'], [0],
> -    [[(!tcp.dst[0] || !tcp.dst[1] || !tcp.dst[2] || !tcp.dst[3] ||
> !tcp.dst[4] || !tcp.dst[5] || !tcp.dst[6] || !tcp.dst[7] || !tcp.dst[8] ||
> !tcp.dst[9] || !tcp.dst[10] || !tcp.dst[11] || !tcp.dst[12] || !tcp.dst[13]
> || !tcp.dst[14] || !tcp.dst[15]) && ip.proto == 0x6 && (eth.type == 0x800
> || eth.type == 0x86dd)
> -]])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- is_chassis_resident simplification])
> -simplify() {
> -    echo "$1" | ovstest test-ovn simplify-expr
> -}
> -AT_CHECK([simplify 'is_chassis_resident("eth1")'], [0], [1
> -])
> -AT_CHECK([simplify 'is_chassis_resident("eth2")'], [0], [0
> -])
> -AT_CHECK([simplify '!is_chassis_resident("eth1")'], [0], [0
> -])
> -AT_CHECK([simplify '!is_chassis_resident("eth2")'], [0], [1
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term numeric expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=3
> --svars=0 --bits=1 4], [0],
> -  [Tested normalizing 1874026 expressions of 4 terminals with 3 numeric
> vars (each 1 bits) in terms of operators == != < <= > >=.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term string expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=0
> --svars=3 --bits=1 4], [0],
> -  [Tested normalizing 11242 expressions of 4 terminals with 3 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term mixed expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=1
> --bits=1 --svars=2 4], [0],
> -  [Tested normalizing 175978 expressions of 4 terminals with 1 numeric
> vars (each 1 bits) in terms of operators == != < <= > >= and 2 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 5-term numeric expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=3
> --svars=0 --bits=1 --relops='==' 5], [0],
> -  [Tested normalizing 1317600 expressions of 5 terminals with 3 numeric
> vars (each 1 bits) in terms of operators ==.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 5-term string expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=0
> --svars=3 --bits=1 --relops='==' 5], [0],
> -  [Tested normalizing 368550 expressions of 5 terminals with 3 string
> vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 5-term mixed expression normalization])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=normalize --nvars=1
> --svars=1 --bits=1 --relops='==' 5], [0],
> -  [Tested normalizing 216000 expressions of 5 terminals with 1 numeric
> vars (each 1 bits) in terms of operators == and 1 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term numeric expressions to flows])
> -AT_KEYWORDS([expression])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=flow --nvars=2
> --svars=0 --bits=2 --relops='==' 4], [0],
> -  [Tested converting to flows 175978 expressions of 4 terminals with 2
> numeric vars (each 2 bits) in terms of operators ==.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term string expressions to flows])
> -AT_KEYWORDS([expression])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=flow --nvars=0
> --svars=4 4], [0],
> -  [Tested converting to flows 21978 expressions of 4 terminals with 4
> string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4-term mixed expressions to flows])
> -AT_KEYWORDS([expression])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=flow --nvars=1 --bits=2
> --svars=1 --relops='==' 4], [0],
> -  [Tested converting to flows 48312 expressions of 4 terminals with 1
> numeric vars (each 2 bits) in terms of operators == and 1 string vars.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 3-term numeric expressions to flows])
> -AT_KEYWORDS([expression])
> -AT_CHECK([ovstest test-ovn exhaustive --operation=flow --nvars=3
> --svars=0 --bits=3 --relops='==' 3], [0],
> -  [Tested converting to flows 41328 expressions of 3 terminals with 3
> numeric vars (each 3 bits) in terms of operators ==.
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- converting expressions to flows -- string fields])
> -AT_KEYWORDS([expression])
> -expr_to_flow () {
> -    echo "$1" | ovstest test-ovn expr-to-flows | sort
> -}
> -AT_CHECK([expr_to_flow 'inport == "eth0"'], [0], [reg14=0x5
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth1"'], [0], [reg14=0x6
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth2"'], [0], [(no flows)
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth0" && ip'], [0], [dnl
> -ip,reg14=0x5
> -ipv6,reg14=0x5
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth1" && ip'], [0], [dnl
> -ip,reg14=0x6
> -ipv6,reg14=0x6
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth2" && ip'], [0], [(no flows)
> -])
> -AT_CHECK([expr_to_flow 'inport == {"eth0", "eth1", "eth2", "LOCAL"}'],
> [0],
> -[reg14=0x5
> -reg14=0x6
> -reg14=0xfffe
> -])
> -AT_CHECK([expr_to_flow 'inport == {"eth0", "eth1", "eth2"} && ip'], [0],
> [dnl
> -ip,reg14=0x5
> -ip,reg14=0x6
> -ipv6,reg14=0x5
> -ipv6,reg14=0x6
> -])
> -AT_CHECK([expr_to_flow 'inport == "eth0" && inport == "eth1"'], [0], [dnl
> -(no flows)
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- converting expressions to flows -- address sets])
> -AT_KEYWORDS([expression])
> -expr_to_flow () {
> -    echo "$1" | ovstest test-ovn expr-to-flows | sort
> -}
> -AT_CHECK([expr_to_flow 'ip4.src == {10.0.0.1, 10.0.0.2, 10.0.0.3}'], [0],
> [dnl
> -ip,nw_src=10.0.0.1
> -ip,nw_src=10.0.0.2
> -ip,nw_src=10.0.0.3
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == $set1'], [0], [dnl
> -ip,nw_src=10.0.0.1
> -ip,nw_src=10.0.0.2
> -ip,nw_src=10.0.0.3
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == {1.2.3.4, $set1}'], [0], [dnl
> -ip,nw_src=1.2.3.4
> -ip,nw_src=10.0.0.1
> -ip,nw_src=10.0.0.2
> -ip,nw_src=10.0.0.3
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == {1.2.0.0/20, 5.5.5.0/24, $set1}'],
> [0], [dnl
> -ip,nw_src=1.2.0.0/20
> -ip,nw_src=10.0.0.1
> -ip,nw_src=10.0.0.2
> -ip,nw_src=10.0.0.3
> -ip,nw_src=5.5.5.0/24
> -])
> -AT_CHECK([expr_to_flow 'ip6.src == {::1, ::2, ::3}'], [0], [dnl
> -ipv6,ipv6_src=::1
> -ipv6,ipv6_src=::2
> -ipv6,ipv6_src=::3
> -])
> -AT_CHECK([expr_to_flow 'ip6.src == {::1, $set2, ::4}'], [0], [dnl
> -ipv6,ipv6_src=::1
> -ipv6,ipv6_src=::2
> -ipv6,ipv6_src=::3
> -ipv6,ipv6_src=::4
> -])
> -AT_CHECK([expr_to_flow 'eth.src == {00:00:00:00:00:01, 00:00:00:00:00:02,
> 00:00:00:00:00:03}'], [0], [dnl
> -dl_src=00:00:00:00:00:01
> -dl_src=00:00:00:00:00:02
> -dl_src=00:00:00:00:00:03
> -])
> -AT_CHECK([expr_to_flow 'eth.src == {$set3}'], [0], [dnl
> -dl_src=00:00:00:00:00:01
> -dl_src=00:00:00:00:00:02
> -dl_src=00:00:00:00:00:03
> -])
> -AT_CHECK([expr_to_flow 'eth.src == {00:00:00:00:00:01, $set3,
> ba:be:be:ef:de:ad, $set3}'], [0], [dnl
> -dl_src=00:00:00:00:00:01
> -dl_src=00:00:00:00:00:02
> -dl_src=00:00:00:00:00:03
> -dl_src=ba:be:be:ef:de:ad
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == {$set4}'], [0], [dnl
> -(no flows)
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == {1.2.3.4, $set4}'], [0], [dnl
> -ip,nw_src=1.2.3.4
> -])
> -AT_CHECK([expr_to_flow 'ip4.src == 1.2.3.4 || ip4.src == {$set4}'], [0],
> [dnl
> -ip,nw_src=1.2.3.4
> -])
> -AT_CHECK([expr_to_flow 'ip4.src != {$set4}'], [0], [dnl
> -
> -])
> -AT_CHECK([expr_to_flow 'ip4.src != {1.0.0.0/8, $set4}'], [0], [dnl
> -ip,nw_src=0.0.0.0/1.0.0.0
> -ip,nw_src=128.0.0.0/1
> -ip,nw_src=16.0.0.0/16.0.0.0
> -ip,nw_src=2.0.0.0/2.0.0.0
> -ip,nw_src=32.0.0.0/32.0.0.0
> -ip,nw_src=4.0.0.0/4.0.0.0
> -ip,nw_src=64.0.0.0/64.0.0.0
> -ip,nw_src=8.0.0.0/8.0.0.0
> -])
> -AT_CHECK([expr_to_flow 'ip4.src != 1.0.0.0/8 && ip4.src != {$set4}'],
> [0], [dnl
> -ip,nw_src=0.0.0.0/1.0.0.0
> -ip,nw_src=128.0.0.0/1
> -ip,nw_src=16.0.0.0/16.0.0.0
> -ip,nw_src=2.0.0.0/2.0.0.0
> -ip,nw_src=32.0.0.0/32.0.0.0
> -ip,nw_src=4.0.0.0/4.0.0.0
> -ip,nw_src=64.0.0.0/64.0.0.0
> -ip,nw_src=8.0.0.0/8.0.0.0
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- converting expressions to flows -- port groups])
> -AT_KEYWORDS([expression])
> -expr_to_flow () {
> -    echo "$1" | ovstest test-ovn expr-to-flows | sort
> -}
> -AT_CHECK([expr_to_flow 'outport == @pg1'], [0], [dnl
> -reg15=0x11
> -reg15=0x12
> -reg15=0x13
> -])
> -AT_CHECK([expr_to_flow 'outport == {@pg_empty}'], [0], [dnl
> -(no flows)
> -])
> -AT_CHECK([expr_to_flow 'outport == {"lsp1", @pg_empty}'], [0], [dnl
> -reg15=0x11
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- converting expressions to flows -- conjunction])
> -AT_KEYWORDS([conjunction])
> -expr_to_flow () {
> -    echo "$1" | ovstest test-ovn expr-to-flows | sort
> -}
> -
> -lflow="ip4.src == {10.0.0.1, 10.0.0.2, 10.0.0.3} && \
> -ip4.dst == {20.0.0.1, 20.0.0.2, 20.0.0.3}"
> -AT_CHECK([expr_to_flow "$lflow"], [0], [dnl
> -conj_id=1,ip
> -ip,nw_dst=20.0.0.1: conjunction(1, 0/2)
> -ip,nw_dst=20.0.0.2: conjunction(1, 0/2)
> -ip,nw_dst=20.0.0.3: conjunction(1, 0/2)
> -ip,nw_src=10.0.0.1: conjunction(1, 1/2)
> -ip,nw_src=10.0.0.2: conjunction(1, 1/2)
> -ip,nw_src=10.0.0.3: conjunction(1, 1/2)
> -])
> -
> -lflow="ip && (!ct.est || (ct.est && ct_label.blocked == 1))"
> -AT_CHECK([expr_to_flow "$lflow"], [0], [dnl
> -ct_state=+est+trk,ct_label=0x1/0x1,ip
> -ct_state=+est+trk,ct_label=0x1/0x1,ipv6
> -ct_state=-est+trk,ip
> -ct_state=-est+trk,ipv6
> -])
> -
> -lflow="ip4.src == {10.0.0.1, 10.0.0.2, 10.0.0.3} && \
> -ip4.dst == {20.0.0.1, 20.0.0.2}"
> -AT_CHECK([expr_to_flow "$lflow"], [0], [dnl
> -conj_id=1,ip
> -ip,nw_dst=20.0.0.1: conjunction(1, 0/2)
> -ip,nw_dst=20.0.0.2: conjunction(1, 0/2)
> -ip,nw_src=10.0.0.1: conjunction(1, 1/2)
> -ip,nw_src=10.0.0.2: conjunction(1, 1/2)
> -ip,nw_src=10.0.0.3: conjunction(1, 1/2)
> -])
> -
> -lflow="ip4 && ip4.src == {10.0.0.1, 10.0.0.2, 10.0.0.3} && \
> -ip4.dst == {20.0.0.1, 20.0.0.2, 20.0.0.3} && \
> -tcp.dst >= 1000 && tcp.dst <= 1010"
> -
> -AT_CHECK([expr_to_flow "$lflow"], [0], [dnl
> -conj_id=1,tcp
> -tcp,nw_dst=20.0.0.1: conjunction(1, 0/3)
> -tcp,nw_dst=20.0.0.2: conjunction(1, 0/3)
> -tcp,nw_dst=20.0.0.3: conjunction(1, 0/3)
> -tcp,nw_src=10.0.0.1: conjunction(1, 1/3)
> -tcp,nw_src=10.0.0.2: conjunction(1, 1/3)
> -tcp,nw_src=10.0.0.3: conjunction(1, 1/3)
> -tcp,tp_dst=0x3ea/0xfffe: conjunction(1, 2/3)
> -tcp,tp_dst=0x3ec/0xfffc: conjunction(1, 2/3)
> -tcp,tp_dst=0x3f0/0xfffe: conjunction(1, 2/3)
> -tcp,tp_dst=1000: conjunction(1, 2/3)
> -tcp,tp_dst=1001: conjunction(1, 2/3)
> -tcp,tp_dst=1010: conjunction(1, 2/3)
> -])
> -
> -lflow="ip4 && ip4.src == {10.0.0.4, 10.0.0.5, 10.0.0.6} && \
> -((ip4.dst == {20.0.0.4, 20.0.0.7, 20.0.0.8} && tcp.dst >= 1000 && \
> -tcp.dst <= 2000 && tcp.src >=1000 && tcp.src <= 2000) \
> -|| ip4.dst == 20.0.0.5 || ip4.dst == 20.0.0.6)"
> -
> -AT_CHECK([expr_to_flow "$lflow"], [0], [dnl
> -conj_id=1,tcp
> -ip,nw_src=10.0.0.4,nw_dst=20.0.0.5
> -ip,nw_src=10.0.0.4,nw_dst=20.0.0.6
> -ip,nw_src=10.0.0.5,nw_dst=20.0.0.5
> -ip,nw_src=10.0.0.5,nw_dst=20.0.0.6
> -ip,nw_src=10.0.0.6,nw_dst=20.0.0.5
> -ip,nw_src=10.0.0.6,nw_dst=20.0.0.6
> -tcp,nw_dst=20.0.0.4: conjunction(1, 0/4)
> -tcp,nw_dst=20.0.0.7: conjunction(1, 0/4)
> -tcp,nw_dst=20.0.0.8: conjunction(1, 0/4)
> -tcp,nw_src=10.0.0.4: conjunction(1, 1/4)
> -tcp,nw_src=10.0.0.5: conjunction(1, 1/4)
> -tcp,nw_src=10.0.0.6: conjunction(1, 1/4)
> -tcp,tp_dst=0x3ea/0xfffe: conjunction(1, 2/4)
> -tcp,tp_dst=0x3ec/0xfffc: conjunction(1, 2/4)
> -tcp,tp_dst=0x3f0/0xfff0: conjunction(1, 2/4)
> -tcp,tp_dst=0x400/0xfe00: conjunction(1, 2/4)
> -tcp,tp_dst=0x600/0xff00: conjunction(1, 2/4)
> -tcp,tp_dst=0x700/0xff80: conjunction(1, 2/4)
> -tcp,tp_dst=0x780/0xffc0: conjunction(1, 2/4)
> -tcp,tp_dst=0x7c0/0xfff0: conjunction(1, 2/4)
> -tcp,tp_dst=1000: conjunction(1, 2/4)
> -tcp,tp_dst=1001: conjunction(1, 2/4)
> -tcp,tp_dst=2000: conjunction(1, 2/4)
> -tcp,tp_src=0x3ea/0xfffe: conjunction(1, 3/4)
> -tcp,tp_src=0x3ec/0xfffc: conjunction(1, 3/4)
> -tcp,tp_src=0x3f0/0xfff0: conjunction(1, 3/4)
> -tcp,tp_src=0x400/0xfe00: conjunction(1, 3/4)
> -tcp,tp_src=0x600/0xff00: conjunction(1, 3/4)
> -tcp,tp_src=0x700/0xff80: conjunction(1, 3/4)
> -tcp,tp_src=0x780/0xffc0: conjunction(1, 3/4)
> -tcp,tp_src=0x7c0/0xfff0: conjunction(1, 3/4)
> -tcp,tp_src=1000: conjunction(1, 3/4)
> -tcp,tp_src=1001: conjunction(1, 3/4)
> -tcp,tp_src=2000: conjunction(1, 3/4)
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- action parsing])
> -dnl Unindented text is input (a set of OVN logical actions).
> -dnl Indented text is expected output.
> -AT_DATA([test-cases.txt],
> -[[# drop
> -drop;
> -    encodes as drop
> -drop; next;
> -    Syntax error at `next' expecting end of input.
> -next; drop;
> -    Syntax error at `drop' expecting action.
> -
> -# output
> -output;
> -    encodes as resubmit(,64)
> -
> -# next
> -next;
> -    encodes as resubmit(,19)
> -next(11);
> -    formats as next;
> -    encodes as resubmit(,19)
> -next(0);
> -    encodes as resubmit(,8)
> -next(23);
> -    encodes as resubmit(,31)
> -
> -next();
> -    Syntax error at `)' expecting "pipeline" or "table".
> -next(10;
> -    Syntax error at `;' expecting `)'.
> -next(24);
> -    "next" action cannot advance beyond table 23.
> -
> -next(table=11);
> -    formats as next;
> -    encodes as resubmit(,19)
> -next(pipeline=ingress);
> -    formats as next;
> -    encodes as resubmit(,19)
> -next(table=11, pipeline=ingress);
> -    formats as next;
> -    encodes as resubmit(,19)
> -next(pipeline=ingress, table=11);
> -    formats as next;
> -    encodes as resubmit(,19)
> -
> -next(pipeline=egress);
> -    "next" action cannot advance from ingress to egress pipeline (use
> "output" action instead)
> -
> -next(table=10);
> -    formats as next(10);
> -    encodes as resubmit(,18)
> -
> -# Loading a constant value.
> -tcp.dst=80;
> -    formats as tcp.dst = 80;
> -    encodes as set_field:80->tcp_dst
> -    has prereqs ip.proto == 0x6 && (eth.type == 0x800 || eth.type ==
> 0x86dd)
> -eth.dst[40] = 1;
> -    encodes as set_field:01:00:00:00:00:00/01:00:00:00:00:00->eth_dst
> -vlan.pcp = 2;
> -    encodes as set_field:0x4000/0xe000->vlan_tci
> -    has prereqs vlan.tci[12]
> -vlan.tci[13..15] = 2;
> -    encodes as set_field:0x4000/0xe000->vlan_tci
> -inport = "";
> -    encodes as set_field:0->reg14
> -ip.ttl=4;
> -    formats as ip.ttl = 4;
> -    encodes as set_field:4->nw_ttl
> -    has prereqs eth.type == 0x800 || eth.type == 0x86dd
> -outport="eth0"; next; outport="LOCAL"; next;
> -    formats as outport = "eth0"; next; outport = "LOCAL"; next;
> -    encodes as
> set_field:0x5->reg15,resubmit(,19),set_field:0xfffe->reg15,resubmit(,19)
> -
> -inport[1] = 1;
> -    Cannot select subfield of string field inport.
> -ip.proto[1] = 1;
> -    Cannot select subfield of nominal field ip.proto.
> -eth.dst[40] == 1;
> -    Syntax error at `==' expecting `=' or `<->'.
> -ip = 1;
> -    Predicate symbol ip used where lvalue required.
> -ip.proto = 6;
> -    Field ip.proto is not modifiable.
> -inport = {"a", "b"};
> -    Syntax error at `{' expecting constant.
> -inport = {};
> -    Syntax error at `{' expecting constant.
> -bad_prereq = 123;
> -    Error parsing expression `xyzzy' encountered as prerequisite or
> predicate of initial expression: Syntax error at `xyzzy' expecting field
> name.
> -self_recurse = 123;
> -    Error parsing expression `self_recurse != 0' encountered as
> prerequisite or predicate of initial expression: Error parsing expression
> `self_recurse != 0' encountered as prerequisite or predicate of initial
> expression: Recursive expansion of symbol `self_recurse'.
> -vlan.present = 0;
> -    Predicate symbol vlan.present used where lvalue required.
> -
> -# Moving one field into another.
> -reg0=reg1;
> -    formats as reg0 = reg1;
> -    encodes as move:NXM_NX_XXREG0[64..95]->NXM_NX_XXREG0[96..127]
> -vlan.pcp = reg0[0..2];
> -    encodes as move:NXM_NX_XXREG0[96..98]->NXM_OF_VLAN_TCI[13..15]
> -    has prereqs vlan.tci[12]
> -reg0[10] = vlan.pcp[1];
> -    encodes as move:NXM_OF_VLAN_TCI[14]->NXM_NX_XXREG0[106]
> -    has prereqs vlan.tci[12]
> -outport = inport;
> -    encodes as move:NXM_NX_REG14[]->NXM_NX_REG15[]
> -
> -reg0[0] = vlan.present;
> -    Predicate symbol vlan.present used where lvalue required.
> -reg0 = reg1[0..10];
> -    Can't assign 11-bit value to 32-bit destination.
> -inport = reg0;
> -    Can't assign integer field (reg0) to string field (inport).
> -inport = big_string;
> -    String fields inport and big_string are incompatible for assignment.
> -ip.proto = reg0[0..7];
> -    Field ip.proto is not modifiable.
> -
> -# Exchanging fields.
> -reg0 <-> reg1;
> -    encodes as
> push:NXM_NX_XXREG0[64..95],push:NXM_NX_XXREG0[96..127],pop:NXM_NX_XXREG0[64..95],pop:NXM_NX_XXREG0[96..127]
> -vlan.pcp <-> reg0[0..2];
> -    encodes as
> push:NXM_NX_XXREG0[96..98],push:NXM_OF_VLAN_TCI[13..15],pop:NXM_NX_XXREG0[96..98],pop:NXM_OF_VLAN_TCI[13..15]
> -    has prereqs vlan.tci[12]
> -reg0[10] <-> vlan.pcp[1];
> -    encodes as
> push:NXM_OF_VLAN_TCI[14],push:NXM_NX_XXREG0[106],pop:NXM_OF_VLAN_TCI[14],pop:NXM_NX_XXREG0[106]
> -    has prereqs vlan.tci[12]
> -outport <-> inport;
> -    encodes as
> push:NXM_NX_REG14[],push:NXM_NX_REG15[],pop:NXM_NX_REG14[],pop:NXM_NX_REG15[]
> -
> -reg0[0] <-> vlan.present;
> -    Predicate symbol vlan.present used where lvalue required.
> -reg0 <-> reg1[0..10];
> -    Can't exchange 32-bit field with 11-bit field.
> -inport <-> reg0;
> -    Can't exchange string field (inport) with integer field (reg0).
> -inport <-> big_string;
> -    String fields inport and big_string are incompatible for exchange.
> -ip.proto <-> reg0[0..7];
> -    Field ip.proto is not modifiable.
> -reg0[0..7] <-> ip.proto;
> -    Field ip.proto is not modifiable.
> -
> -# TTL decrement.
> -ip.ttl--;
> -    encodes as dec_ttl
> -    has prereqs ip
> -ip.ttl
> -    Syntax error at end of input expecting `--'.
> -
> -# load balancing.
> -ct_lb;
> -    encodes as ct(table=19,zone=NXM_NX_REG13[0..15],nat)
> -    has prereqs ip
> -ct_lb();
> -    formats as ct_lb;
> -    encodes as ct(table=19,zone=NXM_NX_REG13[0..15],nat)
> -    has prereqs ip
> -ct_lb(192.168.1.2:80, 192.168.1.3:80);
> -    encodes as group:1
> -    has prereqs ip
> -ct_lb(192.168.1.2, 192.168.1.3, );
> -    formats as ct_lb(192.168.1.2, 192.168.1.3);
> -    encodes as group:2
> -    has prereqs ip
> -ct_lb(fd0f::2, fd0f::3, );
> -    formats as ct_lb(fd0f::2, fd0f::3);
> -    encodes as group:3
> -    has prereqs ip
> -
> -ct_lb(192.168.1.2:);
> -    Syntax error at `)' expecting port number.
> -ct_lb(192.168.1.2:123456);
> -    Syntax error at `123456' expecting port number.
> -ct_lb(foo);
> -    Syntax error at `foo' expecting IP address.
> -ct_lb([192.168.1.2]);
> -    Syntax error at `192.168.1.2' expecting IPv6 address.
> -
> -# ct_next
> -ct_next;
> -    encodes as ct(table=19,zone=NXM_NX_REG13[0..15])
> -    has prereqs ip
> -
> -# ct_commit
> -ct_commit;
> -    encodes as ct(commit,zone=NXM_NX_REG13[0..15])
> -    has prereqs ip
> -ct_commit();
> -    formats as ct_commit;
> -    encodes as ct(commit,zone=NXM_NX_REG13[0..15])
> -    has prereqs ip
> -ct_commit(ct_mark=1);
> -    formats as ct_commit(ct_mark=0x1);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1->ct_mark))
> -    has prereqs ip
> -ct_commit(ct_mark=1/1);
> -    formats as ct_commit(ct_mark=0x1/0x1);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1/0x1->ct_mark))
> -    has prereqs ip
> -ct_commit(ct_label=1);
> -    formats as ct_commit(ct_label=0x1);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1->ct_label))
> -    has prereqs ip
> -ct_commit(ct_label=1/1);
> -    formats as ct_commit(ct_label=0x1/0x1);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1/0x1->ct_label))
> -    has prereqs ip
> -ct_commit(ct_mark=1, ct_label=2);
> -    formats as ct_commit(ct_mark=0x1, ct_label=0x2);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1->ct_mark,set_field:0x2->ct_label))
> -    has prereqs ip
> -
> -ct_commit(ct_label=0x01020304050607080910111213141516);
> -    formats as ct_commit(ct_label=0x1020304050607080910111213141516);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1020304050607080910111213141516->ct_label))
> -    has prereqs ip
> -ct_commit(ct_label=0x181716151413121110090807060504030201);
> -    formats as ct_commit(ct_label=0x16151413121110090807060504030201);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x16151413121110090807060504030201->ct_label))
> -    has prereqs ip
>
> -ct_commit(ct_label=0x1000000000000000000000000000000/0x1000000000000000000000000000000);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0x1000000000000000000000000000000/0x1000000000000000000000000000000->ct_label))
> -    has prereqs ip
> -ct_commit(ct_label=18446744073709551615);
> -    formats as ct_commit(ct_label=0xffffffffffffffff);
> -    encodes as
> ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0xffffffffffffffff->ct_label))
> -    has prereqs ip
> -ct_commit(ct_label=18446744073709551616);
> -    Decimal constants must be less than 2**64.
> -
> -# ct_dnat
> -ct_dnat;
> -    encodes as ct(table=19,zone=NXM_NX_REG11[0..15],nat)
> -    has prereqs ip
> -ct_dnat(192.168.1.2);
> -    encodes as
> ct(commit,table=19,zone=NXM_NX_REG11[0..15],nat(dst=192.168.1.2))
> -    has prereqs ip
> -
> -ct_dnat(192.168.1.2, 192.168.1.3);
> -    Syntax error at `,' expecting `)'.
> -ct_dnat(foo);
> -    Syntax error at `foo' expecting IPv4 address.
> -ct_dnat(foo, bar);
> -    Syntax error at `foo' expecting IPv4 address.
> -ct_dnat();
> -    Syntax error at `)' expecting IPv4 address.
> -
> -# ct_snat
> -ct_snat;
> -    encodes as ct(table=19,zone=NXM_NX_REG12[0..15],nat)
> -    has prereqs ip
> -ct_snat(192.168.1.2);
> -    encodes as
> ct(commit,table=19,zone=NXM_NX_REG12[0..15],nat(src=192.168.1.2))
> -    has prereqs ip
> -
> -ct_snat(192.168.1.2, 192.168.1.3);
> -    Syntax error at `,' expecting `)'.
> -ct_snat(foo);
> -    Syntax error at `foo' expecting IPv4 address.
> -ct_snat(foo, bar);
> -    Syntax error at `foo' expecting IPv4 address.
> -ct_snat();
> -    Syntax error at `)' expecting IPv4 address.
> -
> -# ct_clear
> -ct_clear;
> -    encodes as ct_clear
> -
> -# clone
> -clone { ip4.dst = 255.255.255.255; output; }; next;
> -    encodes as
> clone(set_field:255.255.255.255->ip_dst,resubmit(,64)),resubmit(,19)
> -    has prereqs eth.type == 0x800
> -
> -# arp
> -arp { eth.dst = ff:ff:ff:ff:ff:ff; output; }; output;
> -    encodes as
> controller(userdata=00.00.00.00.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip4
> -arp { };
> -    formats as arp { drop; };
> -    encodes as controller(userdata=00.00.00.00.00.00.00.00)
> -    has prereqs ip4
> -
> -# get_arp
> -get_arp(outport, ip4.dst);
> -    encodes as
> push:NXM_NX_REG0[],push:NXM_OF_IP_DST[],pop:NXM_NX_REG0[],set_field:00:00:00:00:00:00->eth_dst,resubmit(,65),pop:NXM_NX_REG0[]
> -    has prereqs eth.type == 0x800
> -get_arp(inport, reg0);
> -    encodes as
> push:NXM_NX_REG15[],push:NXM_NX_REG0[],push:NXM_NX_XXREG0[96..127],push:NXM_NX_REG14[],pop:NXM_NX_REG15[],pop:NXM_NX_REG0[],set_field:00:00:00:00:00:00->eth_dst,resubmit(,65),pop:NXM_NX_REG0[],pop:NXM_NX_REG15[]
> -
> -get_arp;
> -    Syntax error at `;' expecting `('.
> -get_arp();
> -    Syntax error at `)' expecting field name.
> -get_arp(inport);
> -    Syntax error at `)' expecting `,'.
> -get_arp(inport ip4.dst);
> -    Syntax error at `ip4.dst' expecting `,'.
> -get_arp(inport, ip4.dst;
> -    Syntax error at `;' expecting `)'.
> -get_arp(inport, eth.dst);
> -    Cannot use 48-bit field eth.dst[0..47] where 32-bit field is required.
> -get_arp(inport, outport);
> -    Cannot use string field outport where numeric field is required.
> -get_arp(reg0, ip4.dst);
> -    Cannot use numeric field reg0 where string field is required.
> -
> -# put_arp
> -put_arp(inport, arp.spa, arp.sha);
> -    encodes as
> push:NXM_NX_REG0[],push:NXM_OF_ETH_SRC[],push:NXM_NX_ARP_SHA[],push:NXM_OF_ARP_SPA[],pop:NXM_NX_REG0[],pop:NXM_OF_ETH_SRC[],controller(userdata=00.00.00.01.00.00.00.00),pop:NXM_OF_ETH_SRC[],pop:NXM_NX_REG0[]
> -    has prereqs eth.type == 0x806 && eth.type == 0x806
> -
> -# put_dhcp_opts
> -reg1[0] = put_dhcp_opts(offerip = 1.2.3.4, router = 10.0.0.1);
> -    encodes as
> controller(userdata=00.00.00.02.00.00.00.00.00.01.de.10.00.00.00.40.01.02.03.04.03.04.0a.00.00.01,pause)
> -reg2[5] =
> put_dhcp_opts(offerip=10.0.0.4,router=10.0.0.1,netmask=255.255.254.0,mtu=1400,domain_name="
> ovn.org",wpad="https://example.org",bootfile_name="
> https://127.0.0.1/boot.ipxe",path_prefix="/tftpboot");
> -    formats as reg2[5] = put_dhcp_opts(offerip = 10.0.0.4, router =
> 10.0.0.1, netmask = 255.255.254.0, mtu = 1400, domain_name = "ovn.org",
> wpad = "https://example.org", bootfile_name = "https://127.0.0.1/boot.ipxe",
> path_prefix = "/tftpboot");
> -    encodes as
> controller(userdata=00.00.00.02.00.00.00.00.00.01.de.10.00.00.00.25.0a.00.00.04.03.04.0a.00.00.01.01.04.ff.ff.fe.00.1a.02.05.78.0f.07.6f.76.6e.2e.6f.72.67.fc.13.68.74.74.70.73.3a.2f.2f.65.78.61.6d.70.6c.65.2e.6f.72.67.43.1b.68.74.74.70.73.3a.2f.2f.31.32.37.2e.30.2e.30.2e.31.2f.62.6f.6f.74.2e.69.70.78.65.d2.09.2f.74.66.74.70.62.6f.6f.74,pause)
> -reg0[15] =
> put_dhcp_opts(offerip=10.0.0.4,router=10.0.0.1,netmask=255.255.255.0,mtu=1400,ip_forward_enable=1,default_ttl=121,dns_server={8.8.8.8,7.7.7.7},classless_static_route={
> 30.0.0.0/24,10.0.0.4,40.0.0.0/16,10.0.0.6,0.0.0.0/0,10.0.0.1
> },ethernet_encap=1,router_discovery=0,tftp_server_address={10.0.0.4,10.0.0.5});
> -    formats as reg0[15] = put_dhcp_opts(offerip = 10.0.0.4, router =
> 10.0.0.1, netmask = 255.255.255.0, mtu = 1400, ip_forward_enable = 1,
> default_ttl = 121, dns_server = {8.8.8.8, 7.7.7.7}, classless_static_route
> = {30.0.0.0/24, 10.0.0.4, 40.0.0.0/16, 10.0.0.6, 0.0.0.0/0, 10.0.0.1},
> ethernet_encap = 1, router_discovery = 0, tftp_server_address = {10.0.0.4,
> 10.0.0.5});
> -    encodes as
> controller(userdata=00.00.00.02.00.00.00.00.00.01.de.10.00.00.00.6f.0a.00.00.04.03.04.0a.00.00.01.01.04.ff.ff.ff.00.1a.02.05.78.13.01.01.17.01.79.06.08.08.08.08.08.07.07.07.07.79.14.18.1e.00.00.0a.00.00.04.10.28.00.0a.00.00.06.00.0a.00.00.01.24.01.01.1f.01.00.96.08.0a.00.00.04.0a.00.00.05,pause)
> -
> -reg1[0..1] = put_dhcp_opts(offerip = 1.2.3.4, router = 10.0.0.1);
> -    Cannot use 2-bit field reg1[0..1] where 1-bit field is required.
> -reg1[0] = put_dhcp_opts();
> -    put_dhcp_opts requires offerip to be specified.
> -reg1[0] = put_dhcp_opts(x = 1.2.3.4, router = 10.0.0.1);
> -    Syntax error at `x' expecting DHCPv4 option name.
> -reg1[0] = put_dhcp_opts(router = 10.0.0.1);
> -    put_dhcp_opts requires offerip to be specified.
> -reg1[0] = put_dhcp_opts(offerip=1.2.3.4, "hi");
> -    Syntax error at `"hi"'.
> -reg1[0] = put_dhcp_opts(offerip=1.2.3.4, xyzzy);
> -    Syntax error at `xyzzy' expecting DHCPv4 option name.
> -reg1[0] = put_dhcp_opts(offerip="xyzzy");
> -    DHCPv4 option offerip requires numeric value.
> -reg1[0] = put_dhcp_opts(offerip=1.2.3.4, domain_name=1.2.3.4);
> -    DHCPv4 option domain_name requires string value.
> -
> -# nd_ns
> -nd_ns { nd.target = xxreg0; output; };
> -    encodes as
> controller(userdata=00.00.00.09.00.00.00.00.ff.ff.00.18.00.00.23.20.00.06.00.80.00.00.00.00.00.01.de.10.00.01.2e.10.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00)
> -    has prereqs ip6
> -
> -nd_ns { };
> -    formats as nd_ns { drop; };
> -    encodes as controller(userdata=00.00.00.09.00.00.00.00)
> -    has prereqs ip6
> -
> -# nd_na
> -nd_na { eth.src = 12:34:56:78:9a:bc; nd.tll = 12:34:56:78:9a:bc; outport
> = inport; inport = ""; /* Allow sending out inport. */ output; };
> -    formats as nd_na { eth.src = 12:34:56:78:9a:bc; nd.tll =
> 12:34:56:78:9a:bc; outport = inport; inport = ""; output; };
> -    encodes as
> controller(userdata=00.00.00.03.00.00.00.00.00.19.00.10.80.00.08.06.12.34.56.78.9a.bc.00.00.00.19.00.10.80.00.42.06.12.34.56.78.9a.bc.00.00.ff.ff.00.18.00.00.23.20.00.06.00.20.00.00.00.00.00.01.1c.04.00.01.1e.04.00.19.00.10.00.01.1c.04.00.00.00.00.00.00.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00)
> -    has prereqs nd_ns
> -# nd_na_router
> -nd_na_router { eth.src = 12:34:56:78:9a:bc; nd.tll = 12:34:56:78:9a:bc;
> outport = inport; inport = ""; /* Allow sending out inport. */ output; };
> -    formats as nd_na_router { eth.src = 12:34:56:78:9a:bc; nd.tll =
> 12:34:56:78:9a:bc; outport = inport; inport = ""; output; };
> -    encodes as
> controller(userdata=00.00.00.0c.00.00.00.00.00.19.00.10.80.00.08.06.12.34.56.78.9a.bc.00.00.00.19.00.10.80.00.42.06.12.34.56.78.9a.bc.00.00.ff.ff.00.18.00.00.23.20.00.06.00.20.00.00.00.00.00.01.1c.04.00.01.1e.04.00.19.00.10.00.01.1c.04.00.00.00.00.00.00.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00)
> -    has prereqs nd_ns
> -
> -# get_nd
> -get_nd(outport, ip6.dst);
> -    encodes as
> push:NXM_NX_XXREG0[],push:NXM_NX_IPV6_DST[],pop:NXM_NX_XXREG0[],set_field:00:00:00:00:00:00->eth_dst,resubmit(,65),pop:NXM_NX_XXREG0[]
> -    has prereqs eth.type == 0x86dd
> -get_nd(inport, xxreg0);
> -    encodes as
> push:NXM_NX_REG15[],push:NXM_NX_REG14[],pop:NXM_NX_REG15[],set_field:00:00:00:00:00:00->eth_dst,resubmit(,65),pop:NXM_NX_REG15[]
> -get_nd;
> -    Syntax error at `;' expecting `('.
> -get_nd();
> -    Syntax error at `)' expecting field name.
> -get_nd(inport);
> -    Syntax error at `)' expecting `,'.
> -get_nd(inport ip6.dst);
> -    Syntax error at `ip6.dst' expecting `,'.
> -get_nd(inport, ip6.dst;
> -    Syntax error at `;' expecting `)'.
> -get_nd(inport, eth.dst);
> -    Cannot use 48-bit field eth.dst[0..47] where 128-bit field is
> required.
> -get_nd(inport, outport);
> -    Cannot use string field outport where numeric field is required.
> -get_nd(xxreg0, ip6.dst);
> -    Cannot use numeric field xxreg0 where string field is required.
> -
> -# put_nd
> -put_nd(inport, nd.target, nd.sll);
> -    encodes as
> push:NXM_NX_XXREG0[],push:NXM_OF_ETH_SRC[],push:NXM_NX_ND_SLL[],push:NXM_NX_ND_TARGET[],pop:NXM_NX_XXREG0[],pop:NXM_OF_ETH_SRC[],controller(userdata=00.00.00.04.00.00.00.00),pop:NXM_OF_ETH_SRC[],pop:NXM_NX_XXREG0[]
> -    has prereqs (icmp6.type == 0x87 || icmp6.type == 0x88) && eth.type ==
> 0x86dd && ip.proto == 0x3a && (eth.type == 0x800 || eth.type == 0x86dd) &&
> icmp6.code == 0 && eth.type == 0x86dd && ip.proto == 0x3a && (eth.type ==
> 0x800 || eth.type == 0x86dd) && ip.ttl == 0xff && (eth.type == 0x800 ||
> eth.type == 0x86dd) && icmp6.type == 0x87 && eth.type == 0x86dd && ip.proto
> == 0x3a && (eth.type == 0x800 || eth.type == 0x86dd) && icmp6.code == 0 &&
> eth.type == 0x86dd && ip.proto == 0x3a && (eth.type == 0x800 || eth.type ==
> 0x86dd) && ip.ttl == 0xff && (eth.type == 0x800 || eth.type == 0x86dd)
> -
> -# put_dhcpv6_opts
> -reg1[0] = put_dhcpv6_opts(ia_addr = ae70::4, server_id =
> 00:00:00:00:10:02);
> -    encodes as
> controller(userdata=00.00.00.05.00.00.00.00.00.01.de.10.00.00.00.40.00.05.00.10.ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.04.00.02.00.06.00.00.00.00.10.02,pause)
> -reg1[0] = put_dhcpv6_opts();
> -    encodes as
> controller(userdata=00.00.00.05.00.00.00.00.00.01.de.10.00.00.00.40,pause)
> -reg1[0] = put_dhcpv6_opts(dns_server={ae70::1,ae70::2});
> -    formats as reg1[0] = put_dhcpv6_opts(dns_server = {ae70::1, ae70::2});
> -    encodes as
> controller(userdata=00.00.00.05.00.00.00.00.00.01.de.10.00.00.00.40.00.17.00.20.ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.01.ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.02,pause)
> -reg1[0] = put_dhcpv6_opts(server_id=12:34:56:78:9a:bc,
> dns_server={ae70::1,ae89::2});
> -    formats as reg1[0] = put_dhcpv6_opts(server_id = 12:34:56:78:9a:bc,
> dns_server = {ae70::1, ae89::2});
> -    encodes as
> controller(userdata=00.00.00.05.00.00.00.00.00.01.de.10.00.00.00.40.00.02.00.06.12.34.56.78.9a.bc.00.17.00.20.ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.01.ae.89.00.00.00.00.00.00.00.00.00.00.00.00.00.02,pause)
> -reg1[0] = put_dhcpv6_opts(domain_search = "ovn.org");
> -    encodes as
> controller(userdata=00.00.00.05.00.00.00.00.00.01.de.10.00.00.00.40.00.18.00.07.6f.76.6e.2e.6f.72.67,pause)
> -reg1[0] = put_dhcpv6_opts(x = 1.2.3.4);
> -    Syntax error at `x' expecting DHCPv6 option name.
> -reg1[0] = put_dhcpv6_opts(ia_addr=ae70::4, "hi");
> -    Syntax error at `"hi"'.
> -reg1[0] = put_dhcpv6_opts(ia_addr=ae70::4, xyzzy);
> -    Syntax error at `xyzzy' expecting DHCPv6 option name.
> -reg1[0] = put_dhcpv6_opts(ia_addr="ae70::4");
> -    DHCPv6 option ia_addr requires numeric value.
> -reg1[0] = put_dhcpv6_opts(ia_addr=ae70::4, domain_search=ae70::1);
> -    DHCPv6 option domain_search requires string value.
> -
> -# set_queue
> -set_queue(0);
> -    encodes as set_queue:0
> -set_queue(61440);
> -    encodes as set_queue:61440
> -set_queue(65535);
> -    Queue ID 65535 for set_queue is not in valid range 0 to 61440.
> -
> -# dns_lookup
> -reg1[0] = dns_lookup();
> -    encodes as
> controller(userdata=00.00.00.06.00.00.00.00.00.01.de.10.00.00.00.40,pause)
> -    has prereqs udp
> -reg1[0] = dns_lookup("foo");
> -    dns_lookup doesn't take any parameters
> -
> -# set_meter
> -set_meter(0);
> -    Rate 0 for set_meter is not in valid.
> -set_meter(1);
> -    encodes as meter:1
> -set_meter(100, 1000);
> -    encodes as meter:2
> -set_meter(100, 1000, );
> -    Syntax error at `,' expecting `)'.
> -set_meter(4294967295, 4294967295);
> -    encodes as meter:3
> -
> -# log
> -log(verdict=allow, severity=warning);
> -    encodes as controller(userdata=00.00.00.07.00.00.00.00.00.04)
> -log(name="test1", verdict=drop, severity=info);
> -    encodes as
> controller(userdata=00.00.00.07.00.00.00.00.01.06.74.65.73.74.31)
> -log(verdict=drop, severity=info, meter="meter1");
> -    encodes as
> controller(userdata=00.00.00.07.00.00.00.00.01.06,meter_id=4)
> -log(name="test1", verdict=drop, severity=info, meter="meter1");
> -    encodes as
> controller(userdata=00.00.00.07.00.00.00.00.01.06.74.65.73.74.31,meter_id=4)
> -log(verdict=drop);
> -    formats as log(verdict=drop, severity=info);
> -    encodes as controller(userdata=00.00.00.07.00.00.00.00.01.06)
> -log(verdict=bad_verdict, severity=info);
> -    Syntax error at `bad_verdict' unknown verdict.
> -log(verdict=drop, severity=bad_severity);
> -    Syntax error at `bad_severity' unknown severity.
> -log(severity=notice);
> -    Syntax error at `;' expecting verdict.
> -
> -# put_nd_ra_opts
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", mtu = 1500, prefix =
> aef0::/64, slla = ae:01:02:03:04:05);
> -    encodes as
> controller(userdata=00.00.00.08.00.00.00.00.00.01.de.10.00.00.00.40.86.00.00.00.ff.00.ff.ff.00.00.00.00.00.00.00.00.05.01.00.00.00.00.05.dc.03.04.40.c0.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.ae.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.01.01.ae.01.02.03.04.05,pause)
> -    has prereqs ip6
> -reg1[0] = put_nd_ra_opts(addr_mode = "dhcpv6_stateful", slla =
> ae:01:02:03:04:10, mtu = 1450);
> -    encodes as
> controller(userdata=00.00.00.08.00.00.00.00.00.01.de.10.00.00.00.40.86.00.00.00.ff.80.ff.ff.00.00.00.00.00.00.00.00.01.01.ae.01.02.03.04.10.05.01.00.00.00.00.05.aa,pause)
> -    has prereqs ip6
> -reg1[0] = put_nd_ra_opts(addr_mode = "dhcpv6_stateless", slla =
> ae:01:02:03:04:06, prefix = aef0::/64);
> -    encodes as
> controller(userdata=00.00.00.08.00.00.00.00.00.01.de.10.00.00.00.40.86.00.00.00.ff.40.ff.ff.00.00.00.00.00.00.00.00.01.01.ae.01.02.03.04.06.03.04.40.c0.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.ae.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00,pause)
> -    has prereqs ip6
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", mtu = 1500, prefix =
> aef0::/64);
> -    slla option not present
> -reg1[0] = put_nd_ra_opts(addr_mode = "dhcpv6_stateful", mtu = 1450,
> prefix = aef0::/64, prefix = bef0::/64, slla = ae:01:02:03:04:10);
> -    encodes as
> controller(userdata=00.00.00.08.00.00.00.00.00.01.de.10.00.00.00.40.86.00.00.00.ff.80.ff.ff.00.00.00.00.00.00.00.00.05.01.00.00.00.00.05.aa.03.04.40.80.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.ae.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.03.04.40.80.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.be.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.01.01.ae.01.02.03.04.10,pause)
> -    has prereqs ip6
> -reg1[0] = put_nd_ra_opts(addr_mode = "dhcpv6_stateful", mtu = 1450,
> prefix = aef0::/64, prefix = bef0::/64, slla = ae:01:02:03:04:10);
> -    encodes as
> controller(userdata=00.00.00.08.00.00.00.00.00.01.de.10.00.00.00.40.86.00.00.00.ff.80.ff.ff.00.00.00.00.00.00.00.00.05.01.00.00.00.00.05.aa.03.04.40.80.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.ae.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.03.04.40.80.ff.ff.ff.ff.ff.ff.ff.ff.00.00.00.00.be.f0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.01.01.ae.01.02.03.04.10,pause)
> -    has prereqs ip6
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", slla = ae:01:02:03:04:10);
> -    prefix option needs to be set when address mode is
> slaac/dhcpv6_stateless.
> -reg1[0] = put_nd_ra_opts(addr_mode = "dhcpv6_stateless", slla =
> ae:01:02:03:04:10);
> -    prefix option needs to be set when address mode is
> slaac/dhcpv6_stateless.
> -reg1[0] = put_nd_ra_opts(addr_mode = dhcpv6_stateless, prefix =
> aef0::/64, slla = ae:01:02:03:04:10);
> -    Syntax error at `dhcpv6_stateless' expecting constant.
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", mtu = 1500, prefix =
> aef0::, slla = ae:01:02:03:04:10);
> -    Invalid value for "prefix" option
> -reg1[0] = put_nd_ra_opts(addr_mode = "foo", mtu = 1500, slla =
> ae:01:02:03:04:10);
> -    Invalid value for "addr_mode" option
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", mtu = "1500", slla =
> ae:01:02:03:04:10);
> -    IPv6 ND RA option mtu requires numeric value.
> -reg1[0] = put_nd_ra_opts(addr_mode = "slaac", mtu = 10.0.0.4, slla =
> ae:01:02:03:04:10);
> -    Invalid value for "mtu" option
> -
> -# icmp4
> -icmp4 { eth.dst = ff:ff:ff:ff:ff:ff; output; }; output;
> -    encodes as
> controller(userdata=00.00.00.0a.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip4
> -
> -icmp4 { };
> -    formats as icmp4 { drop; };
> -    encodes as controller(userdata=00.00.00.0a.00.00.00.00)
> -    has prereqs ip4
> -
> -# icmp4 with icmp4.frag_mtu
> -icmp4 { eth.dst = ff:ff:ff:ff:ff:ff; icmp4.frag_mtu = 1500; output; };
> output;
> -    encodes as
> controller(userdata=00.00.00.0a.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.28.00.00.23.20.00.25.00.00.00.00.00.00.00.03.00.0e.00.00.00.0d.00.00.00.00.05.dc.00.00.00.04.00.04.00.00.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip4
> -
> -# icmp4_error
> -icmp4_error { eth.dst = ff:ff:ff:ff:ff:ff; output; }; output;
> -    encodes as
> controller(userdata=00.00.00.0e.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip4
> -
> -icmp4_error { };
> -    formats as icmp4_error { drop; };
> -    encodes as controller(userdata=00.00.00.0e.00.00.00.00)
> -    has prereqs ip4
> -
> -# icmp4_error with icmp4.frag_mtu
> -icmp4_error { eth.dst = ff:ff:ff:ff:ff:ff; icmp4.frag_mtu = 1500; output;
> }; output;
> -    encodes as
> controller(userdata=00.00.00.0e.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.28.00.00.23.20.00.25.00.00.00.00.00.00.00.03.00.0e.00.00.00.0d.00.00.00.00.05.dc.00.00.00.04.00.04.00.00.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip4
> -
> -icmp4.frag_mtu = 1500;
> -    encodes as controller(userdata=00.00.00.0d.00.00.00.00.05.dc,pause)
> -
> -# icmp6
> -icmp6 { eth.dst = ff:ff:ff:ff:ff:ff; output; }; output;
> -    encodes as
> controller(userdata=00.00.00.0a.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs ip6
> -
> -icmp6 { };
> -    formats as icmp6 { drop; };
> -    encodes as controller(userdata=00.00.00.0a.00.00.00.00)
> -    has prereqs ip6
> -
> -# tcp_reset
> -tcp_reset { eth.dst = ff:ff:ff:ff:ff:ff; output; }; output;
> -    encodes as
> controller(userdata=00.00.00.0b.00.00.00.00.00.19.00.10.80.00.06.06.ff.ff.ff.ff.ff.ff.00.00.ff.ff.00.10.00.00.23.20.00.0e.ff.f8.40.00.00.00),resubmit(,64)
> -    has prereqs tcp
> -
> -tcp_reset { };
> -    formats as tcp_reset { drop; };
> -    encodes as controller(userdata=00.00.00.0b.00.00.00.00)
> -    has prereqs tcp
> -
> -# trigger_event
> -trigger_event(event = "empty_lb_backends", vip = "10.0.0.1:80", protocol
> = "tcp", load_balancer = "12345678-abcd-9876-fedc-11119f8e7d6c");
> -    encodes as
> controller(userdata=00.00.00.0f.00.00.00.00.00.00.00.00.00.01.00.0b.31.30.2e.30.2e.30.2e.31.3a.38.30.00.02.00.03.74.63.70.00.03.00.24.31.32.33.34.35.36.37.38.2d.61.62.63.64.2d.39.38.37.36.2d.66.65.64.63.2d.31.31.31.31.39.66.38.65.37.64.36.63)
> -
> -# Testing invalid vip results in extra error messages from socket-util.c
> -trigger_event(event = "empty_lb_backends", vip = "10.0.0.1:80", protocol
> = "sctp", load_balancer = "12345678-abcd-9876-fedc-11119f8e7d6c");
> -    Load balancer protocol 'sctp' is not 'tcp' or 'udp'
> -trigger_event(event = "empty_lb_backends", vip = "10.0.0.1:80", protocol
> = "tcp", load_balancer = "bacon");
> -    Load balancer 'bacon' is not a UUID
> -
> -# IGMP
> -igmp;
> -    encodes as controller(userdata=00.00.00.10.00.00.00.00)
> -
> -# Contradictionary prerequisites (allowed but not useful):
> -ip4.src = ip6.src[0..31];
> -    encodes as move:NXM_NX_IPV6_SRC[0..31]->NXM_OF_IP_SRC[]
> -    has prereqs eth.type == 0x800 && eth.type == 0x86dd
> -ip4.src <-> ip6.src[0..31];
> -    encodes as
> push:NXM_NX_IPV6_SRC[0..31],push:NXM_OF_IP_SRC[],pop:NXM_NX_IPV6_SRC[0..31],pop:NXM_OF_IP_SRC[]
> -    has prereqs eth.type == 0x800 && eth.type == 0x86dd
> -
> -# check_pkt_larger
> -reg0[0] = check_pkt_larger(1500);
> -    encodes as check_pkt_larger(1500)->NXM_NX_XXREG0[96]
> -
> -reg0 = check_pkt_larger(1500);
> -    Cannot use 32-bit field reg0[0..31] where 1-bit field is required.
> -
> -reg0 = check_pkt_larger(foo);
> -    Cannot use 32-bit field reg0[0..31] where 1-bit field is required.
> -
> -reg0[0] = check_pkt_larger(foo);
> -    Syntax error at `foo' expecting `;'.
> -
> -# Miscellaneous negative tests.
> -;
> -    Syntax error at `;'.
> -xyzzy;
> -    Syntax error at `xyzzy' expecting action.
> -next; 123;
> -    Syntax error at `123'.
> -next; xyzzy;
> -    Syntax error at `xyzzy' expecting action.
> -next
> -    Syntax error at end of input expecting `;'.
> -]])
> -sed '/^[[      ]]/d' test-cases.txt > input.txt
> -cp test-cases.txt expout
> -AT_CHECK([ovstest test-ovn parse-actions < input.txt], [0], [expout])
> -AT_CLEANUP
> -
> -AT_BANNER([OVN end-to-end tests])
> -
> -# 3 hypervisors, one logical switch, 3 logical ports per hypervisor
> -AT_SETUP([ovn -- 3 HVs, 1 LS, 3 lports/HV])
> -AT_KEYWORDS([ovnarp])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Create hypervisors hv[123].
> -# Add vif1[123] to hv1, vif2[123] to hv2, vif3[123] to hv3.
> -# Add all of the vifs to a single logical switch lsw0.
> -# Turn on port security on all the vifs except vif[123]1.
> -# Make vif13, vif2[23], vif3[123] destinations for unknown MACs.
> -# Add some ACLs for Ethertypes 1234, 1235, 1236.
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2 3; do
> -        ovs-vsctl add-port br-int vif$i$j -- set Interface vif$i$j
> external-ids:iface-id=lp$i$j options:tx_pcap=hv$i/vif$i$j-tx.pcap
> options:rxq_pcap=hv$i/vif$i$j-rx.pcap ofport-request=$i$j
> -        ovn-nbctl lsp-add lsw0 lp$i$j
> -        if test $j = 1; then
> -            ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j
> 192.168.0.$i$j" unknown
> -        else
> -            if test $j = 3; then
> -                ip_addrs="192.168.0.$i$j fe80::ea2a:eaff:fe28:$i$j/64
> 192.169.0.$i$j"
> -            else
> -                ip_addrs="192.168.0.$i$j"
> -            fi
> -            ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j
> $ip_addrs"
> -            ovn-nbctl lsp-set-port-security lp$i$j f0:00:00:00:00:$i$j
> -        fi
> -    done
> -done
> -ovn-nbctl acl-add lsw0 from-lport 1000 'eth.type == 0x1234' drop
> -ovn-nbctl acl-add lsw0 from-lport 1000 'eth.type == 0x1235 && inport ==
> "lp11"' drop
> -ovn-nbctl acl-add lsw0 to-lport 1000 'eth.type == 0x1236 && outport ==
> "lp33"' drop
> -ovn-nbctl create Address_Set name=set1
> addresses=\"f0:00:00:00:00:11\",\"f0:00:00:00:00:21\",\"f0:00:00:00:00:31\"
> -ovn-nbctl acl-add lsw0 to-lport 1000 'eth.type == 0x1237 && eth.src ==
> $set1 && outport == "lp33"' drop
> -
> -get_lsp_uuid () {
> -    ovn-nbctl lsp-list lsw0 | grep $1 | awk '{ print $1 }'
> -}
> -
> -ovn-nbctl create Port_Group name=pg1 ports=`get_lsp_uuid
> lp22`,`get_lsp_uuid lp33`
> -ovn-nbctl acl-add lsw0 to-lport 1000 'eth.type == 0x1238 && outport ==
> @pg1' drop
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Make sure there is no attempt to adding duplicated flows by
> ovn-controller
> -AT_FAIL_IF([test -n "`grep duplicate hv1/ovn-controller.log`"])
> -AT_FAIL_IF([test -n "`grep duplicate hv2/ovn-controller.log`"])
> -AT_FAIL_IF([test -n "`grep duplicate hv3/ovn-controller.log`"])
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv${1%?}
> -}
> -
> -# test_packet INPORT DST SRC ETHTYPE OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 11 for
> vif11.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        : > $i$j.expected
> -    done
> -done
> -test_packet() {
> -    local inport=$1 packet=$2$3$4; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -# test_arp INPORT SHA SPA TPA [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local inport=$1 sha=$2 spa=$3 tpa=$4 reply_ha=$5
> -    local
> request=ffffffffffff${sha}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -    hv=`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $request
> -
> -    if test X$reply_ha = X; then
> -        # Expect to receive the broadcast ARP on the other logical switch
> ports
> -        # if no reply is expected.
> -        local i j
> -        for i in 1 2 3; do
> -            for j in 1 2 3; do
> -                if test $i$j != $inport; then
> -                    echo $request >> $i$j.expected
> -                fi
> -            done
> -        done
> -    else
> -        # Expect to receive the reply, if any.
> -        local
> reply=${sha}${reply_ha}08060001080006040002${reply_ha}${tpa}${sha}${spa}
> -        echo $reply >> $inport.expected
> -    fi
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast packets are delivered to exactly one logical switch port
> -#    (except that packets destined to their input ports are dropped).
> -#
> -# 2. Broadcast and multicast are delivered to all logical switch ports
> -#    except the input port.
> -#
> -# 3. When port security is turned on, the switch drops packets from the
> wrong
> -#    MAC address.
> -#
> -# 4. The switch drops all packets with a VLAN tag.
> -#
> -# 5. The switch drops all packets with a multicast source address.  (This
> only
> -#    affects behavior when port security is turned off, since otherwise
> port
> -#    security would drop the packet anyway.)
> -#
> -# 6. The switch delivers packets with an unknown destination to logical
> -#    switch ports with "unknown" among their MAC addresses (and port
> -#    security disabled).
> -#
> -# 7. The switch drops unicast packets that violate an ACL.
> -#
> -# 8. The switch drops multicast and broadcast packets that violate an ACL.
> -#
> -# 9. OVN generates responses to ARP requests for known IPs, except for
> -#    requests from a port for the port's own IP.
> -#
> -# 10. No response to ARP requests for unknown IPs.
> -
> -for is in 1 2 3; do
> -    for js in 1 2 3; do
> -        s=$is$js
> -        bcast=
> -        unknown=
> -        bacl2=
> -        bacl3=
> -        for id in 1 2 3; do
> -            for jd in 1 2 3; do
> -                d=$id$jd
> -
> -                if test $d != $s; then unicast=$d; else unicast=; fi
> -                test_packet $s f000000000$d f000000000$s $s$d $unicast
>  #1
> -
> -                if test $d != $s && test $js = 1; then
> -                    impersonate=$d
> -                else
> -                    impersonate=
> -                fi
> -                test_packet $s f000000000$d f00000000055 55$d
> $impersonate #3
> -
> -                if test $d != $s && test $s != 11; then acl2=$d; else
> acl2=; fi
> -                if test $d != $s && test $d != 33; then acl3=$d; else
> acl3=; fi
> -                if test $d = $s || (test $js = 1 && test $d = 33); then
> -                    # Source of 11, 21, or 31 and dest of 33 should be
> dropped
> -                    # due to the 4th ACL that uses address_set(set1).
> -                    acl4=
> -                else
> -                    acl4=$d
> -                fi
> -                if test $d = $s || test $d = 22 || test $d = 33; then
> -                    # dest of 22 and 33 should be dropped
> -                    # due to the 5th ACL that uses port_group(pg1).
> -                    acl5=
> -                else
> -                    acl5=$d
> -                fi
> -                test_packet $s f000000000$d f000000000$s 1234        #7,
> acl1
> -                test_packet $s f000000000$d f000000000$s 1235 $acl2  #7,
> acl2
> -                test_packet $s f000000000$d f000000000$s 1236 $acl3  #7,
> acl3
> -                test_packet $s f000000000$d f000000000$s 1237 $acl4  #7,
> acl4
> -                test_packet $s f000000000$d f000000000$s 1238 $acl5  #7,
> acl5
> -
> -                test_packet $s f000000000$d f00000000055 810000091234
>   #4
> -                test_packet $s f000000000$d 0100000000$s $s$d
>   #5
> -
> -                if test $d != $s && test $jd = 1; then
> -                    unknown="$unknown $d"
> -                fi
> -                bcast="$bcast $unicast"
> -                bacl2="$bacl2 $acl2"
> -                bacl3="$bacl3 $acl3"
> -
> -                sip=`ip_to_hex 192 168 0 $is$js`
> -                tip=`ip_to_hex 192 168 0 $id$jd`
> -                tip_unknown=`ip_to_hex 11 11 11 11`
> -                if test $d != $s; then
> -                    reply_ha=f000000000$d
> -                else
> -                    reply_ha=
> -                fi
> -                test_arp $s f000000000$s $sip $tip $reply_ha
>  #9
> -                test_arp $s f000000000$s $sip $tip_unknown
>  #10
> -
> -                if test $jd = 3; then
> -                    # lsp[123]3 has an additional ip 192.169.0.[123]3.
> -                    tip=`ip_to_hex 192 169 0 $id$jd`
> -                    test_arp $s f000000000$s $sip $tip $reply_ha
>  #9
> -                fi
> -            done
> -        done
> -
> -        # Broadcast and multicast.
> -        test_packet $s ffffffffffff f000000000$s ${s}ff $bcast
>  #2
> -        test_packet $s 010000000000 f000000000$s ${s}ff $bcast
>  #2
> -        if test $js = 1; then
> -            bcast_impersonate=$bcast
> -        else
> -            bcast_impersonate=
> -        fi
> -        test_packet $s 010000000000 f00000000044 44ff $bcast_impersonate
>  #3
> -
> -        test_packet $s f0000000ffff f000000000$s ${s}66 $unknown
>  #6
> -
> -        test_packet $s ffffffffffff f000000000$s 1234                #8,
> acl1
> -        test_packet $s ffffffffffff f000000000$s 1235 $bacl2         #8,
> acl2
> -        test_packet $s ffffffffffff f000000000$s 1236 $bacl3         #8,
> acl3
> -        test_packet $s 010000000000 f000000000$s 1234                #8,
> acl1
> -        test_packet $s 010000000000 f000000000$s 1235 $bacl2         #8,
> acl2
> -        test_packet $s 010000000000 f000000000$s 1236 $bacl3         #8,
> acl3
> -    done
> -done
> -
> -# set address for lp13 with invalid characters.
> -# lp13 should be configured with only 192.168.0.13.
> -ovn-nbctl lsp-set-addresses lp13 "f0:00:00:00:00:13 192.168.0.13 invalid
> 192.169.0.13"
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -sip=`ip_to_hex 192 168 0 11`
> -tip=`ip_to_hex 192 168 0 13`
> -test_arp 11 f00000000011  $sip $tip f00000000013
> -
> -tip=`ip_to_hex 192 169 0 13`
> -#arp request for 192.169.0.13 should be flooded
> -test_arp 11 f00000000011  $sip $tip
> -
> -# dump information and flows with counters
> -ovn-sbctl dump-flows -- list multicast_group
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv3 dump ------"
> -as hv3 ovs-vsctl show
> -as hv3 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        OVN_CHECK_PACKETS([hv$i/vif$i$j-tx.pcap], [$i$j.expected])
> -    done
> -done
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -# 2 hypervisors, one logical switch, 2 logical ports per hypervisor
> -# logical ports bound to chassis encap-ip.
> -AT_SETUP([ovn -- 2 HVs, 1 LS, 2 lports/HV])
> -AT_KEYWORDS([ovnarp])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Create hypervisors hv[12].
> -# Add vif1[12] to hv1, vif2[12] to hv2
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2; do
> -        ovs-vsctl add-port br-int vif$i$j -- set Interface vif$i$j
> external-ids:iface-id=lp$i$j options:tx_pcap=hv$i/vif$i$j-tx.pcap
> options:rxq_pcap=hv$i/vif$i$j-rx.pcap ofport-request=$i$j
> -        ovn-nbctl lsp-add lsw0 lp$i$j
> -        ip_addrs="192.168.0.$i$j"
> -        ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j $ip_addrs"
> -        ovn-nbctl --wait=hv lsp-set-port-security lp$i$j
> f0:00:00:00:00:$i$j
> -    done
> -done
> -
> -get_lsp_uuid () {
> -    ovn-nbctl lsp-list lsw0 | grep $1 | awk '{ print $1 }'
> -}
> -
> -# XXX-Check how to pass lp$i1 in AT_CHECK_UNQUOTED, for now just do it
> -# explictly
> -
> -# For Chassis hv1
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp11],
> [0], [dnl
> -encap               : [[]]
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp12],
> [0], [dnl
> -encap               : [[]]
> -])
> -
> -# For Chassis hv2
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp21],
> [0], [dnl
> -encap               : [[]]
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp22],
> [0], [dnl
> -encap               : [[]]
> -])
> -
> -# Bind the ports to the encap-ip
> -for i in 1 2; do
> -    for j in 1 2; do
> -        as hv$i
> -        ovs-vsctl set Interface vif$i$j external-ids:encap-ip=192.168.0.$i
> -    done
> -done
> -
> -sleep 1
> -
> -# dump port bindings; since we have vxlan and geneve tunnels, we expect
> the
> -# ports to be bound to geneve tunnels.
> -
> -# For Chassis 1
> -encap_rec=`ovn-sbctl --data=bare --no-heading --column _uuid find encap
> chassis_name=hv1 type=geneve ip=192.168.0.1`
> -
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp11],
> [0], [dnl
> -encap               : ${encap_rec}
> -])
> -
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp12],
> [0], [dnl
> -encap               : ${encap_rec}
> -])
> -
> -# For Chassis 2
> -encap_rec=`ovn-sbctl --data=bare --no-heading --column _uuid find encap
> chassis_name=hv2 type=geneve ip=192.168.0.2`
> -
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp21],
> [0], [dnl
> -encap               : ${encap_rec}
> -])
> -
> -AT_CHECK_UNQUOTED([ovn-sbctl  --column encap list port_binding lp22],
> [0], [dnl
> -encap               : ${encap_rec}
> -])
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Make sure there is no attempt to adding duplicated flows by
> ovn-controller
> -AT_FAIL_IF([test -n "`grep duplicate hv1/ovn-controller.log`"])
> -AT_FAIL_IF([test -n "`grep duplicate hv2/ovn-controller.log`"])
> -AT_FAIL_IF([test -n "`grep duplicate hv3/ovn-controller.log`"])
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv${1%?}
> -}
> -
> -# test_packet INPORT DST SRC ETHTYPE OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 11 for
> vif11.
> -for i in 1 2; do
> -    for j in 1 2; do
> -        : > $i$j.expected
> -    done
> -done
> -test_packet() {
> -    local inport=$1 packet=$2$3$4; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast packets are delivered to exactly one logical switch port
> -#    (except that packets destined to their input ports are dropped).
> -
> -for is in 1 2; do
> -    for js in 1 2; do
> -        s=$is$js
> -        bcast=
> -        unknown=
> -        bacl2=
> -        bacl3=
> -        for id in 1 2 3; do
> -            for jd in 1 2 3; do
> -                d=$id$jd
> -
> -                if test $d != $s; then unicast=$d; else unicast=; fi
> -                test_packet $s f000000000$d f000000000$s $s$d $unicast
>  #1
> -            done
> -        done
> -
> -    done
> -done
> -
> -# dump information and flows with counters
> -ovn-sbctl dump-flows -- list multicast_group
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv3 dump ------"
> -as hv3 ovs-vsctl show
> -as hv3 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2; do
> -    for j in 1 2; do
> -        OVN_CHECK_PACKETS([hv$i/vif$i$j-tx.pcap], [$i$j.expected])
> -    done
> -done
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- trace 1 LS, 3 LSPs])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Create a logical switch and some logical ports.
> -# Turn on port security on all lports except ls1.
> -# Make ls1 a destination for unknown MACs.
> -# Add some ACLs for Ethertypes 1234, 1235, 1236.
> -ovn-nbctl ls-add lsw0
> -ovn-sbctl chassis-add hv0 geneve 127.0.0.1
> -for i in 1 2 3; do
> -    ovn-nbctl lsp-add lsw0 lp$i
> -done
> -ovn-nbctl --wait=sb sync
> -for i in 1 2 3; do
> -    ovn-sbctl lsp-bind lp$i hv0
> -    if test $i = 1; then
> -        ovn-nbctl lsp-set-addresses lp$i "f0:00:00:00:00:0$i
> 192.168.0.$i" unknown
> -    else
> -        if test $i = 3; then
> -           ip_addrs="192.168.0.$i fe80::ea2a:eaff:fe28:$i/64 192.169.0.$i"
> -        else
> -           ip_addrs="192.168.0.$i"
> -        fi
> -        ovn-nbctl lsp-set-addresses lp$i "f0:00:00:00:00:0$i $ip_addrs"
> -        ovn-nbctl lsp-set-port-security lp$i f0:00:00:00:00:0$i
> -    fi
> -done
> -ovn-nbctl acl-add lsw0 from-lport 1000 'eth.type == 0x1234' drop
> -ovn-nbctl acl-add lsw0 from-lport 1000 'eth.type == 0x1235 && inport ==
> "lp1"' drop
> -ovn-nbctl acl-add lsw0 to-lport 1000 'eth.type == 0x1236 && outport ==
> "lp3"' drop
> -ovn-nbctl create Address_Set name=set1
> addresses=\"f0:00:00:00:00:01\",\"f0:00:00:00:00:02\"
> -ovn-nbctl acl-add lsw0 to-lport 1000 'eth.type == 0x1237 && eth.src ==
> $set1 && outport == "lp3"' drop
> -
> -ovn-nbctl --wait=sb sync
> -on_exit 'kill `cat ovn-trace.pid`'
> -ovn-trace --detach --pidfile --no-chdir
> -
> -# test_packet INPORT DST SRC [-vlan] [-eth TYPE] OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 11 for
> vif11.
> -test_packet() {
> -    local inport=$1 eth_dst=$2 eth_src=$3; shift; shift; shift
> -    uflow="inport==\"lp$inport\" && eth.dst==$eth_dst &&
> eth.src==$eth_src"
> -    while :; do
> -        case $1 in # (
> -            -vlan) uflow="$uflow && vlan.vid == 1234"; shift ;; # (
> -            -eth) uflow="$uflow && eth.type == 0x$2"; shift; shift ;; # (
> -            *) break ;;
> -        esac
> -    done
> -    for outport; do
> -        echo "output(\"lp$outport\");"
> -    done > expout
> -
> -    AT_CAPTURE_FILE([trace])
> -    AT_CHECK([ovs-appctl -t ovn-trace trace --all lsw0 "$uflow" | tee
> trace | sed '1,/Minimal trace/d'], [0], [expout])
> -}
> -
> -# test_arp INPORT SHA SPA TPA [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local inport=$1 sha=$2 spa=$3 tpa=$4 reply_ha=$5
> -
> -    local request="inport == \"lp$inport\"
> -                   && eth.dst == ff:ff:ff:ff:ff:ff && eth.src == $sha
> -                   && arp.op == 1 && arp.sha == $sha && arp.spa == $spa
> -                   && arp.tha == ff:ff:ff:ff:ff:ff && arp.tpa == $tpa"
> -
> -    if test -z "$reply_ha"; then
> -        reply=
> -        local i
> -        for i in 1 2 3; do
> -            if test $i != $inport; then
> -                reply="${reply}output(\"lp$i\");
> -"
> -            fi
> -        done
> -    else
> -        reply="\
> -eth.dst = $sha;
> -eth.src = $reply_ha;
> -arp.op = 2;
> -arp.tha = $sha;
> -arp.sha = $reply_ha;
> -arp.tpa = $spa;
> -arp.spa = $tpa;
> -output(\"lp$inport\");
> -"
> -    fi
> -
> -    AT_CAPTURE_FILE([trace])
> -    AT_CHECK_UNQUOTED([ovs-appctl -t ovn-trace trace --all lsw0
> "$request" | tee trace | sed '1,/Minimal trace/d'], [0], [$reply])
> -}
> -
> -# Send packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast packets are delivered to exactly one logical switch port
> -#    (except that packets destined to their input ports are dropped).
> -#
> -# 2. Broadcast and multicast are delivered to all logical switch ports
> -#    except the input port.
> -#
> -# 3. When port security is turned on, the switch drops packets from the
> wrong
> -#    MAC address.
> -#
> -# 4. The switch drops all packets with a VLAN tag.
> -#
> -# 5. The switch drops all packets with a multicast source address.  (This
> only
> -#    affects behavior when port security is turned off, since otherwise
> port
> -#    security would drop the packet anyway.)
> -#
> -# 6. The switch delivers packets with an unknown destination to logical
> -#    switch ports with "unknown" among their MAC addresses (and port
> -#    security disabled).
> -#
> -# 7. The switch drops unicast packets that violate an ACL.
> -#
> -# 8. The switch drops multicast and broadcast packets that violate an ACL.
> -#
> -# 9. OVN generates responses to ARP requests for known IPs, except for
> -#    requests from a port for the port's own IP.
> -#
> -# 10. No response to ARP requests for unknown IPs.
> -
> -for s in 1 2 3; do
> -    bcast=
> -    unknown=
> -    bacl2=
> -    bacl3=
> -    for d in 1 2 3; do
> -        echo
> -        echo "lp$s -> lp$d"
> -        if test $d != $s; then unicast=$d; else unicast=; fi
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:0$s $unicast
>   #1
> -
> -        if test $d != $s && test $s = 1; then
> -            impersonate=$d
> -        else
> -            impersonate=
> -        fi
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:55 $impersonate
>  #3
> -
> -        if test $d != $s && test $s != 1; then acl2=$d; else acl2=; fi
> -        if test $d != $s && test $d != 3; then acl3=$d; else acl3=; fi
> -        if test $d = $s || ( (test $s = 1 || test $s = 2) && test $d =
> 3); then
> -            # Source of 1 or 2 and dest of 3 should be dropped
> -            # due to the 4th ACL that uses address_set(set1).
> -            acl4=
> -        else
> -            acl4=$d
> -        fi
> -
> -        #7, acl1 to acl4:
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:0$s -eth 1234
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:0$s -eth 1235
> $acl2
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:0$s -eth 1236
> $acl3
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:0$s -eth 1237
> $acl4
> -
> -        test_packet $s f0:00:00:00:00:0$d f0:00:00:00:00:55 -vlan
>   #4
> -        test_packet $s f0:00:00:00:00:0$d 01:00:00:00:00:0$s
>  #5
> -
> -        if test $d != $s && test $d = 1; then
> -            unknown="$unknown $d"
> -        fi
> -        bcast="$bcast $unicast"
> -        bacl2="$bacl2 $acl2"
> -        bacl3="$bacl3 $acl3"
> -
> -        sip=192.168.0.$s
> -        tip=192.168.0.$d
> -        tip_unknown=11.11.11.11
> -        if test $d != $s; then reply_ha=f0:00:00:00:00:0$d; else
> reply_ha=; fi
> -        test_arp $s f0:00:00:00:00:0$s $sip $tip $reply_ha
>  #9
> -        test_arp $s f0:00:00:00:00:0$s $sip $tip_unknown
>  #10
> -
> -        if test $d = 3; then
> -            # lp3 has an additional ip 192.169.0.[123]3.
> -            tip=192.169.0.$d
> -            test_arp $s f0:00:00:00:00:0$s $sip $tip $reply_ha
>  #9
> -        fi
> -    done
> -
> -    # Broadcast and multicast.
> -    test_packet $s ff:ff:ff:ff:ff:ff f0:00:00:00:00:0$s $bcast
>  #2
> -    test_packet $s 01:00:00:00:00:00 f0:00:00:00:00:0$s $bcast
>  #2
> -    if test $s = 1; then
> -       bcast_impersonate=$bcast
> -    else
> -       bcast_impersonate=
> -    fi
> -    test_packet $s 01:00:00:00:00:00 f0:00:00:00:00:44
> $bcast_impersonate  #3
> -
> -    test_packet $s f0:00:00:00:ff:ff f0:00:00:00:00:0$s $unknown
>  #6
> -
> -    #8, acl1 to acl3:
> -    test_packet $s ff:ff:ff:ff:ff:ff f0:00:00:00:00:0$s -eth 1234
> -    test_packet $s ff:ff:ff:ff:ff:ff f0:00:00:00:00:0$s -eth 1235 $bacl2
> -    test_packet $s ff:ff:ff:ff:ff:ff f0:00:00:00:00:0$s -eth 1236 $bacl3
> -
> -    #8, acl1 to acl3:
> -    test_packet $s 01:00:00:00:00:00 f0:00:00:00:00:0$s -eth 1234
> -    test_packet $s 01:00:00:00:00:00 f0:00:00:00:00:0$s -eth 1235 $bacl2
> -    test_packet $s 01:00:00:00:00:00 f0:00:00:00:00:0$s -eth 1236 $bacl3
> -done
> -
> -AT_CLEANUP
> -
> -# 2 hypervisors, 4 logical ports per HV
> -# 2 locally attached networks (one flat, one vlan tagged over same device)
> -# 2 ports per HV on each network
> -AT_SETUP([ovn -- 2 HVs, 4 lports/HV, localnet ports])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# In this test cases we create 3 switches, all connected to same
> -# physical network (through br-phys on each HV). Each switch has
> -# VIF ports across 2 HVs. Each HV has 5 VIF ports. The first digit
> -# of VIF port name indicates the hypervisor it is bound to, e.g.
> -# lp23 means VIF 3 on hv2.
> -#
> -# Each switch's VLAN tag and their logical switch ports are:
> -#   - ls1:
> -#       - untagged
> -#       - ports: lp11, lp12, lp21, lp22
> -#
> -#   - ls2:
> -#       - tagged with VLAN 101
> -#       - ports: lp13, lp14, lp23, lp24
> -#   - ls3:
> -#       - untagged
> -#       - ports: lp15, lp25
> -#
> -# Note: a localnet port is created for each switch to connect to
> -# physical network.
> -
> -for i in 1 2 3; do
> -    ls_name=ls$i
> -    ovn-nbctl ls-add $ls_name
> -    ln_port_name=ln$i
> -    if test $i -eq 2; then
> -        ovn-nbctl lsp-add $ls_name $ln_port_name "" 101
> -    else
> -        ovn-nbctl lsp-add $ls_name $ln_port_name
> -    fi
> -    ovn-nbctl lsp-set-addresses $ln_port_name unknown
> -    ovn-nbctl lsp-set-type $ln_port_name localnet
> -    ovn-nbctl lsp-set-options $ln_port_name network_name=phys
> -done
> -
> -# lsp_to_ls LSP
> -#
> -# Prints the name of the logical switch that contains LSP.
> -lsp_to_ls () {
> -    case $1 in dnl (
> -        lp?[[12]]) echo ls1 ;; dnl (
> -        lp?[[34]]) echo ls2 ;; dnl (
> -        lp?5) echo ls3 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2 3 4 5; do
> -        ovs-vsctl add-port br-int vif$i$j -- \
> -            set Interface vif$i$j external-ids:iface-id=lp$i$j \
> -                                  options:tx_pcap=hv$i/vif$i$j-tx.pcap \
> -                                  options:rxq_pcap=hv$i/vif$i$j-rx.pcap \
> -                                  ofport-request=$i$j
> -
> -        lsp_name=lp$i$j
> -        ls_name=$(lsp_to_ls $lsp_name)
> -
> -        ovn-nbctl lsp-add $ls_name $lsp_name
> -        ovn-nbctl lsp-set-addresses $lsp_name f0:00:00:00:00:$i$j
> -        ovn-nbctl lsp-set-port-security $lsp_name f0:00:00:00:00:$i$j
> -
> -        OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up $lsp_name` = xup])
> -    done
> -done
> -ovn-nbctl --wait=sb sync
> -ovn-sbctl dump-flows
> -
> -OVN_POPULATE_ARP
> -
> -# XXX This is now the 3rd copy of these functions in this file ...
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv${1%?}
> -}
> -#
> -# test_packet INPORT DST SRC ETHTYPE EOUT LOUT
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  INPORT is specified
> as
> -# logical switch port numbers, e.g. 11 for vif11.
> -#
> -# EOUT is the end-to-end output port, that is, where the packet will end
> up
> -# after possibly bouncing through one or more localnet ports.  LOUT is the
> -# logical output port, which might be a localnet port, as seen by
> ovn-trace
> -# (which doesn't know what localnet ports are connected to and therefore
> can't
> -# figure out the end-to-end answer).
> -for i in 1 2; do
> -    for j in 1 2 3 4 5; do
> -        : > $i$j.expected
> -    done
> -done
> -test_packet() {
> -    local inport=$1 dst=$2 src=$3 eth=$4 eout=$5 lout=$6
> -    echo "$@"
> -
> -    # First try tracing the packet.
> -    uflow="inport==\"lp$inport\" && eth.dst==$dst && eth.src==$src &&
> eth.type==0x$eth"
> -    if test $lout != drop; then
> -        echo "output(\"$lout\");"
> -    fi > expout
> -    AT_CAPTURE_FILE([trace])
> -    AT_CHECK([ovn-trace --all $(lsp_to_ls lp$inport) "$uflow" | tee trace
> | sed '1,/Minimal trace/d'], [0], [expout])
> -
> -    # Then actually send a packet, for an end-to-end test.
> -    local packet=$(echo $dst$src | sed 's/://g')${eth}
> -    hv=`vif_to_hv $inport`
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    if test $eout != drop; then
> -        echo $packet >> ${eout#lp}.expected
> -    fi
> -}
> -
> -# lp11 and lp21 are on the same network (phys, untagged)
> -# and on different hypervisors
> -test_packet 11 f0:00:00:00:00:21 f0:00:00:00:00:11 1121 lp21 lp21
> -test_packet 21 f0:00:00:00:00:11 f0:00:00:00:00:21 2111 lp11 lp11
> -
> -# lp11 and lp12 are on the same network (phys, untagged)
> -# and on the same hypervisor
> -test_packet 11 f0:00:00:00:00:12 f0:00:00:00:00:11 1112 lp12 lp12
> -test_packet 12 f0:00:00:00:00:11 f0:00:00:00:00:12 1211 lp11 lp11
> -
> -# lp13 and lp23 are on the same network (phys, VLAN 101)
> -# and on different hypervisors
> -test_packet 13 f0:00:00:00:00:23 f0:00:00:00:00:13 1323 lp23 lp23
> -test_packet 23 f0:00:00:00:00:13 f0:00:00:00:00:23 2313 lp13 lp13
> -
> -# lp13 and lp14 are on the same network (phys, VLAN 101)
> -# and on the same hypervisor
> -test_packet 13 f0:00:00:00:00:14 f0:00:00:00:00:13 1314 lp14 lp14
> -test_packet 14 f0:00:00:00:00:13 f0:00:00:00:00:14 1413 lp13 lp13
> -
> -# lp11 and lp15 are on the same network (phys, untagged),
> -# same hypervisor, and on different switches
> -test_packet 11 f0:00:00:00:00:15 f0:00:00:00:00:11 1115 lp15 ln1
> -test_packet 15 f0:00:00:00:00:11 f0:00:00:00:00:15 1511 lp11 ln3
> -
> -# lp11 and lp25 are on the same network (phys, untagged),
> -# different hypervisors, and on different switches
> -test_packet 11 f0:00:00:00:00:25 f0:00:00:00:00:11 1125 lp25 ln1
> -test_packet 25 f0:00:00:00:00:11 f0:00:00:00:00:25 2511 lp11 ln3
> -
> -# Ports that should not be able to communicate
> -test_packet 11 f0:00:00:00:00:13 f0:00:00:00:00:11 1113 drop ln1
> -test_packet 11 f0:00:00:00:00:23 f0:00:00:00:00:11 1123 drop ln1
> -test_packet 21 f0:00:00:00:00:13 f0:00:00:00:00:21 2113 drop ln1
> -test_packet 21 f0:00:00:00:00:23 f0:00:00:00:00:21 2123 drop ln1
> -test_packet 13 f0:00:00:00:00:11 f0:00:00:00:00:13 1311 drop ln2
> -test_packet 13 f0:00:00:00:00:21 f0:00:00:00:00:13 1321 drop ln2
> -test_packet 23 f0:00:00:00:00:11 f0:00:00:00:00:23 2311 drop ln2
> -test_packet 23 f0:00:00:00:00:21 f0:00:00:00:00:23 2321 drop ln2
> -
> -# Dump a bunch of info helpful for debugging if there's a failure.
> -
> -echo "------ OVN dump ------"
> -ovn-nbctl show
> -ovn-sbctl show
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2; do
> -    for j in 1 2 3 4 5; do
> -        OVN_CHECK_PACKETS([hv$i/vif$i$j-tx.pcap], [$i$j.expected])
> -    done
> -done
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- vtep: 3 HVs, 1 VIFs/HV, 1 GW, 1 LS])
> -AT_KEYWORDS([vtep])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Configure the Northbound database
> -ovn-nbctl ls-add lsw0
> -
> -ovn-nbctl lsp-add lsw0 lp1
> -ovn-nbctl lsp-set-addresses lp1 f0:00:00:00:00:01
> -
> -ovn-nbctl lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp2 f0:00:00:00:00:02
> -
> -ovn-nbctl lsp-add lsw0 lp-vtep
> -ovn-nbctl lsp-set-type lp-vtep vtep
> -ovn-nbctl lsp-set-options lp-vtep vtep-physical-switch=br-vtep
> vtep-logical-switch=lsw0
> -ovn-nbctl lsp-set-addresses lp-vtep unknown
> -
> -# lpr, lr and lrp1 are used for the ARP request handling test only.
> -ovn-nbctl lsp-add lsw0 lpr
> -ovn-nbctl lr-add lr
> -ovn-nbctl lrp-add lr lrp1 f0:00:00:00:00:f1 192.168.1.1/24
> -ovn-nbctl set Logical_Switch_Port lpr type=router \
> -                             options:router-port=lrp1 \
> -    addresses='"f0:00:00:00:00:f1 192.168.1.1"'
> -
> -
> -net_add n1               # Network to connect hv1, hv2, and vtep
> -net_add n2               # Network to connect vtep and hv3
> -
> -# Create hypervisor hv1 connected to n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1 options:tx_pcap=hv1/vif1-tx.pcap
> options:rxq_pcap=hv1/vif1-rx.pcap ofport-request=1
> -
> -# Create hypervisor hv2 connected to n1
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl add-port br-int vif2 -- set Interface vif2
> external-ids:iface-id=lp2 options:tx_pcap=hv2/vif2-tx.pcap
> options:rxq_pcap=hv2/vif2-rx.pcap ofport-request=1
> -
> -
> -# Start the vtep emulator with a leg in both networks
> -sim_add vtep
> -as vtep
> -
> -ovsdb-tool create "$ovs_base"/vtep/vtep.db
> "$abs_top_srcdir"/vtep/vtep.ovsschema || return 1
> -ovs-appctl -t ovsdb-server ovsdb-server/add-db "$ovs_base"/vtep/vtep.db
> -
> -ovs-vsctl add-br br-phys
> -net_attach n1 br-phys
> -
> -mac=`ovs-vsctl get Interface br-phys mac_in_use | sed s/\"//g`
> -arp_table="$arp_table $sandbox,br-phys,192.168.0.3,$mac"
> -ovs-appctl netdev-dummy/ip4addr br-phys 192.168.0.3/24 >/dev/null ||
> return 1
> -ovs-appctl ovs/route/add 192.168.0.3/24 br-phys >/dev/null || return 1
> -
> -ovs-vsctl add-br br-vtep
> -net_attach n2 br-vtep
> -
> -vtep-ctl add-ps br-vtep
> -vtep-ctl set Physical_Switch br-vtep tunnel_ips=192.168.0.3
> -vtep-ctl add-ls lsw0
> -
> -start_daemon ovs-vtep br-vtep
> -start_daemon ovn-controller-vtep --vtep-db=unix:"$ovs_base"/vtep/db.sock
> --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -
> -OVS_WAIT_UNTIL([vtep-ctl bind-ls br-vtep br-vtep_n2 0 lsw0])
> -
> -OVS_WAIT_UNTIL([test -n "`as vtep vtep-ctl get-replication-mode lsw0 |
> -               grep -- source`"])
> -# It takes more time for the update to be processed by ovs-vtep.
> -sleep 1
> -
> -# Add hv3 on the other side of the vtep
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -net_attach n2 br-phys
> -
> -ovs-vsctl add-port br-phys vif3 -- set Interface vif3
> options:tx_pcap=hv3/vif3-tx.pcap options:rxq_pcap=hv3/vif3-rx.pcap
> ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# test_packet INPORT DST SRC ETHTYPE OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 1 for vif1.
> -for i in 1 2 3; do
> -    : > $i.expected
> -done
> -test_packet() {
> -    local inport=$1 packet=$2$3$4; shift; shift; shift; shift
> -    #hv=hv`echo $inport | sed 's/^\(.\).*/\1/'`
> -    hv=hv$inport
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -# Send packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast packets are delivered to exactly one logical switch port
> -#    (except that packets destined to their input ports are dropped).
> -#
> -# 2. Broadcast and multicast are delivered to all logical switch ports
> -#    except the input port.
> -#
> -# 3. The switch delivers packets with an unknown destination to logical
> -#    switch ports with "unknown" among their MAC addresses (and port
> -#    security disabled).
> -for s in 1 2 3; do
> -    bcast=
> -    unknown=
> -    for d in 1 2 3; do
> -        if test $d != $s; then unicast=$d; else unicast=; fi
> -        test_packet $s f0000000000$d f0000000000$s 00$s$d $unicast
>  #1
> -
> -        # The vtep (vif3) is the only one configured for "unknown"
> -        if test $d != $s && test $d = 3; then
> -            unknown="$unknown $d"
> -        fi
> -        bcast="$bcast $unicast"
> -    done
> -
> -    # Broadcast and multicast.
> -    test_packet $s ffffffffffff f0000000000$s 0${s}ff $bcast
>  #2
> -    test_packet $s 010000000000 f0000000000$s 0${s}ff $bcast
>  #2
> -
> -    test_packet $s f0000000ffff f0000000000$s 0${s}66 $unknown
>  #3
> -done
> -
> -# ARP request should not be responded to by logical switch router
> -# type arp responder on HV1 and HV2 and should reach directly to
> -# vif1 and vif2
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -sha=f00000000003
> -spa=`ip_to_hex 192 168 1 2`
> -tpa=`ip_to_hex 192 168 1 1`
>
> -request=ffffffffffff${sha}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -as hv3 ovs-appctl netdev-dummy/receive vif3 $request
> -echo $request >> 1.expected
> -echo $request >> 2.expected
> -
> -# dump information with counters
> -echo "------ OVN dump ------"
> -ovn-nbctl show
> -ovn-sbctl show
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 show br-int
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-ofctl -O OpenFlow13 show br-int
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv3 dump ------"
> -as hv3 ovs-vsctl show
> -# note: hv3 has no logical port bind, thus it should not have br-int
> -AT_CHECK([as hv3 ovs-ofctl -O OpenFlow13 show br-int], [1], [],
> -[ovs-ofctl: br-int is not a bridge or a socket
> -])
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    OVN_CHECK_PACKETS([hv$i/vif$i-tx.pcap], [$i.expected])
> -done
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1],[hv2],[vtep])
> -OVN_CLEANUP_VSWITCH([hv3])
> -
> -AT_CLEANUP
> -
> -# Similar test to "hardware GW"
> -AT_SETUP([ovn -- 3 HVs, 1 VIFs/HV, 1 software GW, 1 LS])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Configure the Northbound database
> -ovn-nbctl ls-add lsw0
> -
> -ovn-nbctl lsp-add lsw0 lp1
> -ovn-nbctl lsp-set-addresses lp1 f0:00:00:00:00:01
> -
> -ovn-nbctl lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp2 f0:00:00:00:00:02
> -
> -ovn-nbctl lsp-add lsw0 lp-gw
> -ovn-nbctl lsp-set-type lp-gw l2gateway
> -ovn-nbctl lsp-set-options lp-gw network_name=physnet1
> l2gateway-chassis=hv_gw
> -ovn-nbctl lsp-set-addresses lp-gw unknown
> -
> -net_add n1               # Network to connect hv1, hv2, and gw
> -net_add n2               # Network to connect gw and hv3
> -
> -# Create hypervisor hv1 connected to n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1 options:tx_pcap=hv1/vif1-tx.pcap
> options:rxq_pcap=hv1/vif1-rx.pcap ofport-request=1
> -
> -# Create hypervisor hv2 connected to n1
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl add-port br-int vif2 -- set Interface vif2
> external-ids:iface-id=lp2 options:tx_pcap=hv2/vif2-tx.pcap
> options:rxq_pcap=hv2/vif2-rx.pcap ofport-request=1
> -
> -# Create hypervisor hv_gw connected to n1 and n2
> -# connect br-phys bridge to n1; connect hv-gw bridge to n2
> -sim_add hv_gw
> -as hv_gw
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl add-br br-phys2
> -net_attach n2 br-phys2
> -ovs-vsctl set open . external_ids:ovn-bridge-mappings="physnet1:br-phys2"
> -
> -# Add hv3 on the other side of the GW
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -net_attach n2 br-phys
> -ovs-vsctl add-port br-phys vif3 -- set Interface vif3
> options:tx_pcap=hv3/vif3-tx.pcap options:rxq_pcap=hv3/vif3-rx.pcap
> ofport-request=1
> -
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# test_packet INPORT DST SRC ETHTYPE OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as lport numbers, e.g. 1 for vif1.
> -for i in 1 2 3; do
> -    : > $i.expected
> -done
> -test_packet() {
> -    local inport=$1 packet=$2$3$4; shift; shift; shift; shift
> -    #hv=hv`echo $inport | sed 's/^\(.\).*/\1/'`
> -    hv=hv$inport
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -# Send packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast packets are delivered to exactly one lport (except that
> packets
> -#    destined to their input ports are dropped).
> -#
> -# 2. Broadcast and multicast are delivered to all lports except the input
> port.
> -#
> -# 3. The lswitch delivers packets with an unknown destination to lports
> with
> -#    "unknown" among their MAC addresses (and port security disabled).
> -for s in 1 2 3 ; do
> -    bcast=
> -    unknown=
> -    for d in 1 2 3 ; do
> -        if test $d != $s; then unicast=$d; else unicast=; fi
> -        test_packet $s f0000000000$d f0000000000$s 00$s$d $unicast
>  #1
> -
> -        # The vtep (vif3) is the only one configured for "unknown"
> -        if test $d != $s && test $d = 3; then
> -            unknown="$unknown $d"
> -        fi
> -        bcast="$bcast $unicast"
> -    done
> -
> -    test_packet $s ffffffffffff f0000000000$s 0${s}ff $bcast
>  #2
> -    test_packet $s 010000000000 f0000000000$s 0${s}ff $bcast
>  #3
> -    test_packet $s f0000000ffff f0000000000$s 0${s}66 $unknown
>  #4
> -done
> -
> -echo "------ ovn-nbctl show ------"
> -ovn-nbctl show
> -echo "------ ovn-sbctl show ------"
> -ovn-sbctl show
> -
> -echo "------ hv1 ------"
> -as hv1 ovs-vsctl show
> -echo "------ hv1 br-int ------"
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -echo "------ hv1 br-phys ------"
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-phys
> -
> -echo "------ hv2 ------"
> -as hv2 ovs-vsctl show
> -echo "------ hv2 br-int ------"
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -echo "------ hv2 br-phys ------"
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-phys
> -
> -echo "------ hv_gw ------"
> -as hv_gw ovs-vsctl show
> -echo "------ hv_gw br-phys ------"
> -as hv_gw ovs-ofctl -O OpenFlow13 dump-flows br-phys
> -echo "------ hv_gw br-phys2 ------"
> -as hv_gw ovs-ofctl -O OpenFlow13 dump-flows br-phys2
> -
> -echo "------ hv3 ------"
> -as hv3 ovs-vsctl show
> -echo "------ hv3 br-phys ------"
> -as hv3 ovs-ofctl -O OpenFlow13 dump-flows br-phys
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    OVN_CHECK_PACKETS([hv$i/vif$i-tx.pcap], [$i.expected])
> -done
> -AT_CLEANUP
> -
> -# 3 hypervisors, 3 logical switches with 3 logical ports each, 1 logical
> router
> -AT_SETUP([ovn -- 3 HVs, 3 LS, 3 lports/LS, 1 LR])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -#
> -# Three logical switches ls1, ls2, ls3.
> -# One logical router lr0 connected to ls[123],
> -# with nine subnets, three per logical switch:
> -#
> -#    lrp11 on ls1 for subnet 192.168.11.0/24
> -#    lrp12 on ls1 for subnet 192.168.12.0/24
> -#    lrp13 on ls1 for subnet 192.168.13.0/24
> -#    ...
> -#    lrp33 on ls3 for subnet 192.168.33.0/24
> -#
> -# 27 VIFs, 9 per LS, 3 per subnet: lp[123][123][123], where the first two
> -# digits are the subnet and the last digit distinguishes the VIF.
> -for i in 1 2 3; do
> -    ovn-nbctl ls-add ls$i
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            # Add "unknown" to MAC addresses for lp?11, so packets for
> -            # MAC-IP bindings discovered via ARP later have somewhere to
> go.
> -            if test $j$k = 11; then unknown=unknown; else unknown=; fi
> -
> -            ovn-nbctl \
> -                -- lsp-add ls$i lp$i$j$k \
> -                -- lsp-set-addresses lp$i$j$k \
> -                   "f0:00:00:00:0$i:$j$k 192.168.$i$j.$k" $unknown
> -        done
> -    done
> -done
> -
> -ovn-nbctl lr-add lr0
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        ovn-nbctl lrp-add lr0 lrp$i$j 00:00:00:00:ff:$i$j
> 192.168.$i$j.254/24
> -        ovn-nbctl \
> -            -- lsp-add ls$i lrp$i$j-attachment \
> -            -- set Logical_Switch_Port lrp$i$j-attachment type=router \
> -                             options:router-port=lrp$i$j \
> -                             addresses='"00:00:00:00:ff:'$i$j'"'
> -    done
> -done
> -
> -ovn-nbctl set Logical_Switch_Port lrp33-attachment \
> -    addresses='"00:00:00:00:ff:33 192.168.33.254"'
> -
> -# Physical network:
> -#
> -# Three hypervisors hv[123].
> -# lp?1[123] spread across hv[123]: lp?11 on hv1, lp?12 on hv2, lp?13 on
> hv3.
> -# lp?2[123] spread across hv[23]: lp?21 and lp?22 on hv2, lp?23 on hv3.
> -# lp?3[123] all on hv3.
> -
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    case $1 in dnl (
> -        ?11) echo 1 ;; dnl (
> -        ?12 | ?21 | ?22) echo 2 ;; dnl (
> -        ?13 | ?23 | ?3?) echo 3 ;;
> -    esac
> -}
> -
> -# Given the name of a logical port, prints the name of its logical router
> -# port, e.g. "vif_to_lrp 123" yields 12.
> -vif_to_lrp() {
> -    echo ${1%?}
> -}
> -
> -# Given the name of a logical port, prints the name of its logical
> -# switch, e.g. "vif_to_ls 123" yields 1.
> -vif_to_ls() {
> -    echo ${1%??}
> -}
> -
> -net_add n1
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -done
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            hv=`vif_to_hv $i$j$k`
> -                as hv$hv ovs-vsctl \
> -                -- add-port br-int vif$i$j$k \
> -                -- set Interface vif$i$j$k \
> -                    external-ids:iface-id=lp$i$j$k \
> -                    options:tx_pcap=hv$hv/vif$i$j$k-tx.pcap \
> -                    options:rxq_pcap=hv$hv/vif$i$j$k-rx.pcap \
> -                    ofport-request=$i$j$k
> -        done
> -    done
> -done
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 123 for
> vif123.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            : > $i$j$k.expected
> -        done
> -    done
> -done
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    in_ls=`vif_to_ls $inport`
> -    in_lrp=`vif_to_lrp $inport`
> -    for outport; do
> -        out_ls=`vif_to_ls $outport`
> -        if test $in_ls = $out_ls; then
> -            # Ports on the same logical switch receive exactly the same
> packet.
> -            echo $packet
> -        else
> -            # Routing decrements TTL and updates source and dest MAC
> -            # (and checksum).
> -            out_lrp=`vif_to_lrp $outport`
> -            echo
> f00000000${outport}00000000ff${out_lrp}08004500001c00000000"3f1101"00${src_ip}${dst_ip}0035111100080000
> -        fi >> $outport.expected
> -    done
> -}
> -
> -# test_arp INPORT SHA SPA TPA [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local inport=$1 sha=$2 spa=$3 tpa=$4 reply_ha=$5
> -    local
> request=ffffffffffff${sha}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $request
> -    as $hv ovs-appctl ofproto/trace br-int in_port=$inport $request
> -
> -    # Expect to receive the broadcast ARP on the other logical switch
> ports if
> -    # IP address is not configured to the switch patch port.
> -    local i=`vif_to_ls $inport`
> -    local j k
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            # 192.168.33.254 is configured to the switch patch port for
> lrp33,
> -            # so no ARP flooding expected for it.
> -            if test $i$j$k != $inport && test $tpa != `ip_to_hex 192 168
> 33 254`; then
> -                echo $request >> $i$j$k.expected
> -            fi
> -        done
> -    done
> -
> -    # Expect to receive the reply, if any.
> -    if test X$reply_ha != X; then
> -        lrp=`vif_to_lrp $inport`
> -        local
> reply=${sha}00000000ff${lrp}08060001080006040002${reply_ha}${tpa}${sha}${spa}
> -        echo $reply >> $inport.expected
> -    fi
> -}
> -
> -as hv1 ovs-vsctl --columns=name,ofport list interface
> -as hv1 ovn-sbctl list port_binding
> -as hv1 ovn-sbctl list datapath_binding
> -as hv1 ovn-sbctl dump-flows
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Send IP packets between all pairs of source and destination ports:
> -#
> -# 1. Unicast IP packets are delivered to exactly one logical switch port
> -#    (except that packets destined to their input ports are dropped).
> -#
> -# 2. Broadcast IP packets are delivered to all logical switch ports
> -#    except the input port.
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -for is in 1 2 3; do
> -  for js in 1 2 3; do
> -    for ks in 1 2 3; do
> -      bcast=
> -      s=$is$js$ks
> -      smac=f00000000$s
> -      sip=`ip_to_hex 192 168 $is$js $ks`
> -      for id in 1 2 3; do
> -          for jd in 1 2 3; do
> -              for kd in 1 2 3; do
> -                d=$id$jd$kd
> -                dip=`ip_to_hex 192 168 $id$jd $kd`
> -                if test $is = $id; then dmac=f00000000$d; else
> dmac=00000000ff$is$js; fi
> -                if test $d != $s; then unicast=$d; else unicast=; fi
> -
> -                test_ip $s $smac $dmac $sip $dip $unicast #1
> -
> -                if test $id = $is && test $d != $s; then bcast="$bcast
> $d"; fi
> -              done
> -          done
> -        done
> -      test_ip $s $smac ffffffffffff $sip ffffffff $bcast #2
> -      done
> -  done
> -done
> -
> -: > mac_bindings.expected
> -
> -# 3. Send an IP packet from every logical port to every other subnet,
> -#    to an IP address that does not have a static IP-MAC binding.
> -#    This should generate a broadcast ARP request for the destination
> -#    IP address in the destination subnet.
> -#    Moreover generate an ARP reply for each of the IP addresses ARPed
> -for is in 1 2 3; do
> -  for js in 1 2 3; do
> -    for ks in 1 2 3; do
> -      s=$is$js$ks
> -      smac=f00000000$s
> -      sip=`ip_to_hex 192 168 $is$js $ks`
> -      for id in 1 2 3; do
> -        for jd in 1 2 3; do
> -          if test $is$js = $id$jd; then
> -            continue
> -          fi
> -
> -          # Send the packet.
> -          dmac=00000000ff$is$js
> -          # Calculate a 4th octet for the destination that is
> -          # unique per $s, avoids the .1 .2 .3 and .254 IP addresses
> -          # that have static MAC bindings, and fits in the range
> -          # 0-255.
> -          o4=`expr $is '*' 9 + $js '*' 3 + $ks + 10`
> -          dip=`ip_to_hex 192 168 $id$jd $o4`
> -          test_ip $s $smac $dmac $sip $dip
> -
> -          # Every LP on the destination subnet's lswitch should
> -          # receive the ARP request.
> -          lrmac=00000000ff$id$jd
> -          lrip=`ip_to_hex 192 168 $id$jd 254`
> -
> arp=ffffffffffff${lrmac}08060001080006040001${lrmac}${lrip}000000000000${dip}
> -          for jd2 in 1 2 3; do
> -            for kd in 1 2 3; do
> -              echo $arp >> $id$jd2$kd.expected
> -            done
> -          done
> -
> -          hmac=8000000000$o4
> -          rmac=00000000ff$id$jd
> -          echo
> ${hmac}${rmac}08004500001c00000000"3f1101"00${sip}${dip}0035111100080000 >>
> ${id}11.expected
> -
> -          host_mac=8000000000$o4
> -          lrmac=00000000ff$id$jd
> -
> -
> arp_reply=${lrmac}${host_mac}08060001080006040002${host_mac}${dip}${lrmac}${lrip}
> -
> -          hv=hv`vif_to_hv ${id}${jd}1`
> -          as $hv ovs-appctl netdev-dummy/receive vif${id}${jd}1 $arp_reply
> -
> -          host_ip_pretty=192.168.$id$jd.$o4
> -          host_mac_pretty=80:00:00:00:00:$o4
> -          echo lrp$id$jd,$host_ip_pretty,$host_mac_pretty >>
> mac_bindings.expected
> -        done
> -      done
> -    done
> -  done
> -done
> -
> -# Test router replies to ARP requests from all source ports:
> -#
> -# 4. Router replies to query for its MAC address from port's own IP
> address.
> -#
> -# 5. Router replies to query for its MAC address from any random IP
> address
> -#    in its subnet.
> -#
> -# 6. No reply to query for IP address other than router IP.
> -#
> -# 7. No reply to query from another subnet.
> -for i in 1 2 3; do
> -  for j in 1 2 3; do
> -    for k in 1 2 3; do
> -      smac=f00000000$i$j$k               # Source MAC
> -      sip=`ip_to_hex 192 168 $i$j $k`    # Source IP
> -      rip=`ip_to_hex 192 168 $i$j 254`   # Router IP
> -      rmac=00000000ff$i$j                # Router MAC
> -      otherip=`ip_to_hex 192 168 $i$j 55` # Some other IP in subnet
> -      externalip=`ip_to_hex 1 2 3 4`      # Some other IP not in subnet
> -
> -      test_arp $i$j$k $smac $sip        $rip        $rmac      #4
> -      test_arp $i$j$k $smac $otherip    $rip        $rmac      #5
> -      test_arp $i$j$k $smac $sip        $otherip               #6
> -
> -      # When rip is 192.168.33.254, ARP request from externalip won't be
> -      # filtered, because 192.168.33.254 is configured to switch peer port
> -      # for lrp33.
> -      lrp33_rsp=
> -      if test $i = 3 && test $j = 3; then
> -        lrp33_rsp=$rmac
> -      fi
> -      test_arp $i$j$k $smac $externalip $rip        $lrp33_rsp #7
> -
> -      # MAC binding should be learned from ARP request.
> -      host_mac_pretty=f0:00:00:00:0$i:$j$k
> -
> -      host_ip_pretty=192.168.$i$j.$k
> -      echo lrp$i$j,$host_ip_pretty,$host_mac_pretty >>
> mac_bindings.expected
> -
> -      # mac_binding is learned and overwritten so only the last one
> remains.
> -      if test $k = 3; then
> -          # lrp33 will not learn from ARP request, because 192.168.33.254
> is
> -          # configured to switch peer port for lrp33.
> -          if test $i != 3 || test $j != 3; then
> -              host_ip_pretty=192.168.$i$j.55
> -              echo lrp$i$j,$host_ip_pretty,$host_mac_pretty >>
> mac_bindings.expected
> -          fi
> -      fi
> -
> -    done
> -  done
> -done
> -
> -
> -# Allow some time for packet forwarding.
> -# XXX This can be improved.
> -sleep 1
> -
> -# 8. Send an IP packet from every logical port to every other subnet.
> These
> -#    are the same packets already sent as #3, but now the destinations'
> IP-MAC
> -#    bindings have been discovered via ARP, so instead of provoking an ARP
> -#    request, these packets now get routed to their destinations (which
> don't
> -#    have static MAC bindings, so they go to the port we've designated as
> -#    accepting "unknown" MACs.)
> -for is in 1 2 3; do
> -  for js in 1 2 3; do
> -    for ks in 1 2 3; do
> -      s=$is$js$ks
> -      smac=f00000000$s
> -      sip=`ip_to_hex 192 168 $is$js $ks`
> -      for id in 1 2 3; do
> -        for jd in 1 2 3; do
> -          if test $is$js = $id$jd; then
> -            continue
> -          fi
> -
> -          # Send the packet.
> -          dmac=00000000ff$is$js
> -          # Calculate a 4th octet for the destination that is
> -          # unique per $s, avoids the .1 .2 .3 and .254 IP addresses
> -          # that have static MAC bindings, and fits in the range
> -          # 0-255.
> -          o4=`expr $is '*' 9 + $js '*' 3 + $ks + 10`
> -          dip=`ip_to_hex 192 168 $id$jd $o4`
> -          test_ip $s $smac $dmac $sip $dip
> -
> -          # Expect the packet egress.
> -          host_mac=8000000000$o4
> -          outport=${id}11
> -          out_lrp=$id$jd
> -          echo
> ${host_mac}00000000ff${out_lrp}08004500001c00000000"3f1101"00${sip}${dip}0035111100080000
> >> $outport.expected
> -        done
> -      done
> -    done
> -  done
> -done
> -
> -ovn-sbctl -f csv -d bare --no-heading \
> -    -- --columns=logical_port,ip,mac list mac_binding > mac_bindings
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            OVN_CHECK_PACKETS([hv`vif_to_hv $i$j$k`/vif$i$j$k-tx.pcap],
> -                              [$i$j$k.expected])
> -        done
> -    done
> -done
> -
> -# Check the MAC bindings against those expected.
> -AT_CHECK_UNQUOTED([sort < mac_bindings], [0], [`sort <
> mac_bindings.expected`
> -])
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1], [hv2], [hv3])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IP relocation using GARP request])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -#
> -# Two logical switches ls1, ls2.
> -# One logical router lr0 connected to ls[12],
> -# with 2 subnets, 1 per logical switch:
> -#
> -#    lrp1 on ls1 for subnet 192.168.1.1/24
> -#    lrp2 on ls2 for subnet 192.168.2.1/24
> -#
> -# 4 VIFs, 2 per LS lp[12][12], first digit being LS.
> -# VIFs' fixed IP addresses are 192.168.[12].1[12].
> -#
> -# There is a secondary IP 192.168.1.100 that is unknown in NB and learned
> -# through ARP only, and it can move between lp11 and lp12.
> -#
> -ovn-nbctl lr-add lr0
> -for i in 1 2 ; do
> -    ovn-nbctl ls-add ls$i
> -    ovn-nbctl lrp-add lr0 lrp$i 00:00:00:00:ff:0$i 192.168.$i.1/24
> -    ovn-nbctl \
> -        -- lsp-add ls$i lrp$i-attachment \
> -        -- set Logical_Switch_Port lrp$i-attachment type=router \
> -                         options:router-port=lrp$i \
> -                         addresses=router
> -    for j in 1 2; do
> -        ovn-nbctl \
> -            -- lsp-add ls$i lp$i$j \
> -            -- lsp-set-addresses lp$i$j \
> -               "f0:00:00:00:00:$i$j 192.168.$i.1$j"
> -    done
> -done
> -
> -# Physical network:
> -# 2 hypervisors hv[12], lp?1 on hv1, lp?2 on hv2.
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located, e.g. "vif_to_hv 12" yields 2.
> -vif_to_hv() {
> -    echo ${1#?}
> -}
> -
> -# Given the name of a logical port, prints the name of its logical router
> -# port, e.g. "vif_to_lrp 12" yields 1.
> -vif_to_lrp() {
> -    echo ${1%?}
> -}
> -
> -# Given the name of a logical port, prints the name of its logical
> -# switch, e.g. "vif_to_ls 12" yields 1.
> -vif_to_ls() {
> -    echo ${1%?}
> -}
> -
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -done
> -for i in 1 2; do
> -    for j in 1 2; do
> -        hv=`vif_to_hv $i$j`
> -            as hv$hv ovs-vsctl \
> -                -- add-port br-int vif$i$j \
> -                -- set Interface vif$i$j \
> -                    external-ids:iface-id=lp$i$j \
> -                    options:tx_pcap=hv$hv/vif$i$j-tx.pcap \
> -                    options:rxq_pcap=hv$hv/vif$i$j-rx.pcap \
> -                    ofport-request=$i$j
> -    done
> -done
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 12 for
> vif12.
> -for i in 1 2; do
> -    for j in 1 2; do
> -        : > $i$j.expected
> -    done
> -done
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    in_ls=`vif_to_ls $inport`
> -    in_lrp=`vif_to_lrp $inport`
> -    for outport; do
> -        out_ls=`vif_to_ls $outport`
> -        if test $in_ls = $out_ls; then
> -            # Ports on the same logical switch receive exactly the same
> packet.
> -            echo $packet
> -        else
> -            # Routing decrements TTL and updates source and dest MAC
> -            # (and checksum).
> -            out_lrp=`vif_to_lrp $outport`
> -            echo
> f000000000${outport}00000000ff0${out_lrp}08004500001c00000000"3f1101"00${src_ip}${dst_ip}0035111100080000
> -        fi >> $outport.expected
> -    done
> -}
> -
> -# test_arp INPORT SHA SPA TPA [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local inport=$1 sha=$2 spa=$3 tpa=$4 reply_ha=$5
> -    local
> request=ffffffffffff${sha}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $request
> -
> -    # Expect to receive the broadcast ARP on the other logical switch
> ports if
> -    # IP address is not configured to the switch patch port.
> -    local i=`vif_to_ls $inport`
> -    local j
> -    for j in 1 2; do
> -        if test $i$j != $inport; then
> -            echo $request >> $i$j$k.expected
> -        fi
> -    done
> -
> -    # Expect to receive the reply, if any.
> -    if test X$reply_ha != X; then
> -        lrp=`vif_to_lrp $inport`
> -        local
> reply=${sha}00000000ff0${lrp}08060001080006040002${reply_ha}${tpa}${sha}${spa}
> -        echo $reply >> $inport.expected
> -    fi
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# lp11 send GARP request to announce ownership of 192.168.1.100.
> -
> -sha=f00000000011
> -spa=`ip_to_hex 192 168 1 100`
> -tpa=$spa
> -test_arp 11 $sha $spa $tpa
> -OVS_WAIT_UNTIL([test `ovn-sbctl find mac_binding ip="192.168.1.100" | wc
> -l` -gt 0])
> -ovn-nbctl --wait=hv sync
> -
> -# Send an IP packet from lp21 to 192.168.1.100, which should go to lp11.
> -
> -smac=f00000000021
> -dmac=00000000ff02
> -sip=`ip_to_hex 192 168 2 11`
> -dip=`ip_to_hex 192 168 1 100`
> -test_ip 21 $smac $dmac $sip $dip 11
> -
> -# lp12 send GARP request to announce ownership of 192.168.1.100.
> -
> -sha=f00000000012
> -test_arp 12 $sha $spa $tpa
> -OVS_WAIT_UNTIL([ovn-sbctl find mac_binding ip="192.168.1.100" | grep
> f0:00:00:00:00:12])
> -ovn-nbctl --wait=hv sync
> -# give to the hv the time to send queued ip packets
> -sleep 1
> -
> -# Send an IP packet from lp21 to 192.168.1.100, which should go to lp12.
> -
> -test_ip 21 $smac $dmac $sip $dip 12
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2; do
> -    for j in 1 2; do
> -        OVN_CHECK_PACKETS([hv`vif_to_hv $i$j`/vif$i$j-tx.pcap],
> -                          [$i$j.expected])
> -    done
> -done
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1], [hv2])
> -
> -AT_CLEANUP
> -
> -# 3 hypervisors, one logical switch, 3 logical ports per hypervisor
> -AT_SETUP([ovn -- portsecurity : 3 HVs, 1 LS, 3 lports/HV])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Create hypervisors hv[123].
> -# Add vif1[123] to hv1, vif2[123] to hv2, vif3[123] to hv3.
> -# Add all of the vifs to a single logical switch lsw0.
> -# Turn off port security on vifs vif[123]1
> -# Turn on l2 port security on vifs vif[123]2
> -# Turn of l2 and l3 port security on vifs vif[123]3
> -# Make vif13, vif2[23], vif3[123] destinations for unknown MACs.
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2 3; do
> -        ovs-vsctl add-port br-int vif$i$j -- set Interface vif$i$j
> external-ids:iface-id=lp$i$j options:tx_pcap=hv$i/vif$i$j-tx.pcap
> options:rxq_pcap=hv$i/vif$i$j-rx.pcap ofport-request=$i$j
> -        ovn-nbctl lsp-add lsw0 lp$i$j
> -        if test $j = 1; then
> -            ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j
> 192.168.0.$i$j" unknown
> -        elif test $j = 2; then
> -            ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j
> 192.168.0.$i$j"
> -            ovn-nbctl lsp-set-port-security lp$i$j f0:00:00:00:00:$i$j
> -        else
> -            extra_addr="f0:00:00:00:0$i:$i$j fe80::ea2a:eaff:fe28:$i$j"
> -            ovn-nbctl lsp-set-addresses lp$i$j "f0:00:00:00:00:$i$j
> 192.168.0.$i$j" "$extra_addr"
> -            ovn-nbctl lsp-set-port-security lp$i$j "f0:00:00:00:00:$i$j
> 192.168.0.$i$j" "$extra_addr"
> -        fi
> -    done
> -done
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv${1%?}
> -}
> -
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        : > $i$j.expected
> -    done
> -done
> -
> -# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -#
> -# This shell function causes an ip packet to be received on INPORT.
> -# The packet's content has Ethernet destination DST and source SRC
> -# (each exactly 12 hex digits) and Ethernet type ETHTYPE (4 hex digits).
> -# The OUTPORTs (zero or more) list the VIFs on which the packet should
> -# be received.  INPORT and the OUTPORTs are specified as logical switch
> -# port numbers, e.g. 11 for vif11.
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -# test_arp INPORT SHA SPA TPA DROP [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local inport=$1 smac=$2 sha=$3 spa=$4 tpa=$5 drop=$6 reply_ha=$7
> -    local
> request=ffffffffffff${smac}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -    hv=`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $request
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $request
> -    if test $drop != 1; then
> -        if test X$reply_ha = X; then
> -            # Expect to receive the broadcast ARP on the other logical
> switch ports
> -            # if no reply is expected.
> -            local i j
> -            for i in 1 2 3; do
> -                for j in 1 2 3; do
> -                    if test $i$j != $inport; then
> -                        echo $request >> $i$j.expected
> -                    fi
> -                done
> -            done
> -        else
> -            # Expect to receive the reply, if any.
> -            local
> reply=${smac}${reply_ha}08060001080006040002${reply_ha}${tpa}${sha}${spa}
> -            echo $reply >> $inport.expected
> -        fi
> -    fi
> -}
> -
> -# test_ipv6 INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -# This function is similar to test_ip() except that it sends
> -# ipv6 packet
> -test_ipv6() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}86dd6000000000083aff${src_ip}${dst_ip}0000000000000000
> -    shift; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -# test_icmpv6 INPORT  SRC_MAC DST_MAC SRC_IP DST_IP ICMP_TYPE OUTPORT...
> -# This function is similar to test_ipv6() except it specifies the ICMPv6
> type
> -# of the test packet
> -test_icmpv6() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5 icmp_type=$6
> -    local
> packet=${dst_mac}${src_mac}86dd6000000000083aff${src_ip}${dst_ip}${icmp_type}00000000000000
> -    shift; shift; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# no port security
> -sip=`ip_to_hex 192 168 0 12`
> -tip=`ip_to_hex 192 168 0 13`
> -# the arp packet should be allowed even if lp[123]1 is
> -# not configured with mac f00000000023 and ip 192.168.0.12
> -for i in 1 2 3; do
> -    test_arp ${i}1 f00000000023 f00000000023 $sip $tip 0 f00000000013
> -    for j in 1 2 3; do
> -        if test $i != $j; then
> -            test_ip ${i}1 f000000000${i}1 f000000000${j}1 $sip $tip ${j}1
> -        fi
> -    done
> -done
> -
> -# l2 port security
> -sip=`ip_to_hex 192 168 0 12`
> -tip=`ip_to_hex 192 168 0 13`
> -
> -# arp packet should be allowed since lp22 is configured with
> -# mac f00000000022
> -test_arp 22 f00000000022 f00000000022 $sip $tip 0 f00000000013
> -
> -# arp packet should not be allowed since lp32 is not configured with
> -# mac f00000000021
> -test_arp 32 f00000000021 f00000000021 $sip $tip 1
> -
> -# arp packet with sha set to f00000000021 should not be allowed
> -# for lp12
> -test_arp 12 f00000000012 f00000000021 $sip $tip 1
> -
> -# ip packets should be allowed and received since lp[123]2 do not
> -# have l3 port security
> -sip=`ip_to_hex 192 168 0 55`
> -tip=`ip_to_hex 192 168 0 66`
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        if test $i != $j; then
> -            test_ip ${i}2 f000000000${i}2 f000000000${j}2 $sip $tip ${j}2
> -        fi
> -    done
> -done
> -
> -# ipv6 packets should be received by lp[123]2
> -# lp[123]1 can send ipv6 traffic as there is no port security
> -sip=fe800000000000000000000000000000
> -tip=ff020000000000000000000000000000
> -
> -for i in 1 2 3; do
> -    test_ipv6 ${i}1 f000000000${i}1 f000000000${i}2 $sip $tip ${i}2
> -done
> -
> -
> -# l2 and l3 port security
> -sip=`ip_to_hex 192 168 0 13`
> -tip=`ip_to_hex 192 168 0 22`
> -# arp packet should be allowed since lp13 is configured with
> -# f00000000013 and 192.168.0.13
> -test_arp 13 f00000000013 f00000000013 $sip $tip 0 f00000000022
> -
> -# the arp packet should be dropped because lp23 is not configured
> -# with mac f00000000022
> -sip=`ip_to_hex 192 168 0 13`
> -tip=`ip_to_hex 192 168 0 22`
> -test_arp 23 f00000000022 f00000000022 $sip $tip 1
> -
> -# the arp packet should be dropped because lp33 is not configured
> -# with ip 192.168.0.55
> -spa=`ip_to_hex 192 168 0 55`
> -tpa=`ip_to_hex 192 168 0 22`
> -test_arp 33 f00000000031 f00000000031 $spa $tpa 1
> -
> -# ip packets should not be received by lp[123]3 since
> -# l3 port security is enabled
> -sip=`ip_to_hex 192 168 0 55`
> -tip=`ip_to_hex 192 168 0 66`
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        test_ip ${i}2 f000000000${i}2 f000000000${j}3 $sip $tip
> -    done
> -done
> -
> -# ipv6 packets should be dropped for lp[123]3 since
> -# it is configured with only ipv4 address
> -sip=fe800000000000000000000000000000
> -tip=ff020000000000000000000000000000
> -
> -for i in 1 2 3; do
> -    test_ipv6 ${i}3 f000000000${i}3 f00000000022 $sip $tip
> -done
> -
> -# ipv6 packets should not be received by lp[123]3 with mac
> f000000000$[123]3
> -# lp[123]1 can send ipv6 traffic as there is no port security
> -for i in 1 2 3; do
> -    test_ipv6 ${i}1 f000000000${i}1 f000000000${i}3 $sip $tip
> -done
> -
> -# lp13 has extra port security with mac f0000000113 and ipv6 addr
> -# fe80::ea2a:eaff:fe28:0012
> -
> -# ipv4 packet should be dropped for lp13 with mac f0000000113
> -sip=`ip_to_hex 192 168 0 13`
> -tip=`ip_to_hex 192 168 0 23`
> -test_ip 13 f00000000113 f00000000023 $sip $tip
> -
> -# ipv6 packet should be received by lp[123]3 with mac f00000000${i}${i}3
> -# and ip6.dst as fe80::ea2a:eaff:fe28:0${i}${i}3.
> -# lp11 can send ipv6 traffic as there is no port security
> -sip=ee800000000000000000000000000000
> -for i in 1 2 3; do
> -    tip=fe80000000000000ea2aeafffe2800${i}3
> -    test_ipv6 11 f00000000011 f00000000${i}${i}3 $sip $tip ${i}3
> -done
> -
> -
> -# ipv6 packet should not be received by lp33 with mac f0000000333
> -# and ip6.dst as fe80::ea2a:eaff:fe28:0023 as it is
> -# configured with fe80::ea2a:eaff:fe28:0033
> -# lp11 can send ipv6 traffic as there is no port security
> -
> -sip=ee800000000000000000000000000000
> -tip=fe80000000000000ea2aeafffe280023
> -test_ipv6 11 f00000000011 f00000000333 $sip $tip
> -
> -# ipv6 packet should be allowed for lp[123]3 with mac f0000000${i}${i}3
> -# and ip6.src fe80::ea2a:eaff:fe28:0${i}${i}3 and ip6.src ::.
> -# and should be dropped for any other ip6.src
> -# lp21 can receive ipv6 traffic as there is no port security
> -
> -tip=ee800000000000000000000000000000
> -for i in 1 2 3; do
> -    sip=fe80000000000000ea2aeafffe2800${i}3
> -    test_ipv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip $tip 21
> -
> -    # Test ICMPv6 MLD reports (v1 and v2) and NS for DAD
> -    sip=00000000000000000000000000000000
> -    test_icmpv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip
> ff020000000000000000000000160000 83 21
> -    test_icmpv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip
> ff020000000000000000000000160000 8f 21
> -    test_icmpv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip
> ff0200000000000000ea2aeafffe2800 87 21
> -    # Traffic to non-multicast traffic should be dropped
> -    test_icmpv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip $tip 83
> -    # Traffic of other ICMPv6 types should be dropped
> -    test_icmpv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip
> ff020000000000000000000000160000 80
> -
> -    # should be dropped
> -    sip=ae80000000000000ea2aeafffe2800aa
> -    test_ipv6 ${i}3 f00000000${i}${i}3 f00000000021 $sip $tip
> -done
> -
> -# configure lsp13 to send and received IPv4 packets with an address range
> -ovn-nbctl lsp-set-port-security lp13 "f0:00:00:00:00:13 192.168.0.13
> 20.0.0.4/24 10.0.0.0/24"
> -
> -sleep 2
> -
> -sip=`ip_to_hex 10 0 0 13`
> -tip=`ip_to_hex 192 168 0 22`
> -# arp packet with inner ip 10.0.0.13 should be allowed for lsp13
> -test_arp 13 f00000000013 f00000000013 $sip $tip 0 f00000000022
> -
> -sip=`ip_to_hex 10 0 0 14`
> -tip=`ip_to_hex 192 168 0 23`
> -# IPv4 packet from lsp13 with src ip 10.0.0.14 destined to lsp23
> -# with dst ip 192.168.0.23 should be allowed
> -test_ip 13 f00000000013 f00000000023 $sip $tip 23
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 10 0 0 15`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 10.0.0.15 should be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip 13
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 20 0 0 4`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 20.0.0.4 should be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip 13
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 20 0 0 5`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 20.0.0.5 should not be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 20 0 0 255`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 20.0.0.255 should be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip 13
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 192 168 0 255`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 192.168.0.255 should not be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip
> -
> -sip=`ip_to_hex 192 168 0 33`
> -tip=`ip_to_hex 224 0 0 4`
> -# IPv4 packet from lsp33 with src ip 192.168.0.33 destined to lsp13
> -# with dst ip 224.0.0.4  should be received by lsp13
> -test_ip 33 f00000000033 f00000000013 $sip $tip 13
> -
> -#dump information including flow counters
> -ovn-nbctl show
> -ovn-sbctl dump-flows -- list multicast_group
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 show br-int
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-ofctl -O OpenFlow13 show br-int
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -echo "------ hv3 dump ------"
> -as hv3 ovs-vsctl show
> -as hv3 ovs-ofctl -O OpenFlow13 show br-int
> -as hv3 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        OVN_CHECK_PACKETS([hv$i/vif$i$j-tx.pcap], [$i$j.expected])
> -    done
> -done
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 2 LS, 1 lport/LS, 2 peer LRs])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other as peers in
> 20.0.0.0/24
> -# network. R1 has a switchs ls1 (191.168.1.0/24) connected to it.
> -# R2 has ls2 (172.16.1.0/24) connected to it.
> -
> -ls1_lp1_mac="f0:00:00:01:02:03"
> -rp_ls1_mac="00:00:00:01:02:03"
> -rp_ls2_mac="00:00:00:01:02:04"
> -ls2_lp1_mac="f0:00:00:01:02:04"
> -
> -ls1_lp1_ip="192.168.1.2"
> -ls2_lp1_ip="172.16.1.2"
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lr-add R2
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
> -
> -ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1
> type=router \
> -  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
> -
> -# Connect ls2 to R2
> -ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
> -
> -ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2
> type=router \
> -  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
> -
> -# Connect R1 to R2
> -ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 20.0.0.1/24 peer=R2_R1
> -ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 20.0.0.2/24 peer=R1_R2
> -
> -ovn-nbctl lr-route-add R1 "0.0.0.0/0" 20.0.0.2
> -ovn-nbctl lr-route-add R2 "0.0.0.0/0" 20.0.0.1
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
> -
> -# Create logical port ls2-lp1 in ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Packet to send.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_lp1_mac &&
> eth.dst==$rp_ls1_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$ls1_lp1_ip && ip4.dst==$ls2_lp1_ip
> &&
> -        udp && udp.src==53 && udp.dst==4369"
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -
> -# Packet to Expect
> -# The TTL should be decremented by 2.
> -packet="eth.src==$rp_ls2_mac && eth.dst==$ls2_lp1_mac &&
> -        ip4 && ip.ttl==62 && ip4.src==$ls1_lp1_ip && ip4.dst==$ls2_lp1_ip
> &&
> -        udp && udp.src==53 && udp.dst==4369"
> -echo $packet | ovstest test-ovn expr-to-packets > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> -grep "reg0 == 172.16.1.2" | wc -l], [0], [1
> -])
> -
> -# Disable the ls2-lp1 port.
> -ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
> -
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> -grep "reg0 == 172.16.1.2" | wc -l], [0], [0
> -])
> -
> -# Generate the packet destined for ls2-lp1 and it should not be delivered.
> -# Packet to send.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_lp1_mac &&
> eth.dst==$rp_ls1_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$ls1_lp1_ip && ip4.dst==$ls2_lp1_ip
> &&
> -        udp && udp.src==53 && udp.dst==4369"
> -
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -# The 2nd packet sent shound not be received.
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -
> -AT_SETUP([ovn -- 1 HV, 1 LS, 2 lport/LS, 1 LR])
> -AT_KEYWORDS([router-admin-state])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR - R1 has switch ls1 with two subnets attached to it (
> 191.168.1.0/24
> -# and 172.16.1.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add ls1
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 00:00:00:01:02:03 192.168.1.1/24 172.16.1.1/24
> -ovn-nbctl <http://172.16.1.1/24-ovn-nbctl> lsp-add ls1 rp-ls1 -- set
> Logical_Switch_Port rp-ls1 type=router \
> -          options:router-port=ls1 addresses=\"00:00:00:01:02:03\"
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> -          -- lsp-set-addresses ls1-lp1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port ls1-lp2 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp2 \
> -          -- lsp-set-addresses ls1-lp2 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Create one hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int vif1 -- \
> -    set interface vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif2 -- \
> -    set interface vif2 external-ids:iface-id=ls1-lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Send ip packets between the two ports.
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Packet to send.
> -src_mac="f00000010203"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vif1 $packet
> -
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -
> -#Disable router R1
> -ovn-nbctl set Logical_Router R1 enabled=false
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -as hv1 ovs-appctl netdev-dummy/receive vif1 $packet
> -
> -# Packet to Expect
> -expect_src_mac="000000010203"
> -expect_dst_mac="f00000010204"
> -echo
> "${expect_dst_mac}${expect_src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -
> -AT_SETUP([ovn -- 1 HV, 2 LSs, 1 lport/LS, 1 LR])
> -AT_KEYWORDS([router-admin-state])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR - R1 has switch ls1 (191.168.1.0/24) connected to it,
> -# and has switch ls2 (172.16.1.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 00:00:00:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1
> type=router \
> -          options:router-port=ls1 addresses=\"00:00:00:01:02:03\"
> -
> -# Connect ls2 to R1
> -ovn-nbctl lrp-add R1 ls2 00:00:00:01:02:04 172.16.1.1/24
> -ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2
> type=router \
> -          options:router-port=ls2 addresses=\"00:00:00:01:02:04\"
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port ls2-lp1 in ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Create one hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int vif1 -- \
> -    set interface vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif2 -- \
> -    set interface vif2 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Send ip packets between the two ports.
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Packet to send.
> -src_mac="f00000010203"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vif1 $packet
> -
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -#Disable router R1
> -ovn-nbctl set Logical_Router R1 enabled=false
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Allow some time for the disabling of logical router R1 to propagate.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -as hv1 ovs-appctl netdev-dummy/receive vif1 $packet
> -
> -# Packet to Expect
> -expect_src_mac="000000010204"
> -expect_dst_mac="f00000010204"
> -echo
> "${expect_dst_mac}${expect_src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 3 LS, 1 lport/LS, 2 peer LRs, static routes])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other as peers in
> 20.0.0.0/24
> -# network. R1 has switchess foo (192.168.1.0/24)
> -# connected to it.
> -# R2 has alice (172.16.1.0/24) and bob (172.16.2.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lr-add R2
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add bob
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:00:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -          options:router-port=foo addresses=\"00:00:00:01:02:03\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:00:01:02:04 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -          type=router options:router-port=alice
> addresses=\"00:00:00:01:02:04\"
> -
> -# Connect bob to R2
> -ovn-nbctl lrp-add R2 bob 00:00:00:01:02:05 172.16.2.1/24
> -ovn-nbctl lsp-add bob rp-bob -- set Logical_Switch_Port rp-bob
> type=router \
> -          options:router-port=bob addresses=\"00:00:00:01:02:05\"
> -
> -# Connect R1 to R2
> -ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 20.0.0.1/24 peer=R2_R1
> -ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 20.0.0.2/24 peer=R1_R2
> -
> -#install static routes
> -ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> -ovn-nbctl lr-route-add R2 172.16.2.0/24 20.0.0.2 R1_R2
> -ovn-nbctl lr-route-add R2 192.168.1.0/24 20.0.0.1
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Create logical port bob1 in bob
> -ovn-nbctl lsp-add bob bob1 \
> --- lsp-set-addresses bob1 "f0:00:00:01:02:05 172.16.2.2"
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=bob1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="f00000010203"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -# Send ip packets between foo1 and bob1
> -src_mac="f00000010203"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 2 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump ----------"
> -as hv2 ovs-ofctl dump-flows br-int
> -
> -# Packet to Expect at bob1
> -src_mac="000000010205"
> -dst_mac="f00000010205"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 2 2`
> -echo
> "${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Packet to Expect at alice1
> -src_mac="000000010204"
> -dst_mac="f00000010204"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
> -echo
> "${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send gratuitous arp on localnet])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0
> -
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0])
> -AT_CHECK([ovs-vsctl add-port br-eth0 snoopvif -- set Interface snoopvif
> options:tx_pcap=hv/snoopvif-tx.pcap options:rxq_pcap=hv/snoopvif-rx.pcap])
> -
> -# Create a vif.
> -AT_CHECK([ovn-nbctl lsp-add lsw0 localvif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses localvif1 "f0:00:00:00:00:01
> 192.168.1.2"])
> -AT_CHECK([ovn-nbctl lsp-set-port-security localvif1 "f0:00:00:00:00:01"])
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add lsw0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl lsp-set-options ln_port network_name=physnet1])
> -
> -AT_CHECK([ovs-vsctl add-port br-int localvif1 -- set Interface localvif1
> external_ids:iface-id=localvif1])
> -
> -# Wait for packet to be received.
> -echo
> "fffffffffffff0000000000108060001080006040001f00000000001c0a80102000000000000c0a80102"
> > expected
> -OVN_CHECK_PACKETS([hv/snoopvif-tx.pcap], [expected])
> -
> -# Check GARP packet when restart openflow connection.
> -as hv
> -OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
> -
> -OVS_WAIT_UNTIL([grep -c "waiting 4 seconds before reconnect"
> hv/ovn-controller.log])
> -
> -as hv
> -start_daemon ovs-vswitchd --enable-dummy=system -vvconn -vofproto_dpif
> -vunixctl
> -
> -# Wait for packet to be received.
> -echo
> "fffffffffffff0000000000108060001080006040001f00000000001c0a80102000000000000c0a80102"
> > expected
> -OVN_CHECK_PACKETS([hv/snoopvif-tx.pcap], [expected])
> -
> -# Delete the localnet ports.
> -AT_CHECK([ovs-vsctl del-port localvif1])
> -AT_CHECK([ovn-nbctl lsp-del ln_port])
> -
> -OVN_CLEANUP([hv])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 3 LRs connected via LS, static routes])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Three LRs - R1, R2 and R3 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24)
> -# connected to it. R2 has alice (172.16.1.0/24) and R3 has bob (
> 10.32.1.0/24)
> -# connected to it.
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lr-add R2
> -ovn-nbctl lr-add R3
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add bob
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -    options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect bob to R3
> -ovn-nbctl lrp-add R3 bob 00:00:03:01:02:03 10.32.1.1/24
> -ovn-nbctl lsp-add bob rp-bob -- set Logical_Switch_Port rp-bob \
> -    type=router options:router-port=bob addresses=\"00:00:03:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Connect R3 to join
> -ovn-nbctl lrp-add R3 R3_join 00:00:04:01:02:05 20.0.0.3/24
> -ovn-nbctl lsp-add join r3-join -- set Logical_Switch_Port r3-join \
> -    type=router options:router-port=R3_join
> addresses='"00:00:04:01:02:05"'
> -
> -#install static routes
> -ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> -ovn-nbctl lr-route-add R1 10.32.1.0/24 20.0.0.3
> -
> -ovn-nbctl lr-route-add R2 192.168.1.0/24 20.0.0.1
> -ovn-nbctl lr-route-add R2 10.32.1.0/24 20.0.0.3
> -
> -ovn-nbctl lr-route-add R3 192.168.1.0/24 20.0.0.1
> -ovn-nbctl lr-route-add R3 172.16.1.0/24 20.0.0.2
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Create logical port bob1 in bob
> -ovn-nbctl lsp-add bob bob1 \
> --- lsp-set-addresses bob1 "f0:00:00:01:02:05 10.32.1.2"
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=bob1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -as hv1 ovs-appctl ofproto/trace br-int in_port=1 $packet
> -
> -# Send ip packets between foo1 and bob1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 10 32 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -echo "----------------------------"
> -
> -# Packet to Expect at bob1
> -src_mac="000003010203"
> -dst_mac="f00000010205"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 10 32 1 2`
> -echo
> "${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Packet to Expect at alice1
> -src_mac="000002010203"
> -dst_mac="f00000010204"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
> -echo
> "${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- dhcpv4 : 1 HV, 2 LS, 2 LSPs/LS])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -ovn-nbctl lsp-add ls1 ls1-lp2 \
> --- lsp-set-addresses ls1-lp2 "f0:00:00:00:00:02 10.0.0.6 20.0.0.4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp2 "f0:00:00:00:00:02 10.0.0.6
> 20.0.0.4"
> -
> -ovn-nbctl ls-add ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "f0:00:00:00:00:03 30.0.0.6 40.0.0.4"
> -ovn-nbctl lsp-set-port-security ls2-lp1 "f0:00:00:00:00:03 30.0.0.6
> 40.0.0.4"
> -ovn-nbctl lsp-add ls2 ls2-lp2 \
> --- lsp-set-addresses ls2-lp2 "f0:00:00:00:00:04 30.0.0.7"
> -ovn-nbctl lsp-set-port-security ls2-lp2 "f0:00:00:00:00:04 30.0.0.7"
> -
> -d1="$(ovn-nbctl create DHCP_Options cidr=10.0.0.0/24 \
> -options="\"server_id\"=\"10.0.0.1\" \"server_mac\"=\"ff:10:00:00:00:01\" \
> -\"lease_time\"=\"3600\" \"router\"=\"10.0.0.1\"")"
> -
> -ovn-nbctl lsp-set-dhcpv4-options ls1-lp1 ${d1}
> -ovn-nbctl lsp-set-dhcpv4-options ls1-lp2 ${d1}
> -
> -d2="$(ovn-nbctl create DHCP_Options cidr=30.0.0.0/24 \
> -options="\"server_id\"=\"30.0.0.1\" \"server_mac\"=\"ff:10:00:00:00:02\" \
> -\"lease_time\"=\"3600\"")"
> -
> -ovn-nbctl lsp-set-dhcpv4-options ls2-lp2 ${d2}
> -
> -net_add n1
> -sim_add hv1
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=ls1-lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -ovs-vsctl -- add-port br-int hv1-vif3 -- \
> -    set interface hv1-vif3 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=hv1/vif3-tx.pcap \
> -    options:rxq_pcap=hv1/vif3-rx.pcap \
> -    ofport-request=3
> -
> -ovs-vsctl -- add-port br-int hv1-vif4 -- \
> -    set interface hv1-vif4 external-ids:iface-id=ls2-lp2 \
> -    options:tx_pcap=hv1/vif4-tx.pcap \
> -    options:rxq_pcap=hv1/vif4-rx.pcap \
> -    ofport-request=4
> -
> -OVN_POPULATE_ARP
> -
> -sleep 2
> -
> -as hv1 ovs-vsctl show
> -
> -# This shell function sends a DHCP request packet
> -# test_dhcp INPORT SRC_MAC DHCP_TYPE OFFER_IP REQUEST_IP ...
> -test_dhcp() {
> -    local inport=$1 src_mac=$2 dhcp_type=$3 ciaddr=$4 offer_ip=$5
> request_ip=$6 use_ip=$7
> -    shift; shift; shift; shift; shift; shift; shift;
> -    if test $use_ip != 0; then
> -        src_ip=$1
> -        dst_ip=$2
> -        shift; shift;
> -    else
> -        src_ip=`ip_to_hex 0 0 0 0`
> -        dst_ip=`ip_to_hex 255 255 255 255`
> -    fi
> -    if test $request_ip != 0; then
> -        ip_len=0120
> -        udp_len=010b
> -    else
> -        ip_len=011a
> -        udp_len=0106
> -    fi
> -    local
> request=ffffffffffff${src_mac}08004510${ip_len}0000000080110000${src_ip}${dst_ip}
> -    # udp header and dhcp header
> -    request=${request}00440043${udp_len}0000
> -
> request=${request}010106006359aa7600000000${ciaddr}000000000000000000000000${src_mac}
> -    # client hardware padding
> -    request=${request}00000000000000000000
> -    # server hostname
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -    # boot file name
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -    # dhcp magic cookie
> -    request=${request}63825363
> -    # dhcp message type
> -    request=${request}3501${dhcp_type}
> -    # dhcp unknown option
> -    request=${request}d70701020304050607
> -    # dhcp pad option
> -    request=${request}00
> -    if test $request_ip != 0; then
> -        # dhcp requested ip
> -        request=${request}3204${request_ip}
> -    fi
> -    # dhcp end option
> -    request=${request}ff
> -
> -    for port in $inport "$@"; do
> -        : >> $port.expected
> -    done
> -    if test $offer_ip != 0; then
> -        local srv_mac=$1 srv_ip=$2 dhcp_reply_type=$3
> expected_dhcp_opts=$4
> -        # total IP length will be the IP length of the request packet
> -        # (which is 272 in our case) + 8 (padding bytes) +
> (expected_dhcp_opts / 2)
> -        ip_len=`expr 280 + ${#expected_dhcp_opts} / 2`
> -        udp_len=`expr $ip_len - 20`
> -        ip_len=$(printf "%x" $ip_len)
> -        udp_len=$(printf "%x" $udp_len)
> -        # $ip_len var will be in 3 digits i.e 134. So adding a '0' before
> $ip_len
> -        local
> reply=${src_mac}${srv_mac}080045100${ip_len}000000008011XXXX${srv_ip}${offer_ip}
> -        # udp header and dhcp header.
> -        # $udp_len var will be in 3 digits. So adding a '0' before
> $udp_len
> -
> reply=${reply}004300440${udp_len}0000020106006359aa7600000000${ciaddr}
> -        # your ip address; 0 for NAK
> -        if test $dhcp_reply_type = 06; then
> -            reply=${reply}00000000
> -        else
> -            reply=${reply}${offer_ip}
> -        fi
> -        # next server ip address, relay agent ip address, client mac
> address
> -        reply=${reply}0000000000000000${src_mac}
> -        # client hardware padding
> -        reply=${reply}00000000000000000000
> -        # server hostname
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -        # boot file name
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -        # dhcp magic cookie
> -        reply=${reply}63825363
> -
> reply=${reply}3501${dhcp_reply_type}${expected_dhcp_opts}00000000ff00000000
> -        echo $reply >> $inport.expected
> -    else
> -        for outport; do
> -            echo $request >> $outport.expected
> -        done
> -    fi
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif$inport $request
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -AT_CAPTURE_FILE([ofctl_monitor0.log])
> -as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0.log
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Send DHCPDISCOVER.
> -offer_ip=`ip_to_hex 10 0 0 4`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=`ip_to_hex 0 0 0 0`
> -request_ip=0
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 1 f00000000001 01 $ciaddr $offer_ip $request_ip 0 ff1000000001
> $server_ip 02 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 1.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > 1.packets
> -cat 1.expected | cut -c -48 > expout
> -AT_CHECK([cat 1.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 1.expected | cut -c 53- > expout
> -AT_CHECK([cat 1.packets | cut -c 53-], [0], [expout])
> -
> -# ovs-ofctl also resumes the packets and this causes other ports to
> receive
> -# the DHCP request packet. So reset the pcap files so that its easier to
> test.
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST in the SELECTING/INIT-REBOOT state with the offered IP
> -# address in the Requested IP Address option.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=`ip_to_hex 0 0 0 0`
> -request_ip=$offer_ip
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 0 ff1000000001
> $server_ip 05 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST in the SELECTING/INIT-REBOOT state with a mismatched
> IP in
> -# the Requested IP Address option, expect a DHCPNAK.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=`ip_to_hex 0 0 0 0`
> -request_ip=`ip_to_hex 10 0 0 7`
> -expected_dhcp_opts=""
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 0 ff1000000001
> $server_ip 06 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 3.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send Invalid DHCPv4 packet on ls1-lp2. It should be received by
> ovn-controller
> -# but should be resumed without the reply.
> -# ls1-lp1 (vif1-tx.pcap) should receive the DHCPv4 request packet twice,
> -# one from ovn-controller and the other from "ovs-ofctl resume."
> -ciaddr=`ip_to_hex 0 0 0 0`
> -offer_ip=0
> -request_ip=0
> -test_dhcp 2 f00000000002 08 $ciaddr $offer_ip $request_ip 0 1 1
> -
> -# NXT_RESUMEs should be 4.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -# vif1-tx.pcap should have received the DHCPv4 (invalid) request packet
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [1.expected])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPv4 packet on ls2-lp1. It doesn't have any DHCPv4 options
> defined.
> -# ls2-lp2 (vif4-tx.pcap) should receive the DHCPv4 request packet once.
> -
> -ciaddr=`ip_to_hex 0 0 0 0`
> -test_dhcp 3 f00000000003 01 $ciaddr 0 0 4 0
> -
> -# Send DHCPv4 packet on ls2-lp2. "router" DHCPv4 option is not defined for
> -# this lport.
> -ciaddr=`ip_to_hex 0 0 0 0`
> -test_dhcp 4 f00000000004 01 $ciaddr 0 0 3 0
> -
> -# NXT_RESUMEs should be 4.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -#OVN_CHECK_PACKETS([hv1/vif3-tx.pcap], [3.expected])
> -#OVN_CHECK_PACKETS([hv1/vif4-tx.pcap], [4.expected])
> -
> -# Send DHCPREQUEST in the RENEWING/REBINDING state with ip4.src set to
> 10.0.0.6
> -# and ip4.dst set to 10.0.0.1.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=$offer_ip
> -request_ip=0
> -src_ip=$offer_ip
> -dst_ip=$server_ip
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 1 $src_ip
> $dst_ip ff1000000001 $server_ip 05 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 5.
> -OVS_WAIT_UNTIL([test 5 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST in the RENEWING/REBINDING state with ip4.src set to
> 10.0.0.6
> -# and ip4.dst set to 255.255.255.255.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=$offer_ip
> -request_ip=0
> -src_ip=$offer_ip
> -dst_ip=`ip_to_hex 255 255 255 255`
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 1 $src_ip
> $dst_ip ff1000000001 $server_ip 05 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 6.
> -OVS_WAIT_UNTIL([test 6 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST in the RENEWING/REBINDING state with a mismatched IP
> in the
> -# ciaddr, expect a DHCPNAK.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=`ip_to_hex 10 0 0 7`
> -request_ip=0
> -src_ip=$offer_ip
> -dst_ip=`ip_to_hex 255 255 255 255`
> -expected_dhcp_opts=""
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 1 $src_ip
> $dst_ip ff1000000001 $server_ip 06 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 7.
> -OVS_WAIT_UNTIL([test 7 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST in the RENEWING/REBINDING state without a specifyied
> ciaddr,
> -# expect a DHCPNAK.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -ciaddr=`ip_to_hex 0 0 0 0`
> -request_ip=0
> -src_ip=$offer_ip
> -dst_ip=`ip_to_hex 255 255 255 255`
> -expected_dhcp_opts=""
> -test_dhcp 2 f00000000002 03 $ciaddr $offer_ip $request_ip 1 $src_ip
> $dst_ip ff1000000001 $server_ip 06 $expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 8.
> -OVS_WAIT_UNTIL([test 8 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Send DHCPREQUEST with ip4.src set to 10.0.0.6 and ip4.dst set to
> 10.0.0.4.
> -# The packet should not be received by ovn-controller.
> -ciaddr=`ip_to_hex 0 0 0 0`
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 4`
> -test_dhcp 2 f00000000002 03 $ciaddr 0 0 1 $src_ip $dst_ip 1
> -
> -# NXT_RESUMEs should be 8.
> -OVS_WAIT_UNTIL([test 8 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -# vif1-tx.pcap should have received the DHCPv4 request packet
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [1.expected])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- dhcpv6 : 1 HV, 2 LS, 5 LSPs])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4 ae70::4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp1 "f0:00:00:00:00:01 10.0.0.4
> ae70::4"
> -
> -ovn-nbctl lsp-add ls1 ls1-lp2 \
> --- lsp-set-addresses ls1-lp2 "f0:00:00:00:00:02 ae70::5"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp2 "f0:00:00:00:00:02 ae70::5"
> -
> -ovn-nbctl lsp-add ls1 ls1-lp3 \
> --- lsp-set-addresses ls1-lp3 "f0:00:00:00:00:22 ae70::22"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp3 "f0:00:00:00:00:22 ae70::22"
> -
> -d1="$(ovn-nbctl create DHCP_Options cidr="ae70\:\:/64" \
> -options="\"server_id\"=\"00:00:00:10:00:01\"")"
> -
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp1 ${d1}
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp2 ${d1}
> -
> -d2="$(ovn-nbctl create DHCP_Options cidr="ae70\:\:/64" \
> -options="\"dhcpv6_stateless\"=\"true\"
> \"server_id\"=\"00:00:00:10:00:01\"")"
> -
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp3 ${d2}
> -
> -ovn-nbctl ls-add ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "f0:00:00:00:00:03 be70::3"
> -ovn-nbctl lsp-set-port-security ls2-lp1 "f0:00:00:00:00:03 be70::3"
> -ovn-nbctl lsp-add ls2 ls2-lp2 \
> --- lsp-set-addresses ls2-lp2 "f0:00:00:00:00:04 be70::4"
> -ovn-nbctl lsp-set-port-security ls2-lp2 "f0:00:00:00:00:04 be70::4"
> -
> -net_add n1
> -sim_add hv1
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=ls1-lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -ovs-vsctl -- add-port br-int hv1-vif3 -- \
> -    set interface hv1-vif3 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=hv1/vif3-tx.pcap \
> -    options:rxq_pcap=hv1/vif3-rx.pcap \
> -    ofport-request=3
> -
> -ovs-vsctl -- add-port br-int hv1-vif4 -- \
> -    set interface hv1-vif4 external-ids:iface-id=ls2-lp2 \
> -    options:tx_pcap=hv1/vif4-tx.pcap \
> -    options:rxq_pcap=hv1/vif4-rx.pcap \
> -    ofport-request=4
> -
> -ovs-vsctl -- add-port br-int hv1-vif5 -- \
> -    set interface hv1-vif5 external-ids:iface-id=ls1-lp3 \
> -    options:tx_pcap=hv1/vif5-tx.pcap \
> -    options:rxq_pcap=hv1/vif5-rx.pcap \
> -    ofport-request=5
> -
> -OVN_POPULATE_ARP
> -
> -sleep 2
> -
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -# This shell function sends a DHCPv6 request packet
> -# test_dhcpv6 INPORT SRC_MAC SRC_LLA DHCPv6_MSG_TYPE OFFER_IP OUTPORT...
> -# The OUTPORTs (zero or more) list the VIFs on which the original DHCPv6
> -# packet should be received twice (one from ovn-controller and the other
> -# from the "ovs-ofctl monitor br-int resume"
> -test_dhcpv6() {
> -    local inport=$1 src_mac=$2 src_lla=$3 msg_code=$4 offer_ip=$5
> -    if test $msg_code != 0b; then
> -        req_len=2a
> -    else
> -        req_len=1a
> -    fi
> -    local
> request=ffffffffffff${src_mac}86dd0000000000${req_len}1101${src_lla}
> -    # dst ip ff02::1:2
> -    request=${request}ff020000000000000000000000010002
> -    # udp header and dhcpv6 header
> -    request=${request}0222022300${req_len}ffff${msg_code}010203
> -    # Client identifier
> -    request=${request}0001000a00030001${src_mac}
> -    # Add IA-NA (Identity Association for Non Temporary Address) if
> msg_code
> -    # is not 11 (information request packet)
> -    if test $msg_code != 0b; then
> -        request=${request}0003000c0102030400000e1000001518
> -    fi
> -    shift; shift; shift; shift; shift;
> -    if test $offer_ip != 0; then
> -        local server_mac=000000100001
> -        local server_lla=fe80000000000000020000fffe100001
> -        local reply_code=07
> -        if test $msg_code = 01; then
> -            reply_code=02
> -        fi
> -        local msg_len=54
> -        if test $offer_ip = 1; then
> -            msg_len=28
> -        fi
> -        local
> reply=${src_mac}${server_mac}86dd0000000000${msg_len}1101${server_lla}${src_lla}
> -        # udp header and dhcpv6 header
> -        reply=${reply}0223022200${msg_len}ffff${reply_code}010203
> -        # Client identifier
> -        reply=${reply}0001000a00030001${src_mac}
> -        # IA-NA
> -        if test $offer_ip != 1; then
> -
> reply=${reply}0003002801020304ffffffffffffffff00050018${offer_ip}ffffffffffffffff
> -        fi
> -        # Server identifier
> -        reply=${reply}0002000a00030001${server_mac}
> -        echo $reply | trim_zeros >> $inport.expected
> -    else
> -        for outport; do
> -            echo $request | trim_zeros >> $outport.expected
> -        done
> -    fi
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif$inport $request
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -AT_CAPTURE_FILE([ofctl_monitor0.log])
> -as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0.log
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -src_mac=f00000000001
> -src_lla=fe80000000000000f20000fffe000001
> -offer_ip=ae700000000000000000000000000004
> -test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip
> -
> -# NXT_RESUMEs should be 1.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap |
> trim_zeros > 1.packets
> -# cat 1.expected | trim_zeros > expout
> -cat 1.expected | cut -c -120 > expout
> -AT_CHECK([cat 1.packets | cut -c -120], [0], [expout])
> -# Skipping the UDP checksum
> -cat 1.expected | cut -c 125- > expout
> -AT_CHECK([cat 1.packets | cut -c 125-], [0], [expout])
> -
> -rm  1.expected
> -
> -# Send invalid packet on ls1-lp2. ovn-controller should resume the packet
> -# without any modifications and the packet should be received by ls1-lp1.
> -# ls1-lp1 will receive the packet twice, one from the ovn-controller
> after the
> -# resume and the other from ovs-ofctl monitor resume.
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -
> -src_mac=f00000000002
> -src_lla=fe80000000000000f20000fffe000002
> -offer_ip=ae700000000000000000000000000005
> -# Set invalid msg_type
> -
> -test_dhcpv6 2 $src_mac $src_lla 10 0 1 1
> -
> -# NXT_RESUMEs should be 2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -# vif2-tx.pcap should not have received the DHCPv6 reply packet
> -rm 2.packets
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap |
> trim_zeros > 2.packets
> -AT_CHECK([cat 2.packets], [0], [])
> -
> -# vif1-tx.pcap should have received the DHCPv6 (invalid) request packet
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap |
> trim_zeros > 1.packets
> -cat 1.expected > expout
> -AT_CHECK([cat 1.packets], [0], [expout])
> -
> -# Send DHCPv6 packet on ls2-lp1. native DHCPv6 is disabled on this port.
> -# There should be no DHCPv6 reply from ovn-controller and the request
> packet
> -# should be received by ls2-lp2.
> -
> -src_mac=f00000000003
> -src_lla=fe80000000000000f20000fffe000003
> -test_dhcpv6 3 $src_mac $src_lla 01 0 4
> -
> -# NXT_RESUMEs should be 2 only.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -# vif3-tx.pcap should not have received the DHCPv6 reply packet
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap |
> trim_zeros > 3.packets
> -AT_CHECK([cat 3.packets], [0], [])
> -
> -# vif4-tx.pcap should have received the DHCPv6 request packet
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif4-tx.pcap |
> trim_zeros > 4.packets
> -cat 4.expected > expout
> -AT_CHECK([cat 4.packets], [0], [expout])
> -
> -# Send DHCPv6 packet on ls1-lp3. native DHCPv6 works as stateless mode
> for this port.
> -# The DHCPv6 reply shouldn't contain offer_ip.
> -src_mac=f00000000022
> -src_lla=fe80000000000000f20000fffe000022
> -reset_pcap_file hv1-vif5 hv1/vif5
> -test_dhcpv6 5 $src_mac $src_lla 01 1 5
> -
> -# NXT_RESUMEs should be 3.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif5-tx.pcap |
> trim_zeros > 5.packets
> -# Skipping the UDP checksum
> -cat 5.expected | cut -c 1-120,125- > expout
> -AT_CHECK([cat 5.packets | cut -c 1-120,125- ], [0], [expout])
> -
> -# Send DHCPv6 information request (code 11) on ls1-lp3. The DHCPv6 reply
> -# shouldn't contain offer_ip
> -src_mac=f00000000022
> -src_lla=fe80000000000000f20000fffe000022
> -reset_pcap_file hv1-vif5 hv1/vif5
> -rm -f 5.expected
> -test_dhcpv6 5 $src_mac $src_lla 0b 1 5
> -
> -# NXT_RESUMEs should be 4.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif5-tx.pcap |
> -trim_zeros > 5.packets
> -# Skipping the UDP checksum
> -cat 5.expected | cut -c 1-120,125- > expout
> -AT_CHECK([cat 5.packets | cut -c 1-120,125- ], [0], [expout])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 2 LRs connected via LS, gateway router])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24)
> -# connected to it. R2 has alice (172.16.1.0/24) connected to it.
> -# R2 is a gateway router.
> -
> -
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis="hv2"
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -
> -#install static routes
> -ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
> -ip_prefix=172.16.1.0/24 nexthop=20.0.0.2 -- add Logical_Router \
> -R1 static_routes @lrt
> -
> -ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
> -ip_prefix=192.168.1.0/24 nexthop=20.0.0.1 -- add Logical_Router \
> -R2 static_routes @lrt
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 2
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -ovn-sbctl list chassis
> -ovn-sbctl list encap
> -echo "---------------------"
> -
> -# Packet to Expect at alice1
> -src_mac="000002010203"
> -dst_mac="f00000010204"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
> -
> -
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -as hv1 ovs-appctl ofproto/trace br-int in_port=1 $packet
> -
> -echo "------ hv1 dump after packet 1 ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump after packet 1 ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -echo "----------------------------"
> -
> -echo $expected > expected
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Delete the router and re-create it. Things should work as before.
> -ovn-nbctl  lr-del R2
> -ovn-nbctl create Logical_Router name=R2 options:chassis="hv2"
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -
> -ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
> -ip_prefix=192.168.1.0/24 nexthop=20.0.0.1 -- add Logical_Router \
> -R2 static_routes @lrt
> -
> -# Wait for ovn-controller to catch up.
> -sleep 1
> -
> -# Send the packet again.
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -echo "------ hv1 dump after packet 2 ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump after packet 2 ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -echo "----------------------------"
> -
> -echo $expected >> expected
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- icmp_reply: 1 HVs, 2 LSs, 1 lport/LS, 1 LR])
> -AT_KEYWORDS([router-icmp-reply])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR - R1 has switch ls1 (191.168.1.0/24) connected to it,
> -# and has switch ls2 (172.16.1.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 00:00:00:01:02:f1 192.168.1.1/24
> -ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 \
> -    type=router options:router-port=ls1 addresses=\"00:00:00:01:02:f1\"
> -
> -# Connect ls2 to R1
> -ovn-nbctl lrp-add R1 ls2 00:00:00:01:02:f2 172.16.1.1/24
> -ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 \
> -    type=router options:router-port=ls2 addresses=\"00:00:00:01:02:f2\"
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "00:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port ls2-lp1 in ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "00:00:00:01:02:04 172.16.1.2"
> -
> -# Create one hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int vif1 -- \
> -    set interface vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif2 -- \
> -    set interface vif2 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -for i in 1 2; do
> -    : > vif$i.expected
> -done
> -# test_ipv4_icmp_request INPORT ETH_SRC ETH_DST IPV4_SRC IPV4_DST
> IP_CHKSUM ICMP_CHKSUM [EXP_IP_CHKSUM EXP_ICMP_CHKSUM]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ICMPv4
> -# request with ETH_SRC, ETH_DST, IPV4_SRC, IPV4_DST, IP_CHSUM and
> -# ICMP_CHKSUM as specified.  If EXP_IP_CHKSUM and EXP_ICMP_CHKSUM are
> -# provided, then it should be the ip and icmp checksums of the packet
> -# responded; otherwise, no reply is expected.
> -# In the absence of an ip checksum calculation helpers, this relies
> -# on the caller to provide the checksums for the ip and icmp headers.
> -# XXX This should be more systematic.
> -#
> -# INPORT is an lport number, e.g. 11 for vif11.
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC and IPV4_DST are each 8 hex digits.
> -# IP_CHSUM and ICMP_CHKSUM are each 4 hex digits.
> -# EXP_IP_CHSUM and EXP_ICMP_CHKSUM are each 4 hex digits.
> -test_ipv4_icmp_request() {
> -    local inport=$1 eth_src=$2 eth_dst=$3 ipv4_src=$4 ipv4_dst=$5
> ip_chksum=$6 icmp_chksum=$7
> -    local exp_ip_chksum=$8 exp_icmp_chksum=$9
> -    shift; shift; shift; shift; shift; shift; shift
> -    shift; shift
> -
> -    # Use ttl to exercise section 4.2.2.9 of RFC1812
> -    local ip_ttl=01
> -    local icmp_id=5fbf
> -    local icmp_seq=0001
> -    local icmp_data=$(seq 1 56 | xargs printf "%02x")
> -    local icmp_type_code_request=0800
> -    local
> icmp_payload=${icmp_type_code_request}${icmp_chksum}${icmp_id}${icmp_seq}${icmp_data}
> -    local
> packet=${eth_dst}${eth_src}08004500005400004000${ip_ttl}01${ip_chksum}${ipv4_src}${ipv4_dst}${icmp_payload}
> -
> -    as hv1 ovs-appctl netdev-dummy/receive vif$inport $packet
> -    if test X$exp_icmp_chksum != X; then
> -        # Expect to receive the reply, if any. In same port where packet
> was sent.
> -        # Note: src and dst fields are expected to be reversed.
> -        local icmp_type_code_response=0000
> -        local reply_icmp_ttl=fe
> -        local
> reply_icmp_payload=${icmp_type_code_response}${exp_icmp_chksum}${icmp_id}${icmp_seq}${icmp_data}
> -        local
> reply=${eth_src}${eth_dst}08004500005400004000${reply_icmp_ttl}01${exp_ip_chksum}${ipv4_dst}${ipv4_src}${reply_icmp_payload}
> -        echo $reply >> vif$inport.expected
> -    fi
> -}
> -
> -# Send ping packet to router's ip addresses, from each of the 2 logical
> ports.
> -rtr_l1_ip=$(ip_to_hex 192 168 1 1)
> -rtr_l2_ip=$(ip_to_hex 172 16 1 1)
> -l1_ip=$(ip_to_hex 192 168 1 2)
> -l2_ip=$(ip_to_hex 172 16 1 2)
> -
> -# Ping router ip address that is on same subnet as the logical port
> -test_ipv4_icmp_request 1 000000010203 0000000102f1 $l1_ip $rtr_l1_ip 0000
> 8510 02ff 8d10
> -test_ipv4_icmp_request 2 000000010204 0000000102f2 $l2_ip $rtr_l2_ip 0000
> 8510 02ff 8d10
> -
> -# Ping router ip address that is on the other side of the logical ports
> -test_ipv4_icmp_request 1 000000010203 0000000102f1 $l1_ip $rtr_l2_ip 0000
> 8510 02ff 8d10
> -test_ipv4_icmp_request 2 000000010204 0000000102f2 $l2_ip $rtr_l1_ip 0000
> 8510 02ff 8d10
> -
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list logical_flow
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Now check the packets actually received against the ones expected.
> -for inport in 1 2; do
> -    OVN_CHECK_PACKETS([hv1/vif${inport}-tx.pcap], [vif$inport.expected])
> -done
> -
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- policy-based routing: 1 HVs, 2 LSs, 1 lport/LS, 1 LR])
> -AT_KEYWORDS([policy-based-routing])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR - R1 has switch ls1 (191.168.1.0/24) connected to it,
> -# and has switch ls2 (172.16.1.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -ovn-nbctl ls-add ls3
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 00:00:00:01:02:f1 192.168.1.1/24
> -ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 \
> -    type=router options:router-port=ls1 addresses=\"00:00:00:01:02:f1\"
> -
> -# Connect ls2 to R1
> -ovn-nbctl lrp-add R1 ls2 00:00:00:01:02:f2 172.16.1.1/24
> -ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 \
> -    type=router options:router-port=ls2 addresses=\"00:00:00:01:02:f2\"
> -
> -# Connect ls3 to R1
> -ovn-nbctl lrp-add R1 ls3 00:00:00:01:02:f3 20.20.1.1/24
> -ovn-nbctl lsp-add ls3 rp-ls3 -- set Logical_Switch_Port rp-ls3 \
> -    type=router options:router-port=ls3 addresses=\"00:00:00:01:02:f3\"
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "00:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port ls2-lp1 in ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "00:00:00:01:02:04 172.16.1.2"
> -
> -# Create logical port ls3-lp1 in ls3
> -ovn-nbctl lsp-add ls3 ls3-lp1 \
> --- lsp-set-addresses ls3-lp1 "00:00:00:01:02:05 20.20.1.2"
> -
> -# Create one hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add pbr-hv
> -as pbr-hv
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -ovs-vsctl -- add-port br-int vif1 -- \
> -    set interface vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=pbr-hv/vif1-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif2 -- \
> -    set interface vif2 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=pbr-hv/vif2-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif2-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif3 -- \
> -    set interface vif3 external-ids:iface-id=ls3-lp1 \
> -    options:tx_pcap=pbr-hv/vif3-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif3-rx.pcap \
> -    ofport-request=1
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ls1_ro_mac=00:00:00:01:02:f1
> -ls1_ro_ip=192.168.1.1
> -
> -ls2_ro_mac=00:00:00:01:02:f2
> -ls2_ro_ip=172.16.1.1
> -
> -ls3_ro_mac=00:00:00:01:02:f3
> -
> -ls1_p1_mac=00:00:00:01:02:03
> -ls1_p1_ip=192.168.1.2
> -
> -ls2_p1_mac=00:00:00:01:02:04
> -ls2_p1_ip=172.16.1.2
> -
> -ls3_p1_mac=00:00:00:01:02:05
> -
> -# Create a drop policy
> -ovn-nbctl lr-policy-add R1 10 "ip4.src==192.168.1.0/24 && ip4.dst==
> 172.16.1.0/24" drop
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | grep "192.168.1.0" |
> wc -l], [0], [dnl
> -1
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip4 && ip.ttl==64 && ip4.src==$ls1_p1_ip && ip4.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Check if packet hit the drop policy
> -AT_CHECK([ovs-ofctl dump-flows br-int | \
> -    grep "nw_src=192.168.1.0/24,nw_dst=172.16.1.0/24 actions=drop" | \
> -    grep "priority=10" | \
> -    grep "n_packets=1" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Expected to drop the packet.
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" pbr-hv/vif2-tx.pcap >
> vif2.packets
> -rcvd_packet=`cat vif2.packets`
> -AT_FAIL_IF([rcvd_packet = ""])
> -
> -# Override drop policy with allow
> -ovn-nbctl lr-policy-add R1 20 "ip4.src==192.168.1.0/24 && ip4.dst==
> 172.16.1.0/24" allow
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | grep "192.168.1.0" |
> wc -l], [0], [dnl
> -2
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip4 && ip.ttl==64 && ip4.src==$ls1_p1_ip && ip4.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Check if packet hit the allow policy
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | \
> -    grep "192.168.1.0" | \
> -    grep "priority=20" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$ls2_ro_mac && eth.dst==$ls2_p1_mac &&
> -       ip4 && ip.ttl==63 && ip4.src==$ls1_p1_ip && ip4.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > expected
> -
> -OVN_CHECK_PACKETS([pbr-hv/vif2-tx.pcap], [expected])
> -
> -# Override allow policy with reroute
> -ovn-nbctl lr-policy-add R1 30 "ip4.src==192.168.1.0/24 && ip4.dst==
> 172.16.1.0/24" reroute 20.20.1.2
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | \
> -    grep "192.168.1.0" | \
> -    grep "priority=30" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip4 && ip.ttl==64 && ip4.src==$ls1_p1_ip && ip4.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -echo "southbound flows"
> -
> -ovn-sbctl dump-flows | grep lr_in_policy
> -echo "ovs flows"
> -ovs-ofctl dump-flows br-int
> -# Check if packet hit the allow policy
> -AT_CHECK([ovs-ofctl dump-flows br-int | \
> -    grep "nw_src=192.168.1.0/24,nw_dst=172.16.1.0/24" | \
> -    grep "priority=30" | \
> -    grep "n_packets=1" | wc -l], [0], [dnl
> -1
> -])
> -echo "packet hit reroute policy"
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$ls3_ro_mac && eth.dst==$ls3_p1_mac &&
> -       ip4 && ip.ttl==63 && ip4.src==$ls1_p1_ip && ip4.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > 3.expected
> -
> -OVN_CHECK_PACKETS([pbr-hv/vif3-tx.pcap], [3.expected])
> -
> -OVN_CLEANUP([pbr-hv])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- policy-based routing IPv6: 1 HVs, 3 LSs, 1 lport/LS, 1
> LR])
> -AT_KEYWORDS([policy-based-routing])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR - R1 has switch ls1 (191.168.1.0/24) connected to it,
> -# and has switch ls2 (172.16.1.0/24) connected to it.
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -ovn-nbctl ls-add ls3
> -
> -# Connect ls1 to R1
> -ovn-nbctl lrp-add R1 ls1 00:00:00:01:02:f1 2001::1/64
> -ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 \
> -    type=router options:router-port=ls1 addresses=\"00:00:00:01:02:f1\"
> -
> -# Connect ls2 to R1
> -ovn-nbctl lrp-add R1 ls2 00:00:00:01:02:f2 2002::1/64
> -ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 \
> -    type=router options:router-port=ls2 addresses=\"00:00:00:01:02:f2\"
> -
> -# Connect ls3 to R1
> -ovn-nbctl lrp-add R1 ls3 00:00:00:01:02:f3 2003::1/64
> -ovn-nbctl lsp-add ls3 rp-ls3 -- set Logical_Switch_Port rp-ls3 \
> -    type=router options:router-port=ls3 addresses=\"00:00:00:01:02:f3\"
> -
> -# Create logical port ls1-lp1 in ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "00:00:00:01:02:03 2001::2"
> -
> -# Create logical port ls2-lp1 in ls2
> -ovn-nbctl lsp-add ls2 ls2-lp1 \
> --- lsp-set-addresses ls2-lp1 "00:00:00:01:02:04 2002::2"
> -
> -# Create logical port ls3-lp1 in ls3
> -ovn-nbctl lsp-add ls3 ls3-lp1 \
> --- lsp-set-addresses ls3-lp1 "00:00:00:01:02:05 2003::2"
> -
> -# Create one hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add pbr-hv
> -as pbr-hv
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -ovs-vsctl -- add-port br-int vif1 -- \
> -    set interface vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=pbr-hv/vif1-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif2 -- \
> -    set interface vif2 external-ids:iface-id=ls2-lp1 \
> -    options:tx_pcap=pbr-hv/vif2-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif2-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int vif3 -- \
> -    set interface vif3 external-ids:iface-id=ls3-lp1 \
> -    options:tx_pcap=pbr-hv/vif3-tx.pcap \
> -    options:rxq_pcap=pbr-hv/vif3-rx.pcap \
> -    ofport-request=1
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ls1_ro_mac=00:00:00:01:02:f1
> -ls1_ro_ip=2001::1
> -
> -ls2_ro_mac=00:00:00:01:02:f2
> -ls2_ro_ip=2002::1
> -
> -ls3_ro_mac=00:00:00:01:02:f3
> -
> -ls1_p1_mac=00:00:00:01:02:03
> -ls1_p1_ip=2001::2
> -
> -ls2_p1_mac=00:00:00:01:02:04
> -ls2_p1_ip=2002::2
> -
> -ls3_p1_mac=00:00:00:01:02:05
> -
> -# Create a drop policy
> -ovn-nbctl lr-policy-add R1 10 "ip6.src==2001::/64 && ip6.dst==2002::/64"
> drop
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | grep "2001" | wc
> -l], [0], [dnl
> -1
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip6 && ip.ttl==64 && ip6.src==$ls1_p1_ip && ip6.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Check if packet hit the drop policy
> -AT_CHECK([ovs-ofctl dump-flows br-int | \
> -    grep "ipv6_src=2001::/64,ipv6_dst=2002::/64 actions=drop" | \
> -    grep "priority=10" | \
> -    grep "n_packets=1" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Expected to drop the packet.
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" pbr-hv/vif2-tx.pcap >
> vif2.packets
> -rcvd_packet=`cat vif2.packets`
> -AT_FAIL_IF([rcvd_packet = ""])
> -
> -# Override drop policy with allow
> -ovn-nbctl lr-policy-add R1 20 "ip6.src==2001::/64 && ip6.dst==2002::/64"
> allow
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | grep "2001" | wc
> -l], [0], [dnl
> -2
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip6 && ip.ttl==64 && ip6.src==$ls1_p1_ip && ip6.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Check if packet hit the allow policy
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | \
> -    grep "2001" | \
> -    grep "priority=20" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$ls2_ro_mac && eth.dst==$ls2_p1_mac &&
> -       ip6 && ip.ttl==63 && ip6.src==$ls1_p1_ip && ip6.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > expected
> -
> -OVN_CHECK_PACKETS([pbr-hv/vif2-tx.pcap], [expected])
> -
> -# Override allow policy with reroute
> -ovn-nbctl lr-policy-add R1 30 "ip6.src==2001::/64 && ip6.dst==2002::/64"
> reroute 2003::2
> -
> -# Check logical flow
> -AT_CHECK([ovn-sbctl dump-flows | grep lr_in_policy | \
> -    grep "2001" | \
> -    grep "priority=30" | wc -l], [0], [dnl
> -1
> -])
> -
> -# Send packet.
> -packet="inport==\"ls1-lp1\" && eth.src==$ls1_p1_mac &&
> eth.dst==$ls1_ro_mac &&
> -       ip6 && ip.ttl==64 && ip6.src==$ls1_p1_ip && ip6.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -as pbr-hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -echo "southbound flows"
> -
> -ovn-sbctl dump-flows | grep lr_in_policy
> -echo "ovs flows"
> -ovs-ofctl dump-flows br-int
> -# Check if packet hit the allow policy
> -AT_CHECK([ovs-ofctl dump-flows br-int | \
> -    grep "ipv6_src=2001::/64,ipv6_dst=2002::/64" | \
> -    grep "priority=30" | \
> -    grep "n_packets=1" | wc -l], [0], [dnl
> -1
> -])
> -echo "packet hit reroute policy"
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$ls3_ro_mac && eth.dst==$ls3_p1_mac &&
> -       ip6 && ip.ttl==63 && ip6.src==$ls1_p1_ip && ip6.dst==$ls2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > 3.expected
> -
> -OVN_CHECK_PACKETS([pbr-hv/vif3-tx.pcap], [3.expected])
> -
> -OVN_CLEANUP([pbr-hv])
> -AT_CLEANUP
> -
> -# 1 hypervisor, 1 port
> -# make sure that the port state is properly set to up and back down
> -# when created and deleted.
> -AT_SETUP([ovn -- port state up and down])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl lsp-add ls1 lp1
> -ovn-nbctl lsp-set-addresses lp1 unknown
> -
> -net_add n1
> -sim_add hv1
> -as hv1 ovs-vsctl add-br br-phys
> -as hv1 ovn_attach n1 br-phys 192.168.0.1
> -
> -as hv1 ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp1` = xup])
> -
> -as hv1 ovs-vsctl del-port br-int vif1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp1` = xdown])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -# 1 hypervisor, 1 port
> -# make sure that the OF rules created to support a datapath are
> added/cleared
> -# when logical switch is created and removed.
> -AT_SETUP([ovn -- datapath rules added/removed])
> -AT_KEYWORDS([cleanup])
> -ovn_start
> -
> -net_add n1
> -sim_add hv1
> -as hv1 ovs-vsctl add-br br-phys
> -as hv1 ovn_attach n1 br-phys 192.168.0.1
> -
> -# This shell function checks if OF rules in br-int have clauses
> -# related to OVN datapaths. The caller determines if it should find
> -# a match in the output, or not.
> -#
> -# EXPECT_DATAPATH param determines whether flows that refer to
> -#                 datapath to should be present or not. 0 means
> -#                 they should not be.
> -# STAGE_INFO param is a simple string to help identify the stage
> -#            in the test when this function was invoked.
> -test_datapath_in_of_rules() {
> -    local expect_datapath=$1 stage_info=$2
> -    echo "------ ovn-nbctl show ${stage_info} ------"
> -    ovn-nbctl show
> -    echo "------ ovn-sbctl show ${stage_info} ------"
> -    ovn-sbctl show
> -    echo "------ OF rules ${stage_info} ------"
> -    AT_CHECK([ovs-ofctl dump-flows br-int], [0], [stdout])
> -    # if there is a datapath mentioned in the output, check for the
> -    # magic keyword that represents one, based on the exit status of
> -    # a quiet grep
> -    if test $expect_datapath != 0; then
> -       AT_CHECK([grep -q -i 'metadata=' stdout], [0], [ignore-nolog])
> -    else
> -       AT_CHECK([grep -q -i 'metadata=' stdout], [1], [ignore-nolog])
> -    fi
> -}
> -
> -test_datapath_in_of_rules 0 "before ls+port create"
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl lsp-add ls1 lp1
> -ovn-nbctl lsp-set-addresses lp1 unknown
> -
> -as hv1 ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp1` = xup])
> -
> -test_datapath_in_of_rules 1 "after port is bound"
> -
> -as hv1 ovs-vsctl del-port br-int vif1
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp1` = xdown])
> -
> -ovn-nbctl lsp-set-addresses lp1
> -ovn-nbctl lsp-del lp1
> -ovn-nbctl ls-del ls1
> -
> -# wait for earlier changes to take effect
> -AT_CHECK([ovn-nbctl --timeout=3 --wait=sb sync], [0], [ignore])
> -
> -# ensure OF rules are no longer present. There used to be a bug here.
> -test_datapath_in_of_rules 0 "after lport+ls removal"
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- nd_na ])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -#TODO: since patch port for IPv6 logical router port is not ready not,
> -#  so we are not going to test vifs on different lswitches cases. Try
> -#  to update for that once relevant stuff implemented.
> -
> -# In this test cases we create 1 lswitch, it has 2 VIF ports attached
> -# with. NS packet we test, from one VIF for another VIF, will be replied
> -# by local ovn-controller, but not by target VIF.
> -
> -# Create hypervisors and logical switch lsw0.
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -# Add vif1 to hv1 and lsw0, turn on l2 port security on vif1.
> -ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1 options:tx_pcap=hv1/vif1-tx.pcap
> options:rxq_pcap=hv1/vif1-rx.pcap ofport-request=1
> -ovn-nbctl lsp-add lsw0 lp1
> -ovn-nbctl lsp-set-addresses lp1 "fa:16:3e:94:05:98 192.168.0.3
> fd81:ce49:a948:0:f816:3eff:fe94:598"
> -ovn-nbctl lsp-set-port-security lp1 "fa:16:3e:94:05:98 192.168.0.3
> fd81:ce49:a948:0:f816:3eff:fe94:598"
> -
> -# Add vif2 to hv1 and lsw0, turn on l2 port security on vif2.
> -ovs-vsctl add-port br-int vif2 -- set Interface vif2
> external-ids:iface-id=lp2 options:tx_pcap=hv1/vif2-tx.pcap
> options:rxq_pcap=hv1/vif2-rx.pcap ofport-request=2
> -ovn-nbctl lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp2 "fa:16:3e:a1:f9:ae 192.168.0.4
> fd81:ce49:a948:0:f816:3eff:fea1:f9ae"
> -ovn-nbctl lsp-set-port-security lp2 "fa:16:3e:a1:f9:ae 192.168.0.4
> fd81:ce49:a948:0:f816:3eff:fea1:f9ae"
> -
> -# Add ACL rule for ICMPv6 on lsw0
> -ovn-nbctl acl-add lsw0 from-lport 1002 'ip6 && icmp6'  allow-related
> -ovn-nbctl acl-add lsw0 to-lport 1002 'outport == "lp1" && ip6 && icmp6'
> allow-related
> -ovn-nbctl acl-add lsw0 to-lport 1002 'outport == "lp2" && ip6 && icmp6'
> allow-related
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv1${1%?}
> -}
> -for i in 1 2; do
> -    : > $i.expected
> -done
> -
> -# Complete Neighbor Solicitation packet and Neighbor Advertisement packet
> -# vif1 -> NS -> vif2.  vif1 <- NA <- ovn-controller.
> -# vif2 will not receive NS packet, since ovn-controller will reply for it.
>
> -ns_packet=3333ffa1f9aefa163e94059886dd6000000000203afffd81ce49a9480000f8163efffe940598fd81ce49a9480000f8163efffea1f9ae8700e01160000000fd81ce49a9480000f8163efffea1f9ae0101fa163e940598
>
> -na_packet=fa163e940598fa163ea1f9ae86dd6000000000203afffd81ce49a9480000f8163efffea1f9aefd81ce49a9480000f8163efffe9405988800e9ed60000000fd81ce49a9480000f8163efffea1f9ae0201fa163ea1f9ae
> -
> -as hv1 ovs-appctl netdev-dummy/receive vif1 $ns_packet
> -echo $na_packet >> 1.expected
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-ofctl -O OpenFlow13 show br-int
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -
> -for i in 1 2; do
> -    OVN_CHECK_PACKETS([hv1/vif$i-tx.pcap], [$i.expected])
> -done
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- address sets modification/removal smoke test])
> -ovn_start
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -row=`ovn-nbctl create Address_Set name=set1 addresses=\"1.1.1.1\"`
> -ovn-nbctl set Address_Set $row name=set1 addresses=\"1.1.1.1,1.1.1.2\"
> -ovn-nbctl destroy Address_Set $row
> -
> -sleep 1
> -
> -# A bug previously existed in the address set support code
> -# that caused ovn-controller to crash after an address set
> -# was updated and then removed.  This test case ensures
> -# that ovn-controller is at least still running after
> -# creating, updating, and deleting an address set.
> -AT_CHECK([ovs-appctl -t ovn-controller version], [0], [ignore])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ipam])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Add a port to a switch that does not have a subnet set, then set the
> -# subnet which should result in an address being allocated for the port.
> -ovn-nbctl --wait=hv set NB_Global . options:mac_prefix="0a:00:00:00:00:00"
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 p0 -- lsp-set-addresses p0 dynamic
> -ovn-nbctl --wait=sb add Logical-Switch sw0 other_config subnet=
> 192.168.1.0/24
> -AT_CHECK([ovn-nbctl <http://192.168.1.0/24-AT_CHECK(%5Bovn-nbctl> get
> Logical-Switch-Port p0 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:03 192.168.1.2"
> -])
> -
> -# Add 9 more ports to sw0, addresses should all be unique.
> -for n in `seq 1 9`; do
> -    ovn-nbctl --wait=sb lsp-add sw0 "p$n" -- lsp-set-addresses "p$n"
> dynamic
> -done
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p1 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:04 192.168.1.3"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p2 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:05 192.168.1.4"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p3 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:06 192.168.1.5"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p4 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:07 192.168.1.6"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p5 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:08 192.168.1.7"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p6 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:09 192.168.1.8"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p7 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:0a 192.168.1.9"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p8 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:0b 192.168.1.10"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p9 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:0c 192.168.1.11"
> -])
> -
> -# Trying similar tests with a second switch. MAC addresses should be
> unique
> -# across both switches but IP's only need to be unique within the same
> switch.
> -ovn-nbctl ls-add sw1
> -ovn-nbctl lsp-add sw1 p10 -- lsp-set-addresses p10 dynamic
> -ovn-nbctl --wait=sb add Logical-Switch sw1 other_config subnet=
> 192.168.1.0/24
> -AT_CHECK([ovn-nbctl <http://192.168.1.0/24-AT_CHECK(%5Bovn-nbctl> get
> Logical-Switch-Port p10 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:0d 192.168.1.2"
> -])
> -
> -for n in `seq 11 19`; do
> -    ovn-nbctl --wait=sb lsp-add sw1 "p$n" -- lsp-set-addresses "p$n"
> dynamic
> -done
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p11 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:0e 192.168.1.3"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p12 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:0f 192.168.1.4"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p13 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:10 192.168.1.5"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p14 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:11 192.168.1.6"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p15 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:12 192.168.1.7"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p16 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:13 192.168.1.8"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p17 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:14 192.168.1.9"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p18 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:15 192.168.1.10"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p19 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:16 192.168.1.11"
> -])
> -
> -# Change a port's address to test for multiple ip's for a single address
> entry
> -# and addresses set by the user.
> -ovn-nbctl lsp-set-addresses p0 "0a:00:00:a8:01:17 192.168.1.2
> 192.168.1.12 192.168.1.14"
> -ovn-nbctl --wait=sb lsp-add sw0 p20 -- lsp-set-addresses p20 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p20 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:18 192.168.1.13"
> -])
> -
> -# Test for logical router port address management.
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl -- --id=@lrp create Logical_Router_port name=sw0 \
> -network="192.168.1.1/24" mac=\"0a:00:00:a8:01:19\" \
> --- add Logical_Router R1 ports @lrp -- lsp-add sw0 rp-sw0 \
> --- set Logical_Switch_Port rp-sw0 type=router options:router-port=sw0
> -ovn-nbctl --wait=sb lsp-add sw0 p21 -- lsp-set-addresses p21 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p21 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:1a 192.168.1.15"
> -])
> -
> -# Test for address reuse after logical port is deleted.
> -ovn-nbctl lsp-del p0
> -ovn-nbctl --wait=sb lsp-add sw0 p23 -- lsp-set-addresses p23 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p23 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:03 192.168.1.2"
> -])
> -
> -# Test for multiple addresses to one logical port.
> -ovn-nbctl lsp-add sw0 p25 -- lsp-set-addresses p25 \
> -"0a:00:00:a8:01:1b 192.168.1.12" "0a:00:00:a8:01:1c 192.168.1.14"
> -ovn-nbctl --wait=sb lsp-add sw0 p26 -- lsp-set-addresses p26 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p26 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:17 192.168.1.16"
> -])
> -
> -# Test for exhausting subnet address space.
> -ovn-nbctl ls-add sw2 -- add Logical-Switch sw2 other_config subnet=
> 172.16.1.0/30
> -ovn-nbctl <http://172.16.1.0/30-ovn-nbctl> --wait=sb lsp-add sw2 p27 --
> lsp-set-addresses p27 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p27 dynamic_addresses], [0],
> -     ["0a:00:00:10:01:03 172.16.1.2"
> -])
> -
> -ovn-nbctl --wait=sb lsp-add sw2 p28 -- lsp-set-addresses p28 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p28 dynamic_addresses], [0],
> -     ["0a:00:00:00:00:01"
> -])
> -
> -# Test that address management does not add duplicate MAC for lsp/lrp
> peers.
> -ovn-nbctl create Logical_Router name=R2
> -ovn-nbctl ls-add sw3
> -ovn-nbctl lsp-add sw3 p29 -- lsp-set-addresses p29 \
> -"0a:00:00:a8:01:18"
> -ovn-nbctl -- --id=@lrp create Logical_Router_port name=sw3 \
> -network="192.168.2.1/24" mac=\"0a:00:00:a8:01:18\" \
> --- add Logical_Router R2 ports @lrp -- lsp-add sw3 rp-sw3 \
> --- set Logical_Switch_Port rp-sw3 type=router options:router-port=sw3
> -ovn-nbctl --wait=sb lsp-add sw0 p30 -- lsp-set-addresses p30 dynamic
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p30 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:1d 192.168.1.17"
> -])
> -
> -# Test static MAC address with dynamically allocated IP
> -ovn-nbctl --wait=sb lsp-add sw0 p31 -- lsp-set-addresses p31 \
> -"fe:dc:ba:98:76:54 dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p31 dynamic_addresses], [0],
> -     ["fe:dc:ba:98:76:54 192.168.1.18"
> -])
> -
> -# Update the static MAC address with dynamically allocated IP and check
> -# if the MAC address is updated in
> 'Logical_Switch_Port.dynamic_adddresses'
> -ovn-nbctl --wait=sb lsp-set-addresses p31 "fe:dc:ba:98:76:55 dynamic"
> -
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p31 dynamic_addresses], [0],
> -     ["fe:dc:ba:98:76:55 192.168.1.18"
> -])
> -
> -ovn-nbctl --wait=sb lsp-set-addresses p31 "dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p31 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:1e 192.168.1.18"
> -])
> -
> -ovn-nbctl --wait=sb lsp-set-addresses p31 "fe:dc:ba:98:76:56 dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p31 dynamic_addresses], [0],
> -     ["fe:dc:ba:98:76:56 192.168.1.18"
> -])
> -
> -
> -# Test the exclude_ips from the IPAM list
> -ovn-nbctl --wait=sb set logical_switch sw0 \
> -other_config:exclude_ips="192.168.1.19 192.168.1.21
> 192.168.1.23..192.168.1.50"
> -
> -ovn-nbctl --wait=sb lsp-add sw0 p32 -- lsp-set-addresses p32 \
> -"dynamic"
> -# 192.168.1.20 should be assigned as 192.168.1.19 is excluded.
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p32 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:1e 192.168.1.20"
> -])
> -
> -ovn-nbctl --wait=sb lsp-add sw0 p33 -- lsp-set-addresses p33 \
> -"dynamic"
> -# 192.168.1.22 should be assigned as 192.168.1.21 is excluded.
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p33 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:1f 192.168.1.22"
> -])
> -
> -ovn-nbctl --wait=sb lsp-add sw0 p34 -- lsp-set-addresses p34 \
> -"dynamic"
> -# 192.168.1.51 should be assigned as 192.168.1.23-192.168.1.50 is
> excluded.
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p34 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:34 192.168.1.51"
> -])
> -
> -# Now clear the exclude_ips list. 192.168.1.19 should be assigned.
> -ovn-nbctl --wait=sb set Logical-switch sw0
> other_config:exclude_ips="invalid"
> -ovn-nbctl --wait=sb lsp-add sw0 p35 -- lsp-set-addresses p35 \
> -"dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p35 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:20 192.168.1.19"
> -])
> -
> -# Set invalid data in exclude_ips list. It should be ignored.
> -ovn-nbctl --wait=sb set Logical-switch sw0
> other_config:exclude_ips="182.168.1.30"
> -ovn-nbctl --wait=sb lsp-add sw0 p36 -- lsp-set-addresses p36 \
> -"dynamic"
> -# 192.168.1.21 should be assigned as that's the next free one.
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p36 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:21 192.168.1.21"
> -])
> -
> -# Clear the dynamic addresses assignment request.
> -ovn-nbctl --wait=sb clear logical_switch_port p36 addresses
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p36 dynamic_addresses], [0],
> -         [[[]]
> -])
> -
> -# Set IPv6 prefix
> -ovn-nbctl --wait=sb set Logical-switch sw0
> other_config:ipv6_prefix="aef0::"
> -ovn-nbctl --wait=sb lsp-add sw0 p37 -- lsp-set-addresses p37 \
> -"dynamic"
> -
> -# With prefix aef0 and mac 0a:00:00:00:00:26, the dynamic IPv6 should be
> -# - aef0::800:ff:fe00:26 (EUI64)
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p37 dynamic_addresses], [0],
> -     ["0a:00:00:a8:01:21 192.168.1.21 aef0::800:ff:fea8:121"
> -])
> -
> -ovn-nbctl --wait=sb ls-add sw4
> -ovn-nbctl --wait=sb set Logical-switch sw4
> other_config:ipv6_prefix="bef0::" \
> --- set Logical-switch sw4 other_config:subnet=192.168.2.0/30
> -ovn-nbctl --wait=sb lsp-add sw4 p38 -- lsp-set-addresses p38 \
> -"dynamic"
> -
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p38 dynamic_addresses], [0],
> -     ["0a:00:00:a8:02:03 192.168.2.2 bef0::800:ff:fea8:203"
> -])
> -
> -ovn-nbctl --wait=sb lsp-add sw4 p39 -- lsp-set-addresses p39 \
> -"f0:00:00:00:10:12 dynamic"
> -
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p39 dynamic_addresses], [0],
> -     ["f0:00:00:00:10:12 bef0::f200:ff:fe00:1012"
> -])
> -
> -# Test the case where IPv4 addresses are exhausted and IPv6 prefix is set
> -# p40 should not have an IPv4 address since the pool is exhausted
> -ovn-nbctl --wait=sb lsp-add sw4 p40 -- lsp-set-addresses p40 \
> -"dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p40 dynamic_addresses], [0],
> -         ["0a:00:00:00:00:02 bef0::800:ff:fe00:2"
> -])
> -
> -# Test dynamic changes on switch ports.
> -#
> -ovn-nbctl --wait=sb ls-add sw5
> -ovn-nbctl --wait=sb lsp-add sw5 p41 -- lsp-set-addresses p41 \
> -"dynamic"
> -# p41 will start with nothing
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         [[[]]
> -])
> -
> -# Set a subnet. Now p41 should have an ipv4 address, too
> -ovn-nbctl --wait=sb add Logical-Switch sw5 other_config subnet=
> 192.168.1.0/24
> -AT_CHECK([ovn-nbctl <http://192.168.1.0/24-AT_CHECK(%5Bovn-nbctl> get
> Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["0a:00:00:a8:01:22 192.168.1.2"
> -])
> -
> -# Clear the other_config. The IPv4 address should be gone
> -ovn-nbctl --wait=sb clear Logical-Switch sw5 other_config
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         [[[]]
> -])
> -
> -# Set an IPv6 prefix. Now p41 should have an IPv6 address.
> -ovn-nbctl --wait=sb set Logical-Switch sw5
> other_config:ipv6_prefix="aef0::"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["0a:00:00:00:00:03 aef0::800:ff:fe00:3"
> -])
> -
> -# Change the MAC address to a static one. The IPv6 address should update.
> -ovn-nbctl --wait=sb lsp-set-addresses p41 "f0:00:00:00:10:2b dynamic"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["f0:00:00:00:10:2b aef0::f200:ff:fe00:102b"
> -])
> -
> -# Change the IPv6 prefix. The IPv6 address should update.
> -ovn-nbctl --wait=sb set Logical-Switch sw5
> other_config:ipv6_prefix="bef0::"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["f0:00:00:00:10:2b bef0::f200:ff:fe00:102b"
> -])
> -
> -# Clear the other_config. The IPv6 address should be gone
> -ovn-nbctl --wait=sb clear Logical-Switch sw5 other_config
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         [[[]]
> -])
> -
> -# Set the subnet again. Now p41 should get the IPv4 address again.
> -ovn-nbctl --wait=sb add Logical-Switch sw5 other_config subnet=
> 192.168.1.0/24
> -AT_CHECK([ovn-nbctl <http://192.168.1.0/24-AT_CHECK(%5Bovn-nbctl> get
> Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["f0:00:00:00:10:2b 192.168.1.2"
> -])
> -
> -# Add an excluded IP address that conflicts with p41. p41 should update.
> -ovn-nbctl --wait=sb add Logical-Switch sw5 other_config \
> -exclude_ips="192.168.1.2"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["f0:00:00:00:10:2b 192.168.1.3"
> -])
> -
> -# Add static ip address
> -ovn-nbctl --wait=sb lsp-set-addresses p41 "dynamic 192.168.1.100"
> -ovn-nbctl list Logical-Switch-Port p41
> -ovn-nbctl --wait=sb lsp-add sw5 p42 -- lsp-set-addresses p42 \
> -"dynamic 192.168.1.101"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p41 dynamic_addresses], [0],
> -         ["0a:00:00:a8:01:65 192.168.1.100"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p42 dynamic_addresses], [0],
> -         ["0a:00:00:a8:01:66 192.168.1.101"
> -])
> -
> -# define a mac address prefix
> -ovn-nbctl ls-add sw6
> -ovn-nbctl --wait=hv set NB_Global . options:mac_prefix="00:11:22:33:44:55"
> -ovn-nbctl --wait=sb set Logical-Switch sw6 other_config:subnet=
> 192.168.100.0/24
> -for <http://192.168.100.0/24-for> n in $(seq 1 3); do
> -    ovn-nbctl --wait=sb lsp-add sw6 "p5$n" -- lsp-set-addresses "p5$n"
> dynamic
> -done
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p51 dynamic_addresses], [0],
> -    ["00:11:22:a8:64:03 192.168.100.2"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p52 dynamic_addresses], [0],
> -    ["00:11:22:a8:64:04 192.168.100.3"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p53 dynamic_addresses], [0],
> -    ["00:11:22:a8:64:05 192.168.100.4"
> -])
> -
> -# verify configuration order does not break IPAM/MACAM
> -ovn-nbctl ls-add sw7
> -for n in $(seq 1 3); do
> -    ovn-nbctl --wait=sb lsp-add sw7 "p7$n" -- lsp-set-addresses "p7$n"
> dynamic
> -done
> -ovn-nbctl --wait=sb set Logical-Switch sw7
> other_config:ipv6_prefix="bef0::"
> -p71_addr=$(ovn-nbctl get Logical-Switch-Port p71 dynamic_addresses)
> -p72_addr=$(ovn-nbctl get Logical-Switch-Port p72 dynamic_addresses)
> -p73_addr=$(ovn-nbctl get Logical-Switch-Port p73 dynamic_addresses)
> -AT_CHECK([test "$p71_addr" != "$p72_addr"], [0], [])
> -AT_CHECK([test "$p71_addr" != "$p73_addr"], [0], [])
> -AT_CHECK([test "$p72_addr" != "$p73_addr"], [0], [])
> -
> -# request to assign mac only
> -#
> -ovn-nbctl ls-add sw8
> -ovn-nbctl --wait=sb set Logical-Switch sw8 other_config:mac_only=true
> -for n in $(seq 1 3); do
> -    ovn-nbctl --wait=sb lsp-add sw8 "p8$n" -- lsp-set-addresses "p8$n"
> dynamic
> -done
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p81 dynamic_addresses], [0],
> -    ["00:11:22:00:00:06"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p82 dynamic_addresses], [0],
> -    ["00:11:22:00:00:07"
> -])
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p83 dynamic_addresses], [0],
> -    ["00:11:22:00:00:08"
> -])
> -
> -# clear mac_prefix and check it is allocated in a random manner
> -ovn-nbctl --wait=hv remove NB_Global . options mac_prefix
> -ovn-nbctl ls-add sw9
> -ovn-nbctl --wait=sb set Logical-Switch sw9 other_config:mac_only=true
> -ovn-nbctl --wait=sb lsp-add sw9 p91 -- lsp-set-addresses p91 dynamic
> -
> -mac_prefix=$(ovn-nbctl --wait=sb get NB_Global . options:mac_prefix | tr
> -d \")
> -port_addr=$(ovn-nbctl get Logical-Switch-Port p91 dynamic_addresses | tr
> -d \")
> -AT_CHECK([test "$port_addr" = "${mac_prefix}:00:00:09"], [0], [])
> -
> -ovn-nbctl --wait=hv set NB_Global . options:mac_prefix="00:11:22"
> -ovn-nbctl ls-add sw10
> -ovn-nbctl --wait=sb set Logical-Switch sw10
> other_config:ipv6_prefix="ae01::"
> -ovn-nbctl --wait=sb lsp-add sw10 p101 -- lsp-set-addresses p101 "dynamic
> ae01::1"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p101 dynamic_addresses], [0],
> -    ["00:11:22:00:00:0a ae01::1"
> -])
> -
> -ovn-nbctl --wait=sb set Logical-Switch sw10 other_config:subnet=
> 192.168.110.0/24
> -ovn-nbctl <http://192.168.110.0/24-ovn-nbctl> --wait=sb lsp-add sw10
> p102 -- lsp-set-addresses p102 "dynamic 192.168.110.10 ae01::2"
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p102 dynamic_addresses], [0],
> -    ["00:11:22:a8:6e:0b 192.168.110.10 ae01::2"
> -])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as northd-backup
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ipam connectivity])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl lr-add R1
> -
> -# Test for a ping using dynamically allocated addresses.
> -ovn-nbctl --wait=hv set NB_Global . options:mac_prefix="0a:00:00:00:00:00"
> -ovn-nbctl ls-add foo -- add Logical_Switch foo other_config subnet=
> 192.168.1.0/24
> -ovn-nbctl <http://192.168.1.0/24-ovn-nbctl> ls-add alice -- add
> Logical_Switch alice other_config subnet=192.168.2.0/24
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:00:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -          options:router-port=foo \
> -          -- lsp-set-addresses rp-foo router
> -
> -# Connect alice to R1
> -ovn-nbctl lrp-add R1 alice 00:00:00:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice
> type=router \
> -          options:router-port=alice addresses=\"00:00:00:01:02:04\"
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl --wait=sb lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "dynamic"
> -AT_CHECK([ovn-nbctl --timeout=10 wait-until Logical-Switch-Port foo1
> dynamic_addresses='"0a:00:00:a8:01:03 192.168.1.2"'], [0])
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl --wait=sb lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "dynamic"
> -AT_CHECK([ovn-nbctl --timeout=10 wait-until Logical-Switch-Port alice1
> dynamic_addresses='"0a:00:00:a8:02:03 192.168.2.2"'])
> -
> -# Create logical port foo2 in foo
> -ovn-nbctl --wait=sb lsp-add foo foo2 \
> --- lsp-set-addresses foo2 "dynamic"
> -AT_CHECK([ovn-nbctl --timeout=10 wait-until Logical-Switch-Port foo2
> dynamic_addresses='"0a:00:00:a8:01:04 192.168.1.3"'])
> -
> -# Create a hypervisor and create OVS ports corresponding to logical ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=foo2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -ovs-vsctl -- add-port br-int hv1-vif3 -- \
> -    set interface hv1-vif3 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv1/vif3-tx.pcap \
> -    options:rxq_pcap=hv1/vif3-rx.pcap \
> -    ofport-request=3
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and foo2
> -src_mac="0a0000a80103"
> -dst_mac="0a0000a80104"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 1 3`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="0a0000a80103"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Packet to Expect at foo2
> -src_mac="0a0000a80103"
> -dst_mac="0a0000a80104"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 1 3`
>
> -expected=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap >
> received1.packets
> -echo $expected > expout
> -AT_CHECK([cat received1.packets], [0], [expout])
> -
> -# Packet to Expect at alice1
> -src_mac="000000010204"
> -dst_mac="0a0000a80203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap >
> received2.packets
> -echo $expected > expout
> -AT_CHECK([cat received2.packets], [0], [expout])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ovs-vswitchd restart])
> -AT_KEYWORDS([vswitchd])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -net_add n1
> -sim_add hv1
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -OVN_POPULATE_ARP
> -sleep 2
> -
> -as hv1 ovs-vsctl show
> -
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl dump-flows br-int
> -total_flows=`as hv1 ovs-ofctl dump-flows br-int | wc -l`
> -
> -echo "Total flows before vswitchd restart = " $total_flows
> -
> -# Code taken from ovs-save utility
> -save_flows () {
> -    echo "ovs-ofctl add-flows br-int - << EOF" > restore_flows.sh
> -    as hv1 ovs-ofctl dump-flows "br-int" | sed -e '/NXST_FLOW/d' \
> -            -e 's/\(idle\|hard\)_age=[^,]*,//g' >> restore_flows.sh
> -    echo "EOF" >> restore_flows.sh
> -}
> -
> -restart_vswitchd () {
> -    restore_flows=$1
> -
> -    if test $restore_flows = true; then
> -        save_flows
> -    fi
> -
> -    as hv1
> -    OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
> -
> -    if test $restore_flows = true; then
> -        as hv1
> -        ovs-vsctl --no-wait set open_vswitch .
> other_config:flow-restore-wait="true"
> -    fi
> -
> -    as hv1
> -    start_daemon ovs-vswitchd --enable-dummy=system -vvconn
> -vofproto_dpif -vunixctl
> -    ovs-ofctl dump-flows br-int
> -
> -    if test $restore_flows = true; then
> -        sh ./restore_flows.sh
> -        echo "Flows after restore"
> -        as hv1
> -        ovs-ofctl dump-flows br-int
> -        ovs-vsctl --no-wait --if-exists remove open_vswitch .
> other_config \
> -            flow-restore-wait="true"
> -    fi
> -}
> -
> -# Save the flows, restart vswitchd and restore the flows
> -restart_vswitchd true
> -OVS_WAIT_UNTIL([
> -    total_flows_after_restart=`as hv1 ovs-ofctl dump-flows br-int | wc -l`
> -    echo "Total flows after vswitchd restart = "
> $total_flows_after_restart
> -    test "${total_flows}" = "${total_flows_after_restart}"
> -])
> -
> -# Restart vswitchd without restoring
> -restart_vswitchd false
> -OVS_WAIT_UNTIL([
> -    total_flows_after_restart=`as hv1 ovs-ofctl dump-flows br-int | wc -l`
> -    echo "Total flows after vswitchd restart = "
> $total_flows_after_restart
> -    test "${total_flows}" = "${total_flows_after_restart}"
> -])
> -
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send arp for nexthop])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Topology: Two LSs - ls1 and ls2 are connected via router r0
> -
> -# Create logical switches
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -
> -# Create  router
> -ovn-nbctl create Logical_Router name=lr0
> -
> -# Add router ls1p1 port to gateway router
> -ovn-nbctl lrp-add lr0 lrp-ls1lp1 f0:00:00:00:00:01 192.168.0.1/24
> -ovn-nbctl <http://192.168.0.1/24-ovn-nbctl> lsp-add ls1 ls1lp1 -- set
> Logical_Switch_Port ls1lp1  \
> -    type=router options:router-port=lrp-ls1lp1 \
> -    addresses='"f0:00:00:00:00:01 192.168.0.1"'
> -
> -# Add router ls2p2 port to gateway router
> -ovn-nbctl lrp-add lr0 lrp-ls2lp1 f0:00:00:00:00:02 192.168.1.1/24
> -ovn-nbctl <http://192.168.1.1/24-ovn-nbctl> lsp-add ls2 ls2lp1 -- set
> Logical_Switch_Port ls2lp1 \
> -    type=router options:router-port=lrp-ls2lp1 \
> -    addresses='"f0:00:00:00:00:02 192.168.1.1"'
> -
> -# Set default gateway (nexthop) to 192.168.1.254
> -ovn-nbctl lr-route-add lr0 "0.0.0.0/0" 192.168.1.254 lrp-ls2lp1
> -
> -# Create logical port ls1lp2 in ls1
> -ovn-nbctl lsp-add ls1 ls1lp2 \
> --- lsp-set-addresses ls1lp2 "f0:00:00:00:00:03 192.168.0.2"
> -
> -# Create logical port ls2lp2 in ls2
> -ovn-nbctl lsp-add ls2 ls2lp2 \
> --- lsp-set-addresses ls2lp2 "f0:00:00:00:00:04 192.168.1.10"
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-ls1lp2 -- \
> -    set interface hv1-ls1lp2 external-ids:iface-id=ls1lp2 \
> -    options:tx_pcap=hv1/ls1lp2-tx.pcap \
> -    options:rxq_pcap=hv1/ls1lp2-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv1-ls2lp2 -- \
> -    set interface hv1-ls2lp2 external-ids:iface-id=ls2lp2 \
> -    options:tx_pcap=hv1/ls2lp2-tx.pcap \
> -    options:rxq_pcap=hv1/ls2lp2-rx.pcap \
> -    ofport-request=2
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -ovn-sbctl list chassis
> -ovn-sbctl list encap
> -echo "---------------------"
> -
> -echo "------Flows dump-----"
> -as hv1
> -ovs-ofctl dump-flows
> -echo "---------------------"
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -src_mac="f00000000003"
> -dst_mac="f00000000001"
> -src_ip=`ip_to_hex 192 168 0 2`
> -dst_ip=`ip_to_hex 8 8 8 8`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -# Send IP packet destined to 8.8.8.8 from lsp1lp2
> -as hv1 ovs-appctl netdev-dummy/receive hv1-ls1lp2 $packet
> -
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -# ARP packet should be received with Target IP Address set to
> 192.168.1.254 and
> -# not 8.8.8.8
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ls2lp2-tx.pcap |
> trim_zeros > packets
>
> -expected="fffffffffffff0000000000208060001080006040001f00000000002c0a80101000000000000c0a801fe"
> -echo $expected > expout
> -AT_CHECK([cat packets], [0], [expout])
> -cat packets
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send gratuitous arp for nat ips in localnet])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -# Create logical switch
> -ovn-nbctl ls-add ls0
> -# Create gateway router
> -ovn-nbctl create Logical_Router name=lr0 options:chassis=hv1
> -# Add router port to gateway router
> -ovn-nbctl lrp-add lr0 lrp0 f0:00:00:00:00:01 192.168.0.1/24
> -ovn-nbctl lsp-add ls0 lrp0-rp -- set Logical_Switch_Port lrp0-rp \
> -    type=router options:router-port=lrp0 addresses='"f0:00:00:00:00:01"'
> -# Add nat-address option
> -ovn-nbctl lsp-set-options lrp0-rp router-port=lrp0
> nat-addresses="f0:00:00:00:00:01 192.168.0.2"
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0
> -
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0])
> -AT_CHECK([ovs-vsctl add-port br-eth0 snoopvif -- set Interface snoopvif
> options:tx_pcap=hv1/snoopvif-tx.pcap options:rxq_pcap=hv1/snoopvif-rx.pcap])
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add ls0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl lsp-set-options ln_port network_name=physnet1])
> -
> -# Wait until the patch ports are created in hv1 to connect br-int to
> br-eth0
> -OVS_WAIT_UNTIL([test 1 = `as hv1 ovs-vsctl show | \
> -grep "Port patch-br-int-to-ln_port" | wc -l`])
> -
> -# Wait for packet to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 50])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros > packets
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80001000000000000c0a80001"
> -echo $expected > expout
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80002000000000000c0a80002"
> -echo $expected >> expout
> -AT_CHECK([sort packets], [0], [expout])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send gratuitous arp with nat-addresses router in
> localnet])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -# Create logical switch
> -ovn-nbctl ls-add ls0
> -# Create gateway router
> -ovn-nbctl create Logical_Router name=lr0 options:chassis=hv1
> -# Add router port to gateway router
> -ovn-nbctl lrp-add lr0 lrp0 f0:00:00:00:00:01 192.168.0.1/24
> -ovn-nbctl lsp-add ls0 lrp0-rp -- set Logical_Switch_Port lrp0-rp \
> -    type=router options:router-port=lrp0 addresses='"f0:00:00:00:00:01"'
> -# Add nat-address option
> -ovn-nbctl lsp-set-options lrp0-rp router-port=lrp0 nat-addresses="router"
> -# Add NAT rules
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 192.168.0.1 10.0.0.0/24])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 192.168.0.2 10.0.0.1])
> -# Add load balancers
> -AT_CHECK([ovn-nbctl lb-add lb0 192.168.0.3:80 10.0.0.2:80,10.0.0.3:80])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
> -AT_CHECK([ovn-nbctl lb-add lb1 192.168.0.3:8080 10.0.0.2:8080,
> 10.0.0.3:8080])
> -AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl \
> -    -- add-br br-phys \
> -    -- add-br br-eth0
> -
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0])
> -AT_CHECK([ovs-vsctl add-port br-eth0 snoopvif -- set Interface snoopvif
> options:tx_pcap=hv1/snoopvif-tx.pcap options:rxq_pcap=hv1/snoopvif-rx.pcap])
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add ls0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl lsp-set-options ln_port network_name=physnet1])
> -
> -# Wait until the patch ports are created to connect br-int to br-eth0
> -OVS_WAIT_UNTIL([test 1 = `ovs-vsctl show | \
> -grep "Port patch-br-int-to-ln_port" | wc -l`])
> -
> -ovn-sbctl list port_binding lrp0-rp
> -echo "*****"
> -ovn-nbctl list logical_switch_port lrp0-rp
> -ovn-nbctl list logical_router_port lrp0
> -ovn-nbctl show
> -# Wait for packet to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 50])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros > packets
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80001000000000000c0a80001"
> -echo $expected > expout
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80002000000000000c0a80002"
> -echo $expected >> expout
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80003000000000000c0a80003"
> -echo $expected >> expout
> -AT_CHECK([sort packets], [0], [expout])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- delete mac bindings])
> -ovn_start
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl -- add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -# Create logical switch ls0
> -ovn-nbctl ls-add ls0
> -# Create ports lp0, lp1 in ls0
> -ovn-nbctl lsp-add ls0 lp0
> -ovn-nbctl lsp-add ls0 lp1
> -ovn-nbctl lsp-set-addresses lp0 "f0:00:00:00:00:01 192.168.0.1"
> -ovn-nbctl lsp-set-addresses lp1 "f0:00:00:00:00:02 192.168.0.2"
> -dp_uuid=`ovn-sbctl find datapath | grep uuid | cut -f2 -d ":" | cut -f2
> -d " "`
> -ovn-sbctl create MAC_Binding ip=10.0.0.1 datapath=$dp_uuid
> logical_port=lp0 mac="mac1"
> -ovn-sbctl create MAC_Binding ip=10.0.0.1 datapath=$dp_uuid
> logical_port=lp1 mac="mac2"
> -ovn-sbctl find MAC_Binding
> -# Delete port lp0 and check that its MAC_Binding is deleted.
> -ovn-nbctl lsp-del lp0
> -ovn-sbctl find MAC_Binding
> -OVS_WAIT_UNTIL([test `ovn-sbctl find MAC_Binding logical_port=lp0 | wc
> -l` = 0])
> -# Delete logical switch ls0 and check that its MAC_Binding is deleted.
> -ovn-nbctl ls-del ls0
> -ovn-sbctl find MAC_Binding
> -OVS_WAIT_UNTIL([test `ovn-sbctl find MAC_Binding | wc -l` = 0])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- conntrack zone allocation])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# 2 logical switches "foo" (192.168.1.0/24) and "bar" (172.16.1.0/24)
> -# connected to a router R1.
> -# foo has foo1 to act as a client.
> -# bar has bar1, bar2, bar3 to act as servers.
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -for i in foo1 bar1 bar2 bar3; do
> -    ovs-vsctl -- add-port br-int $i -- \
> -        set interface $i external-ids:iface-id=$i \
> -        options:tx_pcap=hv1/$i-tx.pcap \
> -        options:rxq_pcap=hv1/$i-rx.pcap
> -done
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 172.16.1.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port bar1, bar2 and bar3 in bar
> -for i in `seq 1 3`; do
> -    ip=`expr $i + 1`
> -    ovn-nbctl lsp-add bar bar$i \
> -    -- lsp-set-addresses bar$i "f0:00:0a:01:02:$i 172.16.1.$ip"
> -done
> -
> -OVS_WAIT_UNTIL([test `ovs-ofctl dump-flows br-int table=0 | grep REG13 |
> wc -l` -eq 4])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- tag allocation])
> -ovn_start
> -
> -AT_CHECK([ovn-nbctl ls-add ls0])
> -AT_CHECK([ovn-nbctl lsp-add ls0 parent1])
> -AT_CHECK([ovn-nbctl lsp-add ls0 parent2])
> -AT_CHECK([ovn-nbctl ls-add ls1])
> -
> -dnl When a tag is provided, no allocation is done
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c0 parent1 3])
> -AT_CHECK([ovn-nbctl lsp-get-tag c0], [0], [3
> -])
> -dnl The same 'tag' gets created in southbound database.
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c0"], [0], [3
> -])
> -
> -dnl Allocate tags and see it getting created in both NB and SB
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c1 parent1 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c1], [0], [1
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c1"], [0], [1
> -])
> -
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c2 parent1 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c2], [0], [2
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c2"], [0], [2
> -])
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c3 parent1 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c3], [0], [4
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c3"], [0], [4
> -])
> -
> -dnl A different parent.
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c4 parent2 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c4], [0], [1
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c4"], [0], [1
> -])
> -
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c5 parent2 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c5], [0], [2
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c5"], [0], [2
> -])
> -
> -dnl Delete a logical port and create a new one.
> -AT_CHECK([ovn-nbctl --wait=sb lsp-del c1])
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c6 parent1 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag c6], [0], [1
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c6"], [0], [1
> -])
> -
> -dnl Restart northd to see that the same allocation remains.
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -start_daemon ovn-northd \
> -    --ovnnb-db=unix:"$ovs_base"/ovn-nb/ovn-nb.sock \
> -    --ovnsb-db=unix:"$ovs_base"/ovn-sb/ovn-sb.sock
> -
> -dnl Create a switch to make sure that ovn-northd has run through the main
> loop.
> -AT_CHECK([ovn-nbctl --wait=sb ls-add ls-dummy])
> -AT_CHECK([ovn-nbctl lsp-get-tag c0], [0], [3
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c6], [0], [1
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c2], [0], [2
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c3], [0], [4
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c4], [0], [1
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c5], [0], [2
> -])
> -
> -dnl Create a switch port with a tag that has already been allocated.
> -dnl It should go through fine with a duplicate tag.
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls1 c7 parent2 2])
> -AT_CHECK([ovn-nbctl lsp-get-tag c7], [0], [2
> -])
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="c7"], [0], [2
> -])
> -AT_CHECK([ovn-nbctl lsp-get-tag c5], [0], [2
> -])
> -
> -AT_CHECK([ovn-nbctl ls-add ls2])
> -dnl When there is no parent_name provided (for say, 'localnet'),
> 'tag_request'
> -dnl gets copied to 'tag'
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls2 local0 "" 25])
> -AT_CHECK([ovn-nbctl lsp-get-tag local0], [0], [25
> -])
> -dnl The same 'tag' gets created in southbound database.
> -AT_CHECK([ovn-sbctl --data=bare --no-heading --columns=tag find
> port_binding \
> -logical_port="local0"], [0], [25
> -])
> -dnl If 'tag_request' is 0 for localnet, nothing gets written to 'tag'
> -AT_CHECK([ovn-nbctl --wait=sb lsp-add ls2 local1 "" 0])
> -AT_CHECK([ovn-nbctl lsp-get-tag local1])
> -dnl change the tag_request.
> -AT_CHECK([ovn-nbctl --wait=sb  set logical_switch_port local1
> tag_request=50])
> -AT_CHECK([ovn-nbctl lsp-get-tag local1], [0], [50
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- lsp deletion and broadcast-flow deletion on localnet])
> -ovn_start
> -ovn-nbctl ls-add lsw0
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -    ovs-vsctl add-br br-eth0
> -    AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-eth0])
> -done
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add lsw0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl lsp-set-options ln_port network_name=physnet1])
> -
> -
> -# Create 3 vifs.
> -AT_CHECK([ovn-nbctl lsp-add lsw0 localvif1])
> -AT_CHECK([ovn-nbctl lsp-set-addresses localvif1 "f0:00:00:00:00:01
> 192.168.1.1"])
> -AT_CHECK([ovn-nbctl lsp-set-port-security localvif1 "f0:00:00:00:00:01"])
> -AT_CHECK([ovn-nbctl lsp-add lsw0 localvif2])
> -AT_CHECK([ovn-nbctl lsp-set-addresses localvif2 "f0:00:00:00:00:02
> 192.168.1.2"])
> -AT_CHECK([ovn-nbctl lsp-set-port-security localvif2 "f0:00:00:00:00:02"])
> -AT_CHECK([ovn-nbctl lsp-add lsw0 localvif3])
> -AT_CHECK([ovn-nbctl lsp-set-addresses localvif3 "f0:00:00:00:00:03
> 192.168.1.3"])
> -AT_CHECK([ovn-nbctl lsp-set-port-security localvif3 "f0:00:00:00:00:03"])
> -
> -# Bind the localvif1 to hv1.
> -as hv1
> -AT_CHECK([ovs-vsctl add-port br-int localvif1 -- set Interface localvif1
> external_ids:iface-id=localvif1])
> -
> -# On hv1, check that there are no flows outputting bcast to tunnel
> -OVS_WAIT_UNTIL([test `ovs-ofctl dump-flows br-int table=32 | ofctl_strip
> | grep output | wc -l` -eq 0])
> -
> -# On hv2, check that no flow outputs bcast to tunnel to hv1.
> -as hv2
> -OVS_WAIT_UNTIL([test `ovs-ofctl dump-flows br-int table=32 | ofctl_strip
> | grep output | wc -l` -eq 0])
> -
> -# Now bind vif2 on hv2.
> -AT_CHECK([ovs-vsctl add-port br-int localvif2 -- set Interface localvif2
> external_ids:iface-id=localvif2])
> -
> -# At this point, the broadcast flow on vif2 should be deleted.
> -# because, there is now a localnet vif bound (table=32 programming logic)
> -OVS_WAIT_UNTIL([test `ovs-ofctl dump-flows br-int table=32 | ofctl_strip
> | grep output | wc -l` -eq 0])
> -
> -# Verify that the local net patch port exists on hv2.
> -OVS_WAIT_UNTIL([test `ovs-vsctl show | grep "Port
> patch-br-int-to-ln_port" | wc -l` -eq 1])
> -
> -# Now bind vif3 on hv2.
> -AT_CHECK([ovs-vsctl add-port br-int localvif3 -- set Interface localvif3
> external_ids:iface-id=localvif3])
> -
> -# Verify that the local net patch port still exists on hv2
> -OVS_WAIT_UNTIL([test `ovs-vsctl show | grep "Port
> patch-br-int-to-ln_port" | wc -l` -eq 1])
> -
> -# Delete localvif2
> -AT_CHECK([ovn-nbctl lsp-del localvif2])
> -
> -# Verify that the local net patch port still exists on hv2,
> -# because, localvif3 is still bound.
> -OVS_WAIT_UNTIL([test `ovs-vsctl show | grep "Port
> patch-br-int-to-ln_port" | wc -l` -eq 1])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -
> -AT_SETUP([ovn -- ACL logging])
> -AT_KEYWORDS([ovn])
> -ovn_start
> -
> -net_add n1
> -
> -sim_add hv
> -as hv
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -for i in lp1 lp2; do
> -    ovs-vsctl -- add-port br-int $i -- \
> -        set interface $i external-ids:iface-id=$i \
> -        options:tx_pcap=hv/$i-tx.pcap \
> -        options:rxq_pcap=hv/$i-rx.pcap
> -done
> -
> -lp1_mac="f0:00:00:00:00:01"
> -lp1_ip="192.168.1.2"
> -
> -lp2_mac="f0:00:00:00:00:02"
> -lp2_ip="192.168.1.3"
> -
> -ovn-nbctl ls-add lsw0
> -ovn-nbctl --wait=sb lsp-add lsw0 lp1
> -ovn-nbctl --wait=sb lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp1 $lp1_mac
> -ovn-nbctl lsp-set-addresses lp2 $lp2_mac
> -ovn-nbctl --wait=sb sync
> -
> -ovn-nbctl acl-add lsw0 to-lport 1000 'tcp.dst==80' drop
> -ovn-nbctl --log --severity=alert --name=drop-flow acl-add lsw0 to-lport
> 1000 'tcp.dst==81' drop
> -
> -ovn-nbctl acl-add lsw0 to-lport 1000 'tcp.dst==82' allow
> -ovn-nbctl --log --severity=info --name=allow-flow acl-add lsw0 to-lport
> 1000 'tcp.dst==83' allow
> -
> -ovn-nbctl acl-add lsw0 to-lport 1000 'tcp.dst==84' allow-related
> -ovn-nbctl --log acl-add lsw0 to-lport 1000 'tcp.dst==85' allow-related
> -
> -ovn-nbctl acl-add lsw0 to-lport 1000 'tcp.dst==86' reject
> -ovn-nbctl --wait=hv --log --severity=alert --name=reject-flow acl-add
> lsw0 to-lport 1000 'tcp.dst==87' reject
> -
> -ovn-sbctl dump-flows
> -
> -
> -# Send packet that should be dropped without logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4360 && tcp.dst==80"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should be dropped with logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4361 && tcp.dst==81"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should be allowed without logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4362 && tcp.dst==82"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should be allowed with logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4363 && tcp.dst==83"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should allow related flows without logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4364 && tcp.dst==84"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should allow related flows with logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4365 && tcp.dst==85"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should be rejected without logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4366 && tcp.dst==86"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Send packet that should be rejected with logging.
> -packet="inport==\"lp1\" && eth.src==$lp1_mac && eth.dst==$lp2_mac &&
> -        ip4 && ip.ttl==64 && ip4.src==$lp1_ip && ip4.dst==$lp2_ip &&
> -        tcp && tcp.flags==2 && tcp.src==4367 && tcp.dst==87"
> -as hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -OVS_WAIT_UNTIL([ test 4 = $(grep -c 'acl_log' hv/ovn-controller.log) ])
> -
> -AT_CHECK([grep 'acl_log' hv/ovn-controller.log | sed 's/.*name=/name=/'],
> [0], [dnl
> -name="drop-flow", verdict=drop, severity=alert:
> tcp,vlan_tci=0x0000,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02,nw_src=192.168.1.2,nw_dst=192.168.1.3,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=4361,tp_dst=81,tcp_flags=syn
> -name="allow-flow", verdict=allow, severity=info:
> tcp,vlan_tci=0x0000,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02,nw_src=192.168.1.2,nw_dst=192.168.1.3,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=4363,tp_dst=83,tcp_flags=syn
> -name="<unnamed>", verdict=allow, severity=info:
> tcp,vlan_tci=0x0000,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02,nw_src=192.168.1.2,nw_dst=192.168.1.3,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=4365,tp_dst=85,tcp_flags=syn
> -name="reject-flow", verdict=reject, severity=alert:
> tcp,vlan_tci=0x0000,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02,nw_src=192.168.1.2,nw_dst=192.168.1.3,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=4367,tp_dst=87,tcp_flags=syn
> -])
> -
> -OVN_CLEANUP([hv])
> -AT_CLEANUP
> -
> -
> -AT_SETUP([ovn -- ACL rate-limited logging])
> -AT_KEYWORDS([ovn])
> -ovn_start
> -
> -net_add n1
> -
> -sim_add hv
> -as hv
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -for i in lp1 lp2; do
> -    ovs-vsctl -- add-port br-int $i -- \
> -        set interface $i external-ids:iface-id=$i \
> -        options:tx_pcap=hv/$i-tx.pcap \
> -        options:rxq_pcap=hv/$i-rx.pcap
> -done
> -
> -lp1_mac="f0:00:00:00:00:01"
> -lp1_ip="192.168.1.2"
> -
> -lp2_mac="f0:00:00:00:00:02"
> -lp2_ip="192.168.1.3"
> -
> -ovn-nbctl ls-add lsw0
> -ovn-nbctl --wait=sb lsp-add lsw0 lp1
> -ovn-nbctl --wait=sb lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp1 $lp1_mac
> -ovn-nbctl lsp-set-addresses lp2 $lp2_mac
> -ovn-nbctl --wait=sb sync
> -
> -
> -# Add an ACL that rate-limits logs at 10 per second.
> -ovn-nbctl meter-add http-rl1 drop 10 pktps
> -ovn-nbctl --log --severity=alert --meter=http-rl1 --name=http-acl1
> acl-add lsw0 to-lport 1000 'tcp.dst==80' drop
> -
> -# Add an ACL that rate-limits logs at 5 per second.
> -ovn-nbctl meter-add http-rl2 drop 5 pktps
> -ovn-nbctl --log --severity=alert --meter=http-rl2 --name=http-acl2
> acl-add lsw0 to-lport 1000 'tcp.dst==81' allow
> -
> -# Add an ACL that doesn't rate-limit logs.
> -ovn-nbctl --log --severity=alert --name=http-acl3 acl-add lsw0 to-lport
> 1000 'tcp.dst==82' drop
> -ovn-nbctl --wait=hv sync
> -
> -# For each ACL, send 100 packets.
> -for i in `seq 1 100`; do
> -    ovs-appctl netdev-dummy/receive lp1
> 'in_port(1),eth(src=f0:00:00:00:00:01,dst=f0:00:00:00:00:02),eth_type(0x0800),ipv4(src=192.168.1.2,dst=192.168.1.3,proto=6,tos=0,ttl=64,frag=no),tcp(src=7777,dst=80)'
> -
> -    ovs-appctl netdev-dummy/receive lp1
> 'in_port(1),eth(src=f0:00:00:00:00:01,dst=f0:00:00:00:00:02),eth_type(0x0800),ipv4(src=192.168.1.2,dst=192.168.1.3,proto=6,tos=0,ttl=64,frag=no),tcp(src=7777,dst=81)'
> -
> -    ovs-appctl netdev-dummy/receive lp1
> 'in_port(1),eth(src=f0:00:00:00:00:01,dst=f0:00:00:00:00:02),eth_type(0x0800),ipv4(src=192.168.1.2,dst=192.168.1.3,proto=6,tos=0,ttl=64,frag=no),tcp(src=7777,dst=82)'
> -done
> -
> -# The rate at which packets are sent is highly system-dependent, so we
> -# can't count on precise drop counts.  To work around that, we just
> -# check that exactly 100 "http-acl3" actions were logged and that there
> -# were more "http-acl1" actions than "http-acl2" ones.
> -OVS_WAIT_UNTIL([ test 100 = $(grep -c 'http-acl3' hv/ovn-controller.log)
> ])
> -
> -# On particularly slow or overloaded systems, the transmission rate may
> -# be lower than the configured meter rate.  To prevent false test
> -# failures, we check the duration count of the meter, and if it's
> -# greater than nine seconds, just skip the test.
> -d_secs=$(as hv ovs-ofctl -O OpenFlow13 meter-stats br-int | grep
> "meter:1" | sed 's/.* duration:\([[0-9]]\{1,\}\)\.[[0-9]]\+s .*/\1/')
> -
> -echo "Meter duration: $d_secs"
> -AT_SKIP_IF([test $d_secs -gt 9])
> -
> -# Print some information that may help debugging.
> -as hv ovs-appctl -t ovn-controller meter-table-list
> -as hv ovs-ofctl -O OpenFlow13 meter-stats br-int
> -
> -n_acl1=$(grep -c 'http-acl1' hv/ovn-controller.log)
> -n_acl2=$(grep -c 'http-acl2' hv/ovn-controller.log)
> -n_acl3=$(grep -c 'http-acl3' hv/ovn-controller.log)
> -
> -AT_CHECK([ test $n_acl3 -gt $n_acl1 ], [0], [])
> -AT_CHECK([ test $n_acl1 -gt $n_acl2 ], [0], [])
> -
> -OVN_CLEANUP([hv])
> -AT_CLEANUP
> -
> -
> -AT_SETUP([ovn -- DSCP marking and meter check])
> -AT_KEYWORDS([ovn])
> -ovn_start
> -
> -ovn-nbctl ls-add lsw0
> -ovn-nbctl --wait=sb lsp-add lsw0 lp1
> -ovn-nbctl --wait=sb lsp-add lsw0 lp2
> -ovn-nbctl --wait=sb lsp-add lsw0 lp3
> -ovn-nbctl lsp-set-addresses lp1 f0:00:00:00:00:01
> -ovn-nbctl lsp-set-addresses lp2 f0:00:00:00:00:02
> -ovn-nbctl lsp-set-addresses lp3 f0:00:00:00:00:03
> -ovn-nbctl lsp-set-port-security lp1 f0:00:00:00:00:01
> -ovn-nbctl lsp-set-port-security lp2 f0:00:00:00:00:02
> -ovn-nbctl --wait=sb sync
> -net_add n1
> -sim_add hv
> -as hv
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1 options:tx_pcap=vif1-tx.pcap
> options:rxq_pcap=vif1-rx.pcap ofport-request=1
> -ovs-vsctl add-port br-int vif2 -- set Interface vif2
> external-ids:iface-id=lp2 options:tx_pcap=vif2-tx.pcap
> options:rxq_pcap=vif2-rx.pcap ofport-request=2
> -
> -AT_CAPTURE_FILE([trace])
> -ovn_trace () {
> -    ovn-trace --all "$@" | tee trace | sed '1,/Minimal trace/d'
> -}
> -
> -# Extracts nw_tos from the final flow from ofproto/trace output and prints
> -# it on stdout.  Prints "none" if no nw_tos was included.
> -get_final_nw_tos() {
> -    if flow=$(grep '^Final flow:' stdout); then :; else
> -       # The output didn't have a final flow.
> -       return 99
> -    fi
> -
> -    tos=$(echo "$flow" | sed -n 's/.*nw_tos=\([[0-9]]\{1,\}\).*/\1/p')
> -    case $tos in
> -        '') echo none ;;
> -        *) echo $tos ;;
> -    esac
> -}
> -
> -# check_tos TOS
> -#
> -# Checks that a packet from 1.1.1.1 to 1.1.1.2 gets its DSCP set to TOS.
> -check_tos() {
> -    # First check with ovn-trace for logical flows.
> -    echo "checking for tos $1"
> -    (if test $1 != 0; then echo "ip.dscp = $1;"; fi;
> -     echo 'output("lp2");') > expout
> -    AT_CHECK_UNQUOTED([ovn_trace lsw0 'inport == "lp1" && eth.src ==
> f0:00:00:00:00:01 && eth.dst == f0:00:00:00:00:02 && ip4.src == 1.1.1.1 &&
> ip4.dst == 1.1.1.2'], [0], [expout])
> -
> -    # Then re-check with ofproto/trace for a physical packet.
> -    AT_CHECK([ovs-appctl ofproto/trace br-int
> 'in_port=1,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02,dl_type=0x800,nw_src=1.1.1.1,nw_dst=1.1.1.2'],
> [0], [stdout-nolog])
> -    AT_CHECK_UNQUOTED([get_final_nw_tos], [0], [`expr $1 \* 4`
> -])
> -}
> -
> -# check at L2
> -AT_CHECK([ovn_trace lsw0 'inport == "lp1" && eth.src == f0:00:00:00:00:01
> && eth.dst == f0:00:00:00:00:02'], [0], [output("lp2");
> -])
> -AT_CHECK([ovs-appctl ofproto/trace br-int
> 'in_port=1,dl_src=f0:00:00:00:00:01,dl_dst=f0:00:00:00:00:02'], [0],
> [stdout-nolog])
> -AT_CHECK([get_final_nw_tos], [0], [none
> -])
> -
> -# check at L3 without dscp marking
> -check_tos 0
> -
> -# Mark DSCP with a valid value
> -qos_id=$(ovn-nbctl --wait=hv -- --id=@lp1-qos create QoS priority=100
> action=dscp=48 match="inport\=\=\"lp1\"\ &&\ is_chassis_resident(\"lp1\")"
> direction="from-lport" -- set Logical_Switch lsw0 qos_rules=@lp1-qos)
> -AT_CHECK([as hv ovn-nbctl qos-list lsw0 | wc -l], [0], [1
> -])
> -check_tos 48
> -
> -# check at hv without qos meter
> -AT_CHECK([as hv ovs-ofctl dump-flows br-int -O OpenFlow13 | grep meter |
> wc -l], [0], [0
> -])
> -
> -# Update the meter rate
> -ovn-nbctl --wait=hv set QoS $qos_id bandwidth=rate=100
> -
> -# check at hv with a qos meter table
> -AT_CHECK([as hv ovs-ofctl dump-meters br-int -O OpenFlow13 | grep
> rate=100 | wc -l], [0], [1
> -])
> -AT_CHECK([as hv ovs-ofctl dump-flows br-int -O OpenFlow13 | grep meter |
> wc -l], [0], [1
> -])
> -
> -# Update the DSCP marking
> -ovn-nbctl --wait=hv set QoS $qos_id action=dscp=63
> -check_tos 63
> -
> -# Update the meter rate
> -ovn-nbctl --wait=hv set QoS $qos_id
> bandwidth=rate=4294967295,burst=4294967295
> -
> -# check at hv with a qos meter table
> -AT_CHECK([as hv ovs-ofctl dump-meters br-int -O OpenFlow13 | grep
> burst_size=4294967295 | wc -l], [0], [1
> -])
> -AT_CHECK([as hv ovs-ofctl dump-flows br-int -O OpenFlow13 | grep meter |
> wc -l], [0], [1
> -])
> -
> -ovn-nbctl --wait=hv set QoS $qos_id match="outport\=\=\"lp2\""
> direction="to-lport"
> -check_tos 63
> -
> -# Disable DSCP marking
> -ovn-nbctl --wait=hv qos-del lsw0
> -AT_CHECK([as hv ovn-nbctl qos-list lsw0 | wc -l], [0], [0
> -])
> -check_tos 0
> -
> -# check at hv without qos meter
> -AT_CHECK([as hv ovs-ofctl dump-flows br-int -O OpenFlow13 | grep meter |
> wc -l], [0], [0
> -])
> -
> -# check meter with chassis not resident
> -ovn-nbctl qos-add lsw0 to-lport 1001 'inport=="lp3" &&
> is_chassis_resident("lp3")' rate=11123 burst=111230
> -AT_CHECK([as hv ovn-nbctl qos-list lsw0 | wc -l], [0], [1
> -])
> -
> -# check no meter table
> -AT_CHECK([as hv ovs-ofctl dump-flows br-int -O OpenFlow13 | grep meter |
> wc -l], [0], [0
> -])
> -AT_CHECK([as hv ovs-ofctl dump-meters br-int -O OpenFlow13 | grep
> rate=11123 | wc -l], [0], [0
> -])
> -
> -OVN_CLEANUP([hv])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- read-only sb db:ptcp access])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -
> -: > .$1.db.~lock~
> -ovsdb-tool create ovn-sb.db "$abs_top_srcdir"/ovn/ovn-sb.ovsschema
> -
> -# Add read-only remote to sb ovsdb-server
> -AT_CHECK(
> -  [ovsdb-tool transact ovn-sb.db \
> -     ['["OVN_Southbound",
> -       {"op": "insert",
> -        "table": "SB_Global",
> -        "row": {
> -          "connections": ["set", [["named-uuid", "xyz"]]]}},
> -       {"op": "insert",
> -        "table": "Connection",
> -        "uuid-name": "xyz",
> -        "row": {"target": "ptcp:0:127.0.0.1",
> -               "read_only": true}}]']], [0], [ignore], [ignore])
> -
> -start_daemon ovsdb-server --remote=punix:ovn-sb.sock
> --remote=db:OVN_Southbound,SB_Global,connections ovn-sb.db
> -
> -PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT])
> -
> -# read-only accesses should succeed
> -AT_CHECK([ovn-sbctl --db=tcp:127.0.0.1:$TCP_PORT list SB_Global], [0],
> [stdout], [ignore])
> -AT_CHECK([ovn-sbctl --db=tcp:127.0.0.1:$TCP_PORT list Connection], [0],
> [stdout], [ignore])
> -
> -# write access should fail
> -AT_CHECK([ovn-sbctl --db=tcp:127.0.0.1:$TCP_PORT chassis-add ch vxlan
> 1.2.4.8], [1], [ignore],
> -[ovn-sbctl: transaction error: {"details":"insert operation not allowed
> when database server is in read only mode","error":"not allowed"}
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- read-only sb db:pssl access])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -AT_SKIP_IF([test "$HAVE_OPENSSL" = no])
> -PKIDIR="$(cd $abs_top_builddir/tests && pwd)"
> -AT_SKIP_IF([expr "$PKIDIR" : ".*[      '\"
> -\\]"])
> -
> -: > .$1.db.~lock~
> -ovsdb-tool create ovn-sb.db "$abs_top_srcdir"/ovn/ovn-sb.ovsschema
> -
> -# Add read-only remote to sb ovsdb-server
> -AT_CHECK(
> -  [ovsdb-tool transact ovn-sb.db \
> -     ['["OVN_Southbound",
> -       {"op": "insert",
> -        "table": "SB_Global",
> -        "row": {
> -          "connections": ["set", [["named-uuid", "xyz"]]]}},
> -       {"op": "insert",
> -        "table": "Connection",
> -        "uuid-name": "xyz",
> -        "row": {"target": "pssl:0:127.0.0.1",
> -               "read_only": true}}]']], [0], [ignore], [ignore])
> -
> -start_daemon ovsdb-server --remote=punix:ovn-sb.sock \
> -
> --remote=db:OVN_Southbound,SB_Global,connections \
> -                          --private-key="$PKIDIR/testpki-privkey2.pem" \
> -                          --certificate="$PKIDIR/testpki-cert2.pem" \
> -                          --ca-cert="$PKIDIR/testpki-cacert.pem" \
> -                          ovn-sb.db
> -
> -PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT])
> -
> -# read-only accesses should succeed
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -                    list SB_Global], [0], [stdout], [ignore])
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -                    list Connection], [0], [stdout], [ignore])
> -
> -# write access should fail
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -                    chassis-add ch vxlan 1.2.4.8], [1], [ignore],
> -[ovn-sbctl: transaction error: {"details":"insert operation not allowed
> when database server is in read only mode","error":"not allowed"}
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- nb connection/ssl commands])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -AT_SKIP_IF([test "$HAVE_OPENSSL" = no])
> -PKIDIR="$(cd $abs_top_builddir/tests && pwd)"
> -AT_SKIP_IF([expr "$PKIDIR" : ".*[      '\"
> -\\]"])
> -
> -: > .$1.db.~lock~
> -ovsdb-tool create ovn-nb.db "$abs_top_srcdir"/ovn/ovn-nb.ovsschema
> -
> -# Start nb db server using db connection/ssl entries (unpopulated
> initially)
> -start_daemon ovsdb-server --remote=punix:ovnnb_db.sock \
> -
> --remote=db:OVN_Northbound,NB_Global,connections \
> -                          --private-key=db:OVN_Northbound,SSL,private_key
> \
> -                          --certificate=db:OVN_Northbound,SSL,certificate
> \
> -                          --ca-cert=db:OVN_Northbound,SSL,ca_cert \
> -                          ovn-nb.db
> -
> -# Populate SSL configuration entries in nb db
> -AT_CHECK(
> -    [ovn-nbctl set-ssl $PKIDIR/testpki-privkey.pem \
> -                       $PKIDIR/testpki-cert.pem \
> -                       $PKIDIR/testpki-cacert.pem], [0], [stdout],
> [ignore])
> -
> -# Populate a passive SSL connection in nb db
> -AT_CHECK([ovn-nbctl set-connection pssl:0:127.0.0.1], [0], [stdout],
> [ignore])
> -
> -PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT])
> -
> -# Verify SSL connetivity to nb db server
> -AT_CHECK([ovn-nbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          list NB_Global],
> -         [0], [stdout], [ignore])
> -AT_CHECK([ovn-nbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          list Connection],
> -         [0], [stdout], [ignore])
> -AT_CHECK([ovn-nbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          get-connection],
> -         [0], [stdout], [ignore])
> -
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- sb connection/ssl commands])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -AT_SKIP_IF([test "$HAVE_OPENSSL" = no])
> -PKIDIR="$(cd $abs_top_builddir/tests && pwd)"
> -AT_SKIP_IF([expr "$PKIDIR" : ".*[      '\"
> -\\]"])
> -
> -: > .$1.db.~lock~
> -ovsdb-tool create ovn-sb.db "$abs_top_srcdir"/ovn/ovn-sb.ovsschema
> -
> -# Start sb db server using db connection/ssl entries (unpopulated
> initially)
> -start_daemon ovsdb-server --remote=punix:ovnsb_db.sock \
> -
> --remote=db:OVN_Southbound,SB_Global,connections \
> -                          --private-key=db:OVN_Southbound,SSL,private_key
> \
> -                          --certificate=db:OVN_Southbound,SSL,certificate
> \
> -                          --ca-cert=db:OVN_Southbound,SSL,ca_cert \
> -                          ovn-sb.db
> -
> -# Populate SSL configuration entries in sb db
> -AT_CHECK(
> -    [ovn-sbctl set-ssl $PKIDIR/testpki-privkey.pem \
> -                       $PKIDIR/testpki-cert.pem \
> -                       $PKIDIR/testpki-cacert.pem], [0], [stdout],
> [ignore])
> -
> -# Populate a passive SSL connection in sb db
> -AT_CHECK([ovn-sbctl set-connection pssl:0:127.0.0.1], [0], [stdout],
> [ignore])
> -
> -PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT])
> -
> -# Verify SSL connetivity to sb db server
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          list SB_Global],
> -         [0], [stdout], [ignore])
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          list Connection],
> -         [0], [stdout], [ignore])
> -AT_CHECK([ovn-sbctl --db=ssl:127.0.0.1:$TCP_PORT \
> -                    --private-key=$PKIDIR/testpki-privkey.pem \
> -                    --certificate=$PKIDIR/testpki-cert.pem \
> -                    --ca-cert=$PKIDIR/testpki-cacert.pem \
> -          get-connection],
> -         [0], [stdout], [ignore])
> -
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- nested containers])
> -ovn_start
> -
> -# Physical network:
> -# 2 HVs. HV1 has 2 VMs - "VM1" and "bar3". HV2 has 1 VM - "VM2"
> -
> -# Logical network:
> -# 3 Logical switches - "mgmt" (172.16.1.0/24), "foo" (192.168.1.0/24)
> -# and "bar" (192.168.2.0/24). They are all connected to router R1.
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl ls-add mgmt
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -
> -# Connect mgmt to R1
> -ovn-nbctl lrp-add R1 mgmt 00:00:00:01:02:02 172.16.1.1/24
> -ovn-nbctl lsp-add mgmt rp-mgmt -- set Logical_Switch_Port rp-mgmt
> type=router \
> -          options:router-port=mgmt addresses=\"00:00:00:01:02:02\"
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:00:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -          options:router-port=foo addresses=\"00:00:00:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:00:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar
> type=router \
> -          options:router-port=bar addresses=\"00:00:00:01:02:04\"
> -
> -# "mgmt" has VM1 and VM2 connected
> -ovn-nbctl lsp-add mgmt vm1 \
> --- lsp-set-addresses vm1 "f0:00:00:01:02:03 172.16.1.2"
> -
> -ovn-nbctl lsp-add mgmt vm2 \
> --- lsp-set-addresses vm2 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# "foo1" and "foo2" are containers belonging to switch "foo"
> -# "foo1" has "VM1" as parent_port and "foo2" has "VM2" as parent_port.
> -ovn-nbctl lsp-add foo foo1 vm1 1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:05 192.168.1.2"
> -
> -ovn-nbctl lsp-add foo foo2 vm2 2 \
> --- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
> -
> -# "bar1" and "bar2" are containers belonging to switch "bar"
> -# "bar1" has "VM1" as parent_port and "bar2" has "VM2" as parent_port.
> -ovn-nbctl lsp-add bar bar1 vm1 2 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:07 192.168.2.2"
> -
> -ovn-nbctl lsp-add bar bar2 vm2 1 \
> --- lsp-set-addresses bar2 "f0:00:00:01:02:08 192.168.2.3"
> -
> -# bar3 is a standalone VM belonging to switch "bar"
> -ovn-nbctl lsp-add bar bar3 \
> --- lsp-set-addresses bar3 "f0:00:00:01:02:09 192.168.2.4"
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int vm1 -- \
> -    set interface vm1 external-ids:iface-id=vm1 \
> -    options:tx_pcap=hv1/vm1-tx.pcap \
> -    options:rxq_pcap=hv1/vm1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int bar3 -- \
> -    set interface bar3 external-ids:iface-id=bar3 \
> -    options:tx_pcap=hv1/bar3-tx.pcap \
> -    options:rxq_pcap=hv1/bar3-rx.pcap \
> -    ofport-request=2
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int vm2 -- \
> -    set interface vm2 external-ids:iface-id=vm2 \
> -    options:tx_pcap=hv2/vm2-tx.pcap \
> -    options:rxq_pcap=hv2/vm2-rx.pcap \
> -    ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and foo2 (same switch, different HVs and
> -# different VLAN tags).
> -src_mac="f00000010205"
> -dst_mac="f00000010206"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 1 3`
>
> -packet=${dst_mac}${src_mac}8100000108004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at foo2
>
> -packet=${dst_mac}${src_mac}8100000208004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -echo  $packet > expected
> -OVN_CHECK_PACKETS([hv2/vm2-tx.pcap], [expected])
> -
> -# Send ip packets between foo1 and bar2 (different switch, different HV)
> -src_mac="f00000010205"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 3`
>
> -packet=${dst_mac}${src_mac}8100000108004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at bar2
> -src_mac="000000010204"
> -dst_mac="f00000010208"
>
> -packet=${dst_mac}${src_mac}8100000108004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -echo  $packet >> expected
> -OVN_CHECK_PACKETS([hv2/vm2-tx.pcap], [expected])
> -
> -# Send ip packets between foo1 and bar1
> -# (different switch, loopback to same vm but different tag)
> -src_mac="f00000010205"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -packet=${dst_mac}${src_mac}8100000108004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at bar1
> -src_mac="000000010204"
> -dst_mac="f00000010207"
>
> -packet=${dst_mac}${src_mac}8100000208004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -echo  $packet > expected1
> -OVN_CHECK_PACKETS([hv1/vm1-tx.pcap], [expected1])
> -
> -# Send ip packets between bar1 and bar3
> -# (same switch. But one is container and another is a standalone VM)
> -src_mac="f00000010207"
> -dst_mac="f00000010209"
> -src_ip=`ip_to_hex 192 168 2 2`
> -dst_ip=`ip_to_hex 192 168 2 3`
>
> -packet=${dst_mac}${src_mac}8100000208004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at bar3
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -echo  $packet > expected
> -OVN_CHECK_PACKETS([hv1/bar3-tx.pcap], [expected])
> -
> -# Send ip packets between foo1 and vm1.
> -(different switch, container to the VM hosting it.)
> -src_mac="f00000010205"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 2`
>
> -packet=${dst_mac}${src_mac}8100000108004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at vm1
> -src_mac="000000010202"
> -dst_mac="f00000010203"
>
> -packet=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -echo  $packet >> expected1
> -OVN_CHECK_PACKETS([hv1/vm1-tx.pcap], [expected1])
> -
> -# Send packets from vm1 to bar1.
> -(different switch, A hosting VM to a container inside it)
> -src_mac="f00000010203"
> -dst_mac="000000010202"
> -src_ip=`ip_to_hex 172 16 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at vm1
> -src_mac="000000010204"
> -dst_mac="f00000010207"
>
> -packet=${dst_mac}${src_mac}8100000208004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -echo  $packet >> expected1
> -OVN_CHECK_PACKETS([hv1/vm1-tx.pcap], [expected1])
> -
> -# Send broadcast packet from foo1. foo1 should not receive the same
> packet.
> -src_mac="f00000010205"
> -dst_mac="ffffffffffff"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 255 255 255 255`
>
> -packet=${dst_mac}${src_mac}8100000108004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive vm1 $packet
> -
> -# expected packet at VM1
> -OVN_CHECK_PACKETS([hv1/vm1-tx.pcap], [expected1])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 3 HVs, 3 LRs connected via LS, source IP based routes])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Three LRs - R1, R2 and R3 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and bar
> -# (192.168.2.0/24) connected to it.
> -#
> -# R2 and R3 are gateway routers.
> -# R2 has alice (172.16.1.0/24) and R3 has bob (172.16.1.0/24)
> -# connected to it. Note how both alice and bob have the same subnet
> behind it.
> -# We are trying to simulate external network via those 2 switches. In real
> -# world the switch ports of these switches will have addresses set as
> "unknown"
> -# to make them learning switches. Or those switches will be "localnet"
> ones.
> -
> -# Create three hypervisors and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=bar1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-int hv3-vif1 -- \
> -    set interface hv3-vif1 external-ids:iface-id=bob1 \
> -    options:tx_pcap=hv3/vif1-tx.pcap \
> -    options:rxq_pcap=hv3/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis="hv2"
> -ovn-nbctl create Logical_Router name=R3 options:chassis="hv3"
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add bob
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -    options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar
> type=router \
> -    options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect bob to R3
> -ovn-nbctl lrp-add R3 bob 00:00:03:01:02:03 172.16.1.2/24
> -ovn-nbctl lsp-add bob rp-bob -- set Logical_Switch_Port rp-bob \
> -    type=router options:router-port=bob addresses=\"00:00:03:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Connect R3 to join
> -ovn-nbctl lrp-add R3 R3_join 00:00:04:01:02:05 20.0.0.3/24
> -ovn-nbctl lsp-add join r3-join -- set Logical_Switch_Port r3-join \
> -    type=router options:router-port=R3_join
> addresses='"00:00:04:01:02:05"'
> -
> -# Install static routes with source ip address as the policy for routing.
> -# We want traffic from 'foo' to go via R2 and traffic of 'bar' to go via
> R3.
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.1.0/24 20.0.0.2
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.2.0/24 20.0.0.3
> -
> -# Install static routes with destination ip address as the policy for
> routing.
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -
> -ovn-nbctl lr-route-add R3 192.168.0.0/16 20.0.0.1
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port bar1 in bar
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:04 192.168.2.2"
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.3"
> -
> -# Create logical port bob1 in bob
> -ovn-nbctl lsp-add bob bob1 \
> --- lsp-set-addresses bob1 "f0:00:00:01:02:06 172.16.1.4"
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -# Send ip packets between foo1 and bar1
> -# (East-west traffic should flow normally)
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 3`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -as hv1 ovs-appctl ofproto/trace br-int in_port=1 $packet
> -
> -# Send ip packets between bar1 and bob1
> -src_mac="f00000010204"
> -dst_mac="000001010204"
> -src_ip=`ip_to_hex 192 168 2 2`
> -dst_ip=`ip_to_hex 172 16 1 4`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif2 $packet
> -#as hv1 ovs-appctl ofproto/trace br-int in_port=2 $packet
> -
> -# Packet to expect at bar1
> -src_mac="000001010204"
> -dst_mac="f00000010204"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 192 168 2 2`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -echo $expected > expected
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -
> -# Packet to Expect at alice1
> -src_mac="000002010203"
> -dst_mac="f00000010205"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 3`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
> -echo $expected > expected
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Packet to Expect at bob1
> -src_mac="000003010203"
> -dst_mac="f00000010206"
> -src_ip=`ip_to_hex 192 168 2 2`
> -dst_ip=`ip_to_hex 172 16 1 4`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
> -echo $expected > expected
> -OVN_CHECK_PACKETS([hv3/vif1-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- dns lookup : 1 HV, 2 LS, 2 LSPs/LS])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4 aef0::4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp1 "f0:00:00:00:00:01 10.0.0.4
> aef0::4"
> -
> -ovn-nbctl lsp-add ls1 ls1-lp2 \
> --- lsp-set-addresses ls1-lp2 "f0:00:00:00:00:02 10.0.0.6 20.0.0.4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp2 "f0:00:00:00:00:02 10.0.0.6
> 20.0.0.4"
> -
> -DNS1=`ovn-nbctl create DNS records={}`
> -DNS2=`ovn-nbctl create DNS records={}`
> -
> -ovn-nbctl set DNS $DNS1 records:vm1.ovn.org="10.0.0.4 aef0::4"
> -ovn-nbctl set DNS $DNS1 records:vm2.ovn.org="10.0.0.6 20.0.0.4"
> -ovn-nbctl set DNS $DNS2 records:vm3.ovn.org="40.0.0.4"
> -
> -ovn-nbctl set Logical_switch ls1 dns_records="$DNS1"
> -
> -net_add n1
> -sim_add hv1
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=ls1-lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -OVN_POPULATE_ARP
> -sleep 2
> -as hv1 ovs-vsctl show
> -
> -echo "*************************"
> -ovn-sbctl list DNS
> -echo "*************************"
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -# set_dns_params host_name
> -# Sets the dns_req_data and dns_resp_data
> -set_dns_params() {
> -    local hname=$1
> -    local ttl=00000e10
> -    an_count=0001
> -    type=0001
> -    case $hname in
> -    vm1)
> -        # vm1.ovn.org
> -        query_name=03766d31036f766e036f726700
> -        # IPv4 address - 10.0.0.4
> -        expected_dns_answer=${query_name}00010001${ttl}00040a000004
> -        ;;
> -    vm2)
> -        # vm2.ovn.org
> -        query_name=03766d32036f766e036f726700
> -        # IPv4 address - 10.0.0.6
> -        expected_dns_answer=${query_name}00010001${ttl}00040a000006
> -        # IPv4 address - 20.0.0.4
> -
> expected_dns_answer=${expected_dns_answer}${query_name}00010001${ttl}000414000004
> -        an_count=0002
> -        ;;
> -    vm3)
> -        # vm3.ovn.org
> -        query_name=03766d33036f766e036f726700
> -        # IPv4 address - 40.0.0.4
> -        expected_dns_answer=${query_name}00010001${ttl}000428000004
> -        ;;
> -    vm1_ipv6_only)
> -        # vm1.ovn.org
> -        query_name=03766d31036f766e036f726700
> -        # IPv6 address - aef0::4
> -        type=001c
> -
> expected_dns_answer=${query_name}${type}0001${ttl}0010aef00000000000000000000000000004
> -        ;;
> -    vm1_ipv4_v6)
> -        # vm1.ovn.org
> -        query_name=03766d31036f766e036f726700
> -        type=00ff
> -        an_count=0002
> -        # IPv4 address - 10.0.0.4
> -        # IPv6 address - aef0::4
> -        expected_dns_answer=${query_name}00010001${ttl}00040a000004
> -
> expected_dns_answer=${expected_dns_answer}${query_name}001c0001${ttl}0010
> -
> expected_dns_answer=${expected_dns_answer}aef00000000000000000000000000004
> -        ;;
> -    vm1_invalid_type)
> -        # vm1.ovn.org
> -        query_name=03766d31036f766e036f726700
> -        # IPv6 address - aef0::4
> -        type=0002
> -        ;;
> -    vm1_incomplete)
> -        # set type to none
> -        type=''
> -    esac
> -    # TTL - 3600
> -    local dns_req_header=010201200001000000000000
> -    local dns_resp_header=010281200001${an_count}00000000
> -    dns_req_data=${dns_req_header}${query_name}${type}0001
> -
> dns_resp_data=${dns_resp_header}${query_name}${type}0001${expected_dns_answer}
> -}
> -
> -# This shell function sends a DNS request packet
> -# test_dns INPORT SRC_MAC DST_MAC SRC_IP DST_IP DNS_QUERY EXPEC
> -test_dns() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5 dns_reply=$6
> -    local dns_query_data=$7
> -    shift; shift; shift; shift; shift; shift; shift;
> -    # Packet size => IPv4 header (20) + UDP header (8) +
> -    #                DNS data (header + query)
> -    ip_len=`expr 28 + ${#dns_query_data} / 2`
> -    udp_len=`expr $ip_len - 20`
> -    ip_len=$(printf "%x" $ip_len)
> -    udp_len=$(printf "%x" $udp_len)
> -    local request=${dst_mac}${src_mac}0800450000${ip_len}0000000080110000
> -    request=${request}${src_ip}${dst_ip}9234003500${udp_len}0000
> -    # dns data
> -    request=${request}${dns_query_data}
> -
> -    if test $dns_reply != 0; then
> -        local dns_reply=$1
> -        ip_len=`expr 28 + ${#dns_reply} / 2`
> -        udp_len=`expr $ip_len - 20`
> -        ip_len=$(printf "%x" $ip_len)
> -        udp_len=$(printf "%x" $udp_len)
> -        local
> reply=${src_mac}${dst_mac}0800450000${ip_len}0000000080110000
> -
> reply=${reply}${dst_ip}${src_ip}0035923400${udp_len}0000${dns_reply}
> -        echo $reply >> $inport.expected
> -    else
> -        for outport; do
> -            echo $request >> $outport.expected
> -        done
> -    fi
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif$inport $request
> -}
> -
> -test_dns6() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5 dns_reply=$6
> -    local dns_query_data=$7
> -    shift; shift; shift; shift; shift; shift; shift;
> -    # Packet size => UDP header (8) +
> -    #                DNS data (header + query)
> -    ip_len=`expr 8 + ${#dns_query_data} / 2`
> -    udp_len=$ip_len
> -    ip_len=$(printf "%x" $ip_len)
> -    udp_len=$(printf "%x" $udp_len)
> -    local
> request=${dst_mac}${src_mac}86dd6000000000${ip_len}11ff${src_ip}${dst_ip}
> -    request=${request}9234003500${udp_len}0000
> -    #dns data
> -    request=${request}${dns_query_data}
> -
> -    if test $dns_reply != 0; then
> -        local dns_reply=$1
> -        ip_len=`expr 8 + ${#dns_reply} / 2`
> -        udp_len=$ip_len
> -        ip_len=$(printf "%x" $ip_len)
> -        udp_len=$(printf "%x" $udp_len)
> -        local
> reply=${src_mac}${dst_mac}86dd6000000000${ip_len}11ff${dst_ip}${src_ip}
> -        reply=${reply}0035923400${udp_len}0000${dns_reply}
> -        echo $reply >> $inport.expected
> -    else
> -        for outport; do
> -            echo $request >> $outport.expected
> -        done
> -    fi
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif$inport $request
> -}
> -
> -AT_CAPTURE_FILE([ofctl_monitor0.log])
> -as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0.log
> -
> -set_dns_params vm2
> -src_ip=`ip_to_hex 10 0 0 4`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=1
> -test_dns 1 f00000000001 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 1.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > 1.packets
> -cat 1.expected | cut -c -48 > expout
> -AT_CHECK([cat 1.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 1.expected | cut -c 53- > expout
> -AT_CHECK([cat 1.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -set_dns_params vm1
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=1
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Clear the query name options for ls1-lp2
> -ovn-nbctl --wait=hv remove DNS $DNS1 records vm2.ovn.org
> -
> -set_dns_params vm2
> -src_ip=`ip_to_hex 10 0 0 4`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=0
> -test_dns 1 f00000000001 f00000000002 $src_ip $dst_ip $dns_reply
> $dns_req_data
> -
> -# NXT_RESUMEs should be 3.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > 1.packets
> -AT_CHECK([cat 1.packets], [0], [])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Clear the query name for ls1-lp1
> -# Since ls1 has no query names configued,
> -# ovn-northd should not add the DNS flows.
> -ovn-nbctl --wait=hv remove DNS $DNS1 records vm1.ovn.org
> -
> -set_dns_params vm1
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=0
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data
> -
> -# NXT_RESUMEs should be 3 only.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -AT_CHECK([cat 2.packets], [0], [])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Test IPv6 (AAAA records) using IPv4 packet.
> -# Add back the DNS options for ls1-lp1.
> -ovn-nbctl --wait=hv set DNS $DNS1 records:vm1.ovn.org="10.0.0.4 aef0::4"
> -
> -set_dns_params vm1_ipv6_only
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=1
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 4.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Test both IPv4 (A) and IPv6 (AAAA records) using IPv4 packet.
> -set_dns_params vm1_ipv4_v6
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=1
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 5.
> -OVS_WAIT_UNTIL([test 5 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -cat 2.expected | cut -c -48 > expout
> -AT_CHECK([cat 2.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 2.expected | cut -c 53- > expout
> -AT_CHECK([cat 2.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Invalid type.
> -set_dns_params vm1_invalid_type
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=0
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data
> -
> -# NXT_RESUMEs should be 6.
> -OVS_WAIT_UNTIL([test 6 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -AT_CHECK([cat 2.packets], [0], [])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Incomplete DNS packet.
> -set_dns_params vm1_incomplete
> -src_ip=`ip_to_hex 10 0 0 6`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=0
> -test_dns 2 f00000000002 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data
> -
> -# NXT_RESUMEs should be 7.
> -OVS_WAIT_UNTIL([test 7 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -AT_CHECK([cat 2.packets], [0], [])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Add one more DNS record to the ls1.
> -ovn-nbctl --wait=hv set Logical_switch ls1 dns_records="$DNS1 $DNS2"
> -
> -set_dns_params vm3
> -src_ip=`ip_to_hex 10 0 0 4`
> -dst_ip=`ip_to_hex 10 0 0 1`
> -dns_reply=1
> -test_dns 1 f00000000001 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 8.
> -OVS_WAIT_UNTIL([test 8 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > 1.packets
> -cat 1.expected | cut -c -48 > expout
> -AT_CHECK([cat 1.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat 1.expected | cut -c 53- > expout
> -AT_CHECK([cat 1.packets | cut -c 53-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -# Try DNS query over IPv6
> -set_dns_params vm1
> -src_ip=aef00000000000000000000000000004
> -dst_ip=aef00000000000000000000000000001
> -dns_reply=1
> -test_dns6 1 f00000000001 f000000000f0 $src_ip $dst_ip $dns_reply
> $dns_req_data $dns_resp_data
> -
> -# NXT_RESUMEs should be 9.
> -OVS_WAIT_UNTIL([test 9 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap > 1.packets
> -# Skipping the UDP checksum.
> -cat 1.expected | cut -c 1-120,125- > expout
> -AT_CHECK([cat 1.packets | cut -c 1-120,125-], [0], [expout])
> -
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -rm -f 1.expected
> -rm -f 2.expected
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4 HV, 1 LS, 1 LR, packet test with HA distributed router
> gateway port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add gw1
> -as gw1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -sim_add gw2
> -as gw2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.4
> -
> -sim_add ext1
> -as ext1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-int ext1-vif1 -- \
> -    set interface ext1-vif1 external-ids:iface-id=outside1 \
> -    options:tx_pcap=ext1/vif1-tx.pcap \
> -    options:rxq_pcap=ext1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add outside
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect alice to R1 as distributed router gateway port on gw1
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24
> -
> -ovn-nbctl \
> -    --id=@gc0 create Gateway_Chassis name=alice_gw1 \
> -                                     chassis_name=gw1 \
> -                                     priority=20 -- \
> -    --id=@gc1 create Gateway_Chassis name=alice_gw2 \
> -                                     chassis_name=gw2 \
> -                                     priority=10 -- \
> -    set Logical_Router_Port alice 'gateway_chassis=[@gc0, at gc1]'
> -
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port outside1 in outside
> -ovn-nbctl lsp-add outside outside1 \
> --- lsp-set-addresses outside1 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# Create localnet port in alice
> -ovn-nbctl lsp-add alice ln-alice
> -ovn-nbctl lsp-set-addresses ln-alice unknown
> -ovn-nbctl lsp-set-type ln-alice localnet
> -ovn-nbctl lsp-set-options ln-alice network_name=phys
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Create bridge-mappings on gw1, gw2 and ext1, hv1 doesn't need
> -# mapping to the external network, is the one generating packets
> -as gw1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -as gw2 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -as ext1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -AT_CHECK([ovn-nbctl --timeout=3 --wait=sb sync], [0], [ignore])
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 2
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -test_ip_packet()
> -{
> -    local active_gw=$1
> -    local backup_gw=$2
> -    local backup_vswitchd_dead=$3
> -
> -    # Send ip packet between foo1 and outside1
> -    src_mac="f00000010203" # foo1 mac
> -    dst_mac="000001010203" # rp-foo mac (internal router leg)
> -    src_ip=`ip_to_hex 192 168 1 2`
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -    # ARP request packet to expect at outside1
> -
> #arp_request=ffffffffffff${src_mac}08060001080006040001${src_mac}${src_ip}000000000000${dst_ip}
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -    # Send ARP reply from outside1 back to the router
> -    # XXX: note, we could avoid this if we plug this port into a netns
> -    # and setup the IP address into the port, so the kernel would simply
> reply
> -    src_mac="000002010203"
> -    reply_mac="f00000010204"
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -    src_ip=`ip_to_hex 172 16 1 1`
> -
> arp_reply=${src_mac}${reply_mac}08060001080006040002${reply_mac}${dst_ip}${src_mac}${src_ip}
> -
> -    as ext1 ovs-appctl netdev-dummy/receive ext1-vif1 $arp_reply
> -
> -    OVS_WAIT_UNTIL([
> -        test `as $active_gw ovs-ofctl dump-flows br-int | grep table=66 |
> \
> -grep actions=mod_dl_dst:f0:00:00:01:02:04 | wc -l` -eq 1
> -    ])
> -
> -    # Packet to Expect at ext1 chassis, outside1 port
> -    src_mac="000002010203"
> -    dst_mac="f00000010204"
> -    src_ip=`ip_to_hex 192 168 1 2`
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -
> expected=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -    echo $expected > ext1-vif1.expected
> -
> exp_gw_ip_garp=ffffffffffff00000201020308060001080006040001000002010203ac100101000000000000ac100101
> -    echo $exp_gw_ip_garp >> ext1-vif1.expected
> -    as $active_gw reset_pcap_file br-phys_n1 $active_gw/br-phys_n1
> -
> -    if test $backup_vswitchd_dead != 1; then
> -        # Reset the file only if vswitchd in backup gw is alive
> -        as $backup_gw reset_pcap_file br-phys_n1 $backup_gw/br-phys_n1
> -    fi
> -    as ext1 reset_pcap_file ext1-vif1 ext1/vif1
> -
> -    # Resend packet from foo1 to outside1
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -    sleep 1
> -
> -    OVN_CHECK_PACKETS([ext1/vif1-tx.pcap], [ext1-vif1.expected])
> -    $PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> $active_gw/br-phys_n1-tx.pcap  > packets
> -    cat packets | grep $expected > exp
> -    # Its possible that $active_gw/br-phys_n1-tx.pcap may have received
> multiple
> -    # garp packets. So consider only the first packet.
> -    cat packets | grep $exp_gw_ip_garp | head -1 >> exp
> -    AT_CHECK([cat exp], [0], [expout])
> -    rm -f expout
> -    if test $backup_vswitchd_dead != 1; then
> -        # Check for backup gw only if vswitchd is alive
> -        $PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> $backup_gw/br-phys_n1-tx.pcap  > packets
> -        AT_CHECK([grep $expected packets | sort], [0], [])
> -    fi
> -}
> -
> -test_ip_packet gw1 gw2 0
> -
> -ovn-nbctl --timeout=3 --wait=hv \
> -    --id=@gc0 create Gateway_Chassis name=alice_gw1 \
> -                                     chassis_name=gw1 \
> -                                     priority=10 -- \
> -    --id=@gc1 create Gateway_Chassis name=alice_gw2 \
> -                                     chassis_name=gw2 \
> -                                     priority=20 -- \
> -    set Logical_Router_Port alice 'gateway_chassis=[@gc0, at gc1]'
> -
> -test_ip_packet gw2 gw1 0
> -
> -# Get the claim count of both gw1 and gw2.
> -gw1_claim_ct=`grep "cr-alice: Claiming" gw1/ovn-controller.log | wc -l`
> -gw2_claim_ct=`grep "cr-alice: Claiming" gw2/ovn-controller.log | wc -l`
> -
> -# Stop ovs-vswitchd in gw2. gw1 should claim the gateway port.
> -as gw2
> -OVS_APP_EXIT_AND_WAIT([ovs-vswitchd])
> -
> -# gw1 should claim the cr-alice and the claim count of gw1 should be
> -# incremented by 1.
> -gw1_claim_ct=$((gw1_claim_ct+1))
> -
> -OVS_WAIT_UNTIL([test $gw1_claim_ct = `cat gw1/ovn-controller.log \
> -| grep -c "cr-alice: Claiming"`])
> -
> -AT_CHECK([test $gw2_claim_ct = `cat gw2/ovn-controller.log | \
> -grep -c "cr-alice: Claiming"`])
> -
> -test_ip_packet gw1 gw2 1
> -
> -as gw2
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -OVN_CLEANUP([hv1],[gw1],[ext1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 4 HV, 3 LS, 2 LR, packet test with HA distributed router
> gateway port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add gw1
> -as gw1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -sim_add gw2
> -as gw2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.4
> -
> -sim_add ext1
> -as ext1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-int ext1-vif1 -- \
> -    set interface ext1-vif1 external-ids:iface-id=outside1 \
> -    options:tx_pcap=ext1/vif1-tx.pcap \
> -    options:rxq_pcap=ext1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R0
> -ovn-nbctl create Logical_Router name=R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add join
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add outside
> -
> -#Connect foo to R0
> -ovn-nbctl lrp-add R0 R0-foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo foo-R0 -- set Logical_Switch_Port foo-R0 \
> -    type=router options:router-port=R0-foo \
> -    -- lsp-set-addresses foo-R0 router
> -
> -#Connect R0 to join
> -ovn-nbctl lrp-add R0 R0-join 00:00:0d:01:02:03 100.60.1.1/24
> -ovn-nbctl lsp-add join join-R0 -- set Logical_Switch_Port join-R0 \
> -    type=router options:router-port=R0-join \
> -    -- lsp-set-addresses join-R0 router
> -
> -#Connect join to R1
> -ovn-nbctl lrp-add R1 R1-join 00:00:0e:01:02:03 100.60.1.2/24
> -ovn-nbctl lsp-add join join-R1 -- set Logical_Switch_Port join-R1 \
> -    type=router options:router-port=R1-join \
> -    -- lsp-set-addresses join-R1 router
> -
> -#add route rules
> -ovn-nbctl lr-route-add R0 0.0.0.0/0 100.60.1.2
> -ovn-nbctl lr-route-add R1 192.168.0.0/16 100.60.1.1
> -
> -# Connect alice to R1 as distributed router gateway port on gw1
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24
> -
> -ovn-nbctl \
> -    --id=@gc0 create Gateway_Chassis name=alice_gw1 \
> -                                     chassis_name=gw1 \
> -                                     priority=20 -- \
> -    --id=@gc1 create Gateway_Chassis name=alice_gw2 \
> -                                     chassis_name=gw2 \
> -                                     priority=10 -- \
> -    set Logical_Router_Port alice 'gateway_chassis=[@gc0, at gc1]'
> -
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port outside1 in outside
> -ovn-nbctl lsp-add outside outside1 \
> --- lsp-set-addresses outside1 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# Create localnet port in alice
> -ovn-nbctl lsp-add alice ln-alice
> -ovn-nbctl lsp-set-addresses ln-alice unknown
> -ovn-nbctl lsp-set-type ln-alice localnet
> -ovn-nbctl lsp-set-options ln-alice network_name=phys
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Create bridge-mappings on gw1, gw2 and ext1, hv1 doesn't need
> -# mapping to the external network, is the one generating packets
> -as gw1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -as gw2 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -as ext1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -AT_CHECK([ovn-nbctl --timeout=3 --wait=sb sync], [0], [ignore])
> -
> -# hv1 should be in 'ref_chassis' of the ha_chasssi_group as logical
> -# switch 'foo' can reach the router 'R1' (which has gw router port)
> -# via foo1 -> foo -> R0 -> join -> R1
> -hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"`
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$hv1_ch_uuid" = "$ref_ch_list"])
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 2
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -test_ip_packet()
> -{
> -    local active_gw=$1
> -    local backup_gw=$2
> -
> -    # Send ip packet between foo1 and outside1
> -    src_mac="f00000010203" # foo1 mac
> -    dst_mac="000001010203" # foo-R0 mac (internal router leg)
> -    src_ip=`ip_to_hex 192 168 1 2`
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -    # ARP request packet to expect at outside1
> -
> #arp_request=ffffffffffff${src_mac}08060001080006040001${src_mac}${src_ip}000000000000${dst_ip}
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -    # Send ARP reply from outside1 back to the router
> -    # XXX: note, we could avoid this if we plug this port into a netns
> -    # and setup the IP address into the port, so the kernel would simply
> reply
> -    src_mac="000002010203"
> -    reply_mac="f00000010204"
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -    src_ip=`ip_to_hex 172 16 1 1`
> -
> arp_reply=${src_mac}${reply_mac}08060001080006040002${reply_mac}${dst_ip}${src_mac}${src_ip}
> -
> -    as ext1 ovs-appctl netdev-dummy/receive ext1-vif1 $arp_reply
> -
> -    OVS_WAIT_UNTIL([
> -        test `as $active_gw ovs-ofctl dump-flows br-int | grep table=66 |
> \
> -grep actions=mod_dl_dst:f0:00:00:01:02:04 | wc -l` -eq 1
> -    ])
> -
> -    # Packet to Expect at ext1 chassis, outside1 port
> -    src_mac="000002010203"
> -    dst_mac="f00000010204"
> -    src_ip=`ip_to_hex 192 168 1 2`
> -    dst_ip=`ip_to_hex 172 16 1 3`
> -
> expected=${dst_mac}${src_mac}08004500001c000000003e110200${src_ip}${dst_ip}0035111100080000
> -    echo $expected > ext1-vif1.expected
> -
> exp_gw_ip_garp=ffffffffffff00000201020308060001080006040001000002010203ac100101000000000000ac100101
> -    echo $exp_gw_ip_garp >> ext1-vif1.expected
> -
> -    as $active_gw reset_pcap_file br-phys_n1 $active_gw/br-phys_n1
> -    as $backup_gw reset_pcap_file br-phys_n1 $backup_gw/br-phys_n1
> -    as ext1 reset_pcap_file ext1-vif1 ext1/vif1
> -
> -    # Resend packet from foo1 to outside1
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -    OVN_CHECK_PACKETS([ext1/vif1-tx.pcap], [ext1-vif1.expected])
> -    $PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> $active_gw/br-phys_n1-tx.pcap  > packets
> -    cat packets | grep $expected > exp
> -    cat packets | grep $exp_gw_ip_garp | head -1 >> exp
> -    AT_CHECK([cat exp], [0], [expout])
> -
> -    $PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> $backup_gw/br-phys_n1-tx.pcap  > packets
> -    AT_CHECK([grep $expected packets | sort], [0], [])
> -}
> -
> -test_ip_packet gw1 gw2
> -
> -ovn-nbctl --timeout=3 --wait=hv \
> -    --id=@gc0 create Gateway_Chassis name=alice_gw1 \
> -                                     chassis_name=gw1 \
> -                                     priority=10 -- \
> -    --id=@gc1 create Gateway_Chassis name=alice_gw2 \
> -                                     chassis_name=gw2 \
> -                                     priority=20 -- \
> -    set Logical_Router_Port alice 'gateway_chassis=[@gc0, at gc1]'
> -
> -test_ip_packet gw2 gw1
> -
> -OVN_CLEANUP([hv1],[gw1],[gw2],[ext1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 1 LR with distributed router gateway port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR R1 that has switches foo (192.168.1.0/24) and
> -# alice (172.16.1.0/24) connected to it.  The logical port
> -# between R1 and alice has a "redirect-chassis" specified,
> -# i.e. it is the distributed router gateway port.
> -# Switch alice also has a localnet port defined.
> -# An additional switch outside has a localnet port and the
> -# same subnet as alice (172.16.1.0/24).
> -
> -# Physical network:
> -# Three hypervisors hv[123].
> -# hv1 hosts vif foo1.
> -# hv2 is the "redirect-chassis" that hosts the distributed
> -# router gateway port.
> -# hv3 hosts vif outside1.
> -# In order to show that connectivity works only through hv2,
> -# an initial round of tests is run without any bridge-mapping
> -# defined for the localnet on hv2.  These tests are expected
> -# to fail.
> -# Subsequent tests are run after defining the bridge-mapping
> -# for the localnet on hv2. These tests are expected to succeed.
> -
> -# Create three hypervisors and create OVS ports corresponding
> -# to logical ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-int hv3-vif1 -- \
> -    set interface hv3-vif1 external-ids:iface-id=outside1 \
> -    options:tx_pcap=hv3/vif1-tx.pcap \
> -    options:rxq_pcap=hv3/vif1-rx.pcap \
> -    ofport-request=1
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add outside
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect alice to R1 as distributed router gateway port on hv2
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> -    -- set Logical_Router_Port alice options:redirect-chassis="hv2"
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port outside1 in outside
> -ovn-nbctl lsp-add outside outside1 \
> --- lsp-set-addresses outside1 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# Create localnet port in alice
> -ovn-nbctl lsp-add alice ln-alice
> -ovn-nbctl lsp-set-addresses ln-alice unknown
> -ovn-nbctl lsp-set-type ln-alice localnet
> -ovn-nbctl lsp-set-options ln-alice network_name=phys
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Create bridge-mappings on hv1 and hv3, leaving hv2 for later
> -as hv1 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -as hv3 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 2
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -ovn-sbctl list chassis
> -ovn-sbctl list encap
> -echo "------ Gateway_Chassis dump (SBDB) -------"
> -ovn-sbctl list Gateway_Chassis
> -echo "------ Port_Binding chassisredirect -------"
> -ovn-sbctl find Port_Binding type=chassisredirect
> -echo "-------------------------------------------"
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -echo "------ hv3 dump ----------"
> -as hv3 ovs-ofctl show br-int
> -as hv3 ovs-ofctl dump-flows br-int
> -echo "--------------------------"
> -
> -
> -# Check that redirect mapping is programmed only on hv2
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=33 | grep
> =0x3,metadata=0x1 | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=33 | grep
> =0x3,metadata=0x1 | grep load:0x2- | wc -l], [0], [1
> -])
> -# Check that hv1 sends chassisredirect port traffic to hv2
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=32 | grep
> =0x3,metadata=0x1 | grep output | wc -l], [0], [1
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=32 | grep
> =0x3,metadata=0x1 | wc -l], [0], [0
> -])
> -# Check that arp reply on distributed gateway port is only programmed on
> hv2
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep arp | grep load:0x2-
> | grep =0x2,metadata=0x1 | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep arp | grep load:0x2-
> | grep =0x2,metadata=0x1 | wc -l], [0], [1
> -])
> -
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -
> -: > hv2-vif1.expected
> -: > hv3-vif1.expected
> -
> -# test_arp INPORT SHA SPA TPA [REPLY_HA]
> -#
> -# Causes a packet to be received on INPORT.  The packet is an ARP
> -# request with SHA, SPA, and TPA as specified.  If REPLY_HA is provided,
> then
> -# it should be the hardware address of the target to expect to receive in
> an
> -# ARP reply; otherwise no reply is expected.
> -#
> -# INPORT is an logical switch port number, e.g. 11 for vif11.
> -# SHA and REPLY_HA are each 12 hex digits.
> -# SPA and TPA are each 8 hex digits.
> -test_arp() {
> -    local hv=$1 inport=$2 sha=$3 spa=$4 tpa=$5 reply_ha=$6
> -    local
> request=ffffffffffff${sha}08060001080006040001${sha}${spa}ffffffffffff${tpa}
> -    as hv$hv ovs-appctl netdev-dummy/receive hv${hv}-vif$inport $request
> -
> -    if test X$reply_ha != X; then
> -        # Expect to receive the reply, if any.
> -        local
> reply=${sha}${reply_ha}08060001080006040002${reply_ha}${tpa}${sha}${spa}
> -        echo $reply >> hv${hv}-vif$inport.expected
> -    fi
> -}
> -
> -rtr_ip=$(ip_to_hex 172 16 1 1)
> -foo_ip=$(ip_to_hex 192 168 1 2)
> -outside_ip=$(ip_to_hex 172 16 1 3)
> -
> -echo $rtr_ip
> -echo $foo_ip
> -echo $outside_ip
> -
> -# ARP for router IP address from outside1, no response expected
> -test_arp 3 1 f00000010204 $outside_ip $rtr_ip
> -
> -# Now check the packets actually received against the ones expected.
> -OVN_CHECK_PACKETS([hv3/vif1-tx.pcap], [hv3-vif1.expected])
> -
> -# Send ip packet between foo1 and outside1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 3`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -# Now check the packets actually received against the ones expected.
> -OVN_CHECK_PACKETS([hv3/vif1-tx.pcap], [hv3-vif1.expected])
> -
> -# Now add bridge-mappings on hv2, which should make everything work
> -as hv2 ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -# Wait until the patch ports are created in hv2 to connect br-int to
> br-phys
> -OVS_WAIT_UNTIL([test 1 = `as hv2 ovs-vsctl show | \
> -grep "Port patch-br-int-to-ln-alice" | wc -l`])
> -
> -# ARP for router IP address from outside1
> -test_arp 3 1 f00000010204 $outside_ip $rtr_ip 000002010203
> -
> -# hv3-vif1.expected should also have the gw router port garp packet.
>
> -exp_gw_ip_garp=ffffffffffff00000201020308060001080006040001000002010203ac100101000000000000ac100101
> -echo $exp_gw_ip_garp >> hv3-vif1.expected
> -
> -# Now check the packets actually received against the ones expected.
> -OVN_CHECK_PACKETS([hv3/vif1-tx.pcap], [hv3-vif1.expected])
> -
> -# Send ip packet between foo1 and outside1
> -src_mac="f00000010203"
> -dst_mac="000001010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 172 16 1 3`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -# Packet to Expect at outside1
> -src_mac="000002010203"
> -dst_mac="f00000010204"
>
> -expected=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -echo "------ hv1 dump ----------"
> -as hv1 ovs-ofctl show br-int
> -as hv1 ovs-ofctl dump-flows br-int
> -echo "------ hv2 dump ----------"
> -as hv2 ovs-ofctl show br-int
> -as hv2 ovs-ofctl dump-flows br-int
> -echo "------ hv3 dump ----------"
> -as hv3 ovs-ofctl show br-int
> -as hv3 ovs-ofctl dump-flows br-int
> -echo "----------------------------"
> -
> -echo $expected >> hv3-vif1.expected
> -OVN_CHECK_PACKETS([hv3/vif1-tx.pcap], [hv3-vif1.expected])
> -
> -#Check ovn-trace over "chassisredirect" port
> -AT_CAPTURE_FILE([trace])
> -ovn_trace () {
> -    ovn-trace --all "$@" | tee trace | sed '1,/Minimal trace/d'
> -}
> -
> -echo 'ip.ttl--;' > expout
> -echo 'eth.src = 00:00:02:01:02:03;' >> expout
> -echo 'eth.dst = f0:00:00:01:02:04;' >> expout
> -echo 'output("ln-alice");' >> expout
> -AT_CHECK_UNQUOTED([ovn_trace foo 'inport == "foo1" && eth.src ==
> f0:00:00:01:02:03 && eth.dst == 00:00:01:01:02:03 && ip4.src == 192.168.1.2
> && ip4.dst == 172.16.1.3 && ip.ttl == 0xff'], [0], [expout])
> -
> -# Create logical port alice1 in alice on hv1
> -as hv1 ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=1
> -
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.4"
> -
> -# Create logical port foo2 in foo on hv2
> -as hv2 ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=foo2 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovn-nbctl lsp-add foo foo2 \
> --- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -: > hv1-vif2.expected
> -
> -# Send ip packet between alice1 and foo2
> -src_mac="f00000010205"
> -dst_mac="000002010203"
> -src_ip=`ip_to_hex 172 16 1 4`
> -dst_ip=`ip_to_hex 192 168 1 3`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif2 $packet
> -
> -# Packet to Expect at foo2
> -src_mac="000001010203"
> -dst_mac="f00000010206"
> -src_ip=`ip_to_hex 172 16 1 4`
> -dst_ip=`ip_to_hex 192 168 1 3`
>
> -expected=${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000
> -
> -echo $expected >> hv2-vif1.expected
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [hv2-vif1.expected])
> -
> -AT_CHECK([ovn-sbctl --bare --columns _uuid find Port_Binding
> logical_port=cr-alice | wc -l], [0], [1
> -])
> -
> -ovn-nbctl --timeout=3 --wait=sb remove Logical_Router_Port alice options
> redirect-chassis
> -
> -AT_CHECK([ovn-sbctl find Port_Binding logical_port=cr-alice | wc -l],
> [0], [0
> -])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send gratuitous arp for NAT rules on distributed router])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -# Create logical switches
> -ovn-nbctl ls-add ls0
> -ovn-nbctl ls-add ls1
> -# Create distributed router
> -ovn-nbctl create Logical_Router name=lr0
> -# Add distributed gateway port to distributed router
> -ovn-nbctl lrp-add lr0 lrp0 f0:00:00:00:00:01 192.168.0.1/24 \
> -    -- set Logical_Router_Port lrp0 options:redirect-chassis="hv2"
> -ovn-nbctl lsp-add ls0 lrp0-rp -- set Logical_Switch_Port lrp0-rp \
> -    type=router options:router-port=lrp0 addresses="router"
> -# Add router port to ls1
> -ovn-nbctl lrp-add lr0 lrp1 f0:00:00:00:00:02 10.0.0.1/24
> -ovn-nbctl lsp-add ls1 lrp1-rp -- set Logical_Switch_Port lrp1-rp \
> -    type=router options:router-port=lrp1 addresses="router"
> -# Add logical ports for NAT rules
> -ovn-nbctl lsp-add ls1 foo1 \
> --- lsp-set-addresses foo1 "00:00:00:00:00:03 10.0.0.3"
> -ovn-nbctl lsp-add ls1 foo2 \
> --- lsp-set-addresses foo2 "00:00:00:00:00:04 10.0.0.4"
> -# Add nat-addresses option
> -ovn-nbctl lsp-set-options lrp0-rp router-port=lrp0 nat-addresses="router"
> -# Add NAT rules
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 192.168.0.1 10.0.0.0/24])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat 192.168.0.2 10.0.0.2])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.0.3 10.0.0.3
> foo1 f0:00:00:00:00:03])
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 dnat_and_snat 192.168.0.4 10.0.0.4
> foo2 f0:00:00:00:00:04])
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -AT_CHECK([ovs-vsctl add-port br-phys snoopvif -- set Interface snoopvif
> options:tx_pcap=hv1/snoopvif-tx.pcap options:rxq_pcap=hv1/snoopvif-rx.pcap])
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -# Initially test with no bridge-mapping on hv2, expect to receive no
> packets
> -
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -# Initially test with no bridge-mapping on hv3
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add ls0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl --wait=hv lsp-set-options ln_port
> network_name=physnet1])
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 2
> -
> -# Expect no packets when hv2 bridge-mapping is not present
> -: > packets
> -OVN_CHECK_PACKETS([hv1/snoopvif-tx.pcap], [packets])
> -
> -# Add bridge-mapping on hv2
> -AT_CHECK([as hv2 ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -
> -# Wait until the patch ports are created in hv2 to connect br-int to
> br-phys
> -OVS_WAIT_UNTIL([test 1 = `as hv2 ovs-vsctl show | \
> -grep "Port patch-br-int-to-ln_port" | wc -l`])
> -
> -# Wait for packets to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 100])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros > packets
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80001000000000000c0a80001"
> -echo $expected > expout
>
> -expected="fffffffffffff0000000000108060001080006040001f00000000001c0a80002000000000000c0a80002"
> -echo $expected >> expout
> -AT_CHECK([sort packets], [0], [expout])
> -sort packets | cat
> -
> -# Temporarily remove nat-addresses option to avoid race conditions
> -# due to GARP backoff
> -ovn-nbctl lsp-set-options lrp0-rp router-port=lrp0 nat-addresses=""
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -as hv1 reset_pcap_file snoopvif hv1/snoopvif
> -
> -# Add OVS ports for foo1 and foo2 on hv3
> -ovs-vsctl -- add-port br-int hv3-vif1 -- \
> -    set interface hv3-vif1 external-ids:iface-id=foo1 \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv3-vif2 -- \
> -    set interface hv3-vif2 external-ids:iface-id=foo2 \
> -    ofport-request=2
> -
> -# Add bridge-mapping on hv3
> -AT_CHECK([as hv3 ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -
> -# Wait until the patch ports are created in hv3 to connect br-int to
> br-phys
> -OVS_WAIT_UNTIL([test 1 = `as hv3 ovs-vsctl show | \
> -grep "Port patch-br-int-to-ln_port" | wc -l`])
> -
> -# Re-add nat-addresses option
> -ovn-nbctl lsp-set-options lrp0-rp router-port=lrp0 nat-addresses="router"
> -
> -# Wait for packets to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 250])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros > packets
>
> -garp_1="fffffffffffff0000000000308060001080006040001f00000000003c0a80003000000000000c0a80003"
> -echo $garp_1 > expout
>
> -garp_2="fffffffffffff0000000000408060001080006040001f00000000004c0a80004000000000000c0a80004"
> -echo $garp_2 >> expout
> -
> -cat packets | grep $garp_1 | head -1 > exp
> -cat packets | grep $garp_2 | head -1 >> exp
> -AT_CHECK([cat exp], [0], [expout])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -# VLAN traffic for external network redirected through distributed router
> -# gateway port should use vlans(i.e input network vlan tag) across
> hypervisors
> -# instead of tunneling.
> -AT_SETUP([ovn -- vlan traffic for external network with distributed
> router gateway port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# # One LR R1 that has switches foo (192.168.1.0/24) and
> -# # alice (172.16.1.0/24) connected to it.  The logical port
> -# # between R1 and alice has a "redirect-chassis" specified,
> -# # i.e. it is the distributed router gateway port(172.16.1.6).
> -# # Switch alice also has a localnet port defined.
> -# # An additional switch outside has the same subnet as alice
> -# # (172.16.1.0/24), a localnet port and nexthop port(172.16.1.1)
> -# # which will receive the packet destined for external network
> -# # (i.e 8.8.8.8 as destination ip).
> -
> -# Physical network:
> -# # Four hypervisors hv[1234].
> -# # hv1 hosts vif foo1.
> -# # hv2 is the "redirect-chassis" that hosts the distributed router
> gateway port.
> -# # Later to test GARPs for the router port - foo, hv2 and hv4 are added
> to the ha_chassis_group
> -# # hv3 hosts nexthop port vif outside1.
> -# # All other tests connect hypervisors to network n1 through br-phys for
> tunneling.
> -# # But in this test, hv1 won't connect to n1(and no br-phys in hv1), and
> -# # in order to show vlans(instead of tunneling) used between hv1 and hv2,
> -# # a new network n2 created and hv1 and hv2 connected to this network
> through br-ex.
> -# # hv2 and hv3 are still connected to n1 network through br-phys.
> -net_add n1
> -
> -# We are not calling ovn_attach for hv1, to avoid adding br-phys.
> -# Tunneling won't work in hv1 as ovn-encap-ip is not added to any bridge
> in hv1
> -sim_add hv1
> -as hv1
> -ovs-vsctl \
> -    -- set Open_vSwitch . external-ids:system-id=hv1 \
> -    -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -    -- set Open_vSwitch . external-ids:ovn-encap-type=geneve,vxlan \
> -    -- set Open_vSwitch . external-ids:ovn-encap-ip=192.168.0.1 \
> -    -- add-br br-int \
> -    -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true \
> -    -- set Open_vSwitch . external-ids:ovn-bridge-mappings=public:br-ex
> -
> -start_daemon ovn-controller
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings="public:br-ex,phys:br-phys"
> -
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-int hv3-vif1 -- \
> -    set interface hv3-vif1 external-ids:iface-id=outside1 \
> -    options:tx_pcap=hv3/vif1-tx.pcap \
> -    options:rxq_pcap=hv3/vif1-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings="phys:br-phys"
> -
> -sim_add hv4
> -as hv4
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.4
> -ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings="public:br-ex,phys:br-phys"
> -
> -# Create network n2 for vlan connectivity between hv1 and hv2
> -net_add n2
> -
> -as hv1
> -ovs-vsctl add-br br-ex
> -net_attach n2 br-ex
> -
> -as hv2
> -ovs-vsctl add-br br-ex
> -net_attach n2 br-ex
> -
> -as hv4
> -ovs-vsctl add-br br-ex
> -net_attach n2 br-ex
> -
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add outside
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect alice to R1 as distributed router gateway port (172.16.1.6) on
> hv2
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.6/24 \
> -    -- set Logical_Router_Port alice options:redirect-chassis="hv2"
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router \
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port outside1 in outside, which is a nexthop address
> -# for 172.16.1.0/24
> -ovn-nbctl lsp-add outside outside1 \
> --- lsp-set-addresses outside1 "f0:00:00:01:02:04 172.16.1.1"
> -
> -# Set default gateway (nexthop) to 172.16.1.1
> -ovn-nbctl lr-route-add R1 "0.0.0.0/0" 172.16.1.1 alice
> -AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.1.6 192.168.1.1/24])
> -ovn-nbctl set Logical_Switch_Port rp-alice options:nat-addresses=router
> -
> -ovn-nbctl lsp-add foo ln-foo
> -ovn-nbctl lsp-set-addresses ln-foo unknown
> -ovn-nbctl lsp-set-options ln-foo network_name=public
> -ovn-nbctl lsp-set-type ln-foo localnet
> -AT_CHECK([ovn-nbctl set Logical_Switch_Port ln-foo tag=2])
> -
> -# Create localnet port in alice
> -ovn-nbctl lsp-add alice ln-alice
> -ovn-nbctl lsp-set-addresses ln-alice unknown
> -ovn-nbctl lsp-set-type ln-alice localnet
> -ovn-nbctl lsp-set-options ln-alice network_name=phys
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -# Check that there is a logical flow in logical switch foo's pipeline
> -# to set the outport to rp-foo (which is expected).
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl dump-flows foo | grep ls_in_l2_lkup |
> \
> -grep rp-foo | grep -v is_chassis_resident | wc -l`])
> -
> -# Set the option 'reside-on-redirect-chassis' for foo
> -ovn-nbctl set logical_router_port foo
> options:reside-on-redirect-chassis=true
> -# Check that there is a logical flow in logical switch foo's pipeline
> -# to set the outport to rp-foo with the condition is_chassis_redirect.
> -ovn-sbctl dump-flows foo
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl dump-flows foo | grep ls_in_l2_lkup |
> \
> -grep rp-foo | grep is_chassis_resident | wc -l`])
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list nat
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -ovn-sbctl list chassis
> -echo "---------------------"
> -
> -for chassis in hv1 hv2 hv3; do
> -    as $chassis
> -    echo "------ $chassis dump ----------"
> -    ovs-vsctl show br-int
> -    ovs-ofctl show br-int
> -    ovs-ofctl dump-flows br-int
> -    echo "--------------------------"
> -done
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -foo1_ip=$(ip_to_hex 192 168 1 2)
> -gw_ip=$(ip_to_hex 172 16 1 6)
> -dst_ip=$(ip_to_hex 8 8 8 8)
> -nexthop_ip=$(ip_to_hex 172 16 1 1)
> -
> -foo1_mac="f00000010203"
> -foo_mac="000001010203"
> -gw_mac="000002010203"
> -nexthop_mac="f00000010204"
> -
> -# Send ip packet from foo1 to 8.8.8.8
> -src_mac="f00000010203"
> -dst_mac="000001010203"
>
> -packet=${foo_mac}${foo1_mac}08004500001c0000000040110000${foo1_ip}${dst_ip}0035111100080000
> -
> -# Wait for GARPs announcing gw IP to arrive
> -OVS_WAIT_UNTIL([
> -    test `as hv2 ovs-ofctl dump-flows br-int | grep table=66 | \
> -grep actions=mod_dl_dst:f0:00:00:01:02:04 | wc -l` -eq 1
> -    ])
> -
> -# VLAN tagged packet with router port(192.168.1.1) MAC as destination MAC
> -# is expected on bridge connecting hv1 and hv2
>
> -expected=${foo_mac}${foo1_mac}8100000208004500001c0000000040110000${foo1_ip}${dst_ip}0035111100080000
> -echo $expected > hv1-br-ex_n2.expected
> -
> -# Packet to Expect at outside1 i.e nexthop(172.16.1.1) port.
> -# As connection tracking not enabled for this test, snat can't be done on
> the packet.
> -# We still see foo1 as the source ip address. But source mac(gateway MAC)
> and
> -# dest mac(nexthop mac) are properly configured.
>
> -expected=${nexthop_mac}${gw_mac}08004500001c000000003f110100${foo1_ip}${dst_ip}0035111100080000
> -echo $expected > hv3-vif1.expected
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -as hv1 reset_pcap_file br-ex_n2 hv1/br-ex_n2
> -as hv3 reset_pcap_file hv3-vif1 hv3/vif1
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -sleep 2
> -
> -# On hv1, table 32 check that no packet goes via the tunnel port
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=32 \
> -| grep "NXM_NX_TUN_ID" | grep -v n_packets=0 | wc -l], [0], [[0
> -]])
> -
> -ip_packet() {
> -    grep "1010203f00000010203"
> -}
> -
> -# Check vlan tagged packet on the bridge connecting hv1 and hv2 with the
> -# foo1's mac.
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-ex_n2-tx.pcap |
> ip_packet | uniq > hv1-br-ex_n2
> -cat hv1-br-ex_n2.expected > expout
> -AT_CHECK([sort hv1-br-ex_n2], [0], [expout])
> -
> -# Check expected packet on nexthop interface
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv3/vif1-tx.pcap | grep
> ${foo1_ip}${dst_ip} | uniq > hv3-vif1
> -cat hv3-vif1.expected > expout
> -AT_CHECK([sort hv3-vif1], [0], [expout])
> -
> -# Test the GARP for the router port ip - 192.168.1.1
> -ovn-nbctl --wait=sb ha-chassis-group-add hagrp1
> -
> -as hv1 reset_pcap_file hv1-vif1 hv1/vif1
> -as hv2 reset_pcap_file br-ex_n2 hv2/br-ex_n2
> -as hv4 reset_pcap_file br-ex_n2 hv4/br-ex_n2
> -
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 hv2 30
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 hv4 20
> -
> -hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group
> name=hagrp1`
> -ovn-nbctl remove logical_router_port alice options redirect-chassis
> -ovn-nbctl --wait=sb set logical_router_port alice
> ha_chassis_group=$hagrp1_uuid
> -
> -# When hv2 claims the gw router port cr-alice, it should send out
> -# GARP for 192.168.1.1 and it should be received by foo1 on hv1.
> -
> -# foo1 (on hv1) should receive GARP without VLAN tag
>
> -exp_garp_on_foo1="ffffffffffff00000101020308060001080006040001000001010203c0a80101000000000000c0a80101"
> -echo $exp_garp_on_foo1 > foo1.expout
> -
> -# ovn-controller on hv2 should send garp with VLAN tag
>
> -sent_garp="ffffffffffff0000010102038100000208060001080006040001000001010203c0a80101000000000000c0a80101"
> -
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [foo1.expout])
> -# Wait until we receive atleast 1 packet
> -OVS_WAIT_UNTIL([test 1=`$PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> hv2/br-ex_n2-tx.pcap | wc -l`])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-ex_n2-tx.pcap | head
> -1 > packets
> -echo $sent_garp > expout
> -AT_CHECK([cat packets], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv4/br-ex_n2-tx.pcap > empty
> -AT_CHECK([cat empty], [0], [])
> -
> -# Make hv4 master
> -as hv1 reset_pcap_file hv1-vif1 hv1/vif1
> -as hv4 reset_pcap_file br-ex_n2 hv4/br-ex_n2
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 hv4 40
> -
> -# Wait till cr-alice is claimed by hv4
> -hv4_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=hv4)
> -# check that the chassis redirect port has been claimed by the gw1 chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-alice | grep $hv4_chassis | wc -l], [0],[[1
> -]])
> -
> -# Reset the pcap file for hv2/br-ex_n2. From now on ovn-controller in hv2
> -# should not send GARPs for the router ports.
> -as hv2 reset_pcap_file br-ex_n2 hv2/br-ex_n2
> -
> -echo $sent_garp > br-ex_n2.expout
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [foo1.expout])
> -OVN_CHECK_PACKETS([hv4/br-ex_n2-tx.pcap], [br-ex_n2.expout])
> -
> -sleep 2
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-ex_n2-tx.pcap > empty
> -AT_CHECK([cat empty], [0], [])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3], [hv4])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IPv6 ND Router Solicitation responder])
> -AT_KEYWORDS([ovn-nd_ra])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# In this test case we create 1 lswitch with 3 VIF ports attached,
> -# and a lrouter connected to the lswitch.
> -# We generate the Router solicitation packet and verify the Router
> Advertisement
> -# reply packet from the ovn-controller.
> -
> -# Create hypervisor and logical switch lsw0, logical router lr0, attach
> lsw0
> -# onto lr0, set Logical_Router_Port.ipv6_ra_configs:address_mode column to
> -# 'slaac' to allow lrp0 send RA for SLAAC mode.
> -ovn-nbctl ls-add lsw0
> -ovn-nbctl lr-add lr0
> -ovn-nbctl lrp-add lr0 lrp0 fa:16:3e:00:00:01 fdad:1234:5678::1/64
> -ovn-nbctl set Logical_Router_Port lrp0
> ipv6_ra_configs:address_mode="slaac"
> -ovn-nbctl \
> -    -- lsp-add lsw0 lsp0 \
> -    -- set Logical_Switch_Port lsp0 type=router \
> -                     options:router-port=lrp0 \
> -                     addresses='"fa:16:3e:00:00:01 fdad:1234:5678::1"'
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -ovn-nbctl lsp-add lsw0 lp1
> -ovn-nbctl lsp-set-addresses lp1 "fa:16:3e:00:00:02 10.0.0.12
> fdad:1234:5678:0:f816:3eff:fe:2"
> -ovn-nbctl lsp-set-port-security lp1 "fa:16:3e:00:00:02 10.0.0.12
> fdad:1234:5678:0:f816:3eff:fe:2"
> -
> -ovn-nbctl lsp-add lsw0 lp2
> -ovn-nbctl lsp-set-addresses lp2 "fa:16:3e:00:00:03 10.0.0.13
> fdad:1234:5678:0:f816:3eff:fe:3"
> -ovn-nbctl lsp-set-port-security lp2 "fa:16:3e:00:00:03 10.0.0.13
> fdad:1234:5678:0:f816:3eff:fe:3"
> -
> -ovn-nbctl lsp-add lsw0 lp3
> -ovn-nbctl lsp-set-addresses lp3 "fa:16:3e:00:00:04 10.0.0.14
> fdad:1234:5678:0:f816:3eff:fe:4"
> -ovn-nbctl lsp-set-port-security lp3 "fa:16:3e:00:00:04 10.0.0.14
> fdad:1234:5678:0:f816:3eff:fe:4"
> -
> -# Add ACL rule for ICMPv6 on lsw0
> -ovn-nbctl acl-add lsw0 from-lport 1002 'ip6 && icmp6'  allow-related
> -ovn-nbctl acl-add lsw0 to-lport 1002 'outport == "lp1" && ip6 && icmp6'
> allow-related
> -ovn-nbctl acl-add lsw0 to-lport 1002 'outport == "lp2" && ip6 && icmp6'
> allow-related
> -ovn-nbctl acl-add lsw0 to-lport 1002 'outport == "lp3" && ip6 && icmp6'
> allow-related
> -
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -ovs-vsctl -- add-port br-int hv1-vif3 -- \
> -    set interface hv1-vif3 external-ids:iface-id=lp3 \
> -    options:tx_pcap=hv1/vif3-tx.pcap \
> -    options:rxq_pcap=hv1/vif3-rx.pcap \
> -    ofport-request=3
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -# Make sure that ovn-controller has installed the corresponding OF Flow.
> -OVS_WAIT_UNTIL([test 1 = `as hv1 ovs-ofctl dump-flows br-int | grep -c
> "ipv6_dst=ff02::2,nw_ttl=255,icmp_type=133,icmp_code=0"`])
> -
> -# This shell function sends a Router Solicitation packet.
> -# test_ipv6_ra INPORT SRC_MAC SRC_LLA ADDR_MODE MTU RA_PREFIX_OPT
> -test_ipv6_ra() {
> -    local inport=$1 src_mac=$2 src_lla=$3 addr_mode=$4 mtu=$5
> prefix_opt=$6
> -    local
> request=333300000002${src_mac}86dd6000000000103aff${src_lla}ff02000000000000000000000000000285000efc000000000101${src_mac}
> -
> -    local len=24
> -    local mtu_opt=""
> -    if test $mtu != 0; then
> -        len=`expr $len + 8`
> -        mtu_opt=05010000${mtu}
> -    fi
> -
> -    if test ${#prefix_opt} != 0; then
> -        prefix_opt=${prefix_opt}fdad1234567800000000000000000000
> -        len=`expr $len + ${#prefix_opt} / 2`
> -    fi
> -
> -    len=$(printf "%x" $len)
> -    local lrp_mac=fa163e000001
> -    local lrp_lla=fe80000000000000f8163efffe000001
> -    local
> reply=${src_mac}${lrp_mac}86dd6000000000${len}3aff${lrp_lla}${src_lla}8600XXXXff${addr_mode}ffff00000000000000000101${lrp_mac}${mtu_opt}${prefix_opt}
> -    echo $reply >> $inport.expected
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif${inport} $request
> -}
> -
> -AT_CAPTURE_FILE([ofctl_monitor0.log])
> -as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0.log
> -
> -# MTU is not set and the address mode is set to slaac
> -addr_mode=00
> -default_prefix_option_config=030440c0ffffffffffffffff00000000
> -src_mac=fa163e000002
> -src_lla=fe80000000000000f8163efffe000002
> -test_ipv6_ra 1 $src_mac $src_lla $addr_mode 0
> $default_prefix_option_config
> -
> -# NXT_RESUME should be 1.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap  > 1.packets
> -
> -cat 1.expected | cut -c -112 > expout
> -AT_CHECK([cat 1.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum.
> -cat 1.expected | cut -c 117- > expout
> -AT_CHECK([cat 1.packets | cut -c 117-], [0], [expout])
> -
> -rm -f *.expected
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -reset_pcap_file hv1-vif3 hv1/vif3
> -
> -# Set the MTU to 1500
> -ovn-nbctl --wait=hv set Logical_Router_Port lrp0 ipv6_ra_configs:mtu=1500
> -
> -# Make sure that ovn-controller has installed the corresponding OF Flow.
> -OVS_WAIT_UNTIL([test 1 = `as hv1 ovs-ofctl dump-flows br-int | grep -c
> "ipv6_dst=ff02::2,nw_ttl=255,icmp_type=133,icmp_code=0"`])
> -
> -addr_mode=00
> -default_prefix_option_config=030440c0ffffffffffffffff00000000
> -src_mac=fa163e000003
> -src_lla=fe80000000000000f8163efffe000003
> -mtu=000005dc
> -
> -test_ipv6_ra 2 $src_mac $src_lla $addr_mode $mtu
> $default_prefix_option_config
> -
> -# NXT_RESUME should be 2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap  > 2.packets
> -
> -cat 2.expected | cut -c -112 > expout
> -AT_CHECK([cat 2.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum.
> -cat 2.expected | cut -c 117- > expout
> -AT_CHECK([cat 2.packets | cut -c 117-], [0], [expout])
> -
> -rm -f *.expected
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -reset_pcap_file hv1-vif3 hv1/vif3
> -
> -# Set the address mode to dhcpv6_stateful
> -ovn-nbctl --wait=hv set Logical_Router_Port lrp0
> ipv6_ra_configs:address_mode=dhcpv6_stateful
> -# Make sure that ovn-controller has installed the corresponding OF Flow.
> -OVS_WAIT_UNTIL([test 1 = `as hv1 ovs-ofctl dump-flows br-int | grep -c
> "ipv6_dst=ff02::2,nw_ttl=255,icmp_type=133,icmp_code=0"`])
> -
> -addr_mode=80
> -default_prefix_option_config=03044080ffffffffffffffff00000000
> -src_mac=fa163e000004
> -src_lla=fe80000000000000f8163efffe000004
> -mtu=000005dc
> -
> -test_ipv6_ra 3 $src_mac $src_lla $addr_mode $mtu
> $default_prefix_option_config
> -
> -# NXT_RESUME should be 3.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif3-tx.pcap  > 3.packets
> -
> -cat 3.expected | cut -c -112 > expout
> -AT_CHECK([cat 3.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum.
> -cat 3.expected | cut -c 117- > expout
> -AT_CHECK([cat 3.packets | cut -c 117-], [0], [expout])
> -
> -rm -f *.expected
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -reset_pcap_file hv1-vif3 hv1/vif3
> -
> -# Set the address mode to dhcpv6_stateless
> -ovn-nbctl --wait=hv set Logical_Router_Port lrp0
> ipv6_ra_configs:address_mode=dhcpv6_stateless
> -# Make sure that ovn-controller has installed the corresponding OF Flow.
> -OVS_WAIT_UNTIL([test 1 = `as hv1 ovs-ofctl dump-flows br-int | grep -c
> "ipv6_dst=ff02::2,nw_ttl=255,icmp_type=133,icmp_code=0"`])
> -
> -addr_mode=40
> -default_prefix_option_config=030440c0ffffffffffffffff00000000
> -src_mac=fa163e000002
> -src_lla=fe80000000000000f8163efffe000002
> -mtu=000005dc
> -
> -test_ipv6_ra 1 $src_mac $src_lla $addr_mode $mtu
> $default_prefix_option_config
> -
> -# NXT_RESUME should be 4.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap  > 1.packets
> -
> -cat 1.expected | cut -c -112 > expout
> -AT_CHECK([cat 1.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum.
> -cat 1.expected | cut -c 117- > expout
> -AT_CHECK([cat 1.packets | cut -c 117-], [0], [expout])
> -
> -rm -f *.expected
> -reset_pcap_file hv1-vif1 hv1/vif1
> -reset_pcap_file hv1-vif2 hv1/vif2
> -reset_pcap_file hv1-vif3 hv1/vif3
> -
> -# Set the address mode to invalid.
> -ovn-nbctl --wait=hv set Logical_Router_Port lrp0
> ipv6_ra_configs:address_mode=invalid
> -# Make sure that ovn-controller has not installed any OF Flow for IPv6 ND
> RA.
> -OVS_WAIT_UNTIL([test 0 = `as hv1 ovs-ofctl dump-flows br-int | grep -c
> "ipv6_dst=ff02::2,nw_ttl=255,icmp_type=133,icmp_code=0"`])
> -
> -addr_mode=40
> -default_prefix_option_config=""
> -src_mac=fa163e000002
> -src_lla=fe80000000000000f8163efffe000002
> -mtu=000005dc
> -
> -test_ipv6_ra 1 $src_mac $src_lla $addr_mode $mtu
> $default_prefix_option_config
> -
> -# NXT_RESUME should be 4 only.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor*.log | grep -c NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap  > 1.packets
> -AT_CHECK([cat 1.packets], [0], [])
> -
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- /32 router IP address])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# 2 LS 'foo' and 'alice' connected via router R1.
> -# R1 connects to 'alice' with a /32 IP address. We use static routes and
> -# nexthop to push traffic to a logical port in switch 'alice'
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:00:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo
> type=router \
> -          options:router-port=foo addresses=\"00:00:00:01:02:03\"
> -
> -# Connect alice to R1.
> -ovn-nbctl lrp-add R1 alice 00:00:00:01:02:04 172.16.1.1/32
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -          type=router options:router-port=alice
> addresses=\"00:00:00:01:02:04\"
> -
> -# Create logical port foo1 in foo
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical port alice1 in alice
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 10.0.0.2"
> -
> -#install default route in R1 to use alice1's IP address as nexthop
> -ovn-nbctl lr-route-add R1 0.0.0.0/0 10.0.0.2 alice
> -
> -# Create two hypervisor and create OVS ports corresponding to logical
> ports.
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=foo1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=alice1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -# Send ip packets between foo1 and alice1
> -src_mac="f00000010203"
> -dst_mac="000000010203"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 10 0 0 2`
>
> -packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -
> -# Send the first packet to trigger a ARP response and population of
> -# mac_bindings table.
> -as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -OVS_WAIT_UNTIL([test `ovn-sbctl find mac_binding ip="10.0.0.2" | wc -l`
> -gt 0])
> -ovn-nbctl --wait=hv sync
> -
> -# Packet to Expect at 'alice1'
> -src_mac="000000010204"
> -dst_mac="f00000010204"
> -src_ip=`ip_to_hex 192 168 1 2`
> -dst_ip=`ip_to_hex 10 0 0 2`
> -echo
> "${dst_mac}${src_mac}08004500001c000000003f110100${src_ip}${dst_ip}0035111100080000"
> > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 1 lport/HV, localport ports])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -# Add localport to the switch
> -ovn-nbctl lsp-add ls1 lp01
> -ovn-nbctl lsp-set-addresses lp01 f0:00:00:00:00:01
> -ovn-nbctl lsp-set-type lp01 localport
> -
> -net_add n1
> -
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -    ovs-vsctl add-port br-int vif01 -- \
> -        set Interface vif01 external-ids:iface-id=lp01 \
> -                              options:tx_pcap=hv${i}/vif01-tx.pcap \
> -                              options:rxq_pcap=hv${i}/vif01-rx.pcap \
> -                              ofport-request=${i}0
> -
> -    ovs-vsctl add-port br-int vif${i}1 -- \
> -        set Interface vif${i}1 external-ids:iface-id=lp${i}1 \
> -                              options:tx_pcap=hv${i}/vif${i}1-tx.pcap \
> -                              options:rxq_pcap=hv${i}/vif${i}1-rx.pcap \
> -                              ofport-request=${i}1
> -
> -    ovn-nbctl lsp-add ls1 lp${i}1
> -    ovn-nbctl lsp-set-addresses lp${i}1 f0:00:00:00:00:${i}1
> -    ovn-nbctl lsp-set-port-security lp${i}1 f0:00:00:00:00:${i}1
> -
> -        OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp${i}1` = xup])
> -done
> -
> -ovn-nbctl --wait=sb sync
> -ovn-sbctl dump-flows
> -
> -OVN_POPULATE_ARP
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    echo hv${1%?}
> -}
> -#
> -# test_packet INPORT DST SRC ETHTYPE EOUT LOUT DEFHV
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  INPORT is specified
> as
> -# logical switch port numbers, e.g. 11 for vif11.
> -#
> -# EOUT is the end-to-end output port, that is, where the packet will end
> up
> -# after possibly bouncing through one or more localnet ports.  LOUT is the
> -# logical output port, which might be a localnet port, as seen by
> ovn-trace
> -# (which doesn't know what localnet ports are connected to and therefore
> can't
> -# figure out the end-to-end answer).
> -#
> -# DEFHV is the default hypervisor from where the packet is going to be
> sent
> -# if the source port is a localport.
> -for i in 1 2; do
> -    for j in 0 1; do
> -        : > $i$j.expected
> -    done
> -done
> -test_packet() {
> -    local inport=$1 dst=$2 src=$3 eth=$4 eout=$5 lout=$6 defhv=$7
> -    echo "$@"
> -
> -    # First try tracing the packet.
> -    uflow="inport==\"lp$inport\" && eth.dst==$dst && eth.src==$src &&
> eth.type==0x$eth"
> -    if test $lout != drop; then
> -        echo "output(\"$lout\");"
> -    fi > expout
> -    AT_CAPTURE_FILE([trace])
> -    AT_CHECK([ovn-trace --all ls1 "$uflow" | tee trace | sed '1,/Minimal
> trace/d'], [0], [expout])
> -
> -    # Then actually send a packet, for an end-to-end test.
> -    local packet=$(echo $dst$src | sed 's/://g')${eth}
> -    hv=`vif_to_hv $inport`
> -    # If hypervisor 0 (localport) use the defhv parameter
> -    if test $hv = hv0; then
> -        hv=$defhv
> -    fi
> -    vif=vif$inport
> -    as $hv ovs-appctl netdev-dummy/receive $vif $packet
> -    if test $eout != drop; then
> -        echo $packet >> ${eout#lp}.expected
> -    fi
> -}
> -
> -
> -# lp11 and lp21 are on different hypervisors
> -test_packet 11 f0:00:00:00:00:21 f0:00:00:00:00:11 1121 lp21 lp21
> -test_packet 21 f0:00:00:00:00:11 f0:00:00:00:00:21 2111 lp11 lp11
> -
> -# Both VIFs should be able to reach the localport on their own HV
> -test_packet 11 f0:00:00:00:00:01 f0:00:00:00:00:11 1101 lp01 lp01
> -test_packet 21 f0:00:00:00:00:01 f0:00:00:00:00:21 2101 lp01 lp01
> -
> -# Packet sent from localport on same hv should reach the vif
> -test_packet 01 f0:00:00:00:00:11 f0:00:00:00:00:01 0111 lp11 lp11 hv1
> -test_packet 01 f0:00:00:00:00:21 f0:00:00:00:00:01 0121 lp21 lp21 hv2
> -
> -# Packet sent from localport on different hv should be dropped
> -test_packet 01 f0:00:00:00:00:21 f0:00:00:00:00:01 0121 drop lp21 hv1
> -test_packet 01 f0:00:00:00:00:11 f0:00:00:00:00:01 0111 drop lp11 hv2
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2; do
> -    for j in 0 1; do
> -        OVN_CHECK_PACKETS([hv$i/vif$i$j-tx.pcap], [$i$j.expected])
> -    done
> -done
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 1 LR with HA distributed router gateway port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -
> -# create gateways with external network connectivity
> -
> -for i in 1 2; do
> -    sim_add gw$i
> -    as gw$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -    ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -done
> -
> -ovn-nbctl ls-add inside
> -ovn-nbctl ls-add outside
> -
> -# create hypervisors with a vif port each to an internal network
> -
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.1$i
> -    ovs-vsctl -- add-port br-int hv$i-vif1 -- \
> -        set interface hv$i-vif1 external-ids:iface-id=inside$i \
> -        options:tx_pcap=hv$i/vif1-tx.pcap \
> -        options:rxq_pcap=hv$i/vif1-rx.pcap \
> -        ofport-request=1
> -
> -        ovn-nbctl lsp-add inside inside$i \
> -            -- lsp-set-addresses inside$i "f0:00:00:01:22:$i
> 192.168.1.10$i"
> -
> -done
> -
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -
> -# Connect inside to R1
> -ovn-nbctl lrp-add R1 inside 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add inside rp-inside -- set Logical_Switch_Port rp-inside \
> -    type=router options:router-port=inside \
> -    -- lsp-set-addresses rp-inside router
> -
> -# Connect outside to R1 as distributed router gateway port on gw1+gw2
> -ovn-nbctl lrp-add R1 outside 00:00:02:01:02:04 192.168.0.101/24
> -
> -ovn-nbctl --id=@gc0 create Gateway_Chassis \
> -                    name=outside_gw1 chassis_name=gw1 priority=20 -- \
> -          --id=@gc1 create Gateway_Chassis \
> -                    name=outside_gw2 chassis_name=gw2 priority=10 -- \
> -          set Logical_Router_Port outside 'gateway_chassis=[@gc0, at gc1]'
> -
> -ovn-nbctl lsp-add outside rp-outside -- set Logical_Switch_Port
> rp-outside \
> -    type=router options:router-port=outside \
> -    -- lsp-set-addresses rp-outside router
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -echo "---------NB dump-----"
> -ovn-nbctl show
> -echo "---------------------"
> -ovn-nbctl list logical_router
> -echo "---------------------"
> -ovn-nbctl list logical_router_port
> -echo "---------------------"
> -
> -echo "---------SB dump-----"
> -ovn-sbctl list datapath_binding
> -echo "---------------------"
> -ovn-sbctl list port_binding
> -echo "---------------------"
> -ovn-sbctl dump-flows
> -echo "---------------------"
> -ovn-sbctl list chassis
> -ovn-sbctl list encap
> -echo "---------------------"
> -echo "------ Gateway_Chassis dump (SBDB) -------"
> -ovn-sbctl list Gateway_Chassis
> -echo "------ Port_Binding chassisredirect -------"
> -ovn-sbctl find Port_Binding type=chassisredirect
> -echo "-------------------------------------------"
> -
> -# There should be one ha_chassis_group with the name "outside"
> -ha_chassi_grp_name=`ovn-sbctl --bare --columns name find \
> -ha_chassis_group name="outside"`
> -
> -AT_CHECK([test $ha_chassi_grp_name = outside])
> -
> -# There should be 2 ha_chassis rows in SB DB.
> -AT_CHECK([ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | awk '{print $3}' \
> -| grep '-' | wc -l ], [0], [2
> -])
> -
> -ha_ch=`ovn-sbctl --bare --columns ha_chassis  find ha_chassis_group`
> -# Trim the spaces.
> -ha_ch=`echo $ha_ch | sed 's/ //g'`
> -
> -ha_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list ha_chassis | sort`
> -do
> -    ha_ch_list="$ha_ch_list $i"
> -done
> -
> -# Trim the spaces.
> -ha_ch_list=`echo $ha_ch_list | sed 's/ //g'`
> -
> -AT_CHECK([test "$ha_ch_list" = "$ha_ch"])
> -
> -for chassis in gw1 gw2 hv1 hv2; do
> -    as $chassis
> -    echo "------ $chassis dump ----------"
> -    ovs-ofctl show br-int
> -    ovs-ofctl dump-flows br-int
> -    echo "--------------------------"
> -done
> -bfd_dump() {
> -    for chassis in gw1 gw2 hv1 hv2; do
> -        as $chassis
> -        echo "------ $chassis dump (BFD)----"
> -        echo "BFD (from $chassis):"
> -        # dump BFD config and status to the other chassis
> -        for chassis2 in gw1 gw2 hv1 hv2; do
> -            if [[ "$chassis" != "$chassis2" ]]; then
> -                echo " -> $chassis2:"
> -                echo "   $(ovs-vsctl --bare --columns bfd,bfd_status find
> Interface name=ovn-$chassis2-0)"
> -            fi
> -        done
> -        echo "--------------------------"
> -    done
> -}
> -
> -bfd_dump
> -
> -hv1_gw1_ofport=$(as hv1 ovs-vsctl --bare --columns ofport find Interface
> name=ovn-gw1-0)
> -hv1_gw2_ofport=$(as hv1 ovs-vsctl --bare --columns ofport find Interface
> name=ovn-gw2-0)
> -hv2_gw1_ofport=$(as hv2 ovs-vsctl --bare --columns ofport find Interface
> name=ovn-gw1-0)
> -hv2_gw2_ofport=$(as hv2 ovs-vsctl --bare --columns ofport find Interface
> name=ovn-gw2-0)
> -
> -echo $hv1_gw1_ofport
> -echo $hv1_gw2_ofport
> -echo $hv2_gw1_ofport
> -echo $hv2_gw2_ofport
> -
> -echo "--- hv1 ---"
> -as hv1 ovs-ofctl dump-flows br-int table=32
> -
> -echo "--- hv2 ---"
> -as hv2 ovs-ofctl dump-flows br-int table=32
> -
> -gw1_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw1)
> -gw2_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw2)
> -
> -OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv1_gw1_ofport,$hv1_gw2_ofport \
> -| wc -l], [0], [1
> -])
> -
> -OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport \
> -| wc -l], [0], [1
> -])
> -
> -# make sure that flows for handling the outside router port reside on gw1
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# make sure ARP responder flows for outside router port reside on gw1 too
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=9 | \
> -grep arp_tpa=192.168.0.101 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=9 | grep
> arp_tpa=192.168.0.101 | wc -l], [0], [[0
> -]])
> -
> -# check that the chassis redirect port has been claimed by the gw1 chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1
> -]])
> -
> -hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"`
> -hv2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv2"`
> -
> -exp_ref_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list chassis | sort`
> -do
> -    if test $i = $hv1_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    elif test $i = $hv2_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    fi
> -done
> -
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -
> -# at this point, we invert the priority of the gw chassis between gw1 and
> gw2
> -
> -ovn-nbctl --id=@gc0 create Gateway_Chassis \
> -                    name=outside_gw1 chassis_name=gw1 priority=10 -- \
> -          --id=@gc1 create Gateway_Chassis \
> -                    name=outside_gw2 chassis_name=gw2 priority=20 -- \
> -          set Logical_Router_Port outside 'gateway_chassis=[@gc0, at gc1]'
> -
> -
> -# XXX: Let the change propagate down to the ovn-controllers
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -# we make sure that the hypervisors noticed, and inverted the slave ports
> -OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv1_gw2_ofport,$hv1_gw1_ofport \
> -| wc -l], [0], [1
> -])
> -
> -OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv2_gw2_ofport,$hv2_gw1_ofport \
> -| wc -l], [0], [1
> -])
> -
> -# check that the chassis redirect port has been reclaimed by the gw2
> chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw2_chassis | wc -l], [0],[[1
> -]])
> -
> -# check BFD enablement on tunnel ports from gw1 #########
> -as gw1
> -for chassis in gw2 hv1 hv2; do
> -    echo "checking gw1 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -
> -
> -# check BFD enablement on tunnel ports from gw2 ##########
> -as gw2
> -for chassis in gw1 hv1 hv2; do
> -    echo "checking gw2 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -
> -# check BFD enablement on tunnel ports from hv1 ###########
> -as hv1
> -for chassis in gw1 gw2; do
> -    echo "checking hv1 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -# make sure BFD is not enabled to hv2, we don't need it
> -AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-hv2-0],[0],
> -         [[
> -]])
> -
> -
> -# check BFD enablement on tunnel ports from hv2 ##########
> -as hv2
> -for chassis in gw1 gw2; do
> -    echo "checking hv2 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -# make sure BFD is not enabled to hv1, we don't need it
> -AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-hv1-0],[0],
> -         [[
> -]])
> -
> -# make sure that flows for handling the outside router port reside on gw2
> now
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# disconnect GW2 from the network, GW1 should take over
> -as gw2
> -port=${sandbox}_br-phys
> -as main ovs-vsctl del-port n1 $port
> -
> -bfd_dump
> -
> -# make sure that flows for handling the outside router port reside on gw2
> now
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=25 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# check that the chassis redirect port has been reclaimed by the gw1
> chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1
> -]])
> -
> -ovn-nbctl --wait=hv set NB_Global . options:"bfd-min-rx"=2000
> -as gw2
> -for chassis in gw1 hv1 hv2; do
> -    echo "checking gw2 -> $chassis"
> -    OVS_WAIT_UNTIL([
> -    bfd_cfg=$(ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0)
> -    test "$bfd_cfg" = "enable=true min_rx=2000"
> -])
> -done
> -ovn-nbctl --wait=hv set NB_Global . options:"bfd-min-tx"=1500
> -for chassis in gw1 hv1 hv2; do
> -    echo "checking gw2 -> $chassis"
> -    OVS_WAIT_UNTIL([
> -    bfd_cfg=$(ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0)
> -    test "$bfd_cfg" = "enable=true min_rx=2000 min_tx=1500"
> -])
> -done
> -ovn-nbctl remove NB_Global . options "bfd-min-rx"
> -ovn-nbctl --wait=hv set NB_Global . options:"bfd-mult"=5
> -for chassis in gw1 hv1 hv2; do
> -    echo "checking gw2 -> $chassis"
> -    OVS_WAIT_UNTIL([
> -    bfd_cfg=$(ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0)
> -    test "$bfd_cfg" = "enable=true min_tx=1500 mult=5"
> -])
> -done
> -
> -# Delete the inside1 vif. The ref_chassis in ha_chassis_group shouldn't
> have
> -# reference to hv1.
> -as hv1 ovs-vsctl del-port hv1-vif1
> -
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$hv2_ch_uuid" = "$ref_ch_list"])
> -
> -# Delete the inside2 vif.
> -ovn-sbctl show
> -
> -echo "Deleting hv2-vif1"
> -as hv2 ovs-vsctl del-port hv2-vif1
> -
> -# ref_chassis of ha_chassis_group should be empty
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     exp_ref_ch_list=""
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# Delete the Gateway_Chassis for lrp - outside
> -ovn-nbctl clear Logical_Router_Port outside gateway_chassis
> -
> -# There shoud be no ha_chassis_group rows in SB DB.
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis_group | wc -l`])
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`])
> -
> -ovn-nbctl remove NB_Global . options "bfd-min-rx"
> -ovn-nbctl remove NB_Global . options "bfd-min-tx"
> -ovn-nbctl remove NB_Global . options "bfd-mult"
> -
> -# Now test with HA chassis group instead of Gateway chassis in NB DB
> -ovn-nbctl --wait=sb ha-chassis-group-add hagrp1
> -
> -ovn-nbctl list ha_chassis_group
> -ovn-nbctl --bare --columns _uuid find ha_chassis_group name=hagrp1
> -hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group
> name=hagrp1`
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw1 30
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw2 20
> -
> -# ovn-northd should not create HA chassis group and HA chassis rows
> -# unless the HA chassis group in OVN NB DB is associated to
> -# a logical router port.
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl list ha_chassis | wc -l`])
> -
> -# Associate hagrp1 to outside logical router port
> -ovn-nbctl set Logical_Router_Port outside ha_chassis_group=$hagrp1_uuid
> -
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns _uuid \
> -find ha_chassis_group | wc -l`])
> -
> -OVS_WAIT_UNTIL([test 2 = `ovn-sbctl list ha_chassis | grep chassis | \
> -grep -v chassis-name | wc -l`])
> -
> -OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv1_gw1_ofport,$hv1_gw2_ofport \
> -| wc -l], [0], [1
> -])
> -
> -OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv2_gw1_ofport,$hv2_gw2_ofport \
> -| wc -l], [0], [1
> -])
> -
> -# make sure that flows for handling the outside router port reside on gw1
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# make sure ARP responder flows for outside router port reside on gw1 too
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=9 | \
> -grep arp_tpa=192.168.0.101 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=9 | grep
> arp_tpa=192.168.0.101 | wc -l], [0], [[0
> -]])
> -
> -# check that the chassis redirect port has been claimed by the gw1 chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1
> -]])
> -
> -# Re add the ovs ports.
> -for i in 1 2; do
> -    as hv$i
> -    ovs-vsctl -- add-port br-int hv$i-vif1 -- \
> -        set interface hv$i-vif1 external-ids:iface-id=inside$i \
> -        options:tx_pcap=hv$i/vif1-tx.pcap \
> -        options:rxq_pcap=hv$i/vif1-rx.pcap \
> -        ofport-request=1
> -done
> -
> -hv1_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv1"`
> -hv2_ch_uuid=`ovn-sbctl --bare --columns _uuid find chassis name="hv2"`
> -
> -exp_ref_ch_list=''
> -for i in `ovn-sbctl --bare --columns _uuid list chassis | sort`
> -do
> -    if test $i = $hv1_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    elif test $i = $hv2_ch_uuid; then
> -        exp_ref_ch_list="${exp_ref_ch_list}$i"
> -    fi
> -done
> -
> -OVS_WAIT_UNTIL(
> -    [ref_ch_list=`ovn-sbctl --bare --columns ref_chassis find
> ha_chassis_group | sort`
> -     # Trim the spaces.
> -     ref_ch_list=`echo $ref_ch_list | sed 's/ //g'`
> -     test "$exp_ref_ch_list" = "$ref_ch_list"])
> -
> -# Increase the priority of gw2
> -ovn-nbctl --wait=sb ha-chassis-group-add-chassis hagrp1 gw2 40
> -
> -OVS_WAIT_UNTIL([as hv1 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv1_gw2_ofport,$hv1_gw1_ofport \
> -| wc -l], [0], [1
> -])
> -
> -OVS_WAIT_UNTIL([as hv2 ovs-ofctl dump-flows br-int table=32 | \
> -grep active_backup | grep slaves:$hv2_gw2_ofport,$hv2_gw1_ofport \
> -| wc -l], [0], [1
> -])
> -
> -# check that the chassis redirect port has been reclaimed by the gw2
> chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw2_chassis | wc -l], [0],[[1
> -]])
> -
> -# check BFD enablement on tunnel ports from gw1 #########
> -as gw1
> -for chassis in gw2 hv1 hv2; do
> -    echo "checking gw1 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -
> -# check BFD enablement on tunnel ports from gw2 ##########
> -as gw2
> -for chassis in gw1 hv1 hv2; do
> -    echo "checking gw2 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -
> -# check BFD enablement on tunnel ports from hv1 ###########
> -as hv1
> -for chassis in gw1 gw2; do
> -    echo "checking hv1 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -# make sure BFD is not enabled to hv2, we don't need it
> -AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-hv2-0],[0],
> -         [[
> -]])
> -
> -# check BFD enablement on tunnel ports from hv2 ##########
> -as hv2
> -for chassis in gw1 gw2; do
> -    echo "checking hv2 -> $chassis"
> -    AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-$chassis-0],[0],
> -             [[enable=true
> -]])
> -done
> -# make sure BFD is not enabled to hv1, we don't need it
> -AT_CHECK([ovs-vsctl --bare --columns bfd find Interface
> name=ovn-hv1-0],[0],
> -         [[
> -]])
> -
> -# make sure that flows for handling the outside router port reside on gw2
> now
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# disconnect GW2 from the network, GW1 should take over
> -as gw2
> -port=${sandbox}_br-phys
> -as main ovs-vsctl del-port n1 $port
> -
> -bfd_dump
> -
> -# make sure that flows for handling the outside router port reside on gw2
> now
> -OVS_WAIT_UNTIL([as gw1 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[1
> -]])
> -OVS_WAIT_UNTIL([as gw2 ovs-ofctl dump-flows br-int table=24 | \
> -grep 00:00:02:01:02:04 | wc -l], [0], [[0
> -]])
> -
> -# check that the chassis redirect port has been reclaimed by the gw1
> chassis
> -OVS_WAIT_UNTIL([ovn-sbctl --columns chassis --bare find Port_Binding \
> -logical_port=cr-outside | grep $gw1_chassis | wc -l], [0],[[1
> -]])
> -
> -OVN_CLEANUP([gw1],[gw2],[hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- send gratuitous ARP for NAT rules on HA distributed
> router])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -ovn-nbctl ls-add ls0
> -ovn-nbctl ls-add ls1
> -ovn-nbctl create Logical_Router name=lr0
> -ovn-nbctl lrp-add lr0 lrp0 f0:00:00:00:00:01 192.168.0.100/24
> -
> -ovn-nbctl --id=@gc0 create Gateway_Chassis \
> -                    name=outside_gw1 chassis_name=hv2 priority=10 -- \
> -          --id=@gc1 create Gateway_Chassis \
> -                    name=outside_gw2 chassis_name=hv3 priority=1 -- \
> -          set Logical_Router_Port lrp0 'gateway_chassis=[@gc0, at gc1]'
> -
> -ovn-nbctl lsp-add ls0 lrp0-rp -- set Logical_Switch_Port lrp0-rp \
> -    type=router options:router-port=lrp0 addresses="router"
> -ovn-nbctl lrp-add lr0 lrp1 f0:00:00:00:00:02 10.0.0.1/24
> -ovn-nbctl lsp-add ls1 lrp1-rp -- set Logical_Switch_Port lrp1-rp \
> -    type=router options:router-port=lrp1 addresses="router"
> -
> -# Add NAT rules
> -AT_CHECK([ovn-nbctl lr-nat-add lr0 snat 192.168.0.100 10.0.0.0/24])
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -AT_CHECK([ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -AT_CHECK([ovs-vsctl add-port br-phys snoopvif -- set Interface snoopvif
> options:tx_pcap=hv1/snoopvif-tx.pcap options:rxq_pcap=hv1/snoopvif-rx.pcap])
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -AT_CHECK([as hv2 ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -
> -sim_add hv3
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -AT_CHECK([as hv3 ovs-vsctl set Open_vSwitch .
> external-ids:ovn-bridge-mappings=physnet1:br-phys])
> -
> -# Create a localnet port.
> -AT_CHECK([ovn-nbctl lsp-add ls0 ln_port])
> -AT_CHECK([ovn-nbctl lsp-set-addresses ln_port unknown])
> -AT_CHECK([ovn-nbctl lsp-set-type ln_port localnet])
> -AT_CHECK([ovn-nbctl lsp-set-options ln_port network_name=physnet1])
> -
> -# wait for earlier changes to take effect
> -AT_CHECK([ovn-nbctl --timeout=3 --wait=hv sync], [0], [ignore])
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -as hv1 reset_pcap_file snoopvif hv1/snoopvif
> -as hv2 reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -as hv3 reset_pcap_file br-phys_n1 hv3/br-phys_n1
> -# add nat-addresses option
> -ovn-nbctl --wait=hv lsp-set-options lrp0-rp router-port=lrp0
> nat-addresses="router"
> -
> -# Wait for packets to be received through hv2.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 100])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -only_broadcast_from_lrp1() {
> -    grep "fffffffffffff00000000001"
> -}
> -
>
> -garp="fffffffffffff0000000000108060001080006040001f00000000001c0a80064000000000000c0a80064"
> -echo $garp > expout
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv1_snoop_tx
> -echo "packets on hv1-snoopvif:"
> -cat hv1_snoop_tx
> -AT_CHECK([sort hv1_snoop_tx], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv2_br_phys_tx
> -echo "packets on hv2 br-phys tx"
> -cat hv2_br_phys_tx
> -AT_CHECK([grep $garp hv2_br_phys_tx | sort], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv3/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv3_br_phys_tx
> -echo "packets on hv3 br-phys tx"
> -cat hv3_br_phys_tx
> -AT_CHECK([grep $garp hv3_br_phys_tx | sort], [0], [])
> -
> -
> -# at this point, we invert the priority of the gw chassis between hv2 and
> hv3
> -
> -ovn-nbctl --wait=hv \
> -          --id=@gc0 create Gateway_Chassis \
> -                    name=outside_gw1 chassis_name=hv2 priority=1 -- \
> -          --id=@gc1 create Gateway_Chassis \
> -                    name=outside_gw2 chassis_name=hv3 priority=10 -- \
> -          set Logical_Router_Port lrp0 'gateway_chassis=[@gc0, at gc1]'
> -
> -
> -as hv1 reset_pcap_file snoopvif hv1/snoopvif
> -as hv2 reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -as hv3 reset_pcap_file br-phys_n1 hv3/br-phys_n1
> -
> -# Wait for packets to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 100])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq >  hv1_snoopvif_tx
> -AT_CHECK([sort hv1_snoopvif_tx], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv3/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv3_br_phys_tx
> -AT_CHECK([grep $garp hv3_br_phys_tx | sort], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv2_br_phys_tx
> -AT_CHECK([grep $garp hv2_br_phys_tx | sort], [0], [])
> -
> -# change localnet port tag.
> -AT_CHECK([ovn-nbctl set Logical_Switch_Port ln_port tag=2014])
> -
> -# wait for earlier changes to take effect
> -OVS_WAIT_UNTIL([test 1 = `as hv2 ovs-ofctl dump-flows br-int table=65 | \
> -grep "actions=mod_vlan_vid:2014" | wc -l`
> -])
> -
> -OVS_WAIT_UNTIL([test 1 = `as hv3 ovs-ofctl dump-flows br-int table=65 | \
> -grep "actions=mod_vlan_vid:2014" | wc -l`
> -])
> -
> -# update nat-addresses option
> -ovn-nbctl --wait=hv clear logical_switch_port lrp0-rp options
> -
> -#Wait until the Port_Binding.nat_addresses is cleared.
> -OVS_WAIT_UNTIL([test 0 = `ovn-sbctl --bare --columns nat_addresses find
> port_binding \
> -logical_port=lrp0-rp | grep is_chassis | wc -l`])
> -
> -as hv1 reset_pcap_file snoopvif hv1/snoopvif
> -as hv2 reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -as hv3 reset_pcap_file br-phys_n1 hv3/br-phys_n1
> -
> -ovn-nbctl --wait=hv lsp-set-options lrp0-rp router-port=lrp0
> nat-addresses="router"
> -
> -#Wait until the Port_Binding.nat_addresses is set.
> -OVS_WAIT_UNTIL([test 1 = `ovn-sbctl --bare --columns nat_addresses find
> port_binding \
> -logical_port=lrp0-rp | grep is_chassis | wc -l`])
> -
> -# Wait for packets to be received.
> -OVS_WAIT_UNTIL([test `wc -c < "hv1/snoopvif-tx.pcap"` -ge 100])
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
>
> -garp="fffffffffffff00000000001810007de08060001080006040001f00000000001c0a80064000000000000c0a80064"
> -echo $garp > expout
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/snoopvif-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq >  hv1_snoopvif_tx
> -AT_CHECK([sort hv1_snoopvif_tx], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv3/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv3_br_phys_tx
> -AT_CHECK([grep $garp hv3_br_phys_tx | sort], [0], [expout])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-phys_n1-tx.pcap |
> trim_zeros | only_broadcast_from_lrp1 | uniq > hv2_br_phys_tx
> -AT_CHECK([grep $garp hv2_br_phys_tx | sort], [0], [])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ensure one gw controller restart in HA doesn't bounce
> the master])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -
> -# create two gateways with external network connectivity
> -for i in 1 2; do
> -    sim_add gw$i
> -    as gw$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -    ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -done
> -
> -ovn-nbctl ls-add inside
> -ovn-nbctl ls-add outside
> -
> -# create one hypervisors with a vif port the internal network
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.11
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=inside1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovn-nbctl lsp-add inside inside1 \
> -        -- lsp-set-addresses inside1 "f0:00:00:01:22:01 192.168.1.101"
> -
> -
> -OVN_POPULATE_ARP
> -
> -ovn-nbctl create Logical_Router name=R1
> -
> -# Connect inside to R1
> -ovn-nbctl lrp-add R1 inside 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add inside rp-inside -- set Logical_Switch_Port rp-inside \
> -    type=router options:router-port=inside \
> -    -- lsp-set-addresses rp-inside router
> -
> -# Connect outside to R1 as distributed router gateway port on gw1+gw2
> -ovn-nbctl lrp-add R1 outside 00:00:02:01:02:04 192.168.0.101/24
> -
> -ovn-nbctl --id=@gc0 create Gateway_Chassis \
> -                    name=outside_gw1 chassis_name=gw1 priority=20 -- \
> -          --id=@gc1 create Gateway_Chassis \
> -                    name=outside_gw2 chassis_name=gw2 priority=10 -- \
> -          set Logical_Router_Port outside 'gateway_chassis=[@gc0, at gc1]'
> -
> -ovn-nbctl lsp-add outside rp-outside -- set Logical_Switch_Port
> rp-outside \
> -    type=router options:router-port=outside \
> -    -- lsp-set-addresses rp-outside router
> -
> -# Create localnet port in outside
> -ovn-nbctl lsp-add outside ln-outside
> -ovn-nbctl lsp-set-addresses ln-outside unknown
> -ovn-nbctl lsp-set-type ln-outside localnet
> -ovn-nbctl lsp-set-options ln-outside network_name=phys
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -# currently when ovn-controller is restarted, the old entry is deleted
> -# and a new one is created, which leaves the Gateway_Chassis with
> -# an empty chassis for a while. NOTE: restarting ovn-controller in tests
> -# doesn't have the same effect because "name" is conserved, and the
> -# Chassis entry is not replaced.
> -
> -> gw1/ovn-controller.log
> -
> -gw2_chassis=$(ovn-sbctl --bare --columns=_uuid find Chassis name=gw2)
> -ovn-sbctl destroy Chassis $gw2_chassis
> -
> -OVS_WAIT_UNTIL([test 0 = `grep -c "Releasing lport"
> gw1/ovn-controller.log`])
> -
> -OVN_CLEANUP([gw1],[gw2],[hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IPv6 Neighbor Solicitation for unknown MAC])
> -AT_KEYWORDS([ovn-nd_ns for unknown mac])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add sw0_ip6
> -ovn-nbctl lsp-add sw0_ip6 sw0_ip6-port1
> -ovn-nbctl lsp-set-addresses sw0_ip6-port1 \
> -"50:64:00:00:00:02 aef0::5264:00ff:fe00:0002"
> -
> -ovn-nbctl lsp-set-port-security sw0_ip6-port1 \
> -"50:64:00:00:00:02 aef0::5264:00ff:fe00:0002"
> -
> -ovn-nbctl lr-add lr0_ip6
> -ovn-nbctl lrp-add lr0_ip6 lrp0_ip6 00:00:00:00:af:01 aef0:0:0:0:0:0:0:0/64
> -ovn-nbctl lsp-add sw0_ip6 lrp0_ip6-attachment
> -ovn-nbctl lsp-set-type lrp0_ip6-attachment router
> -ovn-nbctl lsp-set-addresses lrp0_ip6-attachment router
> -ovn-nbctl lsp-set-options lrp0_ip6-attachment router-port=lrp0_ip6
> -ovn-nbctl set logical_router_port lrp0_ip6
> ipv6_ra_configs:address_mode=slaac
> -
> -ovn-nbctl ls-add public
> -ovn-nbctl lsp-add public ln-public
> -ovn-nbctl lsp-set-addresses ln-public unknown
> -ovn-nbctl lsp-set-type ln-public localnet
> -ovn-nbctl lsp-set-options ln-public network_name=phys
> -
> -ovn-nbctl lrp-add lr0_ip6 ip6_public 00:00:02:01:02:04 \
> -2001:db8:1:0:200:02ff:fe01:0204/64 \
> --- set Logical_Router_port ip6_public options:redirect-chassis="hv1"
> -
> -#install static route
> -ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route \
> -ip_prefix="\:\:/0" nexthop="2001\:db8\:1\:0\:200\:02ff\:fe01\:1305" \
> --- add Logical_Router lr0_ip6 static_routes @lrt
> -
> -ovn-nbctl lsp-add public rp-ip6_public -- set Logical_Switch_Port \
> -rp-ip6_public  type=router options:router-port=ip6_public \
> --- lsp-set-addresses rp-ip6_public router
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw0_ip6-port1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw0_ip6-port1` = xup])
> -
> -# There should be 2 Neighbor Advertisement flows for the router port
> -# aef0:: ip address in logical switch pipeline with action nd_na_router.
> -AT_CHECK([ovn-sbctl dump-flows sw0_ip6 | grep ls_in_arp_rsp | \
> -grep "nd_na_router" | wc -l], [0], [2
> -])
> -
> -# There should be 4 Neighbor Advertisement flows with action nd_na_router
> -# in the router pipeline for the router lr0_ip6.
> -AT_CHECK([ovn-sbctl dump-flows lr0_ip6 | grep nd_na_router | \
> -wc -l], [0], [4
> -])
> -
> -cr_uuid=`ovn-sbctl find port_binding logical_port=cr-ip6_public | grep
> _uuid | cut -f2 -d ":"`
> -
> -# There is only one chassis.
> -chassis_uuid=`ovn-sbctl list chassis | grep _uuid | cut -f2 -d ":"`
> -OVS_WAIT_UNTIL([test $chassis_uuid = `ovn-sbctl get port_binding $cr_uuid
> chassis`])
> -
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -# Test the IPv6 Neighbor Solicitation (NS) - nd_ns action for unknown MAC
> -# addresses. ovn-controller should generate an IPv6 NS request for IPv6
> -# packets whose MAC is unknown (in the ARP_REQUEST router pipeline stage.
> -# test_ipv6 INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -# This function sends ipv6 packet
> -test_ipv6() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local dst_mcast_mac=$6 mcast_node_ip=$7 nd_target=$8
> -
> -    local
> packet=${dst_mac}${src_mac}86dd6000000000083aff${src_ip}${dst_ip}
> -    packet=${packet}8000000000000000
> -
> -    src_mac=000002010204
> -
> expected_packet=${dst_mcast_mac}${src_mac}86dd6000000000203aff${src_ip}
> -    expected_packet=${expected_packet}${mcast_node_ip}8700XXXX00000000
> -    expected_packet=${expected_packet}${nd_target}0101${src_mac}
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif${inport} $packet
> -    rm -f ipv6_ns.expected
> -    echo $expected_packet >> ipv6_ns.expected
> -}
> -
> -src_mac=506400000002
> -dst_mac=00000000af01
> -src_ip=aef0000000000000526400fffe000002
> -dst_ip=20010db800010000020002fffe010205
> -dst_mcast_mac=3333ff010205
> -mcast_node_ip=ff0200000000000000000001ff010205
> -nd_target=20010db800010000020002fffe010205
> -# Send an IPv6 packet. Generated IPv6 Neighbor solicitation packet
> -# should be received by the ports attached to br-phys.
> -test_ipv6 1 $src_mac $dst_mac $src_ip $dst_ip $dst_mcast_mac \
> -$mcast_node_ip $nd_target
> -
> -OVS_WAIT_WHILE([test 24 = $(wc -c hv1/br-phys_n1-tx.pcap | cut -d " "
> -f1)])
> -OVS_WAIT_WHILE([test 24 = $(wc -c hv1/br-phys-tx.pcap | cut -d " " -f1)])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys_n1-tx.pcap | \
> -trim_zeros > 1.packets
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap | \
> -trim_zeros > 2.packets
> -
> -cat ipv6_ns.expected | cut -c -112 > expout
> -AT_CHECK([cat 1.packets | cut -c -112], [0], [expout])
> -AT_CHECK([cat 2.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum
> -cat ipv6_ns.expected | cut -c 117- > expout
> -AT_CHECK([cat 1.packets | cut -c 117-], [0], [expout])
> -AT_CHECK([cat 2.packets | cut -c 117-], [0], [expout])
> -
> -# Now send a packet with destination ip other than
> -# 2001:db8:1:0:200:02ff:fe01:0204/64 prefix.
> -reset_pcap_file br-phys_n1 hv1/br-phys_n1
> -reset_pcap_file br-phys hv1/br-phys
> -
> -src_mac=506400000002
> -dst_mac=00000000af01
> -src_ip=aef0000000000000526400fffe000002
> -dst_ip=20020ab8000100000200020000020306
> -# multicast mac of the nexthop IP - 2001:db8:1:0:200:02ff:fe01:1305
> -dst_mcast_mac=3333ff011305
> -mcast_node_ip=ff0200000000000000000001ff011305
> -nd_target=20010db800010000020002fffe011305
> -test_ipv6 1 $src_mac $dst_mac $src_ip $dst_ip $dst_mcast_mac \
> -$mcast_node_ip $nd_target
> -
> -OVS_WAIT_WHILE([test 24 = $(wc -c hv1/br-phys_n1-tx.pcap | cut -d " "
> -f1)])
> -OVS_WAIT_WHILE([test 24 = $(wc -c hv1/br-phys-tx.pcap | cut -d " " -f1)])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys_n1-tx.pcap | \
> -trim_zeros > 1.packets
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap | \
> -trim_zeros > 2.packets
> -
> -cat ipv6_ns.expected | cut -c -112 > expout
> -AT_CHECK([cat 1.packets | cut -c -112], [0], [expout])
> -AT_CHECK([cat 2.packets | cut -c -112], [0], [expout])
> -
> -# Skipping the ICMPv6 checksum
> -cat ipv6_ns.expected | cut -c 117- > expout
> -AT_CHECK([cat 1.packets | cut -c 117-], [0], [expout])
> -AT_CHECK([cat 2.packets | cut -c 117-], [0], [expout])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- options:requested-chassis for logical port])
> -ovn_start
> -
> -net_add n1
> -
> -ovn-nbctl ls-add ls0
> -ovn-nbctl lsp-add ls0 lsp0
> -
> -# create two hypervisors, each with one vif port
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.11
> -ovs-vsctl -- add-port br-int hv1-vif0 -- \
> -set Interface hv1-vif0 ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.12
> -ovs-vsctl -- add-port br-int hv2-vif0 -- \
> -set Interface hv2-vif0 ofport-request=1
> -
> -# Allow only chassis hv1 to bind logical port lsp0.
> -ovn-nbctl lsp-set-options lsp0 requested-chassis=hv1
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -# Retrieve hv1 and hv2 chassis UUIDs from southbound database
> -ovn-sbctl wait-until chassis hv1
> -ovn-sbctl wait-until chassis hv2
> -hv1_uuid=$(ovn-sbctl --bare --columns _uuid find chassis name=hv1)
> -hv2_uuid=$(ovn-sbctl --bare --columns _uuid find chassis name=hv2)
> -
> -# (1) Chassis hv2 should not bind lsp0 when requested-chassis is hv1.
> -echo "verifying that hv2 does not bind lsp0 when hv2 physical/logical
> mapping is added"
> -as hv2
> -ovs-vsctl set interface hv2-vif0 external-ids:iface-id=lsp0
> -
> -OVS_WAIT_UNTIL([test 1 = $(grep -c "Not claiming lport lsp0"
> hv2/ovn-controller.log)])
> -AT_CHECK([test x$(ovn-sbctl --bare --columns chassis find port_binding
> logical_port=lsp0) = x], [0], [])
> -
> -# (2) Chassis hv2 should not add flows in OFTABLE_PHY_TO_LOG and
> OFTABLE_LOG_TO_PHY tables.
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [1], [])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=65 | grep output],
> [1], [])
> -
> -# (3) Chassis hv1 should bind lsp0 when physical to logical mapping
> exists on hv1.
> -echo "verifying that hv1 binds lsp0 when hv1 physical/logical mapping is
> added"
> -as hv1
> -ovs-vsctl set interface hv1-vif0 external-ids:iface-id=lsp0
> -
> -OVS_WAIT_UNTIL([test 1 = $(grep -c "Claiming lport lsp0"
> hv1/ovn-controller.log)])
> -AT_CHECK([test x$(ovn-sbctl --bare --columns chassis find port_binding
> logical_port=lsp0) = x"$hv1_uuid"], [0], [])
> -
> -# (4) Chassis hv1 should add flows in OFTABLE_PHY_TO_LOG and
> OFTABLE_LOG_TO_PHY tables.
> -as hv1 ovs-ofctl dump-flows br-int
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [0], [ignore])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep
> actions=output:1], [0], [ignore])
> -
> -# (5) Chassis hv1 should release lsp0 binding and chassis hv2 should bind
> lsp0 when
> -# the requested chassis for lsp0 is changed from hv1 to hv2.
> -echo "verifying that lsp0 binding moves when requested-chassis is changed"
> -
> -ovn-nbctl lsp-set-options lsp0 requested-chassis=hv2
> -OVS_WAIT_UNTIL([test 1 = $(grep -c "Releasing lport lsp0 from this
> chassis" hv1/ovn-controller.log)])
> -OVS_WAIT_UNTIL([test x$(ovn-sbctl --bare --columns chassis find
> port_binding logical_port=lsp0) = x"$hv2_uuid"])
> -
> -# (6) Chassis hv2 should add flows and hv1 should not.
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [0], [ignore])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int table=65 | grep
> actions=output:1], [0], [ignore])
> -
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [1], [])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep output],
> [1], [])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- options:requested-chassis with hostname])
> -
> -ovn_start
> -
> -ovn-nbctl ls-add ls0
> -ovn-nbctl lsp-add ls0 lsp0
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.11
> -ovs-vsctl -- add-port br-int hv1-vif0 -- set Interface hv1-vif0
> ofport-request=1
> -
> -ovn-sbctl wait-until chassis hv1
> -hv1_hostname=$(ovn-sbctl --bare --columns hostname find Chassis name=hv1)
> -echo "hv1_hostname=${hv1_hostname}"
> -ovn-nbctl --wait=hv --timeout=3 lsp-set-options lsp0
> requested-chassis=${hv1_hostname}
> -as hv1 ovs-vsctl set interface hv1-vif0 external-ids:iface-id=lsp0
> -
> -hv1_uuid=$(ovn-sbctl --bare --columns _uuid find Chassis name=hv1)
> -echo "hv1_uuid=${hv1_uuid}"
> -OVS_WAIT_UNTIL([test 1 = $(grep -c "Claiming lport lsp0"
> hv1/ovn-controller.log)])
> -AT_CHECK([test x$(ovn-sbctl --bare --columns chassis find port_binding
> logical_port=lsp0) = x"$hv1_uuid"], [0], [])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [0], [ignore])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep
> actions=output:1], [0], [ignore])
> -
> -ovn-nbctl --wait=hv --timeout=3 lsp-set-options lsp0
> requested-chassis=non-existant-chassis
> -OVS_WAIT_UNTIL([test 1 = $(grep -c "Releasing lport lsp0 from this
> chassis" hv1/ovn-controller.log)])
> -ovn-nbctl --wait=hv --timeout=3 sync
> -AT_CHECK([test x$(ovn-sbctl --bare --columns chassis find port_binding
> logical_port=lsp0) = x], [0], [])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=0 | grep in_port=1],
> [1], [])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int table=65 | grep output],
> [1], [])
> -
> -OVN_CLEANUP([hv1])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IPv6 periodic RA])
> -ovn_start
> -
> -# This test sets up two hypervisors.
> -# hv1 and hv2 run ovn-controllers, and
> -# each has a VIF connected to the same
> -# logical switch in OVN. The logical
> -# switch is connected to a logical
> -# router port that is configured to send
> -# periodic router advertisements.
> -#
> -# The reason for having two ovn-controller
> -# hypervisors is to ensure that the
> -# periodic RAs being sent by each ovn-controller
> -# are kept to their local hypervisors. If the
> -# packets are not kept local, then each port
> -# will receive too many RAs.
> -
> -net_add n1
> -sim_add hv1
> -sim_add hv2
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -
> -ovn-nbctl lr-add ro
> -ovn-nbctl lrp-add ro ro-sw 00:00:00:00:00:01 aef0:0:0:0:0:0:0:1/64
> -
> -ovn-nbctl ls-add sw
> -ovn-nbctl lsp-add sw sw-ro
> -ovn-nbctl lsp-set-type sw-ro router
> -ovn-nbctl lsp-set-options sw-ro router-port=ro-sw
> -ovn-nbctl lsp-set-addresses sw-ro 00:00:00:00:00:01
> -ovn-nbctl lsp-add sw sw-p1
> -ovn-nbctl lsp-set-addresses sw-p1 "00:00:00:00:00:02 aef0::200:ff:fe00:2"
> -ovn-nbctl lsp-add sw sw-p2
> -ovn-nbctl lsp-set-addresses sw-p2 "00:00:00:00:00:03 aef0::200:ff:fe00:3"
> -
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:send_periodic=true
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:address_mode=slaac
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:max_interval=4
> -ovn-nbctl set Logical_Router_Port ro-sw ipv6_ra_configs:min_interval=3
> -
> -for i in 1 2 ; do
> -    as hv$i
> -    ovs-vsctl -- add-port br-int hv$i-vif1 -- \
> -        set interface hv$i-vif1 external-ids:iface-id=sw-p$i \
> -        options:tx_pcap=hv$i/vif1-tx.pcap \
> -        options:rxq_pcap=hv$i/vif1-rx.pcap \
> -        ofport-request=1
> -done
> -
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw-p1` = xup])
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up sw-p2` = xup])
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -
> -}
> -
> -construct_expected_ra() {
> -    local src_mac=000000000001
> -    local dst_mac=333300000001
> -    local src_addr=fe80000000000000020000fffe000001
> -    local dst_addr=ff020000000000000000000000000001
> -
> -    local mtu=$1
> -    local ra_mo=$2
> -    local ra_prefix_la=$3
> -
> -    local slla=0101${src_mac}
> -    local mtu_opt=""
> -    if test $mtu != 0; then
> -        mtu_opt=05010000${mtu}
> -    fi
> -    shift 3
> -
> -    local prefix=""
> -    while [[ $# -gt 0 ]] ; do
> -        local size=$1
> -        local net=$2
> -
> prefix=${prefix}0304${size}${ra_prefix_la}ffffffffffffffff00000000${net}
> -        shift 2
> -    done
> -
> -    local ra=ff${ra_mo}ffff0000000000000000${slla}${mtu_opt}${prefix}
> -    local icmp=8600XXXX${ra}
> -
> -    local ip_len=$(expr ${#icmp} / 2)
> -    ip_len=$(echo "$ip_len" | awk '{printf "%0.4x\n", $0}')
> -
> -    local ip=60000000${ip_len}3aff${src_addr}${dst_addr}${icmp}
> -    local eth=${dst_mac}${src_mac}86dd${ip}
> -    local packet=${eth}
> -    echo $packet >> expected
> -}
> -
> -ra_test() {
> -    construct_expected_ra $@
> -
> -    for i in hv1 hv2 ; do
> -        OVS_WAIT_WHILE([test 24 = $(wc -c $i/vif1-tx.pcap | cut -d " "
> -f1)])
> -
> -        $PYTHON "$top_srcdir/utilities/ovs-pcap.in" $i/vif1-tx.pcap >
> packets
> -
> -        cat expected | cut -c -112 > expout
> -        AT_CHECK([cat packets | cut -c -112], [0], [expout])
> -
> -        # Skip ICMPv6 checksum.
> -        cat expected | cut -c 117- > expout
> -        AT_CHECK([cat packets | cut -c 117-], [0], [expout])
> -
> -        rm -f packets
> -        as $i reset_pcap_file $i-vif1 $i/vif1
> -    done
> -
> -    rm -f expected
> -}
> -
> -# Baseline test with no MTU
> -ra_test 0 00 c0 40 aef00000000000000000000000000000
> -
> -# Now make sure an MTU option makes it
> -ovn-nbctl --wait=hv set Logical_Router_Port ro-sw ipv6_ra_configs:mtu=1500
> -ra_test 000005dc 00 c0 40 aef00000000000000000000000000000
> -
> -# Now test for multiple network prefixes
> -ovn-nbctl --wait=hv set Logical_Router_port ro-sw networks='aef0\:\:1/64
> fd0f\:\:1/48'
> -ra_test 000005dc 00 c0 40 aef00000000000000000000000000000 30
> fd0f0000000000000000000000000000
> -
> -# Test a different address mode now
> -ovn-nbctl --wait=hv set Logical_Router_Port ro-sw
> ipv6_ra_configs:address_mode=dhcpv6_stateful
> -ra_test 000005dc 80 80 40 aef00000000000000000000000000000 30
> fd0f0000000000000000000000000000
> -
> -# And the other address mode
> -ovn-nbctl --wait=hv set Logical_Router_Port ro-sw
> ipv6_ra_configs:address_mode=dhcpv6_stateless
> -ra_test 000005dc 40 c0 40 aef00000000000000000000000000000 30
> fd0f0000000000000000000000000000
> -
> -OVN_CLEANUP([hv1],[hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ACL reject rule test])
> -AT_KEYWORDS([acl-reject])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# test_ip_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_DST IP_CHKSUM
> EXP_IP_CHKSUM EXP_ICMP_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv4 packet with
> -# ETH_SRC, ETH_DST, IPV4_SRC, IPV4_DST, IP_CHKSUM as specified.
> -# EXP_IP_CHKSUM and EXP_ICMP_CHKSUM are the ip and icmp checksums of the
> icmp destination
> -# unreachable frame generated from ACL rule hit
> -#
> -# INPORT is a lport number, e.g. 11 for vif11.
> -# HV is a hypervisor number
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC and IPV4_DST are each 8 hex digits.
> -# IP_CHKSUM, EXP_IP_CHSUM and EXP_ICMP_CHKSUM are each 4 hex digits
> -test_ip_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ipv4_dst=$6
> ip_chksum=$7
> -    local exp_ip_chksum=$8 exp_icmp_chksum=$9
> -    shift 9
> -
> -    local ip_ttl=ff
> -    local
> packet=${eth_dst}${eth_src}08004500001400004000${ip_ttl}01${ip_chksum}${ipv4_src}${ipv4_dst}
> -
> -    local reply_icmp_ttl=ff
> -    local icmp_type_code_response=0301
> -    local icmp_data=00000000
> -    local
> reply_icmp_payload=${icmp_type_code_response}${exp_icmp_chksum}${icmp_data}
> -    local
> reply=${eth_src}${eth_dst}08004500001c00004000${reply_icmp_ttl}01${exp_ip_chksum}${ipv4_dst}${ipv4_src}${reply_icmp_payload}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_ipv6_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_DST
> EXP_ICMP_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv6 packet with
> -# ETH_SRC, ETH_DST, IPV6_SRC, IPV6_DST as specified.
> -# EXP_ICMP_CHKSUM is the icmp6 checksums of the icmp6 destination
> unreachable frame generated from ACL rule hit
> -test_ipv6_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv6_src=$5 ipv6_dst=$6
> exp_icmp_chksum=$7
> -    shift 7
> -
> -    local ip6_hdr=6000000000083aff${ipv6_src}${ipv6_dst}
> -    local packet=${eth_dst}${eth_src}86dd${ip6_hdr}0000000000000000
> -
> -    local
> reply=${eth_src}${eth_dst}86dd6000000000303aff${ipv6_dst}${ipv6_src}0101${exp_icmp_chksum}00000000${ip6_hdr}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_tcp_syn_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_DST
> IP_CHKSUM TCP_SPORT TCP_DPORT TCP_CHKSUM EXP_IP_CHKSUM EXP_TCP_RST_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an TCP syn segment with
> -# ETH_SRC, ETH_DST, IPV4_SRC, IPV4_DST, IP_CHKSUM, TCP_SPORT, TCP_DPORT,
> TCP_CHKSUM  as specified.
> -# EXP_IP_CHKSUM and EXP_TCP_RST_CHKSUM are the ip and tcp checksums of
> the tcp reset segment generated from ACL rule hit
> -#
> -# INPORT is an lport number, e.g. 11 for vif11.
> -# HV is an hypervisor number
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC and IPV4_DST are each 8 hex digits.
> -# TCP_SPORT and TCP_DPORT are 4 hex digits.
> -# IP_CHKSUM, TCP_CHKSUM, EXP_IP_CHSUM and EXP_TCP_RST_CHKSUM are each 4
> hex digits
> -test_tcp_syn_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ipv4_dst=$6
> ip_chksum=$7
> -    local tcp_sport=$8 tcp_dport=$9 tcp_chksum=${10}
> -    local exp_ip_chksum=${11} exp_tcp_rst_chksum=${12}
> -    shift 12
> -
> -    local ip_ttl=ff
> -    local
> packet=${eth_dst}${eth_src}08004500002800004000${ip_ttl}06${ip_chksum}${ipv4_src}${ipv4_dst}${tcp_sport}${tcp_dport}000000010000000050027210${tcp_chksum}0000
> -
> -    local tcp_rst_ttl=ff
> -    local
> reply=${eth_src}${eth_dst}08004500002800004000${tcp_rst_ttl}06${exp_ip_chksum}${ipv4_dst}${ipv4_src}${tcp_dport}${tcp_sport}000000000000000150040000${exp_tcp_rst_chksum}0000
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# Create hypervisors hv[123].
> -# Add vif1[123] to hv1, vif2[123] to hv2, vif3[123] to hv3.
> -# Add all of the vifs to a single logical switch sw0.
> -
> -net_add n1
> -ovn-nbctl ls-add sw0
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2 3; do
> -        ovn-nbctl lsp-add sw0 sw0-p$i$j -- \
> -                lsp-set-addresses sw0-p$i$j "00:00:00:00:00:$i$j
> 192.168.1.$i$j"
> -
> -        ovs-vsctl -- add-port br-int vif$i$j -- \
> -                set interface vif$i$j \
> -                external-ids:iface-id=sw0-p$i$j \
> -                options:tx_pcap=hv$i/vif$i$j-tx.pcap \
> -                options:rxq_pcap=hv$i/vif$i$j-rx.pcap \
> -                ofport-request=$i$j
> -    done
> -done
> -
> -OVN_POPULATE_ARP
> -# allow some time for ovn-northd and ovn-controller to catch up.
> -sleep 1
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -for i in 1 2 3; do
> -    : > vif${i}1.expected
> -done
> -
> -ovn-nbctl --log acl-add sw0 to-lport 1000 "outport == \"sw0-p12\"" reject
> -ovn-nbctl --log acl-add sw0 from-lport 1000 "inport == \"sw0-p11\"" reject
> -ovn-nbctl --log acl-add sw0 from-lport 1000 "inport == \"sw0-p21\"" reject
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -ovn-nbctl --timeout=3 --wait=hv sync
> -
> -test_ip_packet 11 1 000000000011 000000000021 $(ip_to_hex 192 168 1 11)
> $(ip_to_hex 192 168 1 21) 0000 7d8d fcfe
> -test_ip_packet 21 2 000000000021 000000000011 $(ip_to_hex 192 168 1 21)
> $(ip_to_hex 192 168 1 11) 0000 7d8d fcfe
> -test_ip_packet 31 3 000000000031 000000000012 $(ip_to_hex 192 168 1 31)
> $(ip_to_hex 192 168 1 12) 0000 7d82 fcfe
> -
> -test_ipv6_packet 11 1 000000000011 000000000021
> fe80000000000000020001fffe000001 fe80000000000000020001fffe000002 6183
> -
> -test_tcp_syn_packet 11 1 000000000011 000000000021 $(ip_to_hex 192 168 1
> 11) $(ip_to_hex 192 168 1 21) 0000 8b40 3039 0000 7d8d 4486
> -test_tcp_syn_packet 21 2 000000000021 000000000011 $(ip_to_hex 192 168 1
> 21) $(ip_to_hex 192 168 1 11) 0000 8b40 3039 0000 7d8d 4486
> -test_tcp_syn_packet 31 3 000000000031 000000000012 $(ip_to_hex 192 168 1
> 31) $(ip_to_hex 192 168 1 12) 0000 8b40 3039 0000 7d82 4486
> -
> -for i in 1 2 3; do
> -    OVN_CHECK_PACKETS([hv$i/vif${i}1-tx.pcap], [vif${i}1.expected])
> -done
> -
> -OVN_CLEANUP([hv1], [hv2], [hv3])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- Port Groups])
> -AT_KEYWORDS([ovnpg])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -#
> -# Three logical switches ls1, ls2, ls3.
> -# One logical router lr0 connected to ls[123],
> -# with nine subnets, three per logical switch:
> -#
> -#    lrp11 on ls1 for subnet 192.168.11.0/24
> -#    lrp12 on ls1 for subnet 192.168.12.0/24
> -#    lrp13 on ls1 for subnet 192.168.13.0/24
> -#    ...
> -#    lrp33 on ls3 for subnet 192.168.33.0/24
> -#
> -# 27 VIFs, 9 per LS, 3 per subnet: lp[123][123][123], where the first two
> -# digits are the subnet and the last digit distinguishes the VIF.
> -#
> -# This test will create two port groups and uses them in ACL.
> -
> -get_lsp_uuid () {
> -    ovn-nbctl lsp-list ls${1%??} | grep lp$1 | awk '{ print $1 }'
> -}
> -
> -pg1_ports=
> -pg2_ports=
> -for i in 1 2 3; do
> -    ovn-nbctl ls-add ls$i
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            ovn-nbctl \
> -                -- lsp-add ls$i lp$i$j$k \
> -                -- lsp-set-addresses lp$i$j$k \
> -                   "f0:00:00:00:0$i:$j$k 192.168.$i$j.$k"
> -            # logical ports lp[12]?1 belongs to port group pg1
> -            if test $i != 3 && test $k == 1; then
> -                pg1_ports="$pg1_ports `get_lsp_uuid $i$j$k`"
> -            fi
> -            # logical ports lp[23]?2 belongs to port group pg2
> -            if test $i != 1 && test $k == 2; then
> -                pg2_ports="$pg2_ports `get_lsp_uuid $i$j$k`"
> -            fi
> -        done
> -    done
> -done
> -
> -ovn-nbctl lr-add lr0
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        ovn-nbctl lrp-add lr0 lrp$i$j 00:00:00:00:ff:$i$j
> 192.168.$i$j.254/24
> -        ovn-nbctl \
> -            -- lsp-add ls$i lrp$i$j-attachment \
> -            -- set Logical_Switch_Port lrp$i$j-attachment type=router \
> -                             options:router-port=lrp$i$j \
> -                             addresses='"00:00:00:00:ff:'$i$j'"'
> -    done
> -done
> -
> -ovn-nbctl create Port_Group name=pg1 ports="$pg1_ports"
> -ovn-nbctl create Port_Group name=pg2 ports="$pg2_ports"
> -
> -# create ACLs on all lswitches to drop traffic from pg2 to pg1
> -ovn-nbctl acl-add ls1 to-lport 1001 'outport == @pg1 && ip4.src ==
> $pg2_ip4' drop
> -ovn-nbctl acl-add ls2 to-lport 1001 'outport == @pg1 && ip4.src ==
> $pg2_ip4' drop
> -ovn-nbctl acl-add ls3 to-lport 1001 'outport == @pg1 && ip4.src ==
> $pg2_ip4' drop
> -
> -# Physical network:
> -#
> -# Three hypervisors hv[123].
> -# lp?1[123] spread across hv[123]: lp?11 on hv1, lp?12 on hv2, lp?13 on
> hv3.
> -# lp?2[123] spread across hv[23]: lp?21 and lp?22 on hv2, lp?23 on hv3.
> -# lp?3[123] all on hv3.
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    case $1 in dnl (
> -        ?11) echo 1 ;; dnl (
> -        ?12 | ?21 | ?22) echo 2 ;; dnl (
> -        ?13 | ?23 | ?3?) echo 3 ;;
> -    esac
> -}
> -
> -# Given the name of a logical port, prints the name of its logical router
> -# port, e.g. "vif_to_lrp 123" yields 12.
> -vif_to_lrp() {
> -    echo ${1%?}
> -}
> -
> -# Given the name of a logical port, prints the name of its logical
> -# switch, e.g. "vif_to_ls 123" yields 1.
> -vif_to_ls() {
> -    echo ${1%??}
> -}
> -
> -net_add n1
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -done
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            hv=`vif_to_hv $i$j$k`
> -                as hv$hv ovs-vsctl \
> -                -- add-port br-int vif$i$j$k \
> -                -- set Interface vif$i$j$k \
> -                    external-ids:iface-id=lp$i$j$k \
> -                    options:tx_pcap=hv$hv/vif$i$j$k-tx.pcap \
> -                    options:rxq_pcap=hv$hv/vif$i$j$k-rx.pcap \
> -                    ofport-request=$i$j$k
> -        done
> -    done
> -done
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -#
> -# This shell function causes a packet to be received on INPORT.  The
> packet's
> -# content has Ethernet destination DST and source SRC (each exactly 12 hex
> -# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> -# more) list the VIFs on which the packet should be received.  INPORT and
> the
> -# OUTPORTs are specified as logical switch port numbers, e.g. 123 for
> vif123.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            : > $i$j$k.expected
> -        done
> -    done
> -done
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    in_ls=`vif_to_ls $inport`
> -    in_lrp=`vif_to_lrp $inport`
> -    for outport; do
> -        out_ls=`vif_to_ls $outport`
> -        if test $in_ls = $out_ls; then
> -            # Ports on the same logical switch receive exactly the same
> packet.
> -            echo $packet
> -        else
> -            # Routing decrements TTL and updates source and dest MAC
> -            # (and checksum).
> -            out_lrp=`vif_to_lrp $outport`
> -            echo
> f00000000${outport}00000000ff${out_lrp}08004500001c00000000"3f1101"00${src_ip}${dst_ip}0035111100080000
> -        fi >> $outport.expected
> -    done
> -}
> -
> -as hv1 ovs-vsctl --columns=name,ofport list interface
> -as hv1 ovn-sbctl list port_binding
> -as hv1 ovn-sbctl list datapath_binding
> -as hv1 ovn-sbctl list port_group
> -as hv1 ovn-sbctl list address_set
> -as hv1 ovn-sbctl dump-flows
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Send IP packets between all pairs of source and destination ports,
> -# packets matches ACL (pg2 to pg1) should be dropped
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -for is in 1 2 3; do
> -  for js in 1 2 3; do
> -    for ks in 1 2 3; do
> -      bcast=
> -      s=$is$js$ks
> -      smac=f00000000$s
> -      sip=`ip_to_hex 192 168 $is$js $ks`
> -      for id in 1 2 3; do
> -          for jd in 1 2 3; do
> -              for kd in 1 2 3; do
> -                d=$id$jd$kd
> -                dip=`ip_to_hex 192 168 $id$jd $kd`
> -                if test $is = $id; then dmac=f00000000$d; else
> dmac=00000000ff$is$js; fi
> -                if test $d != $s; then unicast=$d; else unicast=; fi
> -
> -                # packets matches ACL should be dropped
> -                if test $id != 3 && test $kd == 1; then
> -                    if test $is != 1 && test $ks == 2; then
> -                        unicast=
> -                    fi
> -                fi
> -                test_ip $s $smac $dmac $sip $dip $unicast #1
> -              done
> -          done
> -        done
> -      done
> -  done
> -done
> -
> -# Allow some time for packet forwarding.
> -# XXX This can be improved.
> -sleep 1
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            OVN_CHECK_PACKETS([hv`vif_to_hv $i$j$k`/vif$i$j$k-tx.pcap],
> -                              [$i$j$k.expected])
> -        done
> -    done
> -done
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1], [hv2], [hv3])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ACLs on Port Groups])
> -AT_KEYWORDS([ovnpg_acl])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -#
> -# Three logical switches ls1, ls2, ls3.
> -# One logical router lr0 connected to ls[123],
> -# with nine subnets, three per logical switch:
> -#
> -#    lrp11 on ls1 for subnet 192.168.11.0/24
> -#    lrp12 on ls1 for subnet 192.168.12.0/24
> -#    lrp13 on ls1 for subnet 192.168.13.0/24
> -#    ...
> -#    lrp33 on ls3 for subnet 192.168.33.0/24
> -#
> -# 27 VIFs, 9 per LS, 3 per subnet: lp[123][123][123], where the first two
> -# digits are the subnet and the last digit distinguishes the VIF.
> -#
> -# This test will create two port groups and ACLs will be applied on them.
> -
> -get_lsp_uuid () {
> -    ovn-nbctl lsp-list ls${1%??} | grep lp$1 | awk '{ print $1 }'
> -}
> -
> -pg1_ports=
> -pg2_ports=
> -for i in 1 2 3; do
> -    ovn-nbctl ls-add ls$i
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            ovn-nbctl \
> -                -- lsp-add ls$i lp$i$j$k \
> -                -- lsp-set-addresses lp$i$j$k \
> -                   "f0:00:00:00:0$i:$j$k 192.168.$i$j.$k"
> -            # logical ports lp[12]?1 belongs to port group pg1
> -            if test $i != 3 && test $k == 1; then
> -                pg1_ports="$pg1_ports `get_lsp_uuid $i$j$k`"
> -            fi
> -            # logical ports lp[23]?2 belongs to port group pg2
> -            if test $i != 1 && test $k == 2; then
> -                pg2_ports="$pg2_ports `get_lsp_uuid $i$j$k`"
> -            fi
> -        done
> -    done
> -done
> -
> -ovn-nbctl lr-add lr0
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        ovn-nbctl lrp-add lr0 lrp$i$j 00:00:00:00:ff:$i$j
> 192.168.$i$j.254/24
> -        ovn-nbctl \
> -            -- lsp-add ls$i lrp$i$j-attachment \
> -            -- set Logical_Switch_Port lrp$i$j-attachment type=router \
> -                             options:router-port=lrp$i$j \
> -                             addresses='"00:00:00:00:ff:'$i$j'"'
> -    done
> -done
> -
> -ovn-nbctl create Port_Group name=pg1 ports="$pg1_ports"
> -ovn-nbctl create Port_Group name=pg2 ports="$pg2_ports"
> -
> -# create ACLs on pg1 to drop traffic from pg2 to pg1
> -ovn-nbctl acl-add pg1 to-lport 1001 'outport == @pg1' drop
> -ovn-nbctl --type=port-group acl-add pg1 to-lport 1002 \
> -        'outport == @pg1 && ip4.src == $pg2_ip4' allow-related
> -
> -# Physical network:
> -#
> -# Three hypervisors hv[123].
> -# lp?1[123] spread across hv[123]: lp?11 on hv1, lp?12 on hv2, lp?13 on
> hv3.
> -# lp?2[123] spread across hv[23]: lp?21 and lp?22 on hv2, lp?23 on hv3.
> -# lp?3[123] all on hv3.
> -
> -# Given the name of a logical port, prints the name of the hypervisor
> -# on which it is located.
> -vif_to_hv() {
> -    case $1 in dnl (
> -        ?11) echo 1 ;; dnl (
> -        ?12 | ?21 | ?22) echo 2 ;; dnl (
> -        ?13 | ?23 | ?3?) echo 3 ;;
> -    esac
> -}
> -
> -# Given the name of a logical port, prints the name of its logical router
> -# port, e.g. "vif_to_lrp 123" yields 12.
> -vif_to_lrp() {
> -    echo ${1%?}
> -}
> -
> -# Given the name of a logical port, prints the name of its logical
> -# switch, e.g. "vif_to_ls 123" yields 1.
> -vif_to_ls() {
> -    echo ${1%??}
> -}
> -
> -net_add n1
> -for i in 1 2 3; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -done
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            hv=`vif_to_hv $i$j$k`
> -                as hv$hv ovs-vsctl \
> -                -- add-port br-int vif$i$j$k \
> -                -- set Interface vif$i$j$k \
> -                    external-ids:iface-id=lp$i$j$k \
> -                    options:tx_pcap=hv$hv/vif$i$j$k-tx.pcap \
> -                    options:rxq_pcap=hv$hv/vif$i$j$k-rx.pcap \
> -                    ofport-request=$i$j$k
> -        done
> -    done
> -done
> -
> -# Pre-populate the hypervisors' ARP tables so that we don't lose any
> -# packets for ARP resolution (native tunneling doesn't queue packets
> -# for ARP resolution).
> -OVN_POPULATE_ARP
> -
> -# Allow some time for ovn-northd and ovn-controller to catch up.
> -# XXX This should be more systematic.
> -sleep 1
> -
> -lsp_to_mac() {
> -    echo f0:00:00:00:0${1:0:1}:${1:1:2}
> -}
> -
> -lrp_to_mac() {
> -    echo 00:00:00:00:ff:$1
> -}
> -
> -# test_icmp INPORT SRC_MAC DST_MAC SRC_IP DST_IP ICMP_TYPE OUTPORT...
> -#
> -# This shell function causes a ICMP packet to be received on INPORT.
> -# The OUTPORTs (zero or more) list the VIFs on which the packet should
> -# be received.  INPORT and the OUTPORTs are specified as logical switch
> -# port numbers, e.g. 123 for vif123.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            : > $i$j$k.expected
> -        done
> -    done
> -done
> -
> -test_icmp() {
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5 icmp_type=$6
> -    local packet="inport==\"lp$inport\" && eth.src==$src_mac &&
> -                  eth.dst==$dst_mac && ip.ttl==64 && ip4.src==$src_ip
> -                  && ip4.dst==$dst_ip && icmp4.type==$icmp_type &&
> -                  icmp4.code==0"
> -    shift; shift; shift; shift; shift; shift
> -    hv=hv`vif_to_hv $inport`
> -    as $hv ovs-appctl -t ovn-controller inject-pkt "$packet"
> -    in_ls=`vif_to_ls $inport`
> -    in_lrp=`vif_to_lrp $inport`
> -    for outport; do
> -        out_ls=`vif_to_ls $outport`
> -        if test $in_ls = $out_ls; then
> -            # Ports on the same logical switch receive exactly the same
> packet.
> -            echo $packet | ovstest test-ovn expr-to-packets
> -        else
> -            # Routing decrements TTL and updates source and dest MAC
> -            # (and checksum).
> -            out_lrp=`vif_to_lrp $outport`
> -            exp_smac=`lrp_to_mac $out_lrp`
> -            exp_dmac=`lsp_to_mac $outport`
> -            exp_packet="eth.src==$exp_smac && eth.dst==$exp_dmac &&
> -                ip.ttl==63 && ip4.src==$src_ip && ip4.dst==$dst_ip &&
> -                icmp4.type==$icmp_type && icmp4.code==0"
> -            echo $exp_packet | ovstest test-ovn expr-to-packets
> -
> -        fi >> $outport.expected
> -    done
> -}
> -
> -as hv1 ovs-vsctl --columns=name,ofport list interface
> -as hv1 ovn-sbctl list port_binding
> -as hv1 ovn-sbctl list datapath_binding
> -as hv1 ovn-sbctl list port_group
> -as hv1 ovn-sbctl list address_set
> -as hv1 ovn-sbctl dump-flows
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Send IP packets between all pairs of source and destination ports,
> -# packets matches ACL1 but not ACL2 should be dropped
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -for is in 1 2 3; do
> -  for js in 1 2 3; do
> -    for ks in 1 2 3; do
> -      bcast=
> -      s=$is$js$ks
> -      slsp_mac=`lsp_to_mac $s`
> -      slrp_mac=`lrp_to_mac $is$js`
> -      sip=192.168.$is$js.$ks
> -      for id in 1 2 3; do
> -          for jd in 1 2 3; do
> -              for kd in 1 2 3; do
> -                d=$id$jd$kd
> -                dlsp_mac=`lsp_to_mac $d`
> -                dlrp_mac=`lrp_to_mac $id$jd`
> -                dip=192.168.$id$jd.$kd
> -                if test $is = $id; then dmac=$dlsp_mac; else
> dmac=$slrp_mac; fi
> -                if test $d != $s; then unicast=$d; else unicast=; fi
> -
> -                # packets matches ACL1 but not ACL2 should be dropped
> -                if test $id != 3 && test $kd == 1; then
> -                    if test $is == 1 || test $ks != 2; then
> -                        unicast=
> -                    fi
> -                fi
> -                # icmp request (type = 8)
> -                test_icmp $s $slsp_mac $dmac $sip $dip 8 $unicast
> -
> -                # if packets are not dropped, test the return traffic
> (icmp echo)
> -                # to make sure stateful works, too.
> -                if test x$unicast != x; then
> -                    if test $is = $id; then dmac=$slsp_mac; else
> dmac=$dlrp_mac; fi
> -                    # icmp echo (type = 0)
> -                    test_icmp $unicast $dlsp_mac $dmac $dip $sip 0 $s
> -                fi
> -              done
> -          done
> -        done
> -      done
> -  done
> -done
> -
> -# Allow some time for packet forwarding.
> -# XXX This can be improved.
> -sleep 1
> -
> -# Now check the packets actually received against the ones expected.
> -for i in 1 2 3; do
> -    for j in 1 2 3; do
> -        for k in 1 2 3; do
> -            OVN_CHECK_PACKETS([hv`vif_to_hv $i$j$k`/vif$i$j$k-tx.pcap],
> -                              [$i$j$k.expected])
> -        done
> -    done
> -done
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1], [hv2], [hv3])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- Address Set generation from Port Groups (static
> addressing)])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -ovn-nbctl lsp-add ls1 lp1
> -ovn-nbctl lsp-add ls1 lp2
> -ovn-nbctl lsp-add ls1 lp3
> -
> -ovn-nbctl lsp-set-addresses lp1 "02:00:00:00:00:01 10.0.0.1 2001:db8::1"
> -ovn-nbctl lsp-set-addresses lp2 "02:00:00:00:00:02 10.0.0.2 2001:db8::2"
> -ovn-nbctl lsp-set-addresses lp3 "02:00:00:00:00:03 10.0.0.3 2001:db8::3"
> -
> -ovn-nbctl create Port_Group name=pg1
> -ovn-nbctl create Port_Group name=pg2
> -
> -ovn-nbctl --id=@p get Logical_Switch_Port lp1 -- add Port_Group pg1 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp2 -- add Port_Group pg1 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp2 -- add Port_Group pg2 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp3 -- add Port_Group pg2 ports
> @p
> -
> -ovn-nbctl --wait=sb sync
> -
> -dnl Check if port group address sets were populated with ports' addresses
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip4 addresses],
> -         [0], [[["10.0.0.1", "10.0.0.2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg2_ip4 addresses],
> -         [0], [[["10.0.0.2", "10.0.0.3"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip6 addresses],
> -         [0], [[["2001:db8::1", "2001:db8::2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg2_ip6 addresses],
> -         [0], [[["2001:db8::2", "2001:db8::3"]]
> -])
> -
> -ovn-nbctl --wait=sb lsp-set-addresses lp1 \
> -    "02:00:00:00:00:01 10.0.0.11 2001:db8::11"
> -
> -dnl Check if updated address got propagated to the port group address sets
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip4 addresses],
> -         [0], [[["10.0.0.11", "10.0.0.2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip6 addresses],
> -         [0], [[["2001:db8::11", "2001:db8::2"]]
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- Address Set generation from Port Groups (dynamic
> addressing)])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl ls-add ls2
> -ovn-nbctl ls-add ls3
> -
> -ovn-nbctl set Logical_Switch ls1 \
> -    other_config:subnet=10.1.0.0/24
> other_config:ipv6_prefix="2001:db8:1::"
> -ovn-nbctl set Logical_Switch ls2 \
> -    other_config:subnet=10.2.0.0/24
> other_config:ipv6_prefix="2001:db8:2::"
> -ovn-nbctl set Logical_Switch ls3 \
> -    other_config:subnet=10.3.0.0/24
> other_config:ipv6_prefix="2001:db8:3::"
> -
> -ovn-nbctl lsp-add ls1 lp1
> -ovn-nbctl lsp-add ls2 lp2
> -ovn-nbctl lsp-add ls3 lp3
> -
> -ovn-nbctl lsp-set-addresses lp1 "02:00:00:00:00:01 dynamic"
> -ovn-nbctl lsp-set-addresses lp2 "02:00:00:00:00:02 dynamic"
> -ovn-nbctl lsp-set-addresses lp3 "02:00:00:00:00:03 dynamic"
> -
> -ovn-nbctl create Port_Group name=pg1
> -ovn-nbctl create Port_Group name=pg2
> -
> -ovn-nbctl --id=@p get Logical_Switch_Port lp1 -- add Port_Group pg1 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp2 -- add Port_Group pg1 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp2 -- add Port_Group pg2 ports
> @p
> -ovn-nbctl --id=@p get Logical_Switch_Port lp3 -- add Port_Group pg2 ports
> @p
> -
> -ovn-nbctl --wait=sb sync
> -
> -dnl Check if port group address sets were populated with ports' addresses
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip4 addresses],
> -         [0], [[["10.1.0.2", "10.2.0.2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg2_ip4 addresses],
> -         [0], [[["10.2.0.2", "10.3.0.2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip6 addresses],
> -         [0], [[["2001:db8:1::ff:fe00:1", "2001:db8:2::ff:fe00:2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg2_ip6 addresses],
> -         [0], [[["2001:db8:2::ff:fe00:2", "2001:db8:3::ff:fe00:3"]]
> -])
> -
> -ovn-nbctl --wait=sb set Logical_Switch ls1 \
> -    other_config:subnet=10.11.0.0/24
> other_config:ipv6_prefix="2001:db8:11::"
> -
> -dnl Check if updated address got propagated to the port group address sets
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip4 addresses],
> -         [0], [[["10.11.0.2", "10.2.0.2"]]
> -])
> -AT_CHECK([ovn-sbctl get Address_Set pg1_ip6 addresses],
> -         [0], [[["2001:db8:11::ff:fe00:1", "2001:db8:2::ff:fe00:2"]]
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ACL conjunction])
> -ovn_start
> -
> -ovn-nbctl ls-add ls1
> -
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp1 "f0:00:00:00:00:01 10.0.0.4"
> -
> -ovn-nbctl lsp-add ls1 ls1-lp2 \
> --- lsp-set-addresses ls1-lp2 "f0:00:00:00:00:02 10.0.0.6"
> -
> -ovn-nbctl lsp-set-port-security ls1-lp2 "f0:00:00:00:00:02 10.0.0.6"
> -
> -net_add n1
> -sim_add hv1
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=ls1-lp2 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=2
> -
> -ovn-nbctl create Address_Set name=set1 \
> -addresses=\"10.0.0.4\",\"10.0.0.5\",\"10.0.0.6\"
> -ovn-nbctl create Address_Set name=set2 \
> -addresses=\"10.0.0.7\",\"10.0.0.8\",\"10.0.0.9\"
> -ovn-nbctl acl-add ls1 to-lport 1002 \
> -'ip4 && ip4.src == $set1 && ip4.dst == $set1' allow
> -ovn-nbctl acl-add ls1 to-lport 1001 \
> -'ip4 && ip4.src == $set1 && ip4.dst == $set2' drop
> -
> -# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> -#
> -# This shell function causes an ip packet to be received on INPORT.
> -# The packet's content has Ethernet destination DST and source SRC
> -# (each exactly 12 hex digits) and Ethernet type ETHTYPE (4 hex digits).
> -# The OUTPORTs (zero or more) list the VIFs on which the packet should
> -# be received.  INPORT and the OUTPORTs are specified as logical switch
> -# port numbers, e.g. 11 for vif11.
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}\
> -${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -    for outport; do
> -        echo $packet >> $outport.expected
> -    done
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -
> -sip=`ip_to_hex 10 0 0 4`
> -dip=`ip_to_hex 10 0 0 6`
> -
> -test_ip 1 f00000000001 f00000000002 $sip $dip 2
> -
> -cat 2.expected > expout
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -AT_CHECK([cat 2.packets], [0], [expout])
> -
> -# There should be total of 12 flows present with conjunction action and 2
> flows
> -# with conj match. Eg.
> -# table=44, priority=2002,conj_id=2,metadata=0x1 actions=resubmit(,45)
> -# table=44, priority=2001,conj_id=3,metadata=0x1 actions=drop
> -# priority=2002,ip,metadata=0x1,nw_dst=10.0.0.6 actions=conjunction(2,2/2)
> -# priority=2002,ip,metadata=0x1,nw_dst=10.0.0.4 actions=conjunction(2,2/2)
> -# priority=2002,ip,metadata=0x1,nw_dst=10.0.0.5 actions=conjunction(2,2/2)
> -# priority=2001,ip,metadata=0x1,nw_dst=10.0.0.7 actions=conjunction(3,2/2)
> -# priority=2001,ip,metadata=0x1,nw_dst=10.0.0.9 actions=conjunction(3,2/2)
> -# priority=2001,ip,metadata=0x1,nw_dst=10.0.0.8 actions=conjunction(3,2/2)
> -# priority=2002,ip,metadata=0x1,nw_src=10.0.0.6 actions=conjunction(2,1/2)
> -# priority=2002,ip,metadata=0x1,nw_src=10.0.0.4 actions=conjunction(2,1/2)
> -# priority=2002,ip,metadata=0x1,nw_src=10.0.0.5 actions=conjunction(2,1/2)
> -# priority=2001,ip,metadata=0x1,nw_src=10.0.0.6 actions=conjunction(3,1/2)
> -# priority=2001,ip,metadata=0x1,nw_src=10.0.0.4 actions=conjunction(3,1/2)
> -# priority=2001,ip,metadata=0x1,nw_src=10.0.0.5 actions=conjunction(3,1/2)
> -
> -OVS_WAIT_UNTIL([test 12 = `as hv1 ovs-ofctl dump-flows br-int | \
> -grep conjunction | wc -l`])
> -OVS_WAIT_UNTIL([test 2 = `as hv1 ovs-ofctl dump-flows br-int | \
> -grep conj_id | wc -l`])
> -
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -# Set the ip address for ls1-lp2 from set2 so that the drop ACL flow is
> hit.
> -ovn-nbctl lsp-set-addresses ls1-lp2 "f0:00:00:00:00:02 10.0.0.7 20.0.0.4"
> -ovn-nbctl lsp-set-port-security ls1-lp2 "f0:00:00:00:00:02 10.0.0.7
> 20.0.0.4"
> -
> -reset_pcap_file hv1-vif2 hv1/vif2
> -
> -rm -f 2.packets
> -
> -sip=`ip_to_hex 10 0 0 4`
> -dip=`ip_to_hex 10 0 0 7`
> -
> -test_ip 1 f00000000001 f00000000002 $sip $dip
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif2-tx.pcap > 2.packets
> -AT_CHECK([cat 2.packets], [0], [])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- TTL exceeded])
> -AT_KEYWORDS([ttl-exceeded])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# test_ip_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_DST IPV4_ROUTER
> IP_CHKSUM EXP_IP_CHKSUM EXP_ICMP_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv4 packet with
> -# ETH_SRC, ETH_DST, IPV4_SRC, IPV4_DST, IP_CHKSUM as specified and TTL
> set to 1.
> -# EXP_IP_CHKSUM and EXP_ICMP_CHKSUM are the ip and icmp checksums of the
> icmp time exceeded frame
> -# generated by OVN logical router
> -#
> -# INPORT is a lport number, e.g. 11 for vif11.
> -# HV is a hypervisor number
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC, IPV4_DST and IPV4_ROUTER are each 8 hex digits.
> -# IP_CHKSUM, EXP_IP_CHSUM and EXP_ICMP_CHKSUM are each 4 hex digits
> -test_ip_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ipv4_dst=$6
> ip_router=$7 ip_chksum=$8
> -    local exp_ip_chksum=$9 exp_icmp_chksum=${10}
> -    shift 10
> -
> -    local ip_ttl=01
> -    local
> packet=${eth_dst}${eth_src}08004500001400004000${ip_ttl}01${ip_chksum}${ipv4_src}${ipv4_dst}
> -
> -    local reply_icmp_ttl=fe
> -    local icmp_type_code_response=0b00
> -    local icmp_data=00000000
> -    local
> reply_icmp_payload=${icmp_type_code_response}${exp_icmp_chksum}${icmp_data}
> -    local
> reply=${eth_src}${eth_dst}08004500001c00004000${reply_icmp_ttl}01${exp_ip_chksum}${ip_router}${ipv4_src}${reply_icmp_payload}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_ip6_packet INPORT HV ETH_SRC ETH_DST IPV6_SRC IPV6_DST IPV6_ROUTER
> EXP_ICMP_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv6
> -# packet with ETH_SRC, ETH_DST, IPV6_SRC and IPV6_DST as specified.
> -# IPV6_ROUTER and EXP_ICMP_CHKSUM are the source IP and checksum of the
> icmpv6 ttl exceeded
> -# packet sent by OVN logical router
> -test_ip6_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv6_src=$5 ipv6_dst=$6
> ipv6_router=$7 exp_icmp_chksum=$8
> -    shift 8
> -
> -    local ip6_hdr=6000000000151101${ipv6_src}${ipv6_dst}
> -    local
> packet=${eth_dst}${eth_src}86dd${ip6_hdr}dbb8303900155bac6b646f65206676676e6d66720a
> -
> -    local
> reply=${eth_src}${eth_dst}86dd6000000000303afe${ipv6_router}${ipv6_src}0300${exp_icmp_chksum}00000000${ip6_hdr}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -for i in 1 2; do
> -    net_add n$i
> -    ovn-nbctl ls-add sw$i
> -
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n$i br-phys 192.168.$i.1
> -
> -    ovn-nbctl lsp-add sw$i sw$i-p${i}0 -- \
> -        lsp-set-addresses sw$i-p${i}0 "00:00:00:00:00:0$i 192.168.$i.1
> 2001:db8:$i::11"
> -
> -    ovs-vsctl -- add-port br-int vif$i -- \
> -        set interface vif$i \
> -        external-ids:iface-id=sw$i-p${i}0 \
> -            options:tx_pcap=hv$i/vif$i-tx.pcap \
> -            options:rxq_pcap=hv$i/vif$i-rx.pcap \
> -            ofport-request=$i
> -done
> -
> -ovn-nbctl lr-add lr0
> -for i in 1 2; do
> -    ovn-nbctl lrp-add lr0 lrp$i 00:00:00:00:ff:0$i 192.168.$i.254/24
> 2001:db8:$i::1/64
> -    ovn-nbctl -- lsp-add sw$i lrp$i-attachment \
> -              -- set Logical_Switch_Port lrp$i-attachment type=router \
> -                options:router-port=lrp$i
> addresses='"00:00:00:00:ff:0'$i' 192.168.'$i'.254 2001:db8:'$i'::1"'
> -done
> -
> -OVN_POPULATE_ARP
> -# allow some time for ovn-northd and ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -
> -test_ip_packet 1 1 000000000001 00000000ff01 $(ip_to_hex 192 168 1 1)
> $(ip_to_hex 192 168 2 1) $(ip_to_hex 192 168 1 254) 0000 7dae f4ff
> -test_ip6_packet 1 1 000000000001 00000000ff01
> 20010db8000100000000000000000011 20010db8000200000000000000000011
> 20010db8000100000000000000000001 d461
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [vif1.expected])
> -
> -OVN_CLEANUP([hv1], [hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- router port unreachable])
> -AT_KEYWORDS([router-port-unreachable])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# test_ip_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_ROUTER
> L4_PROTCOL IP_CHKSUM EXP_IP_CHKSUM EXP_ICMP_CHKSUM EXP_ICMP_CODE
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv4 packet with
> -# ETH_SRC, ETH_DST, IPV4_SRC, IPV4_ROUTER, L4_PROTCOL, IP_CHKSUM as
> specified.
> -# EXP_IP_CHKSUM and EXP_ICMP_CHKSUM are the ip and icmp checksums of the
> icmp frame generated by OVN logical router
> -# EXP_ICMP_CODE are code and type of the icmp frame generated by OVN
> logical router
> -#
> -# INPORT is a lport number, e.g. 11 for vif11.
> -# HV is a hypervisor number
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC and IPV4_ROUTER are each 8 hex digits.
> -# IP_CHKSUM, EXP_IP_CHSUM and EXP_ICMP_CHKSUM are each 4 hex digits
> -test_ip_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ip_router=$6
> l4_proto=$7 ip_chksum=$8
> -    local exp_ip_chksum=$9 exp_icmp_chksum=${10} exp_icmp_code=${11}
> -    shift 11
> -
> -    local ip_ttl=ff
> -    local
> packet=${eth_dst}${eth_src}08004500001400004000${ip_ttl}${l4_proto}${ip_chksum}${ipv4_src}${ip_router}
> -
> -    local reply_icmp_ttl=fe
> -    local icmp_data=00000000
> -    local
> reply_icmp_payload=${exp_icmp_code}${exp_icmp_chksum}${icmp_data}
> -    local
> reply=${eth_src}${eth_dst}08004500001c00004000${reply_icmp_ttl}01${exp_ip_chksum}${ip_router}${ipv4_src}${reply_icmp_payload}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_tcp_syn_packet INPORT HV ETH_SRC ETH_DST IPV4_SRC IPV4_ROUTER
> IP_CHKSUM TCP_SPORT TCP_DPORT TCP_CHKSUM EXP_IP_CHKSUM EXP_TCP_RST_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an TCP syn segment with
> -# ETH_SRC, ETH_DST, IPV4_SRC, IPV4_ROUTER, IP_CHKSUM, TCP_SPORT,
> TCP_DPORT, TCP_CHKSUM  as specified.
> -# EXP_IP_CHKSUM and EXP_TCP_RST_CHKSUM are the ip and tcp checksums of
> the tcp reset segment generated by OVN logical router
> -#
> -# INPORT is an lport number, e.g. 11 for vif11.
> -# HV is an hypervisor number
> -# ETH_SRC and ETH_DST are each 12 hex digits.
> -# IPV4_SRC and IPV4_ROUTER are each 8 hex digits.
> -# TCP_SPORT and TCP_DPORT are 4 hex digits.
> -# IP_CHKSUM, TCP_CHKSUM, EXP_IP_CHSUM and EXP_TCP_RST_CHKSUM are each 4
> hex digits
> -test_tcp_syn_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ip_router=$6
> ip_chksum=$7
> -    local tcp_sport=$8 tcp_dport=$9 tcp_chksum=${10}
> -    local exp_ip_chksum=${11} exp_tcp_rst_chksum=${12}
> -    shift 12
> -
> -    local ip_ttl=ff
> -    local
> packet=${eth_dst}${eth_src}08004500002800004000${ip_ttl}06${ip_chksum}${ipv4_src}${ip_router}${tcp_sport}${tcp_dport}000000010000000050027210${tcp_chksum}0000
> -
> -    local tcp_rst_ttl=fe
> -    local
> reply=${eth_src}${eth_dst}08004500002800004000${tcp_rst_ttl}06${exp_ip_chksum}${ip_router}${ipv4_src}${tcp_dport}${tcp_sport}000000000000000150040000${exp_tcp_rst_chksum}0000
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_tcp6_packet INPORT HV ETH_SRC ETH_DST IPV6_SRC IPV6_ROUTER
> TCP_SPORT TCP_DPORT TCP_CHKSUM EXP_TCP_RST_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is a TCP syn segment with
> -# ETH_SRC, ETH_DST, IPV6_SRC, IPV6_ROUTER, TCP_SPORT, TCP_DPORT and
> TCP_CHKSUM as specified.
> -# EXP_TCP_RST_CHKSUM is the tcp checksums of the tcp reset segment
> generated by OVN logical router
> -test_tcp6_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv6_src=$5 ipv6_router=$6
> -    local tcp_sport=$7 tcp_dport=$8 tcp_chksum=$9
> -    local exp_tcp_rst_chksum=${10}
> -    shift 10
> -
> -    local ip6_hdr=60000000001406ff${ipv6_src}${ipv6_router}
> -    local
> packet=${eth_dst}${eth_src}86dd${ip6_hdr}${tcp_sport}${tcp_dport}000000010000000050027210${tcp_chksum}0000
> -
> -    local
> reply=${eth_src}${eth_dst}86dd60000000001406fe${ipv6_router}${ipv6_src}${tcp_dport}${tcp_sport}000000000000000150040000${exp_tcp_rst_chksum}0000
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -# test_ip6_packet INPORT HV ETH_SRC ETH_DST IPV6_SRC IPV6_DST IPV6_PROTO
> IPV6_LEN DATA EXP_ICMP_CODE EXP_ICMP_CHKSUM
> -#
> -# Causes a packet to be received on INPORT of the hypervisor HV. The
> packet is an IPv6
> -# packet with ETH_SRC, ETH_DST, IPV6_SRC, IPV6_DST, IPV6_PROTO, IPV6_LEN
> and DATA as specified.
> -# EXP_ICMP_CODE and EXP_ICMP_CHKSUM are the code and checksum of the
> icmp6 packet sent by OVN logical router
> -test_ip6_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv6_src=$5 ipv6_dst=$6
> ipv6_proto=$7 ipv6_len=$8 data=$9
> -    local exp_icmp_code=${10} exp_icmp_chksum=${11}
> -    shift 11
> -
> -    local ip6_hdr=60000000${ipv6_len}${ipv6_proto}ff${ipv6_src}${ipv6_dst}
> -    local packet=${eth_dst}${eth_src}86dd${ip6_hdr}${data}
> -
> -    local
> reply=${eth_src}${eth_dst}86dd6000000000303afe${ipv6_dst}${ipv6_src}${exp_icmp_code}${exp_icmp_chksum}00000000${ip6_hdr}
> -    echo $reply >> vif$inport.expected
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive vif$inport $packet
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -for i in 1 2; do
> -    net_add n$i
> -    ovn-nbctl ls-add sw$i
> -
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n$i br-phys 192.168.$i.1
> -
> -    ovn-nbctl lsp-add sw$i sw$i-p${i}0 -- \
> -        lsp-set-addresses sw$i-p${i}0 "00:00:00:00:00:0$i 192.168.$i.1
> 2001:db8:$i::11"
> -
> -    ovs-vsctl -- add-port br-int vif$i -- \
> -        set interface vif$i \
> -        external-ids:iface-id=sw$i-p${i}0 \
> -            options:tx_pcap=hv$i/vif$i-tx.pcap \
> -            options:rxq_pcap=hv$i/vif$i-rx.pcap \
> -            ofport-request=$i
> -done
> -
> -ovn-nbctl lr-add lr0
> -for i in 1 2; do
> -    ovn-nbctl lrp-add lr0 lrp$i 00:00:00:00:ff:0$i 192.168.$i.254/24
> 2001:db8:$i::1/64
> -    ovn-nbctl -- lsp-add sw$i lrp$i-attachment \
> -              -- set Logical_Switch_Port lrp$i-attachment type=router \
> -                options:router-port=lrp$i
> addresses='"00:00:00:00:ff:0'$i' 192.168.'$i'.254 2001:db8:'$i'::1"'
> -done
> -
> -OVN_POPULATE_ARP
> -# allow some time for ovn-northd and ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -
> -test_ip_packet 1 1 000000000001 00000000ff01 $(ip_to_hex 192 168 1 1)
> $(ip_to_hex 192 168 1 254) 11 0000 7dae fcfc 0303
> -test_ip_packet 1 1 000000000001 00000000ff01 $(ip_to_hex 192 168 1 1)
> $(ip_to_hex 192 168 1 254) 84 0000 7dae fcfd 0302
> -test_ip6_packet 1 1 000000000001 00000000ff01
> 20010db8000100000000000000000011 20010db8000100000000000000000001 11 0015
> dbb8303900155bac6b646f65206676676e6d66720a 0104 d570
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [vif1.expected])
> -
> -test_tcp_syn_packet 2 2 000000000002 00000000ff02 $(ip_to_hex 192 168 2
> 1) $(ip_to_hex 192 168 2 254) 0000 8b40 3039 0000 7bae 4486
> -test_ip6_packet 2 2 000000000002 00000000ff02
> 20010db8000200000000000000000011 20010db8000200000000000000000001 84 0004
> 01020304 0103 627e
> -test_tcp6_packet 2 2 000000000002 00000000ff02
> 20010db8000200000000000000000011 20010db8000200000000000000000001 8b40 3039
> 0000 4486
> -OVN_CHECK_PACKETS([hv2/vif2-tx.pcap], [vif2.expected])
> -
> -OVN_CLEANUP([hv1], [hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ovn-controller exit])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -# Logical network:
> -# One Logical Router: ro, with two logical switches sw1 and sw2.
> -# sw1 is for subnet 10.0.0.0/8
> -# sw2 is for subnet 20.0.0.0/8
> -# sw1 has a single port bound on hv1
> -# sw2 has a single port bound on hv2
> -
> -ovn-nbctl lr-add ro
> -ovn-nbctl ls-add sw1
> -ovn-nbctl ls-add sw2
> -
> -sw1_ro_mac=00:00:10:00:00:01
> -sw1_ro_ip=10.0.0.1
> -sw2_ro_mac=00:00:20:00:00:01
> -sw2_ro_ip=20.0.0.1
> -sw1_p1_mac=00:00:10:00:00:02
> -sw1_p1_ip=10.0.0.2
> -sw2_p1_mac=00:00:20:00:00:02
> -sw2_p1_ip=20.0.0.2
> -
> -ovn-nbctl lrp-add ro ro-sw1 $sw1_ro_mac ${sw1_ro_ip}/8
> -ovn-nbctl lrp-add ro ro-sw2 $sw2_ro_mac ${sw2_ro_ip}/8
> -ovn-nbctl lsp-add sw1 sw1-ro -- set Logical_Switch_Port sw1-ro
> type=router \
> -  options:router-port=ro-sw1 addresses=\"$sw1_ro_mac\"
> -ovn-nbctl lsp-add sw2 sw2-ro -- set Logical_Switch_Port sw2-ro
> type=router \
> -  options:router-port=ro-sw2 addresses=\"$sw2_ro_mac\"
> -
> -ovn-nbctl lsp-add sw1 sw1-p1 \
> --- lsp-set-addresses sw1-p1 "$sw1_p1_mac $sw1_p1_ip"
> -
> -ovn-nbctl lsp-add sw2 sw2-p1 \
> --- lsp-set-addresses sw2-p1 "$sw2_p1_mac $sw2_p1_ip"
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw1-p1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=sw2-p1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -OVN_POPULATE_ARP
> -
> -sleep 1
> -
> -packet="inport==\"sw1-p1\" && eth.src==$sw1_p1_mac &&
> eth.dst==$sw1_ro_mac &&
> -       ip4 && ip.ttl==64 && ip4.src==$sw1_p1_ip && ip4.dst==$sw2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -
> -# Start by Sending the packet and make sure it makes it there as expected
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$sw2_ro_mac && eth.dst==$sw2_p1_mac &&
> -       ip4 && ip.ttl==63 && ip4.src==$sw1_p1_ip && ip4.dst==$sw2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Stop ovn-controller on hv2
> -as hv2 ovs-appctl -t ovn-controller exit
> -
> -# Now send the packet again. This time, it should not arrive.
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Start ovn-controller again just so OVN_CLEANUP doesn't complain
> -as hv2 start_daemon ovn-controller
> -
> -OVN_CLEANUP([hv1],[hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- external logical port])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -sim_add hv1
> -sim_add hv2
> -sim_add hv3
> -
> -ovn-nbctl ls-add ls1
> -ovn-nbctl lsp-add ls1 ls1-lp1 \
> --- lsp-set-addresses ls1-lp1 "f0:00:00:00:00:01 10.0.0.4 ae70::4"
> -
> -# Add a couple of external logical port
> -ovn-nbctl lsp-add ls1 ls1-lp_ext1 \
> --- lsp-set-addresses ls1-lp_ext1 "f0:00:00:00:00:03 10.0.0.6 ae70::6"
> -ovn-nbctl lsp-set-port-security ls1-lp_ext1 \
> -"f0:00:00:00:00:03 10.0.0.6 ae70::6"
> -ovn-nbctl lsp-set-type ls1-lp_ext1 external
> -
> -ovn-nbctl lsp-add ls1 ls1-lp_ext2 \
> --- lsp-set-addresses ls1-lp_ext2 "f0:00:00:00:00:04 10.0.0.7 ae70::7"
> -ovn-nbctl lsp-set-port-security ls1-lp_ext2 \
> -"f0:00:00:00:00:04 10.0.0.7 ae70::8"
> -ovn-nbctl lsp-set-type ls1-lp_ext2 external
> -
> -d1="$(ovn-nbctl create DHCP_Options cidr=10.0.0.0/24 \
> -options="\"server_id\"=\"10.0.0.1\" \"server_mac\"=\"ff:10:00:00:00:01\" \
> -\"lease_time\"=\"3600\" \"router\"=\"10.0.0.1\"")"
> -
> -d2="$(ovn-nbctl create DHCP_Options cidr="ae70\:\:/64" \
> -options="\"server_id\"=\"00:00:00:10:00:01\"")"
> -
> -ovn-nbctl lsp-set-dhcpv4-options ls1-lp1 ${d1}
> -ovn-nbctl lsp-set-dhcpv4-options ls1-lp_ext1 ${d1}
> -ovn-nbctl lsp-set-dhcpv4-options ls1-lp_ext2 ${d1}
> -
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp1 ${d2}
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp_ext1 ${d2}
> -ovn-nbctl lsp-set-dhcpv6-options ls1-lp_ext2 ${d2}
> -
> -# Create a logical router and connect it to ls1
> -ovn-nbctl lr-add lr0
> -ovn-nbctl lrp-add lr0 lr0-ls1 a0:10:00:00:00:01 10.0.0.1/24
> -ovn-nbctl lsp-add ls1 ls1-lr0
> -ovn-nbctl set Logical_Switch_Port ls1-lr0 type=router \
> -    options:router-port=lr0-ls1 addresses=router
> -
> -# Create HA chassis group
> -ovn-nbctl ha-chassis-group-add hagrp1
> -ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv1 30
> -
> -hagrp1_uuid=`ovn-nbctl --bare --columns _uuid find ha_chassis_group
> name="hagrp1"`
> -
> -# There should be 1 HA_Chassis rows with chassis sets
> -OVS_WAIT_UNTIL([ovn-sbctl list ha_chassis | grep chassis | awk '{print
> $3}' \
> -| grep '-' | wc -l ], [0], [1
> -])
> -
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-phys hv1-ext1 -- \
> -    set interface hv1-ext1 options:tx_pcap=hv1/ext1-tx.pcap \
> -    options:rxq_pcap=hv1/ext1-rx.pcap \
> -    ofport-request=2
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-phys hv2-ext2 -- \
> -    set interface hv2-ext2 options:tx_pcap=hv2/ext2-tx.pcap \
> -    options:rxq_pcap=hv2/ext2-rx.pcap \
> -    ofport-request=2
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -as hv3
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.3
> -ovs-vsctl -- add-port br-phys hv3-ext3 -- \
> -    set interface hv3-ext3 options:tx_pcap=hv3/ext3-tx.pcap \
> -    options:rxq_pcap=hv3/ext3-rx.pcap \
> -    ofport-request=2
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -# No DHCPv4/v6 flows for the external port - ls1-lp_ext1 - 10.0.0.6 in
> hv1 and
> -# hv2 as ha-chassis-group is not set and no localnet port added to ls1.
> -AT_CHECK([ovn-sbctl dump-flows ls1 | grep "offerip = 10.0.0.6" | \
> -wc -l], [0], [0
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0
> -])
> -
> -hv1_uuid=$(ovn-sbctl list chassis hv1 | grep uuid | awk '{print $3}')
> -hv2_uuid=$(ovn-sbctl list chassis hv2 | grep uuid | awk '{print $3}')
> -hv3_uuid=$(ovn-sbctl list chassis hv3 | grep uuid | awk '{print $3}')
> -
> -# The port_binding row for ls1-lp_ext1 should have empty chassis
> -chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -
> -AT_CHECK([test x$chassis == x], [0], [])
> -
> -# Associate hagrp1 ha-chassis-group to ls1-lp_ext1
> -ovn-nbctl --wait=hv set Logical_Switch_Port ls1-lp_ext1 \
> -ha-chassis-group=$hagrp1_uuid
> -
> -# Get the hagrp1 uuid in SB DB.
> -sb_hagrp1_uuid=`ovn-sbctl --bare --columns _uuid find ha_chassis_group \
> -name="hagrp1"`
> -
> -# Wait till ls1-lp_ext1 port_binding has ha_chassis_group set
> -OVS_WAIT_UNTIL(
> -    [sb_pb_hagrp=`ovn-sbctl --bare --columns ha_chassis_group find \
> -port_binding logical_port=ls1-lp_ext1`
> -     test "$sb_pb_hagrp" = "$sb_hagrp1_uuid"])
> -
> -# No DHCPv4/v6 flows for the external port - ls1-lp_ext1 - 10.0.0.6 in
> hv1 and hv2
> -# as no localnet port added to ls1 yet.
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0
> -])
> -
> -# Add the localnet port to the logical switch ls1
> -ovn-nbctl lsp-add ls1 ln-public
> -ovn-nbctl lsp-set-addresses ln-public unknown
> -ovn-nbctl lsp-set-type ln-public localnet
> -ovn-nbctl --wait=hv lsp-set-options ln-public network_name=phys
> -
> -ln_public_key=$(ovn-sbctl list port_binding ln-public | grep  tunnel_key
> | \
> -awk '{print $3}')
> -
> -# The ls1-lp_ext1 should be bound to hv1 as only hv1 is part of the
> -# ha chassis group.
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = "$hv1_uuid"])
> -
> -# There should be DHCPv4/v6 OF flows for the ls1-lp_ext1 port in hv1
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | grep reg14=0x$ln_public_key | \
> -wc -l], [0], [3
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \
> -grep reg14=0x$ln_public_key | wc -l], [0], [1
> -])
> -
> -# There should be no DHCPv4/v6 flows for ls1-lp_ext1 on hv2
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | wc -l], [0], [0
> -])
> -
> -# No DHCPv4/v6 flows for the external port - ls1-lp_ext2 - 10.0.0.7 in
> hv1 and
> -# hv2 as requested-chassis option is not set.
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.07" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.07" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.07" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.07" | wc -l], [0], [0
> -])
> -
> -as hv1
> -ovs-vsctl show
> -
> -# This shell function sends a DHCP request packet
> -# test_dhcp INPORT SRC_MAC DHCP_TYPE OFFER_IP ...
> -test_dhcp() {
> -    local inport=$1 src_mac=$2 dhcp_type=$3 offer_ip=$4 use_ip=$5
> -    shift; shift; shift; shift; shift;
> -    if test $use_ip != 0; then
> -        src_ip=$1
> -        dst_ip=$2
> -        shift; shift;
> -    else
> -        src_ip=`ip_to_hex 0 0 0 0`
> -        dst_ip=`ip_to_hex 255 255 255 255`
> -    fi
> -    local
> request=ffffffffffff${src_mac}0800451001100000000080110000${src_ip}${dst_ip}
> -    # udp header and dhcp header
> -    request=${request}0044004300fc0000
> -
> request=${request}010106006359aa760000000000000000000000000000000000000000${src_mac}
> -    # client hardware padding
> -    request=${request}00000000000000000000
> -    # server hostname
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -    # boot file name
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -
> request=${request}0000000000000000000000000000000000000000000000000000000000000000
> -    # dhcp magic cookie
> -    request=${request}63825363
> -    # dhcp message type
> -    request=${request}3501${dhcp_type}ff
> -
> -    local srv_mac=$1 srv_ip=$2 expected_dhcp_opts=$3
> -    # total IP length will be the IP length of the request packet
> -    # (which is 272 in our case) + 8 (padding bytes) +
> (expected_dhcp_opts / 2)
> -    ip_len=`expr 280 + ${#expected_dhcp_opts} / 2`
> -    udp_len=`expr $ip_len - 20`
> -    ip_len=$(printf "%x" $ip_len)
> -    udp_len=$(printf "%x" $udp_len)
> -    # $ip_len var will be in 3 digits i.e 134. So adding a '0' before
> $ip_len
> -    local
> reply=${src_mac}${srv_mac}080045100${ip_len}000000008011XXXX${srv_ip}${offer_ip}
> -    # udp header and dhcp header.
> -    # $udp_len var will be in 3 digits. So adding a '0' before $udp_len
> -    reply=${reply}004300440${udp_len}0000020106006359aa760000000000000000
> -    # your ip address
> -    reply=${reply}${offer_ip}
> -    # next server ip address, relay agent ip address, client mac address
> -    reply=${reply}0000000000000000${src_mac}
> -    # client hardware padding
> -    reply=${reply}00000000000000000000
> -    # server hostname
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -    # boot file name
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -
> reply=${reply}0000000000000000000000000000000000000000000000000000000000000000
> -    # dhcp magic cookie
> -    reply=${reply}63825363
> -    # dhcp message type
> -    local dhcp_reply_type=02
> -    if test $dhcp_type = 03; then
> -        dhcp_reply_type=05
> -    fi
> -
> reply=${reply}3501${dhcp_reply_type}${expected_dhcp_opts}00000000ff00000000
> -    echo $reply >> ext1_v4.expected
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport}
> $request
> -}
> -
> -
> -trim_zeros() {
> -    sed 's/\(00\)\{1,\}$//'
> -}
> -
> -# This shell function sends a DHCPv6 request packet
> -# test_dhcpv6 INPORT SRC_MAC SRC_LLA DHCPv6_MSG_TYPE OFFER_IP OUTPORT...
> -# The OUTPORTs (zero or more) list the VIFs on which the original DHCPv6
> -# packet should be received twice (one from ovn-controller and the other
> -# from the "ovs-ofctl monitor br-int resume"
> -test_dhcpv6() {
> -    local inport=$1 src_mac=$2 src_lla=$3 msg_code=$4 offer_ip=$5
> -    local req_pkt_in_expected=$6
> -    local request=ffffffffffff${src_mac}86dd00000000002a1101${src_lla}
> -    # dst ip ff02::1:2
> -    request=${request}ff020000000000000000000000010002
> -    # udp header and dhcpv6 header
> -    request=${request}02220223002affff${msg_code}010203
> -    # Client identifier
> -    request=${request}0001000a00030001${src_mac}
> -    # IA-NA (Identity Association for Non Temporary Address)
> -    request=${request}0003000c0102030400000e1000001518
> -    shift; shift; shift; shift; shift;
> -
> -    local server_mac=000000100001
> -    local server_lla=fe80000000000000020000fffe100001
> -    local reply_code=07
> -    if test $msg_code = 01; then
> -        reply_code=02
> -    fi
> -    local msg_len=54
> -    if test $offer_ip = 1; then
> -        msg_len=28
> -    fi
> -    local reply=${src_mac}${server_mac}86dd0000000000${msg_len}1101
> -    reply=${reply}${server_lla}${src_lla}
> -
> -    # udp header and dhcpv6 header
> -    reply=${reply}0223022200${msg_len}ffff${reply_code}010203
> -    # Client identifier
> -    reply=${reply}0001000a00030001${src_mac}
> -    # IA-NA
> -    if test $offer_ip != 1; then
> -        reply=${reply}0003002801020304ffffffffffffffff00050018${offer_ip}
> -        reply=${reply}ffffffffffffffff
> -    fi
> -    # Server identifier
> -    reply=${reply}0002000a00030001${server_mac}
> -
> -    echo $reply | trim_zeros >> ext${inport}_v6.expected
> -    # The inport also receives the request packet since it is connected
> -    # to the br-phys.
> -    #echo $request >> ext${inport}_v6.expected
> -
> -    as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport}
> $request
> -}
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -AT_CAPTURE_FILE([ofctl_monitor0_hv1.log])
> -as hv1 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv1.log
> -
> -AT_CAPTURE_FILE([ofctl_monitor0_hv2.log])
> -as hv2 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv2.log
> -
> -AT_CAPTURE_FILE([ofctl_monitor0_hv3.log])
> -as hv3 ovs-ofctl monitor br-int resume --detach --no-chdir \
> ---pidfile=ovs-ofctl0.pid 2> ofctl_monitor0_hv3.log
> -
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -# Send DHCPDISCOVER.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -server_mac=ff1000000001
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \
> -$expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 1 in hv1.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 0 in hv2.
> -OVS_WAIT_UNTIL([test 0 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap >
> ext1_v4.packets
> -cat ext1_v4.expected | cut -c -48 > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat ext1_v4.expected | cut -c 53- > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout])
> -
> -# ovs-ofctl also resumes the packets and this causes other ports to
> receive
> -# the DHCP request packet. So reset the pcap files so that its easier to
> test.
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -rm -f ext1_v4.expected
> -rm -f ext1_v4.packets
> -
> -# Send DHCPv6 request
> -src_mac=f00000000003
> -src_lla=fe80000000000000f20000fffe000003
> -offer_ip=ae700000000000000000000000000006
> -test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip
> -
> -# NXT_RESUMEs should be 2 in hv1.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 0 in hv2.
> -OVS_WAIT_UNTIL([test 0 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \
> -sort > ext1_v6.packets
> -cat ext1_v6.expected | cut -c -120 > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout])
> -# Skipping the UDP checksum
> -cat ext1_v6.expected | cut -c 125- > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout])
> -
> -rm -f ext1_v6.expected
> -rm -f ext1_v6.packets
> -
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -# Delete the ha-chassis hv1.
> -ovn-nbctl ha-chassis-group-remove-chassis hagrp1 hv1
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = ""])
> -
> -# Add hv2 to the ha chassis group
> -ovn-nbctl --wait=hv ha-chassis-group-add-chassis hagrp1 hv2 20
> -
> -ovn-sbctl list ha_chassis_group
> -ovn-sbctl list ha_chassis
> -
> -ovn-sbctl find port_binding logical_port=ls1-lp_ext1
> -
> -# The ls1-lp_ext1 should be bound to hv2
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = "$hv2_uuid"])
> -
> -# There should be OF flows for DHCP4/v6 for the ls1-lp_ext1 port in hv2
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | grep reg14=0x$ln_public_key | \
> -wc -l], [0], [3
> -])
> -AT_CHECK([as hv2 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \
> -grep reg14=0x$ln_public_key | wc -l], [0], [1
> -])
> -
> -# There should be no DHCPv4/v6 flows for ls1-lp_ext1 on hv1
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep "0a.00.00.06" | wc -l], [0], [0
> -])
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep table=20 | \
> -grep controller | grep tp_src=546 | grep \
> -"ae.70.00.00.00.00.00.00.00.00.00.00.00.00.00.06" | \
> -grep reg14=0x$ln_public_key | wc -l], [0], [0
> -])
> -
> -# Send DHCPDISCOVER again for hv1/ext1. The DHCP response should come from
> -# hv2 ovn-controller.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -server_mac=ff1000000001
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \
> -$expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 2 in hv1.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 1 in hv2.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap >
> ext1_v4.packets
> -cat ext1_v4.expected | cut -c -48 > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat ext1_v4.expected | cut -c 53- > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout])
> -
> -# ovs-ofctl also resumes the packets and this causes other ports to
> receive
> -# the DHCP request packet. So reset the pcap files so that its easier to
> test.
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -rm -f ext1_v4.expected
> -
> -# Send DHCPv6 request again
> -src_mac=f00000000003
> -src_lla=fe80000000000000f20000fffe000003
> -offer_ip=ae700000000000000000000000000006
> -test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip 1
> -
> -# NXT_RESUMEs should be 2 in hv1.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \
> -sort > ext1_v6.packets
> -cat ext1_v6.expected | cut -c -120 > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout])
> -# Skipping the UDP checksum
> -cat ext1_v6.expected | cut -c 125- > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout])
> -
> -rm -f ext1_v6.expected
> -rm -f ext1_v6.packets
> -
> -as hv1
> -ovs-vsctl show
> -reset_pcap_file hv1-ext1 hv1/ext1
> -reset_pcap_file br-phys_n1 hv1/br-phys_n1
> -reset_pcap_file br-phys hv1/br-phys
> -
> -as hv2
> -ovs-vsctl show
> -reset_pcap_file hv2-ext2 hv2/ext2
> -reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -reset_pcap_file br-phys hv2/br-phys
> -
> -# From  ls1-lp_ext1, send ARP request for the router ip. The ARP
> -# response should come from the router pipeline of hv2.
> -ext1_mac=f00000000003
> -router_mac=a01000000001
> -ext1_ip=`ip_to_hex 10 0 0 6`
> -router_ip=`ip_to_hex 10 0 0 1`
>
> -arp_request=ffffffffffff${ext1_mac}08060001080006040001${ext1_mac}${ext1_ip}000000000000${router_ip}
> -
> -as hv1 ovs-appctl netdev-dummy/receive hv1-ext1 $arp_request
>
> -expected_response=${src_mac}${router_mac}08060001080006040002${router_mac}${router_ip}${ext1_mac}${ext1_ip}
> -echo $expected_response > expout
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap >
> ext1_arp_resp
> -AT_CHECK([cat ext1_arp_resp], [0], [expout])
> -
> -# Verify that the response came from hv2
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv2/br-phys_n1-tx.pcap >
> ext1_arp_resp
> -AT_CHECK([cat ext1_arp_resp], [0], [expout])
> -
> -# Now add 3 ha chassis to the ha chassis group
> -ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv1 30
> -ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 20
> -ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 10
> -
> -# hv1 should be master and claim ls1-lp_ext1
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = "$hv1_uuid"])
> -
> -as hv1
> -ovs-vsctl show
> -reset_pcap_file hv1-ext1 hv1/ext1
> -reset_pcap_file br-phys_n1 hv1/br-phys_n1
> -reset_pcap_file br-phys hv1/br-phys
> -
> -as hv2
> -ovs-vsctl show
> -reset_pcap_file hv2-ext2 hv2/ext2
> -reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -reset_pcap_file br-phys hv2/br-phys
> -
> -as hv3
> -ovs-vsctl show
> -reset_pcap_file hv3-ext3 hv3/ext3
> -reset_pcap_file br-phys_n1 hv3/br-phys_n1
> -reset_pcap_file br-phys hv3/br-phys
> -
> -# Send DHCPDISCOVER.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -server_mac=ff1000000001
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \
> -$expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 3 in hv1.
> -OVS_WAIT_UNTIL([test 3 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap >
> ext1_v4.packets
> -cat ext1_v4.expected | cut -c -48 > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat ext1_v4.expected | cut -c 53- > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout])
> -
> -# ovs-ofctl also resumes the packets and this causes other ports to
> receive
> -# the DHCP request packet. So reset the pcap files so that its easier to
> test.
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -rm -f ext1_v4.expected
> -rm -f ext1_v4.packets
> -
> -# Send DHCPv6 request
> -src_mac=f00000000003
> -src_lla=fe80000000000000f20000fffe000003
> -offer_ip=ae700000000000000000000000000006
> -test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip
> -
> -# NXT_RESUMEs should be 4 in hv1.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \
> -sort > ext1_v6.packets
> -cat ext1_v6.expected | cut -c -120 > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout])
> -# Skipping the UDP checksum
> -cat ext1_v6.expected | cut -c 125- > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout])
> -
> -rm -f ext1_v6.expected
> -rm -f ext1_v6.packets
> -as hv1 reset_pcap_file hv1-ext1 hv1/ext1
> -
> -# Now increase the priority of hv3 so it becomes master.
> -ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 50
> -
> -# hv3 should be master and claim ls1-lp_ext1
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = "$hv3_uuid"])
> -
> -as hv1
> -ovs-vsctl show
> -reset_pcap_file hv1-ext1 hv1/ext1
> -reset_pcap_file br-phys_n1 hv1/br-phys_n1
> -reset_pcap_file br-phys hv1/br-phys
> -
> -as hv2
> -ovs-vsctl show
> -reset_pcap_file hv2-ext2 hv2/ext2
> -reset_pcap_file br-phys_n1 hv2/br-phys_n1
> -reset_pcap_file br-phys hv2/br-phys
> -
> -as hv2
> -ovs-vsctl show
> -reset_pcap_file hv3-ext3 hv3/ext3
> -reset_pcap_file br-phys_n1 hv3/br-phys_n1
> -reset_pcap_file br-phys hv3/br-phys
> -
> -# Send DHCPDISCOVER.
> -offer_ip=`ip_to_hex 10 0 0 6`
> -server_ip=`ip_to_hex 10 0 0 1`
> -server_mac=ff1000000001
> -expected_dhcp_opts=330400000e100104ffffff0003040a00000136040a000001
> -test_dhcp 1 f00000000003 01 $offer_ip 0 $server_mac $server_ip \
> -$expected_dhcp_opts
> -
> -# NXT_RESUMEs should be 4 in hv1.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 1 in hv3.
> -OVS_WAIT_UNTIL([test 1 = `cat ofctl_monitor0_hv3.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap >
> ext1_v4.packets
> -cat ext1_v4.expected | cut -c -48 > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c -48], [0], [expout])
> -# Skipping the IPv4 checksum.
> -cat ext1_v4.expected | cut -c 53- > expout
> -AT_CHECK([cat ext1_v4.packets | cut -c 53-], [0], [expout])
> -
> -# ovs-ofctl also resumes the packets and this causes other ports to
> receive
> -# the DHCP request packet. So reset the pcap files so that its easier to
> test.
> -as hv1
> -reset_pcap_file hv1-ext1 hv1/ext1
> -
> -rm -f ext1_v4.expected
> -rm -f ext1_v4.packets
> -
> -# Send DHCPv6 request
> -src_mac=f00000000003
> -src_lla=fe80000000000000f20000fffe000003
> -offer_ip=ae700000000000000000000000000006
> -test_dhcpv6 1 $src_mac $src_lla 01 $offer_ip
> -
> -# NXT_RESUMEs should be 4 in hv1.
> -OVS_WAIT_UNTIL([test 4 = `cat ofctl_monitor0_hv1.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv2.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv2.log | grep -c
> NXT_RESUME`])
> -
> -# NXT_RESUMEs should be 2 in hv3.
> -OVS_WAIT_UNTIL([test 2 = `cat ofctl_monitor0_hv3.log | grep -c
> NXT_RESUME`])
> -
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/ext1-tx.pcap | \
> -sort > ext1_v6.packets
> -cat ext1_v6.expected | cut -c -120 > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c -120], [0], [expout])
> -# Skipping the UDP checksum
> -cat ext1_v6.expected | cut -c 125- > expout
> -AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout])
> -
> -# disconnect hv3 from the network, hv1 should take over
> -as hv3
> -port=${sandbox}_br-phys
> -as main ovs-vsctl del-port n1 $port
> -
> -# hv1 should be master and claim ls1-lp_ext1
> -OVS_WAIT_UNTIL(
> -    [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
> -logical_port=ls1-lp_ext1`
> -    test "$chassis" = "$hv1_uuid"])
> -
> -OVN_CLEANUP([hv1],[hv2],[hv3])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- Address Set Incremental Processing])
> -AT_KEYWORDS([ovn_as_inc])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.10
> -
> -ovn-nbctl ls-add ls1
> -for i in 1 2; do
> -    ovn-nbctl lsp-add ls1 lp$i \
> -        -- lsp-set-addresses lp$i "f0:00:00:00:00:0$i 192.168.1.$i"
> -    as hv1 ovs-vsctl \
> -        -- add-port br-int vif$i \
> -        -- set Interface vif$i \
> -            external-ids:iface-id=lp$i
> -done
> -
> -for i in 1 2 3; do
> -    as1_uuid=`ovn-nbctl --wait=hv create addr name=as1`
> -    as2_uuid=`ovn-nbctl --wait=hv create addr name=as2`
> -    ovn-nbctl --wait=hv acl-add ls1 to-lport 200 \
> -            'outport=="lp1" && ip4 && ip4.src == {$as1, $as2}'
> allow-related
> -    ovn-nbctl --wait=hv set addr as1 addresses="10.1.2.10"
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.10"], [0],
> [ignore])
> -
> -    # Update address set as1
> -    ovn-nbctl --wait=hv set addr as1 addresses="10.1.2.10 10.1.2.11"
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.11"], [0],
> [ignore])
> -
> -    # Update address set as2
> -    ovn-nbctl --wait=hv set addr as2 addresses="10.1.2.12 10.1.2.13"
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.12"], [0],
> [ignore])
> -
> -    # Add another ACL referencing as1
> -    n_flows_before=`ovs-ofctl dump-flows br-int | grep "10.1.2.10" | wc
> -l`
> -    ovn-nbctl --wait=hv acl-add ls1 to-lport 200 \
> -            'outport=="lp2" && ip4 && ip4.src == $as1' allow-related
> -    n_flows_after=`ovs-ofctl dump-flows br-int | grep "10.1.2.10" | wc -l`
> -    AT_CHECK([test $(expr $n_flows_before \* 2) = $n_flows_after], [0],
> [ignore])
> -
> -    # Remove an ACL
> -    ovn-nbctl --wait=hv acl-del ls1 to-lport 200 \
> -            'outport=="lp2" && ip4 && ip4.src == $as1'
> -    n_flows_after=`ovs-ofctl dump-flows br-int | grep "10.1.2.10" | wc -l`
> -    AT_CHECK([test $n_flows_before = $n_flows_after], [0], [ignore])
> -
> -    # Remove as1 while it is still used by an ACL, the lflows should be
> reparsed and
> -    # parsing should fail.
> -    echo "before del as1"
> -    ovn-nbctl list addr | grep as1
> -    ovn-nbctl --wait=hv destroy addr $as1_uuid
> -    echo "after del as1"
> -    ovn-nbctl list addr | grep as1
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.10"], [1],
> [ignore])
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.12"], [1],
> [ignore])
> -
> -    # Recreate as1
> -    as1_uuid=`ovn-nbctl --wait=hv create addr name=as1`
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.12"], [0],
> [ignore])
> -
> -    # Remove ACLs and address sets
> -    ovn-nbctl --wait=hv destroy addr $as1_uuid -- destroy addr $as2_uuid
> -    AT_CHECK([ovs-ofctl dump-flows br-int | grep "10.1.2.12"], [1],
> [ignore])
> -
> -    ovn-nbctl --wait=hv acl-del ls1
> -done
> -
> -# Gracefully terminate daemons
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ovn-controller restart])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One Logical Router: ro, with two logical switches sw1 and sw2.
> -# sw1 is for subnet 10.0.0.0/8
> -# sw2 is for subnet 20.0.0.0/8
> -# sw1 has a single port bound on hv1
> -# sw2 has a single port bound on hv2
> -
> -ovn-nbctl lr-add ro
> -ovn-nbctl ls-add sw1
> -ovn-nbctl ls-add sw2
> -
> -sw1_ro_mac=00:00:10:00:00:01
> -sw1_ro_ip=10.0.0.1
> -sw2_ro_mac=00:00:20:00:00:01
> -sw2_ro_ip=20.0.0.1
> -sw1_p1_mac=00:00:10:00:00:02
> -sw1_p1_ip=10.0.0.2
> -sw2_p1_mac=00:00:20:00:00:02
> -sw2_p1_ip=20.0.0.2
> -
> -ovn-nbctl lrp-add ro ro-sw1 $sw1_ro_mac ${sw1_ro_ip}/8
> -ovn-nbctl lrp-add ro ro-sw2 $sw2_ro_mac ${sw2_ro_ip}/8
> -ovn-nbctl lsp-add sw1 sw1-ro -- set Logical_Switch_Port sw1-ro
> type=router \
> -  options:router-port=ro-sw1 addresses=\"$sw1_ro_mac\"
> -ovn-nbctl lsp-add sw2 sw2-ro -- set Logical_Switch_Port sw2-ro
> type=router \
> -  options:router-port=ro-sw2 addresses=\"$sw2_ro_mac\"
> -
> -ovn-nbctl lsp-add sw1 sw1-p1 \
> --- lsp-set-addresses sw1-p1 "$sw1_p1_mac $sw1_p1_ip"
> -
> -ovn-nbctl lsp-add sw2 sw2-p1 \
> --- lsp-set-addresses sw2-p1 "$sw2_p1_mac $sw2_p1_ip"
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw1-p1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=sw2-p1 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -OVN_POPULATE_ARP
> -
> -sleep 1
> -
> -packet="inport==\"sw1-p1\" && eth.src==$sw1_p1_mac &&
> eth.dst==$sw1_ro_mac &&
> -       ip4 && ip.ttl==64 && ip4.src==$sw1_p1_ip && ip4.dst==$sw2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -
> -# Start by Sending the packet and make sure it makes it there as expected
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -# Expected packet has TTL decreased by 1
> -expected="eth.src==$sw2_ro_mac && eth.dst==$sw2_p1_mac &&
> -       ip4 && ip.ttl==63 && ip4.src==$sw1_p1_ip && ip4.dst==$sw2_p1_ip &&
> -       udp && udp.src==53 && udp.dst==4369"
> -echo $expected | ovstest test-ovn expr-to-packets > expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -# Stop ovn-controller on hv2 with --restart flag
> -as hv2 ovs-appctl -t ovn-controller exit --restart
> -
> -# Now send the packet again. This time, it should still arrive
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -cat expected expected > expected2
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected2])
> -
> -# Start ovn-controller again just so OVN_CLEANUP doesn't complain
> -as hv2 start_daemon ovn-controller
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ovn-nbctl duplicate addresses])
> -ovn_start
> -
> -# Set up a switch with some switch ports of varying address types
> -ovn-nbctl ls-add sw1
> -ovn-nbctl set logical_switch sw1 other_config:subnet=192.168.0.0/24
> -
> -ovn-nbctl lsp-add sw1 sw1-p1
> -ovn-nbctl lsp-add sw1 sw1-p2
> -ovn-nbctl lsp-add sw1 sw1-p3
> -ovn-nbctl lsp-add sw1 sw1-p4
> -
> -ovn-nbctl lsp-set-addresses sw1-p1 "00:00:00:00:00:01 10.0.0.1 aef0::1"
> "00:00:00:00:00:02 10.0.0.2 aef0::2"
> -ovn-nbctl lsp-set-addresses sw1-p2 "00:00:00:00:00:03 dynamic"
> -ovn-nbctl lsp-set-addresses sw1-p3 "dynamic"
> -ovn-nbctl lsp-set-addresses sw1-p4 "router"
> -ovn-nbctl lsp-set-addresses sw1-p5 "unknown"
> -
> -ovn-nbctl list logical_switch_port
> -
> -# Now try to add duplicate addresses on a new port. These should all fail
> -ovn-nbctl --wait=sb lsp-add sw1 sw1-p5
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> 10.0.0.1"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv4 address 10.0.0.1
> -])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> 10.0.0.2"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv4 address 10.0.0.2
> -])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> aef0::1"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv6 address aef0::1
> -])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> aef0::2"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv6 address aef0::2
> -])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> 192.168.0.2"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv4 address 192.168.0.2
> -])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p5 "00:00:00:00:00:04
> 192.168.0.3"], [1], [],
> -[ovn-nbctl: Error on switch sw1: duplicate IPv4 address 192.168.0.3
> -])
> -
> -# Now try re-setting sw1-p1. This should succeed
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw1-p1 "00:00:00:00:00:01 10.0.0.1
> aef0::1"])
> -
> -# Now create a new switch and try setting IP addresses the same as the
> -# first switch. This should succeed.
> -ovn-nbctl ls-add sw2
> -ovn-nbctl lsp-add sw2 sw2-p1
> -
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw2-p1 "00:00:00:00:00:04
> 10.0.0.1"])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw2-p1 "00:00:00:00:00:04
> 192.168.0.2"])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw2-p1 "00:00:00:00:00:04
> 192.168.0.3"])
> -AT_CHECK([ovn-nbctl lsp-set-addresses sw2-p1 "00:00:00:00:00:04 aef0::1"])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- router - check packet length - icmp defrag])
> -AT_KEYWORDS([check packet length])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 sw0-port1
> -ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3"
> -
> -ovn-nbctl lr-add lr0
> -ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> -ovn-nbctl lsp-add sw0 sw0-lr0
> -ovn-nbctl lsp-set-type sw0-lr0 router
> -ovn-nbctl lsp-set-addresses sw0-lr0 router
> -ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> -
> -ovn-nbctl ls-add public
> -ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24
> -ovn-nbctl <http://172.168.0.100/24-ovn-nbctl> lsp-add public public-lr0
> -ovn-nbctl lsp-set-type public-lr0 router
> -ovn-nbctl lsp-set-addresses public-lr0 router
> -ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> -
> -# localnet port
> -ovn-nbctl lsp-add public ln-public
> -ovn-nbctl lsp-set-type ln-public localnet
> -ovn-nbctl lsp-set-addresses ln-public unknown
> -ovn-nbctl lsp-set-options ln-public network_name=phys
> -
> -ovn-nbctl lrp-set-gateway-chassis lr0-public hv1 20
> -ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw0-port1 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -reset_pcap_file() {
> -     local iface=$1
> -     local pcap_file=$2
> -     ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> - options:rxq_pcap=dummy-rx.pcap
> -     rm -f ${pcap_file}*.pcap
> -     ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> - options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -ip_to_hex() {
> -     printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -test_ip_packet_larger() {
> -    local icmp_pmtu_reply_expected=$1
> -
> -    # Send ip packet from sw0-port1 to outside
> -    src_mac="505400000001" # sw-port1 mac
> -    dst_mac="00000000ff01" # sw0-lr0 mac (internal router leg)
> -    src_ip=`ip_to_hex 10 0 0 3`
> -    dst_ip=`ip_to_hex 172 168 0 3`
> -    # Set the packet length to 100.
> -    pkt_len=0064
> -    packet=${dst_mac}${src_mac}08004500${pkt_len}0000000040010000
> -    orig_packet_l3=${src_ip}${dst_ip}0304000000000000
> -    orig_packet_l3=${orig_packet_l3}000000000000000000000000000000000000
> -    orig_packet_l3=${orig_packet_l3}000000000000000000000000000000000000
> -    orig_packet_l3=${orig_packet_l3}000000000000000000000000000000000000
> -    orig_packet_l3=${orig_packet_l3}000000000000000000000000000000000000
> -    packet=${packet}${orig_packet_l3}
> -
> -
> gw_ip_garp=ffffffffffff00002020121308060001080006040001000020201213aca80064000000000000aca80064
> -
> -    # If icmp_pmtu_reply_expected is 0, it means the packet is lesser than
> -    # the gateway mtu and should be delivered to the provider bridge via
> the
> -    # localnet port.
> -    # If icmp_pmtu_reply_expected is 1, it means the packet is larger than
> -    # the gateway mtu and ovn-controller should drop the packet and
> instead
> -    # generate ICMPv4  Destination Unreachable message with pmtu set to
> 42.
> -    if test $icmp_pmtu_reply_expected = 0; then
> -        # Packet to expect at br-phys.
> -        src_mac="000020201213"
> -        dst_mac="00000012af11"
> -        src_ip=`ip_to_hex 10 0 0 3`
> -        dst_ip=`ip_to_hex 172 168 0 3`
> -        expected=${dst_mac}${src_mac}08004500${pkt_len}000000003f010100
> -        expected=${expected}${src_ip}${dst_ip}0304000000000000
> -        expected=${expected}000000000000000000000000000000000000
> -        expected=${expected}000000000000000000000000000000000000
> -        expected=${expected}000000000000000000000000000000000000
> -        expected=${expected}000000000000000000000000000000000000
> -        echo $expected > br_phys_n1.expected
> -        echo $gw_ip_garp >> br_phys_n1.expected
> -    else
> -        # MTU would be 100 - 18 = 82 (hex 0052)
> -        mtu=0052
> -        src_ip=`ip_to_hex 10 0 0 1`
> -        dst_ip=`ip_to_hex 10 0 0 3`
> -        # pkt len should be 128 (28 (icmp packet) + 100 (orig ip +
> payload))
> -        reply_pkt_len=0080
> -        ip_csum=bd91
> -
> icmp_reply=${src_mac}${dst_mac}08004500${reply_pkt_len}00004000fe016879
> -        icmp_reply=${icmp_reply}${src_ip}${dst_ip}0304${ip_csum}0000${mtu}
> -        icmp_reply=${icmp_reply}4500${pkt_len}000000003f010100
> -        icmp_reply=${icmp_reply}${orig_packet_l3}
> -        echo $icmp_reply > hv1-vif1.expected
> -    fi
> -
> -    as hv1 reset_pcap_file br-phys_n1 hv1/br-phys_n1
> -    as hv1 reset_pcap_file hv1-vif1 hv1/vif1
> -
> -    # Send packet from sw0-port1 to outside
> -    as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 $packet
> -
> -    if test $icmp_pmtu_reply_expected = 0; then
> -        OVN_CHECK_PACKETS([hv1/br-phys_n1-tx.pcap], [br_phys_n1.expected])
> -        $PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/vif1-tx.pcap  >
> pkts
> -        # hv1/vif1-tx.pcap can receive the GARP packet generated by
> ovn-controller
> -        # for the gateway router port. So ignore this packet.
> -        cat pkts | grep -v $gw_ip_garp > packets
> -        AT_CHECK([cat packets], [0], [])
> -    else
> -        OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [hv1-vif1.expected])
> -        $PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> hv1/br-phys_n1-tx.pcap  > \
> -        pkts
> -        # hv1/br-phys_n1-tx.pcap can receive the GARP packet generated by
> ovn-controller
> -        # for the gateway router port. So ignore this packet.
> -        cat pkts | grep -v $gw_ip_garp > packets
> -        AT_CHECK([cat packets], [0], [])
> -    fi
> -}
> -
> -ovn-nbctl show
> -ovn-sbctl show
> -
> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int  \
> -| grep "check_pkt_larger" | wc -l], [0], [[0
> -]])
> -dp_uuid=$(ovn-sbctl find datapath_binding | grep sw0 -B2 | grep _uuid | \
> -awk '{print $3}')
> -ovn-sbctl create MAC_Binding ip=172.168.0.3 datapath=$dp_uuid \
> -logical_port=lr0-public mac="00\:00\:00\:12\:af\:11"
> -
> -# Set the gateway mtu to 100. If the packet length is > 100,
> ovn-controller
> -# should send icmp host not reachable with pmtu set to 100.
> -ovn-nbctl --wait=hv set logical_router_port lr0-public
> options:gateway_mtu=100
> -as hv3 ovs-appctl netdev-dummy/receive hv3-vif1 $arp_reply
> -OVS_WAIT_UNTIL([
> -    test `as hv1 ovs-ofctl dump-flows br-int | grep
> "check_pkt_larger(100)" | \
> -    wc -l` -eq 1
> -])
> -
> -icmp_reply_expected=1
> -test_ip_packet_larger $icmp_reply_expected
> -
> -# Set the gateway mtu to 500.
> -ovn-nbctl --wait=hv set logical_router_port lr0-public
> options:gateway_mtu=500
> -as hv3 ovs-appctl netdev-dummy/receive hv3-vif1 $arp_reply
> -OVS_WAIT_UNTIL([
> -    test `as hv1 ovs-ofctl dump-flows br-int | grep
> "check_pkt_larger(500)" | \
> -    wc -l` -eq 1
> -])
> -
> -# Now the packet should be sent via the localnet port to br-phys.
> -icmp_reply_expected=0
> -test_ip_packet_larger $icmp_reply_expected
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IP packet buffering])
> -AT_KEYWORDS([ip-buffering])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# One LR lr0 that has switches sw0 (192.168.1.0/24) and
> -# sw1 (172.16.1.0/24) connected to it.
> -#
> -# Physical network:
> -# Tw0 hypervisors hv[12].
> -# hv1 hosts vif sw0-p0.
> -# hv1 hosts vif sw1-p0.
> -
> -send_icmp_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ipv4_dst=$6
> ip_chksum=$7 data=$8
> -    shift 8
> -
> -    local ip_ttl=ff
> -    local ip_len=001c
> -    local
> packet=${eth_dst}${eth_src}08004500${ip_len}00004000${ip_ttl}01${ip_chksum}${ipv4_src}${ipv4_dst}${data}
> -    as hv$hv ovs-appctl netdev-dummy/receive hv$hv-vif$inport $packet
> -}
> -
> -send_icmp6_packet() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ipv6_src=$5 ipv6_dst=$6
> ipv6_router=$7 exp_icmp_chksum=$8
> -    shift 8
> -
> -    local ip6_hdr=6000000000083aff${ipv6_src}${ipv6_dst}
> -    local packet=${eth_dst}${eth_src}86dd${ip6_hdr}8000dcb662f00001
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive hv$hv-vif$inport $packet
> -}
> -
> -get_arp_req() {
> -    local eth_src=$1 spa=$2 tpa=$3
> -    local
> request=ffffffffffff${eth_src}08060001080006040001${eth_src}${spa}000000000000${tpa}
> -    echo $request
> -}
> -
> -send_arp_reply() {
> -    local hv=$1 inport=$2 eth_src=$3 eth_dst=$4 spa=$5 tpa=$6
> -    local
> request=${eth_dst}${eth_src}08060001080006040002${eth_src}${spa}${eth_dst}${tpa}
> -    as hv$hv ovs-appctl netdev-dummy/receive hv${hv}-vif$inport $request
> -}
> -
> -send_na() {
> -    local hv=$1 inport=$2 eth_src=$3 eth_dst=$4 src_ip=$5 dst_ip=$6
> -    local ip6_hdr=6000000000203aff${src_ip}${dst_ip}
> -    local
> request=${eth_dst}${eth_src}86dd${ip6_hdr}8800d78440000000${src_ip}0201${eth_src}
> -
> -    as hv$hv ovs-appctl netdev-dummy/receive hv${hv}-vif$inport $request
> -}
> -
> -get_nd() {
> -    local eth_src=$1 src_ip=$2 dst_ip=$3 ta=$4
> -    local ip6_hdr=6000000000203aff${src_ip}${dst_ip}
> -
> request=3333ff000010${eth_src}86dd${ip6_hdr}8700357600000000${ta}0101${eth_src}
> -
> -    echo $request
> -}
> -
> -net_add n1
> -
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw0-p0 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=sw1-p0 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -
> -ovn-nbctl create Logical_Router name=lr0 options:chassis=hv1
> -ovn-nbctl ls-add sw0
> -ovn-nbctl ls-add sw1
> -
> -ovn-nbctl lrp-add lr0 sw0 00:00:01:01:02:03 192.168.1.1/24
> 2001:0:0:0:0:0:0:1/64
> -ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> -    type=router options:router-port=sw0 \
> -    -- lsp-set-addresses rp-sw0 router
> -
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24
> 2002:0:0:0:0:0:0:1/64
> -ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> -    -- lsp-set-addresses rp-sw1 router
> -
> -ovn-nbctl lsp-add sw0 sw0-p0 \
> -    -- lsp-set-addresses sw0-p0 "f0:00:00:01:02:03 192.168.1.2 2001::2"
> -
> -ovn-nbctl lsp-add sw1 sw1-p0 \
> -    -- lsp-set-addresses sw1-p0 unknown
> -
> -OVN_POPULATE_ARP
> -ovn-nbctl --wait=hv sync
> -
> -ip_to_hex() {
> -    printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -src_mac=f00000010203
> -src_ip=$(ip_to_hex 192 168 1 2)
> -src_ip6=20010000000000000000000000000002
> -
> -router_mac0=000001010203
> -router_mac1=000002010203
> -router_ip=$(ip_to_hex 172 16 1 1)
> -router_ip6=20020000000000000000000000000001
> -
> -dst_mac=001122334455
> -dst_ip=$(ip_to_hex 172 16 1 10)
> -dst_ip6=20020000000000000000000000000010
> -
> -data=0800bee4391a0001
> -
> -send_icmp_packet 1 1 $src_mac $router_mac0 $src_ip $dst_ip 0000 $data
> -send_arp_reply 2 1 $dst_mac $router_mac1 $dst_ip $router_ip
> -echo $(get_arp_req $router_mac1 $router_ip $dst_ip) > expected
> -echo
> "${dst_mac}${router_mac1}08004500001c00004000fe010100${src_ip}${dst_ip}${data}"
> >> expected
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -nd_ip=ff0200000000000000000001ff000010
> -ip6_hdr=6000000000083afe${src_ip6}${dst_ip6}
> -
> -send_icmp6_packet 1 1 $src_mac $router_mac0 $src_ip6 $dst_ip6
> -echo $(get_nd $router_mac1 $src_ip6 $nd_ip $dst_ip6) >> expected
> -echo "${dst_mac}${router_mac1}86dd${ip6_hdr}8000dcb662f00001" >> expected
> -send_na 2 1 $dst_mac $router_mac1 $dst_ip6 $router_ip6
> -
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- neighbor update on same HV])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# A public switch (pub) with a localnet port connected to two LRs (lr0
> and lr1)
> -# each with a distributed gateway port.
> -# Two VMs: lp0 on sw0 connected to lr0
> -#          lp1 on sw1 connected to lr1
> -#
> -# This test adds a floating IP to each VM so when they are bound to the
> same
> -# hypervisor, it checks that the GARP sent by ovn-controller causes the
> -# MAC_Binding entries to be updated properly on each logical router.
> -# It will also capture packets on the physical interface to make sure
> that the
> -# GARPs have been sent out to the external network as well.
> -
> -# Create logical switches
> -ovn-nbctl ls-add sw0
> -ovn-nbctl ls-add sw1
> -ovn-nbctl ls-add pub
> -
> -# Created localnet port on public switch
> -ovn-nbctl lsp-add pub ln-pub
> -ovn-nbctl lsp-set-type ln-pub localnet
> -ovn-nbctl lsp-set-addresses ln-pub unknown
> -ovn-nbctl lsp-set-options ln-pub network_name=phys
> -
> -# Create logical routers and connect them to public switch
> -ovn-nbctl create Logical_Router name=lr0
> -ovn-nbctl create Logical_Router name=lr1
> -
> -ovn-nbctl lrp-add lr0 lr0-pub f0:00:00:00:00:01 172.24.4.220/24
> -ovn-nbctl lsp-add pub pub-lr0 -- set Logical_Switch_Port pub-lr0 \
> -    type=router options:router-port=lr0-pub
> options:nat-addresses="router" addresses="router"
> -ovn-nbctl lrp-add lr1 lr1-pub f0:00:00:00:01:01 172.24.4.221/24
> -ovn-nbctl lsp-add pub pub-lr1 -- set Logical_Switch_Port pub-lr1 \
> -    type=router options:router-port=lr1-pub
> options:nat-addresses="router" addresses="router"
> -
> -ovn-nbctl lrp-set-gateway-chassis lr0-pub hv1 10
> -ovn-nbctl lrp-set-gateway-chassis lr1-pub hv1 10
> -
> -# Connect sw0 and sw1 to lr0 and lr1
> -ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.254/24
> -ovn-nbctl lsp-add sw0 sw0-lr0 -- set Logical_Switch_Port sw0-lr0
> type=router \
> -    options:router-port=lr0-sw0 addresses="router"
> -ovn-nbctl lrp-add lr1 lr1-sw1 00:00:00:00:ff:02 20.0.0.254/24
> -ovn-nbctl lsp-add sw1 sw1-lr1 -- set Logical_Switch_Port sw1-lr1
> type=router \
> -    options:router-port=lr1-sw1 addresses="router"
> -
> -
> -# Add SNAT rules
> -ovn-nbctl lr-nat-add lr0 snat 172.24.4.220 10.0.0.0/24
> -ovn-nbctl lr-nat-add lr1 snat 172.24.4.221 20.0.0.0/24
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 172.24.4.1
> -ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -
> -ovs-vsctl add-port br-int vif0 -- set Interface vif0
> external-ids:iface-id=lp0
> -ovs-vsctl add-port br-int vif1 -- set Interface vif1
> external-ids:iface-id=lp1
> -
> -ovn-nbctl lsp-add sw0 lp0
> -ovn-nbctl lsp-add sw1 lp1
> -ovn-nbctl lsp-set-addresses lp0 "50:54:00:00:00:01 10.0.0.10"
> -ovn-nbctl lsp-set-addresses lp1 "50:54:00:00:00:02 20.0.0.10"
> -
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp0` = xup])
> -OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up lp1` = xup])
> -
> -# Create two floating IPs, one for each VIF
> -ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.24.4.100 10.0.0.10
> -ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.24.4.200 20.0.0.10
> -
> -# Check that the MAC_Binding entries have been properly created
> -OVS_WAIT_UNTIL([test `ovn-sbctl find mac_binding logical_port="lr0-pub"
> ip="172.24.4.200" | wc -l` -gt 0])
> -OVS_WAIT_UNTIL([test `ovn-sbctl find mac_binding logical_port="lr1-pub"
> ip="172.24.4.100" | wc -l` -gt 0])
> -
> -# Check that the GARPs went also to the external physical network
> -# Wait until at least 4 packets have arrived and copy them to a separate
> file as
> -# more GARPs are expected in the capture in order to avoid race
> conditions.
> -OVS_WAIT_UNTIL([test `$PYTHON "$top_srcdir/utilities/ovs-pcap.in"
> hv1/br-phys-tx.pcap | wc -l` -gt 4])
> -$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap | head
> -n4 > hv1/br-phys-tx4.pcap
> -
> -# GARP for lp0 172.24.4.100 on lr0-pub MAC (f0:00:00:00:00:01)
> -echo
> "fffffffffffff0000000000108060001080006040001f00000000001ac180464000000000000ac180464"
> > expout
> -# GARP for 172.24.4.220 on lr0-pub (f0:00:00:00:00:01)
> -echo
> "fffffffffffff0000000000108060001080006040001f00000000001ac1804dc000000000000ac1804dc"
> >> expout
> -# GARP for lp1 172.24.4.200 on lr1-pub MAC (f0:00:00:00:01:01)
> -echo
> "fffffffffffff0000000010108060001080006040001f00000000101ac1804c8000000000000ac1804c8"
> >> expout
> -# GARP for 172.24.4.221 on lr1-pub (f0:00:00:00:01:01)
> -echo
> "fffffffffffff0000000010108060001080006040001f00000000101ac1804dd000000000000ac1804dd"
> >> expout
> -AT_CHECK([sort hv1/br-phys-tx4.pcap], [0], [expout])
> -#OVN_CHECK_PACKETS([hv1/br-phys-tx4.pcap], [br-phys.expected])
> -
> -OVN_CLEANUP([hv1])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ipam to non-ipam])
> -ovn_start
> -
> -ovn-nbctl --wait=hv set NB_Global . options:mac_prefix="0a:00:00:00:00:00"
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 p0 -- lsp-set-addresses p0 dynamic
> -ovn-nbctl --wait=sb add Logical-Switch sw0 other_config subnet=
> 192.168.1.0/24
> -
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p0 dynamic_addresses], [0],
> -    ["0a:00:00:a8:01:03 192.168.1.2"
> -])
> -
> -ovn-nbctl --wait=sb lsp-set-addresses p0 router
> -
> -ovn-nbctl get Logical-Switch-Port p0 dynamic_addresses
> -
> -AT_CHECK([ovn-nbctl get Logical-Switch-Port p0 dynamic_addresses], [0],
> [[[]]
> -])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- ipam router ports])
> -ovn_start
> -
> -ovn-nbctl ls-add sw
> -ovn-nbctl set logical_switch sw other-config:subnet=192.168.1.0/24
> -
> -for i in 2 3 4; do
> -    ovn-nbctl lr-add ro$i
> -    ovn-nbctl lsp-add sw swp$i
> -    ovn-nbctl --wait=sb lsp-set-addresses swp$i "02:00:00:00:00:0$i
> dynamic"
> -    cidr=$(ovn-nbctl get logical_switch_port swp$i dynamic_addresses |cut
> -f2 -d' '|cut -f1 -d\")
> -    ovn-nbctl lrp-add ro$i rop$i 02:00:00:00:00:0$i $cidr/24 -- set
> logical_switch_port swp$i type=router options:router-port=rop$i
> addresses=router;
> -    AT_CHECK_UNQUOTED([ovn-nbctl get logical_router_port rop$i networks],
> [0], [[["192.168.1.$i/24"]]
> -])
> -done
> -
> -ovn-nbctl list logical_switch_port
> -ovn-nbctl list logical_router_port
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- test transport zones])
> -ovn_start
> -
> -net_add n1
> -for i in 1 2 3 4 5; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.$i.1
> -done
> -
> -dnl Wait for the changes to be propagated
> -ovn-nbctl --wait=sb --timeout=3 sync
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -dnl Assert that each Chassis has a tunnel formed to every other Chassis
> -as hv1
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv2
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv3-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv3
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv4
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv5-0
> -]])
> -
> -as hv5
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv4-0
> -]])
> -
> -dnl Let's now add some Chassis to different transport zones
> -dnl * hv1: Will be part of two transport zones: tz1 and tz2 so it
> -dnl   should have tunnels formed between the other two Chassis (hv2 and
> hv3)
> -dnl
> -dnl * hv2: Will be part of one transport zone: tz1. It should have a
> tunnel
> -dnl   to hv1 but not to hv3
> -dnl
> -dnl * hv3: Will be part of one transport zone: tz2. It should have a
> tunnel
> -dnl   to hv1 but not to hv2
> -dnl
> -dnl * hv4 and hv5: Will not have any TZ set so they will keep the tunnels
> -dnl   between themselves and remove the tunnels to other Chassis which now
> -dnl   belongs to some TZs
> -dnl
> -as hv1
> -ovs-vsctl set open . external-ids:ovn-transport-zones=tz1,tz2
> -
> -as hv2
> -ovs-vsctl set open . external-ids:ovn-transport-zones=tz1
> -
> -as hv3
> -ovs-vsctl set open . external-ids:ovn-transport-zones=tz2
> -
> -dnl Wait for the changes to be propagated
> -ovn-nbctl --wait=sb --timeout=3 sync
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -as hv1
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv2-0
> -ovn-hv3-0
> -]])
> -
> -as hv2
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -]])
> -
> -as hv3
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -]])
> -
> -as hv4
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv5-0
> -]])
> -
> -as hv5
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv4-0
> -]])
> -
> -dnl Removing the transport zones should make all Chassis to create
> -dnl tunnels between every other Chassis again
> -for i in 1 2 3; do
> -    as hv$i
> -    ovs-vsctl remove open . external-ids ovn-transport-zones
> -done
> -
> -dnl Wait for the changes to be propagated
> -ovn-nbctl --wait=sb --timeout=3 sync
> -ovn-nbctl --wait=hv --timeout=3 sync
> -
> -as hv1
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv2
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv3-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv3
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv4-0
> -ovn-hv5-0
> -]])
> -
> -as hv4
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv5-0
> -]])
> -
> -as hv5
> -AT_CHECK([ovs-vsctl --bare --columns=name find interface type="geneve" |
> awk NF | sort], [0],
> -[[ovn-hv1-0
> -ovn-hv2-0
> -ovn-hv3-0
> -ovn-hv4-0
> -]])
> -
> -OVN_CLEANUP([hv1], [hv2], [hv3])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 HVs, 2 lports/HV, localnet ports, DVR chassis mac])
> -ovn_start
> -
> -
> -# In this test cases we create 2 switches, all connected to same
> -# physical network (through br-phys on each HV). Each switch has
> -# 1 VIF. Each HV has 1 VIF port. The first digit
> -# of VIF port name indicates the hypervisor it is bound to, e.g.
> -# lp23 means VIF 3 on hv2.
> -#
> -# Each switch's VLAN tag and their logical switch ports are:
> -#   - ls1:
> -#       - tagged with VLAN 101
> -#       - ports: lp11
> -#   - ls2:
> -#       - tagged with VLAN 201
> -#       - ports: lp22
> -#
> -# Note: a localnet port is created for each switch to connect to
> -# physical network.
> -
> -for i in 1 2; do
> -    ls_name=ls$i
> -    ovn-nbctl ls-add $ls_name
> -    ln_port_name=ln$i
> -    if test $i -eq 1; then
> -        ovn-nbctl lsp-add $ls_name $ln_port_name "" 101
> -    elif test $i -eq 2; then
> -        ovn-nbctl lsp-add $ls_name $ln_port_name "" 201
> -    fi
> -    ovn-nbctl lsp-set-addresses $ln_port_name unknown
> -    ovn-nbctl lsp-set-type $ln_port_name localnet
> -    ovn-nbctl lsp-set-options $ln_port_name network_name=phys
> -done
> -
> -# lsp_to_ls LSP
> -#
> -# Prints the name of the logical switch that contains LSP.
> -lsp_to_ls () {
> -    case $1 in dnl (
> -        lp?[[11]]) echo ls1 ;; dnl (
> -        lp?[[12]]) echo ls2 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -vif_to_ls () {
> -    case $1 in dnl (
> -        vif?[[11]]) echo ls1 ;; dnl (
> -        vif?[[12]]) echo ls2 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -hv_to_num () {
> -    case $1 in dnl (
> -        hv1) echo 1 ;; dnl (
> -        hv2) echo 2 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -vif_to_num () {
> -    case $1 in dnl (
> -        vif22) echo 22 ;; dnl (
> -        vif21) echo 21 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -vif_to_hv () {
> -    case $1 in dnl (
> -        vif[[1]]?) echo hv1 ;; dnl (
> -        vif[[2]]?) echo hv2 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -vif_to_lrp () {
> -    echo router-to-`vif_to_ls $1`
> -}
> -
> -hv_to_chassis_mac () {
> -     case $1 in dnl (
> -        hv[[1]]) echo aa:bb:cc:dd:ee:11 ;; dnl (
> -        hv[[2]]) echo aa:bb:cc:dd:ee:22 ;; dnl (
> -        *) AT_FAIL_IF([:]) ;;
> -    esac
> -}
> -
> -ip_to_hex() {
> -       printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -net_add n1
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
> -    ovs-vsctl set open .
> external-ids:ovn-chassis-mac-mappings="phys:aa:bb:cc:dd:ee:$i$i"
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    ovs-vsctl add-port br-int vif$i$i -- \
> -        set Interface vif$i$i external-ids:iface-id=lp$i$i \
> -                              options:tx_pcap=hv$i/vif$i$i-tx.pcap \
> -                              options:rxq_pcap=hv$i/vif$i$i-rx.pcap \
> -                              ofport-request=$i$i
> -
> -    lsp_name=lp$i$i
> -    ls_name=$(lsp_to_ls $lsp_name)
> -
> -    ovn-nbctl lsp-add $ls_name $lsp_name
> -    ovn-nbctl lsp-set-addresses $lsp_name "f0:00:00:00:00:$i$i
> 192.168.$i.$i"
> -    ovn-nbctl lsp-set-port-security $lsp_name f0:00:00:00:00:$i$i
> -
> -    OVS_WAIT_UNTIL([test x`ovn-nbctl lsp-get-up $lsp_name` = xup])
> -
> -done
> -
> -ovn-nbctl lr-add router
> -ovn-nbctl lrp-add router router-to-ls1 00:00:01:01:02:03 192.168.1.3/24
> -ovn-nbctl <http://192.168.1.3/24-ovn-nbctl> lrp-add router router-to-ls2
> 00:00:01:01:02:05 192.168.2.3/24
> -
> -ovn-nbctl lsp-add ls1 ls1-to-router -- set Logical_Switch_Port
> ls1-to-router type=router options:router-port=router-to-ls1 --
> lsp-set-addresses ls1-to-router router
> -ovn-nbctl lsp-add ls2 ls2-to-router -- set Logical_Switch_Port
> ls2-to-router type=router options:router-port=router-to-ls2 --
> lsp-set-addresses ls2-to-router router
> -
> -ovn-nbctl --wait=sb sync
> -#ovn-sbctl dump-flows
> -
> -ovn-nbctl show
> -ovn-sbctl show
> -
> -OVN_POPULATE_ARP
> -
> -test_ip() {
> -    # This packet has bad checksums but logical L3 routing doesn't check.
> -    local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> -    local
> packet=${dst_mac}${src_mac}08004500001c0000000040110000${src_ip}${dst_ip}0035111100080000
> -    shift; shift; shift; shift; shift
> -    hv=`vif_to_hv $inport`
> -    hv_num=`hv_to_num $hv`
> -    chassis_mac=`hv_to_chassis_mac $hv`
> -    as $hv ovs-appctl netdev-dummy/receive $inport $packet
> -    #as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> -    in_ls=`vif_to_ls $inport`
> -    in_lrp=`vif_to_lrp $inport`
> -    for outport; do
> -        out_ls=`vif_to_ls $outport`
> -        if test $in_ls = $out_ls; then
> -            # Ports on the same logical switch receive exactly the same
> packet.
> -            echo $packet
> -        else
> -            # Routing decrements TTL and updates source and dest MAC
> -            # (and checksum).
> -            outport_num=`vif_to_num $outport`
> -            out_lrp=`vif_to_lrp $outport`
> -            echo
> f000000000${outport_num}aabbccddee${hv_num}${hv_num}08004500001c00000000"3f1101"00${src_ip}${dst_ip}0035111100080000
> -        fi >> $outport.expected
> -    done
> -}
> -
> -# Dump a bunch of info helpful for debugging if there's a failure.
> -
> -echo "------ OVN dump ------"
> -ovn-nbctl show
> -ovn-sbctl show
> -
> -echo "------ hv1 dump ------"
> -as hv1 ovs-vsctl show
> -as hv1 ovs-vsctl list Open_Vswitch
> -
> -echo "------ hv2 dump ------"
> -as hv2 ovs-vsctl show
> -as hv2 ovs-vsctl list Open_Vswitch
> -
> -echo "Send traffic"
> -sip=`ip_to_hex 192 168 1 1`
> -dip=`ip_to_hex 192 168 2 2`
> -test_ip vif11 f00000000011  000001010203 $sip $dip vif22
> -
> -echo "----------- Post Traffic hv1 dump -----------"
> -as hv1 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -as hv1 ovs-appctl fdb/show br-phys
> -
> -echo "----------- Post Traffic hv2 dump -----------"
> -as hv2 ovs-ofctl -O OpenFlow13 dump-flows br-int
> -as hv2 ovs-appctl fdb/show br-phys
> -
> -OVN_CHECK_PACKETS([hv2/vif22-tx.pcap], [vif22.expected])
> -
> -OVN_CLEANUP([hv1],[hv2])
> -
> -AT_CLEANUP
> -
> -# Run ovn-nbctl in daemon mode, change to a backup database and verify
> that
> -# an insert operation is not allowed.
> -AT_SETUP([ovn -- can't write to a backup database server instance])
> -ovn_start
> -on_exit 'kill $(cat ovn-nbctl.pid)'
> -export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
> -
> -AT_CHECK([ovn-nbctl ls-add sw0])
> -as ovn-nb
> -AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep
> active | wc -l], [0], [1
> -])
> -ovs-appctl -t ovsdb-server ovsdb-server/set-active-ovsdb-server tcp:
> 192.0.2.2:6641
> -ovs-appctl -t ovsdb-server ovsdb-server/connect-active-ovsdb-server
> -AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep -c
> backup], [0], [1
> -])
> -AT_CHECK([ovn-nbctl ls-add sw1], [1], [ignore],
> -[ovn-nbctl: transaction error: {"details":"insert operation not allowed
> when database server is in read only mode","error":"not allowed"}
> -])
> -
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- controller event])
> -AT_KEYWORDS([ovn_controller_event])
> -ovn_start
> -
> -# Create hypervisors hv[12].
> -# Add vif1[12] to hv1, vif2[12] to hv2
> -# Add all of the vifs to a single logical switch sw0.
> -
> -net_add n1
> -ovn-nbctl ls-add sw0
> -for i in 1 2; do
> -    sim_add hv$i
> -    as hv$i
> -    ovs-vsctl add-br br-phys
> -    ovn_attach n1 br-phys 192.168.0.$i
> -
> -    for j in 1 2; do
> -        ovn-nbctl lsp-add sw0 sw0-p$i$j -- \
> -                lsp-set-addresses sw0-p$i$j "00:00:00:00:00:$i$j
> 192.168.1.$i$j"
> -
> -        ovs-vsctl -- add-port br-int vif$i$j -- \
> -                set interface vif$i$j \
> -                external-ids:iface-id=sw0-p$i$j \
> -                options:tx_pcap=hv$i/vif$i$j-tx.pcap \
> -                options:rxq_pcap=hv$i/vif$i$j-rx.pcap \
> -                ofport-request=$i$j
> -    done
> -done
> -
> -ovn-nbctl --wait=hv set NB_Global . options:controller_event=true
> -ovn-nbctl lb-add lb0 192.168.1.100:80 ""
> -ovn-nbctl ls-lb-add sw0 lb0
> -uuid_lb=$(ovn-nbctl --bare --columns=_uuid find load_balancer name=lb0)
> -
> -OVN_POPULATE_ARP
> -ovn-nbctl --timeout=3 --wait=hv sync
> -ovn-sbctl lflow-list
> -as hv1 ovs-ofctl dump-flows br-int
> -
> -packet="inport==\"sw0-p11\" && eth.src==00:00:00:00:00:11 &&
> eth.dst==00:00:00:00:00:21 &&
> -       ip4 && ip.ttl==64 && ip4.src==192.168.1.11 &&
> ip4.dst==192.168.1.100 &&
> -       tcp && tcp.src==10000 && tcp.dst==80"
> -as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"
> -
> -ovn-sbctl list controller_event
> -uuid=$(ovn-sbctl list controller_event | awk '/_uuid/{print $3}')
> -AT_CHECK([ovn-sbctl get controller_event $uuid event_type], [0], [dnl
> -empty_lb_backends
> -])
> -AT_CHECK([ovn-sbctl get controller_event $uuid event_info:vip], [0], [dnl
> -"192.168.1.100:80"
> -])
> -AT_CHECK([ovn-sbctl get controller_event $uuid event_info:protocol], [0],
> [dnl
> -tcp
> -])
> -AT_CHECK_UNQUOTED([ovn-sbctl get controller_event $uuid
> event_info:load_balancer], [0], [dnl
> -"$uuid_lb"
> -])
> -AT_CHECK([ovn-sbctl get controller_event $uuid seq_num], [0], [dnl
> -1
> -])
> -
> -OVN_CLEANUP([hv1], [hv2])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- IGMP snoop/querier])
> -AT_SKIP_IF([test $HAVE_PYTHON = no])
> -ovn_start
> -
> -# Logical network:
> -# Two independent logical switches (sw1 and sw2).
> -# sw1:
> -#   - subnet 10.0.0.0/8
> -#   - 2 ports bound on hv1 (sw1-p11, sw1-p12)
> -#   - 2 ports bound on hv2 (sw1-p21, sw1-p22)
> -# sw2:
> -#   - subnet 20.0.0.0/8
> -#   - 1 port bound on hv1 (sw2-p1)
> -#   - 1 port bound on hv2 (sw2-p2)
> -#   - IGMP Querier from 20.0.0.254
> -
> -reset_pcap_file() {
> -    local iface=$1
> -    local pcap_file=$2
> -    ovs-vsctl -- set Interface $iface options:tx_pcap=dummy-tx.pcap \
> -options:rxq_pcap=dummy-rx.pcap
> -    rm -f ${pcap_file}*.pcap
> -    ovs-vsctl -- set Interface $iface
> options:tx_pcap=${pcap_file}-tx.pcap \
> -options:rxq_pcap=${pcap_file}-rx.pcap
> -}
> -
> -ip_to_hex() {
> -     printf "%02x%02x%02x%02x" "$@"
> -}
> -
> -#
> -# send_igmp_v3_report INPORT HV ETH_SRC IP_SRC IP_CSUM GROUP REC_TYPE
> -#                     IGMP_CSUM OUTFILE
> -#
> -# This shell function causes an IGMPv3 report to be received on INPORT of
> HV.
> -# The packet's content has Ethernet destination 01:00:5E:00:00:22 and
> source
> -# ETH_SRC (exactly 12 hex digits). Ethernet type is set to IP.
> -# GROUP is the IP multicast group to be joined/to leave (based on
> REC_TYPE).
> -# REC_TYPE == 04: join GROUP
> -# REC_TYPE == 03: leave GROUP
> -# The packet hexdump is also stored in OUTFILE.
> -#
> -send_igmp_v3_report() {
> -    local inport=$1 hv=$2 eth_src=$3 ip_src=$4 ip_chksum=$5 group=$6
> -    local rec_type=$7 igmp_chksum=$8 outfile=$9
> -
> -    local eth_dst=01005e000016
> -    local ip_dst=$(ip_to_hex 224 0 0 22)
> -    local ip_ttl=01
> -    local ip_ra_opt=94040000
> -
> -    local igmp_type=2200
> -    local num_rec=00000001
> -    local aux_dlen=00
> -    local num_src=0000
> -
> -    local eth=${eth_dst}${eth_src}0800
> -    local
> ip=46c0002800004000${ip_ttl}02${ip_chksum}${ip_src}${ip_dst}${ip_ra_opt}
> -    local
> igmp=${igmp_type}${igmp_chksum}${num_rec}${rec_type}${aux_dlen}${num_src}${group}
> -    local packet=${eth}${ip}${igmp}
> -
> -    echo ${packet} >> ${outfile}
> -    as $hv ovs-appctl netdev-dummy/receive ${inport} ${packet}
> -}
> -
> -#
> -# store_igmp_v3_query ETH_SRC IP_SRC IP_CSUM OUTFILE
> -#
> -# This shell function builds an IGMPv3 general query from ETH_SRC and
> IP_SRC
> -# and stores the hexdump of the packet in OUTFILE.
> -#
> -store_igmp_v3_query() {
> -    local eth_src=$1 ip_src=$2 ip_chksum=$3 outfile=$4
> -
> -    local eth_dst=01005e000001
> -    local ip_dst=$(ip_to_hex 224 0 0 1)
> -    local ip_ttl=01
> -    local igmp_type=11
> -    local max_resp=0a
> -    local igmp_chksum=eeeb
> -    local addr=00000000
> -
> -    local eth=${eth_dst}${eth_src}0800
> -    local ip=4500002000004000${ip_ttl}02${ip_chksum}${ip_src}${ip_dst}
> -    local igmp=${igmp_type}${max_resp}${igmp_chksum}${addr}000a0000
> -    local packet=${eth}${ip}${igmp}
> -
> -    echo ${packet} >> ${outfile}
> -}
> -
> -#
> -# send_ip_multicast_pkt INPORT HV ETH_SRC ETH_DST IP_SRC IP_DST IP_LEN
> -#    IP_PROTO DATA OUTFILE
> -#
> -# This shell function causes an IP multicast packet to be received on
> INPORT
> -# of HV.
> -# The hexdump of the packet is stored in OUTFILE.
> -#
> -send_ip_multicast_pkt() {
> -    local inport=$1 hv=$2 eth_src=$3 eth_dst=$4 ip_src=$5 ip_dst=$6
> -    local ip_len=$7 ip_chksum=$8 proto=$9 data=${10} outfile=${11}
> -
> -    local ip_ttl=20
> -
> -    local eth=${eth_dst}${eth_src}0800
> -    local
> ip=450000${ip_len}95f14000${ip_ttl}${proto}${ip_chksum}${ip_src}${ip_dst}
> -    local packet=${eth}${ip}${data}
> -
> -    as $hv ovs-appctl netdev-dummy/receive ${inport} ${packet}
> -    echo ${packet} >> ${outfile}
> -}
> -
> -ovn-nbctl ls-add sw1
> -ovn-nbctl ls-add sw2
> -
> -ovn-nbctl lsp-add sw1 sw1-p11
> -ovn-nbctl lsp-add sw1 sw1-p12
> -ovn-nbctl lsp-add sw1 sw1-p21
> -ovn-nbctl lsp-add sw1 sw1-p22
> -ovn-nbctl lsp-add sw2 sw2-p1
> -ovn-nbctl lsp-add sw2 sw2-p2
> -
> -net_add n1
> -sim_add hv1
> -as hv1
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.1
> -ovs-vsctl -- add-port br-int hv1-vif1 -- \
> -    set interface hv1-vif1 external-ids:iface-id=sw1-p11 \
> -    options:tx_pcap=hv1/vif1-tx.pcap \
> -    options:rxq_pcap=hv1/vif1-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv1-vif2 -- \
> -    set interface hv1-vif2 external-ids:iface-id=sw1-p12 \
> -    options:tx_pcap=hv1/vif2-tx.pcap \
> -    options:rxq_pcap=hv1/vif2-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv1-vif3 -- \
> -    set interface hv1-vif3 external-ids:iface-id=sw2-p1 \
> -    options:tx_pcap=hv1/vif3-tx.pcap \
> -    options:rxq_pcap=hv1/vif3-rx.pcap \
> -    ofport-request=1
> -
> -sim_add hv2
> -as hv2
> -ovs-vsctl add-br br-phys
> -ovn_attach n1 br-phys 192.168.0.2
> -ovs-vsctl -- add-port br-int hv2-vif1 -- \
> -    set interface hv2-vif1 external-ids:iface-id=sw1-p21 \
> -    options:tx_pcap=hv2/vif1-tx.pcap \
> -    options:rxq_pcap=hv2/vif1-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv2-vif2 -- \
> -    set interface hv2-vif2 external-ids:iface-id=sw1-p22 \
> -    options:tx_pcap=hv2/vif2-tx.pcap \
> -    options:rxq_pcap=hv2/vif2-rx.pcap \
> -    ofport-request=1
> -ovs-vsctl -- add-port br-int hv2-vif3 -- \
> -    set interface hv2-vif3 external-ids:iface-id=sw2-p2 \
> -    options:tx_pcap=hv2/vif3-tx.pcap \
> -    options:rxq_pcap=hv2/vif3-rx.pcap \
> -    ofport-request=1
> -
> -OVN_POPULATE_ARP
> -
> -# Enable IGMP snooping on sw1.
> -ovn-nbctl set Logical_Switch sw1 other_config:mcast_querier="false"
> -ovn-nbctl set Logical_Switch sw1 other_config:mcast_snoop="true"
> -
> -# No IGMP query should be generated by sw1 (mcast_querier="false").
> -truncate -s 0 expected
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv2/vif2-tx.pcap], [expected])
> -
> -ovn-nbctl --wait=hv sync
> -
> -# Inject IGMP Join for 239.0.1.68 on sw1-p11.
> -send_igmp_v3_report hv1-vif1 hv1 \
> -    000000000001 $(ip_to_hex 10 0 0 1) f9f8 \
> -    $(ip_to_hex 239 0 1 68) 04 e9b9 \
> -    /dev/null
> -# Inject IGMP Join for 239.0.1.68 on sw1-p21.
> -send_igmp_v3_report hv2-vif1 hv2 000000000002 $(ip_to_hex 10 0 0 2) f9f9 \
> -    $(ip_to_hex 239 0 1 68) 04 e9b9 \
> -    /dev/null
> -
> -# Check that the IGMP Group is learned on both hv.
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    test "${total_entries}" = "2"
> -])
> -
> -# Send traffic and make sure it gets forwarded only on the two ports that
> -# joined.
> -truncate -s 0 expected
> -truncate -s 0 expected_empty
> -send_ip_multicast_pkt hv1-vif2 hv1 \
> -    000000000001 01005e000144 \
> -    $(ip_to_hex 10 0 0 42) $(ip_to_hex 239 0 1 68) 1e ca70 11 \
> -    e518e518000a3b3a0000 \
> -    expected
> -
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv2/vif2-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv1/vif3-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv2/vif3-tx.pcap], [expected_empty])
> -
> -# Inject IGMP Leave for 239.0.1.68 on sw1-p11.
> -send_igmp_v3_report hv1-vif1 hv1 \
> -    000000000001 $(ip_to_hex 10 0 0 1) f9f8 \
> -    $(ip_to_hex 239 0 1 68) 03 eab9 \
> -    /dev/null
> -
> -# Check IGMP_Group table on both HV.
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    test "${total_entries}" = "1"
> -])
> -
> -# Send traffic traffic and make sure it gets forwarded only on the port
> that
> -# joined.
> -as hv1 reset_pcap_file hv1-vif1 hv1/vif1
> -as hv2 reset_pcap_file hv2-vif1 hv2/vif1
> -truncate -s 0 expected
> -truncate -s 0 expected_empty
> -send_ip_multicast_pkt hv1-vif2 hv1 \
> -    000000000001 01005e000144 \
> -    $(ip_to_hex 10 0 0 42) $(ip_to_hex 239 0 1 68) 1e ca70 11 \
> -    e518e518000a3b3a0000 \
> -    expected
> -
> -OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv1/vif2-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv2/vif2-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv1/vif3-tx.pcap], [expected_empty])
> -OVN_CHECK_PACKETS([hv2/vif3-tx.pcap], [expected_empty])
> -
> -# Flush IGMP groups.
> -ovn-sbctl ip-multicast-flush sw1
> -ovn-nbctl --wait=hv -t 3 sync
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    test "${total_entries}" = "0"
> -])
> -
> -# Enable IGMP snooping and querier on sw2 and set query interval to
> minimum.
> -ovn-nbctl set Logical_Switch sw2 \
> -    other_config:mcast_snoop="true" \
> -    other_config:mcast_querier="true" \
> -    other_config:mcast_query_interval=1 \
> -    other_config:mcast_eth_src="00:00:00:00:02:fe" \
> -    other_config:mcast_ip4_src="20.0.0.254"
> -
> -# Wait for 1 query interval (1 sec) and check that two queries are
> generated.
> -truncate -s 0 expected
> -store_igmp_v3_query 0000000002fe $(ip_to_hex 20 0 0 254) 84dd expected
> -store_igmp_v3_query 0000000002fe $(ip_to_hex 20 0 0 254) 84dd expected
> -
> -sleep 1
> -OVN_CHECK_PACKETS([hv1/vif3-tx.pcap], [expected])
> -OVN_CHECK_PACKETS([hv2/vif3-tx.pcap], [expected])
> -
> -OVN_CLEANUP([hv1], [hv2])
> -AT_CLEANUP
> diff --git a/tests/ovsdb-cluster-testsuite.at b/tests/
> ovsdb-cluster-testsuite.at
> deleted file mode 100644
> index c411c184e..000000000
> --- a/tests/ovsdb-cluster-testsuite.at
> +++ /dev/null
> @@ -1,10 +0,0 @@
> -AT_INIT
> -
> -m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
> -
> -m4_include([tests/ovs-macros.at])
> -m4_include([tests/ovsdb-macros.at])
> -m4_include([tests/ofproto-macros.at])
> -
> -m4_include([tests/ovsdb-execution.at])
> -m4_include([tests/ovsdb-cluster.at])
> diff --git a/tests/ovsdb-cluster.at b/tests/ovsdb-cluster.at
> deleted file mode 100644
> index 470127270..000000000
> --- a/tests/ovsdb-cluster.at
> +++ /dev/null
> @@ -1,450 +0,0 @@
> -OVS_START_SHELL_HELPERS
> -# ovsdb_check_cluster N_SERVERS SCHEMA_FUNC OUTPUT TRANSACTION...
> -ovsdb_check_cluster () {
> -    local n=$1 schema_func=$2 output=$3
> -    shift; shift; shift
> -
> -    $schema_func > schema
> -    schema=`ovsdb-tool schema-name schema`
> -    AT_CHECK([ovsdb-tool '-vPATTERN:console:%c|%p|%m' create-cluster
> s1.db schema unix:s1.raft], [0], [], [stderr])
> -    AT_CHECK([grep -v 'from ephemeral to persistent' stderr], [1])
> -    cid=`ovsdb-tool db-cid s1.db`
> -    for i in `seq 2 $n`; do
> -        AT_CHECK([ovsdb-tool join-cluster s$i.db $schema unix:s$i.raft
> unix:s1.raft])
> -    done
> -
> -    on_exit 'kill `cat *.pid`'
> -    for i in `seq $n`; do
> -        AT_CHECK([ovsdb-server -vraft -vconsole:off -vsyslog:off --detach
> --no-chdir --log-file=s$i.log --pidfile=s$i.pid --unixctl=s$i
> --remote=punix:s$i.ovsdb s$i.db])
> -    done
> -    for i in `seq $n`; do
> -        AT_CHECK([ovsdb_client_wait unix:s$i.ovsdb $schema connected])
> -    done
> -
> -    for txn
> -    do
> -      AT_CHECK([ovsdb-client --timeout=30 -vjsonrpc -vconsole:off
> -vsyslog:off -vvlog:off --log-file transact
> unix:s1.ovsdb,unix:s2.ovsdb,unix:s3.ovsdb "$txn"], [0], [stdout])
> -      cat stdout >> output
> -    done
> -    AT_CHECK_UNQUOTED([uuidfilt output], [0], [$output])
> -    for i in `seq $n`; do
> -        OVS_APP_EXIT_AND_WAIT_BY_TARGET([`pwd`/s$i], [s$i.pid])
> -    done
> -
> -    AT_CHECK([ovsdb-tool check-cluster s*.db])
> -}
> -OVS_END_SHELL_HELPERS
> -
> -# Test a 1-server cluster.
> -AT_BANNER([OVSDB - clustered transactions (1 server)])
> -m4_define([OVSDB_CHECK_EXECUTION],
> -  [AT_SETUP([$1 - cluster of 1])
> -   AT_KEYWORDS([ovsdb server positive unix cluster cluster1 $5])
> -   ovsdb_check_cluster 1 "$2" '$4' m4_foreach([txn], [$3], ['txn' ])
> -   AT_CLEANUP])
> -EXECUTION_EXAMPLES
> -
> -# Test a 3-server cluster.
> -AT_BANNER([OVSDB - clustered transactions (3 servers)])
> -m4_define([OVSDB_CHECK_EXECUTION],
> -  [AT_SETUP([$1 - cluster of 3])
> -   AT_KEYWORDS([ovsdb server positive unix cluster cluster3 $5])
> -   ovsdb_check_cluster 3 "$2" '$4' m4_foreach([txn], [$3], ['txn' ])
> -   AT_CLEANUP])
> -EXECUTION_EXAMPLES
> -
> -# Test a 5-server cluster.
> -AT_BANNER([OVSDB - clustered transactions (5 servers)])
> -m4_define([OVSDB_CHECK_EXECUTION],
> -  [AT_SETUP([$1 - cluster of 5])
> -   AT_KEYWORDS([ovsdb server positive unix cluster cluster5 $5])
> -   ovsdb_check_cluster 5 "$2" '$4' m4_foreach([txn], [$3], ['txn' ])
> -   AT_CLEANUP])
> -EXECUTION_EXAMPLES
> -
> -
> -OVS_START_SHELL_HELPERS
> -# ovsdb_cluster_failure_test SCHEMA_FUNC OUTPUT TRANSACTION...
> -ovsdb_cluster_failure_test () {
> -    # Initial state: s1 is leader, s2 and s3 are followers
> -    remote_1=$1
> -    remote_2=$2
> -    crash_node=$3
> -    crash_command=$4
> -    if test "$crash_node" == "1"; then
> -        new_leader=$5
> -    fi
> -
> -    cp $top_srcdir/ovn/ovn-nb.ovsschema schema
> -    schema=`ovsdb-tool schema-name schema`
> -    AT_CHECK([ovsdb-tool '-vPATTERN:console:%c|%p|%m' create-cluster
> s1.db schema unix:s1.raft], [0], [], [dnl
> -ovsdb|WARN|schema: changed 2 columns in 'OVN_Northbound' database from
> ephemeral to persistent, including 'status' column in 'Connection' table,
> because clusters do not support ephemeral columns
> -])
> -
> -    n=3
> -    join_cluster() {
> -        local i=$1
> -        others=
> -        for j in `seq 1 $n`; do
> -            if test $i != $j; then
> -                others="$others unix:s$j.raft"
> -            fi
> -        done
> -        AT_CHECK([ovsdb-tool join-cluster s$i.db $schema unix:s$i.raft
> $others])
> -    }
> -    start_server() {
> -        local i=$1
> -        printf "\ns$i: starting\n"
> -        AT_CHECK([ovsdb-server -vjsonrpc -vconsole:off -vsyslog:off
> --detach --no-chdir --log-file=s$i.log --pidfile=s$i.pid --unixctl=s$i
> --remote=punix:s$i.ovsdb s$i.db])
> -    }
> -    connect_server() {
> -        local i=$1
> -        printf "\ns$i: waiting to connect to storage\n"
> -        AT_CHECK([ovsdb_client_wait --log-file=connect$i.log
> unix:s$i.ovsdb $schema connected])
> -    }
> -    cid=`ovsdb-tool db-cid s1.db`
> -    for i in `seq 2 $n`; do join_cluster $i; done
> -
> -    on_exit 'kill `cat *.pid`'
> -    for i in `seq $n`; do start_server $i; done
> -    for i in `seq $n`; do connect_server $i; done
> -
> -    export OVN_NB_DB=unix:s$remote_1.ovsdb,unix:s$remote_2.ovsdb
> -
> -    # To ensure $new_leader node the new leader, we delay election timer
> for
> -    # the other follower.
> -    if test -n "$new_leader"; then
> -        if test "$new_leader" == "2"; then
> -            delay_election_node=3
> -        else
> -            delay_election_node=2
> -        fi
> -        AT_CHECK([ovs-appctl -t "`pwd`"/s$delay_election_node
> cluster/failure-test delay-election], [0], [ignore])
> -    fi
> -    AT_CHECK([ovs-appctl -t "`pwd`"/s$crash_node cluster/failure-test
> $crash_command], [0], [ignore])
> -    AT_CHECK([ovn-nbctl -v --timeout=10 --no-leader-only
> --no-shuffle-remotes create logical_switch name=ls1], [0], [ignore],
> [ignore])
> -
> -    # Make sure that the node really crashed.
> -    AT_CHECK([ls s$crash_node.ovsdb], [2], [ignore], [ignore])
> -    # XXX: Client will fail if remotes contains unix socket that doesn't
> exist (killed).
> -    if test "$remote_1" == "$crash_node"; then
> -        export OVN_NB_DB=unix:s$remote_2.ovsdb
> -    fi
> -    AT_CHECK([ovn-nbctl --no-leader-only ls-list | awk '{ print $2 }'],
> [0], [(ls1)
> -])
> -}
> -OVS_END_SHELL_HELPERS
> -AT_BANNER([OVSDB - cluster failure with pending transaction])
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash before sending
> appendReq, follower-2 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1 crash-before-sending-append-request 2
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash before sending
> appendReq, follower-3 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1 crash-before-sending-append-request 3
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash before sending
> execRep, follower-2 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1
> crash-before-sending-execute-command-reply 2
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash before sending
> execRep, follower-3 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1
> crash-before-sending-execute-command-reply 3
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash after sending
> execRep, follower-2 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1
> crash-after-sending-execute-command-reply 2
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, leader crash after sending
> execRep, follower-3 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 1
> crash-after-sending-execute-command-reply 3
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on leader, leader crash before sending
> appendReq, follower-2 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 1 2 1 crash-before-sending-append-request 2
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on leader, leader crash before sending
> appendReq, follower-3 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 1 2 1 crash-before-sending-append-request 3
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on leader, leader crash after sending
> appendReq, follower-2 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -# XXX: Detect and skip repeated transaction before enabling this test
> -AT_CHECK([exit 77])
> -ovsdb_cluster_failure_test 1 2 1 crash-after-sending-append-request 2
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on leader, leader crash after sending
> appendReq, follower-3 becomes leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -# XXX: Detect and skip repeated transaction before enabling this test
> -AT_CHECK([exit 77])
> -ovsdb_cluster_failure_test 1 2 1 crash-after-sending-append-request 3
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, follower-2 crash before
> sending execReq, reconnect to follower-3])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 3 2
> crash-before-sending-execute-command-request
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, follower-2 crash before
> sending execReq, reconnect to leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 1 2
> crash-before-sending-execute-command-request
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, follower-2 crash after
> sending execReq, reconnect to follower-3])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -# XXX: Detect and skip repeated transaction before enabling this test
> -AT_CHECK([exit 77])
> -ovsdb_cluster_failure_test 2 3 2
> crash-after-sending-execute-command-request
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, follower-2 crash after
> sending execReq, reconnect to leader])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -# XXX: Detect and skip repeated transaction before enabling this test
> -AT_CHECK([exit 77])
> -ovsdb_cluster_failure_test 2 1 2
> crash-after-sending-execute-command-request
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on leader, follower-2 crash after receiving
> appendReq for the update])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 1 1 2
> crash-after-receiving-append-request-update
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB cluster - txn on follower-2, follower-3 crash after
> receiving appendReq for the update])
> -AT_KEYWORDS([ovsdb server negative unix cluster pending-txn])
> -ovsdb_cluster_failure_test 2 2 3
> crash-after-receiving-append-request-update
> -AT_CLEANUP
> -
> -
> -AT_BANNER([OVSDB - cluster tests])
> -
> -# Torture test.
> -OVS_START_SHELL_HELPERS
> -ovsdb_torture_test () {
> -    local n=$1                  # Number of cluster members
> -    local victim=$2             # Cluster member to kill or remove
> -    local variant=$3            # 'kill' and restart or 'remove' and add
> -    cp $top_srcdir/ovn/ovn-sb.ovsschema schema
> -    schema=`ovsdb-tool schema-name schema`
> -    AT_CHECK([ovsdb-tool '-vPATTERN:console:%c|%p|%m' create-cluster
> s1.db schema unix:s1.raft], [0], [], [dnl
> -ovsdb|WARN|schema: changed 2 columns in 'OVN_Southbound' database from
> ephemeral to persistent, including 'status' column in 'Connection' table,
> because clusters do not support ephemeral columns
> -])
> -
> -    join_cluster() {
> -        local i=$1
> -        others=
> -        for j in `seq 1 $n`; do
> -            if test $i != $j; then
> -                others="$others unix:s$j.raft"
> -            fi
> -        done
> -        AT_CHECK([ovsdb-tool join-cluster s$i.db $schema unix:s$i.raft
> $others])
> -    }
> -
> -    start_server() {
> -        local i=$1
> -        printf "\ns$i: starting\n"
> -        AT_CHECK([ovsdb-server -vjsonrpc -vconsole:off -vsyslog:off
> --detach --no-chdir --log-file=s$i.log --pidfile=s$i.pid --unixctl=s$i
> --remote=punix:s$i.ovsdb s$i.db])
> -    }
> -    stop_server() {
> -        local i=$1
> -        printf "\ns$i: stopping\n"
> -        OVS_APP_EXIT_AND_WAIT_BY_TARGET([`pwd`/s$i], [s$i.pid])
> -    }
> -    connect_server() {
> -        local i=$1
> -        printf "\ns$i: waiting to connect to storage\n"
> -        AT_CHECK([ovsdb_client_wait --log-file=connect$i.log
> unix:s$i.ovsdb $schema connected])
> -    }
> -    remove_server() {
> -        local i=$1
> -        printf "\ns$i: removing from cluster\n"
> -        AT_CHECK([ovs-appctl --timeout=30 -t "`pwd`"/s$i cluster/leave
> OVN_Southbound])
> -        printf "\ns$i: waiting for removal to complete\n"
> -        AT_CHECK([ovsdb_client_wait --log-file=remove$i.log
> unix:s$i.ovsdb $schema removed])
> -        stop_server $i
> -    }
> -    add_server() {
> -        local i=$1
> -        rm s$i.db
> -        join_cluster $i
> -        start_server $i
> -        connect_server $i
> -    }
> -
> -    cid=`ovsdb-tool db-cid s1.db`
> -    for i in `seq 2 $n`; do join_cluster $i; done
> -
> -    on_exit 'kill `cat *.pid`'
> -    for i in `seq $n`; do start_server $i; done
> -    for i in `seq $n`; do connect_server $i; done
> -
> -    OVN_SB_DB=unix:s1.ovsdb
> -    for i in `seq 2 $n`; do
> -        OVN_SB_DB=$OVN_SB_DB,unix:s$i.ovsdb
> -    done
> -    export OVN_SB_DB
> -
> -    n1=10 n2=5 n3=50
> -    echo "starting $n1*$n2 ovn-sbctl processes..."
> -    for i in $(seq 0 $(expr $n1 - 1) ); do
> -        (for j in $(seq $n2); do
> -             : > $i-$j.running
> -             txn="add SB_Global . external_ids $i-$j=$i-$j"
> -             for k in $(seq $n3); do
> -                 txn="$txn -- add SB_Global . external_ids
> $i-$j-$k=$i-$j-$k"
> -             done
> -             run_as "ovn-sbctl($i-$j)" ovn-sbctl
> "-vPATTERN:console:ovn-sbctl($i-$j)|%D{%H:%M:%S}|%05N|%c|%p|%m"
> --log-file=$i-$j.log -vfile -vsyslog:off -vtimeval:off --timeout=120
> --no-leader-only $txn
> -             status=$?
> -             if test $status != 0; then
> -                 echo "$i-$j exited with status $status" > $i-$j:$status
> -             fi
> -             rm $i-$j.running
> -         done
> -         : > $i.done)&
> -    done
> -    echo "...done"
> -
> -    echo "waiting for ovn-sbctl processes to exit..."
> -    # Use file instead of var because code inside "while" runs in a
> subshell.
> -    echo 0 > phase
> -    i=0
> -    (while :; do echo; sleep 0.1; done) | while read REPLY; do
> -        printf "t=%2d s:" $i
> -        done=0
> -        for j in $(seq 0 $(expr $n1 - 1)); do
> -            if test -f $j.done; then
> -                printf " $j"
> -                done=$(expr $done + 1)
> -            fi
> -        done
> -        printf '\n'
> -        if test $done = $n1; then
> -            break
> -        fi
> -
> -        case $(cat phase) in # (
> -        0)
> -            if test $done -ge $(expr $n1 / 10); then
> -                if test $variant = kill; then
> -                    stop_server $victim
> -                else
> -                    remove_server $victim
> -                fi
> -                echo 1 > phase
> -                next=$(expr $i + 2)
> -            fi
> -            ;; # (
> -        1)
> -            if test $i -ge $next; then
> -                if test $variant = kill; then
> -                    start_server $victim
> -                    connect_server $victim
> -                else
> -                    add_server $victim
> -                fi
> -                echo 2 > phase
> -            fi
> -            ;;
> -        esac
> -
> -        i=$(expr $i + 1)
> -    done
> -    echo "...done"
> -    AT_CHECK([if test $(cat phase) != 2; then exit 77; fi])
> -
> -    for i in $(seq 0 $(expr $n1 - 1) ); do
> -        for j in `seq $n2`; do
> -            echo "$i-$j=$i-$j"
> -            for k in `seq $n3`; do
> -                echo "$i-$j-$k=$i-$j-$k"
> -            done
> -        done
> -    done | sort > expout
> -    AT_CHECK([ovn-sbctl --timeout=30 --log-file=finalize.log
> -vtimeval:off -vfile -vsyslog:off --bare get SB_Global . external-ids | tr
> ',' '\n' | sed 's/[[{}"" ]]//g' | sort], [0], [expout])
> -
> -    for i in `seq $n`; do
> -        if test $i != $victim || test $(cat phase) != 1; then
> -            stop_server $i
> -        fi
> -    done
> -
> -    # We ignore stdout because non-fatal warnings get printed there.
> -    AT_CHECK([ovsdb-tool check-cluster s*.db], [0], [ignore])
> -}
> -OVS_END_SHELL_HELPERS
> -
> -AT_SETUP([OVSDB 3-server torture test - kill/restart leader])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 1 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 3-server torture test - kill/restart follower 1])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 2 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 3-server torture test - kill/restart follower 2])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 3 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - kill/restart leader])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 1 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - kill/restart follower 1])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 2 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - kill/restart follower 2])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 3 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - kill/restart follower 3])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 4 kill
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - kill/restart follower 4])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 5 kill
> -AT_CLEANUP
> -
> -AT_SETUP([OVSDB 3-server torture test - remove/re-add leader])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 1 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 3-server torture test - remove/re-add follower 1])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 2 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 3-server torture test - remove/re-add follower 2])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster3])
> -ovsdb_torture_test 3 3 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - remove/re-add leader])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 1 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - remove/re-add follower 1])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 2 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - remove/re-add follower 2])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 3 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - remove/re-add follower 3])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 4 remove
> -AT_CLEANUP
> -AT_SETUP([OVSDB 5-server torture test - remove/re-add follower 4])
> -AT_KEYWORDS([ovsdb server positive unix cluster cluster5])
> -ovsdb_torture_test 5 5 remove
> -AT_CLEANUP
> diff --git a/tests/system-kmod-testsuite.at b/tests/
> system-kmod-testsuite.at
> index 2fe2e8f94..3de0290c0 100644
> --- a/tests/system-kmod-testsuite.at
> +++ b/tests/system-kmod-testsuite.at
> @@ -19,11 +19,9 @@ m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
>  m4_include([tests/ovs-macros.at])
>  m4_include([tests/ovsdb-macros.at])
>  m4_include([tests/ofproto-macros.at])
> -m4_include([tests/ovn-macros.at])
>  m4_include([tests/system-common-macros.at])
>  m4_include([tests/system-kmod-macros.at])
>
>  m4_include([tests/system-traffic.at])
>  m4_include([tests/system-layer3-tunnels.at])
> -m4_include([tests/system-ovn.at])
>  m4_include([tests/system-interface.at])
> diff --git a/tests/system-ovn.at b/tests/system-ovn.at
> deleted file mode 100644
> index 10fbd2649..000000000
> --- a/tests/system-ovn.at
> +++ /dev/null
> @@ -1,1663 +0,0 @@
> -AT_BANNER([system-ovn])
> -
> -AT_SETUP([ovn -- 2 LRs connected via LS, gateway router, SNAT and DNAT])
> -AT_KEYWORDS([ovnnat])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and
> -# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24)
> connected
> -# to it.  R2 is a gateway router on which we add NAT rules.
> -#
> -#    foo -- R1 -- join - R2 -- alice
> -#           |
> -#    bar ----
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis=hv1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Static routes.
> -ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \
> -"192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2"
> -
> -# Add a DNAT rule.
> -ovn-nbctl -- --id=@nat create nat type="dnat" logical_ip=192.168.1.2 \
> -    external_ip=30.0.0.2 -- add logical_router R2 nat @nat
> -
> -# Add a SNAT rule
> -ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.2.2 \
> -    external_ip=30.0.0.1 -- add logical_router R2 nat @nat
> -
> -# wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep 'nat(src=30.0.0.1)'])
> -
> -# 'alice1' should be able to ping 'foo1' directly.
> -NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 192.168.1.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# North-South DNAT: 'alice1' should also be able to ping 'foo1' via
> 30.0.0.2
> -NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# Check conntrack entries.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.2,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# South-North SNAT: 'bar1' pings 'alice1'. But 'alice1' receives traffic
> -# from 30.0.0.1
> -NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.2.2,dst=172.16.1.2,id=<cleared>,type=8,code=0),reply=(src=172.16.1.2,dst=30.0.0.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# Add static routes to handle east-west NAT.
> -ovn-nbctl lr-route-add R1 30.0.0.0/24 20.0.0.2
> -
> -# wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -
> -# Flush conntrack entries for easier output parsing of next test.
> -AT_CHECK([ovs-appctl dpctl/flush-conntrack])
> -
> -# East-west DNAT and SNAT: 'bar1' pings 30.0.0.2. 'foo1' receives it.
> -NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 | FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# As we have a static route that sends all packets with destination
> -# 30.0.0.2 to R2, it hits the DNAT rule and converts 30.0.0.2 to
> 192.168.1.2
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.2.2,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=192.168.2.2,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# As we have a SNAT rule that converts 192.168.2.2 to 30.0.0.1, the
> source is
> -# SNATted and 'foo1' receives it.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.2.2,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=30.0.0.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 LRs connected via LS, gateway router, easy SNAT])
> -AT_KEYWORDS([ovnnat])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) connected
> -# to it.  R2 has alice (172.16.1.0/24) connected to it.
> -# R2 is a gateway router on which we add NAT rules.
> -#
> -#    foo -- R1 -- join - R2 -- alice
> -
> -ovn-nbctl lr-add R1
> -ovn-nbctl lr-add R2 -- set Logical_Router R2 options:chassis=hv1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add join
> -
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -
> -# Connect foo to R1
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect alice to R2
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Static routes.
> -ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Add a SNAT rule
> -ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.1.2 \
> -    external_ip=172.16.1.1 -- add logical_router R2 nat @nat
> -
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep 'nat(src=172.16.1.1)'])
> -
> -# South-North SNAT: 'foo1' pings 'alice1'. But 'alice1' receives traffic
> -# from 172.16.1.1
> -NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=8,code=0),reply=(src=172.16.1.2,dst=172.16.1.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- multiple gateway routers, SNAT and DNAT])
> -AT_KEYWORDS([ovnnat])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Three LRs - R1, R2 and R3 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and
> -# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24)
> connected
> -# to it.  R3 has bob (172.16.1.0/24) connected to it. Note how both
> alice and
> -# bob have the same subnet behind it.  We are trying to simulate external
> -# network via those 2 switches. In real world the switch ports of these
> -# switches will have addresses set as "unknown" to make them learning
> switches.
> -# Or those switches will be "localnet" ones.
> -#
> -#    foo -- R1 -- join - R2 -- alice
> -#           |          |
> -#    bar ----          - R3 --- bob
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis=hv1
> -ovn-nbctl create Logical_Router name=R3 options:chassis=hv1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add bob
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect bob to R3
> -ovn-nbctl lrp-add R3 bob 00:00:03:01:02:03 172.16.1.2/24
> -ovn-nbctl lsp-add bob rp-bob -- set Logical_Switch_Port rp-bob \
> -    type=router options:router-port=bob addresses=\"00:00:03:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Connect R3 to join
> -ovn-nbctl lrp-add R3 R3_join 00:00:04:01:02:05 20.0.0.3/24
> -ovn-nbctl lsp-add join r3-join -- set Logical_Switch_Port r3-join \
> -    type=router options:router-port=R3_join
> addresses='"00:00:04:01:02:05"'
> -
> -# Install static routes with source ip address as the policy for routing.
> -# We want traffic from 'foo' to go via R2 and traffic of 'bar' to go via
> R3.
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.1.0/24 20.0.0.2
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.2.0/24 20.0.0.3
> -
> -# Static routes.
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -ovn-nbctl lr-route-add R3 192.168.0.0/16 20.0.0.1
> -
> -# For gateway routers R2 and R3, set a force SNAT rule.
> -ovn-nbctl set logical_router R2 options:dnat_force_snat_ip=20.0.0.2
> -ovn-nbctl set logical_router R3 options:dnat_force_snat_ip=20.0.0.3
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.3/24", "f0:00:00:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \
> -"192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2"
> -
> -# Logical port 'bob1' in switch 'bob'.
> -ADD_NAMESPACES(bob1)
> -ADD_VETH(bob1, bob1, br-int, "172.16.1.4/24", "f0:00:00:01:02:06", \
> -         "172.16.1.2")
> -ovn-nbctl lsp-add bob bob1 \
> --- lsp-set-addresses bob1 "f0:00:00:01:02:06 172.16.1.4"
> -
> -# Router R2
> -# Add a DNAT rule.
> -ovn-nbctl -- --id=@nat create nat type="dnat" logical_ip=192.168.1.2 \
> -    external_ip=30.0.0.2 -- add logical_router R2 nat @nat
> -
> -# Add a SNAT rule
> -ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.1.2 \
> -    external_ip=30.0.0.1 -- add logical_router R2 nat @nat
> -
> -# Router R3
> -# Add a DNAT rule.
> -ovn-nbctl -- --id=@nat create nat type="dnat" logical_ip=192.168.1.2 \
> -    external_ip=30.0.0.3 -- add logical_router R3 nat @nat
> -
> -# Add a SNAT rule
> -ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.2.2 \
> -    external_ip=30.0.0.4 -- add logical_router R3 nat @nat
> -
> -# wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep 'nat(src=30.0.0.4)'])
> -
> -# North-South DNAT: 'alice1' should be able to ping 'foo1' via 30.0.0.2
> -NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# Check conntrack entries.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.3) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.3,dst=30.0.0.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.3,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# But foo1 should receive traffic from 20.0.0.2
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(20.0.0.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.3,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=20.0.0.2,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# North-South DNAT: 'bob1' should be able to ping 'foo1' via 30.0.0.3
> -NS_CHECK_EXEC([bob1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.3 | FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# Check conntrack entries.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.4) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.4,dst=30.0.0.3,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.4,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# But foo1 should receive traffic from 20.0.0.3
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(20.0.0.3) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.4,dst=192.168.1.2,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=20.0.0.3,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# South-North SNAT: 'bar1' pings 'bob1'. But 'bob1' receives traffic
> -# from 30.0.0.4
> -NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.4 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.4) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.2.2,dst=172.16.1.4,id=<cleared>,type=8,code=0),reply=(src=172.16.1.4,dst=30.0.0.4,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# South-North SNAT: 'foo1' pings 'alice1'. But 'alice1' receives traffic
> -# from 30.0.0.1
> -NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.3 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.1.2,dst=172.16.1.3,id=<cleared>,type=8,code=0),reply=(src=172.16.1.3,dst=30.0.0.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- load-balancing])
> -AT_KEYWORDS([ovnlb])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# 2 logical switches "foo" (192.168.1.0/24) and "bar" (172.16.1.0/24)
> -# connected to a router R1.
> -# foo has foo1 to act as a client.
> -# bar has bar1, bar2, bar3 to act as servers.
> -#
> -# Loadbalancer VIPs in 30.0.0.0/24 network.
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 172.16.1.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Create logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Create logical ports 'bar1', 'bar2', 'bar3' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "172.16.1.2/24", "f0:00:0f:01:02:03", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:0f:01:02:03 172.16.1.2"
> -
> -ADD_NAMESPACES(bar2)
> -ADD_VETH(bar2, bar2, br-int, "172.16.1.3/24", "f0:00:0f:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add bar bar2 \
> --- lsp-set-addresses bar2 "f0:00:0f:01:02:04 172.16.1.3"
> -
> -ADD_NAMESPACES(bar3)
> -ADD_VETH(bar3, bar3, br-int, "172.16.1.4/24", "f0:00:0f:01:02:05", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add bar bar3 \
> --- lsp-set-addresses bar3 "f0:00:0f:01:02:05 172.16.1.4"
> -
> -# Config OVN load-balancer with a VIP.
> -uuid=`ovn-nbctl  create load_balancer
> vips:30.0.0.1="172.16.1.2,172.16.1.3,172.16.1.4"`
> -ovn-nbctl set logical_switch foo load_balancer=$uuid
> -
> -# Create another load-balancer with another VIP.
> -uuid=`ovn-nbctl create load_balancer
> vips:30.0.0.3="172.16.1.2,172.16.1.3,172.16.1.4"`
> -ovn-nbctl add logical_switch foo load_balancer $uuid
> -
> -# Config OVN load-balancer with another VIP (this time with ports).
> -ovn-nbctl set load_balancer $uuid vips:'"30.0.0.2:8000"'='"172.16.1.2:80,
> 172.16.1.3:80,172.16.1.4:80"'
> -
> -# Wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
> -grep 'nat(dst=172.16.1.4:80)'])
> -
> -# Start webservers in 'bar1', 'bar2' and 'bar3'.
> -OVS_START_L7([bar1], [http])
> -OVS_START_L7([bar2], [http])
> -OVS_START_L7([bar3], [http])
> -
> -dnl Should work with the virtual IP 30.0.0.1 address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([foo1], [wget 30.0.0.1 -t 5 -T 1 --retry-connrefused -v
> -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.2,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.4,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Should work with the virtual IP 30.0.0.3 address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([foo1], [wget 30.0.0.3 -t 5 -T 1 --retry-connrefused -v
> -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.3) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.3,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.2,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.3,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.3,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.4,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Test load-balancing that includes L4 ports in NAT.
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([foo1], [wget 30.0.0.2:8000 -t 5 -T 1
> --retry-connrefused -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.2,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=172.16.1.4,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- load-balancing - same subnet.])
> -AT_KEYWORDS([ovnlb])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# 1 logical switch "foo" (192.168.1.0/24) connected to router R1.
> -# foo has foo1, foo2, foo3, foo4 as logical ports.
> -#
> -# Loadbalancer VIPs in 30.0.0.0/24 network. Router is needed for default
> -# gateway. We will test load-balancing with foo1 as a client and foo2,
> foo3 and
> -# foo4 as servers.
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl ls-add foo
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Create logical port 'foo1', 'foo2', 'foo3' and 'foo4' in switch 'foo'.
> -ADD_NAMESPACES(foo1, foo2, foo3, foo4)
> -for i in `seq 1 4`; do
> -    j=`expr $i + 1`
> -    ADD_VETH(foo$i, foo$i, br-int, "192.168.1.$j/24",
> "f0:00:00:01:02:0$j", \
> -             "192.168.1.1")
> -    ovn-nbctl lsp-add foo foo$i \
> -        -- lsp-set-addresses foo$i "f0:00:00:01:02:0$j 192.168.1.$j"
> -done
> -
> -# Config OVN load-balancer with a VIP.
> -uuid=`ovn-nbctl  create load_balancer
> vips:30.0.0.1="192.168.1.3,192.168.1.4,192.168.1.5"`
> -ovn-nbctl set logical_switch foo load_balancer=$uuid
> -
> -# Config OVN load-balancer with another VIP (this time with ports).
> -ovn-nbctl set load_balancer $uuid vips:'"30.0.0.2:8000"'='"192.168.1.3:80
> ,192.168.1.4:80,192.168.1.5:80"'
> -
> -# Wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
> -grep 'nat(dst=192.168.1.5:80)'])
> -
> -# Start webservers in 'foo2', 'foo3' and 'foo4'.
> -OVS_START_L7([foo2], [http])
> -OVS_START_L7([foo3], [http])
> -OVS_START_L7([foo4], [http])
> -
> -dnl Should work with the virtual IP address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([foo1], [wget 30.0.0.1 -t 5 -T 1 --retry-connrefused -v
> -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.4,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.5,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Test load-balancing that includes L4 ports in NAT.
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([foo1], [wget 30.0.0.2:8000 -t 5 -T 1
> --retry-connrefused -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.4,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=192.168.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.5,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- load balancing in gateway router])
> -AT_KEYWORDS([ovnlb])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Two LRs - R1 and R2 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and
> -# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24)
> connected
> -# to it.  R2 is a gateway router on which we add load-balancing rules.
> -#
> -#    foo -- R1 -- join - R2 -- alice
> -#           |
> -#    bar ----
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis=hv1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Static routes.
> -ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \
> -"192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2"
> -
> -# Config OVN load-balancer with a VIP.
> -uuid=`ovn-nbctl  create load_balancer
> vips:30.0.0.1="192.168.1.2,192.168.2.2"`
> -ovn-nbctl set logical_router R2 load_balancer=$uuid
> -
> -# Config OVN load-balancer with another VIP (this time with ports).
> -ovn-nbctl set load_balancer $uuid vips:'"30.0.0.2:8000"'='"192.168.1.2:80
> ,192.168.2.2:80"'
> -
> -# Add SNAT rule to make sure that Load-balancing still works with a SNAT
> rule.
> -ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.2.2 \
> -    external_ip=30.0.0.2 -- add logical_router R2 nat @nat
> -
> -
> -# Wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
> -grep 'nat(dst=192.168.2.2:80)'])
> -
> -# Start webservers in 'foo1', 'bar1'.
> -OVS_START_L7([foo1], [http])
> -OVS_START_L7([bar1], [http])
> -
> -dnl Should work with the virtual IP address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([alice1], [wget 30.0.0.1 -t 5 -T 1 --retry-connrefused
> -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.2,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Test load-balancing that includes L4 ports in NAT.
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([alice1], [wget 30.0.0.2:8000 -t 5 -T 1
> --retry-connrefused -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.2) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.2,dst=30.0.0.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- multiple gateway routers, load-balancing])
> -AT_KEYWORDS([ovnlb])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Three LRs - R1, R2 and R3 that are connected to each other via LS "join"
> -# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and
> -# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24)
> connected
> -# to it.  R3 has bob (172.16.1.0/24) connected to it. Note how both
> alice and
> -# bob have the same subnet behind it.  We are trying to simulate external
> -# network via those 2 switches. In real world the switch ports of these
> -# switches will have addresses set as "unknown" to make them learning
> switches.
> -# Or those switches will be "localnet" ones.
> -#
> -#    foo -- R1 -- join - R2 -- alice
> -#           |          |
> -#    bar ----          - R3 --- bob
> -
> -ovn-nbctl create Logical_Router name=R1
> -ovn-nbctl create Logical_Router name=R2 options:chassis=hv1
> -ovn-nbctl create Logical_Router name=R3 options:chassis=hv1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -ovn-nbctl ls-add bob
> -ovn-nbctl ls-add join
> -
> -# Connect foo to R1
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> -
> -# Connect bar to R1
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> -
> -# Connect alice to R2
> -ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice addresses=\"00:00:02:01:02:03\"
> -
> -# Connect bob to R3
> -ovn-nbctl lrp-add R3 bob 00:00:03:01:02:03 172.16.1.2/24
> -ovn-nbctl lsp-add bob rp-bob -- set Logical_Switch_Port rp-bob \
> -    type=router options:router-port=bob addresses=\"00:00:03:01:02:03\"
> -
> -# Connect R1 to join
> -ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> -ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> -    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> -
> -# Connect R2 to join
> -ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> -ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> -    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> -
> -# Connect R3 to join
> -ovn-nbctl lrp-add R3 R3_join 00:00:04:01:02:05 20.0.0.3/24
> -ovn-nbctl lsp-add join r3-join -- set Logical_Switch_Port r3-join \
> -    type=router options:router-port=R3_join
> addresses='"00:00:04:01:02:05"'
> -
> -# Install static routes with source ip address as the policy for routing.
> -# We want traffic from 'foo' to go via R2 and traffic of 'bar' to go via
> R3.
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.1.0/24 20.0.0.2
> -ovn-nbctl --policy="src-ip" lr-route-add R1 192.168.2.0/24 20.0.0.3
> -
> -# Static routes.
> -ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> -ovn-nbctl lr-route-add R3 192.168.0.0/16 20.0.0.1
> -
> -# For gateway routers R2 and R3, set a force SNAT rule.
> -ovn-nbctl set logical_router R2 options:lb_force_snat_ip=20.0.0.2
> -ovn-nbctl set logical_router R3 options:lb_force_snat_ip=20.0.0.3
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.3/24", "f0:00:00:01:02:04", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.3"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \
> -"192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2"
> -
> -# Logical port 'bob1' in switch 'bob'.
> -ADD_NAMESPACES(bob1)
> -ADD_VETH(bob1, bob1, br-int, "172.16.1.4/24", "f0:00:00:01:02:06", \
> -         "172.16.1.2")
> -ovn-nbctl lsp-add bob bob1 \
> --- lsp-set-addresses bob1 "f0:00:00:01:02:06 172.16.1.4"
> -
> -# Config OVN load-balancer with a VIP.
> -uuid=`ovn-nbctl  create load_balancer
> vips:30.0.0.1="192.168.1.2,192.168.2.2"`
> -ovn-nbctl set logical_router R2 load_balancer=$uuid
> -ovn-nbctl set logical_router R3 load_balancer=$uuid
> -
> -# Wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
> -grep 'nat(dst=192.168.2.2)'])
> -
> -# Start webservers in 'foo1', 'bar1'.
> -OVS_START_L7([foo1], [http])
> -OVS_START_L7([bar1], [http])
> -
> -dnl Should work with the virtual IP address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([alice1], [wget 30.0.0.1 -t 5 -T 1 --retry-connrefused
> -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.3,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.3,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.3,dst=30.0.0.1,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.3,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Force SNAT should have worked.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(20.0.0) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.3,dst=192.168.1.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=20.0.0.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.3,dst=192.168.2.2,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=20.0.0.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- load balancing in router with gateway router port])
> -AT_KEYWORDS([ovnlb])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> -# alice is the router gateway port where the R1 LB rules are applied.
> -#
> -#    foo -- R1 -- bar
> -#           |
> -#    alice ----
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> -    -- set Logical_Router_Port alice options:redirect-chassis=hv1
> -
> -# Connect foo to R1
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect bar to R1
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar \
> -    -- lsp-set-addresses rp-bar router
> -
> -# Connect alice to R1
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'foo2' in switch 'foo'.
> -ADD_NAMESPACES(foo2)
> -ADD_VETH(foo2, foo2, br-int, "192.168.1.3/24", "f0:00:00:01:02:06", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo2 \
> --- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:04", \
> -         "192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:04 192.168.2.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:05", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> -
> -# Config OVN load-balancer with a VIP.
> -uuid=`ovn-nbctl  create load_balancer
> vips:172.16.1.10="192.168.1.2,192.168.2.2"`
> -ovn-nbctl set logical_router R1 load_balancer=$uuid
> -
> -# Config OVN load-balancer with another VIP (this time with ports).
> -ovn-nbctl set load_balancer $uuid vips:'"172.16.1.11:8000"'='"
> 192.168.1.2:80,192.168.2.2:80"'
> -
> -# Wait for ovn-controller to catch up.
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
> -grep 'nat(dst=192.168.2.2:80)'])
> -
> -# Start webservers in 'foo1', 'bar1'.
> -OVS_START_L7([foo1], [http])
> -OVS_START_L7([bar1], [http])
> -
> -dnl Should work with the virtual IP address through NAT
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([alice1], [wget 172.16.1.10 -t 5 -T 1
> --retry-connrefused -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.10) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -dnl Test load-balancing that includes L4 ports in NAT.
> -for i in `seq 1 20`; do
> -    echo Request $i
> -    NS_CHECK_EXEC([alice1], [wget 172.16.1.11:8000 -t 5 -T 1
> --retry-connrefused -v -o wget$i.log])
> -done
> -
> -dnl Each server should have at least one connection.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.11) |
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
>
> -tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- DNAT and SNAT on distributed router - N/S])
> -AT_KEYWORDS([ovnnat])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> -# alice is the router gateway port where the R1 NAT rules are applied.
> -#
> -#    foo -- R1 -- alice
> -#           |
> -#    bar ----
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> -    -- set Logical_Router_Port alice options:redirect-chassis=hv1
> -
> -# Connect foo to R1
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect bar to R1
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar \
> -    -- lsp-set-addresses rp-bar router
> -
> -# Connect alice to R1
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'foo2' in switch 'foo'.
> -ADD_NAMESPACES(foo2)
> -ADD_VETH(foo2, foo2, br-int, "192.168.1.3/24", "f0:00:00:01:02:06", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo2 \
> --- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:04", \
> -         "192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:04 192.168.2.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:05", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> -
> -# Add DNAT rules
> -AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.3 192.168.1.2
> foo1 00:00:02:02:03:04])
> -AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.4 192.168.1.3
> foo2 00:00:02:02:03:05])
> -
> -# Add a SNAT rule
> -AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.1.1 192.168.0.0/16])
> -
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep 'nat(src=172.16.1.1)'])
> -
> -# North-South DNAT: 'alice1' pings 'foo1' using 172.16.1.3.
> -NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.3 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that DNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.3) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.2,dst=172.16.1.3,id=<cleared>,type=8,code=0),reply=(src=192.168.1.2,dst=172.16.1.2,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# South-North SNAT: 'foo2' pings 'alice1'. But 'alice1' receives traffic
> -# from 172.16.1.4
> -NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.4) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.1.3,dst=172.16.1.2,id=<cleared>,type=8,code=0),reply=(src=172.16.1.2,dst=172.16.1.4,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# South-North SNAT: 'bar1' pings 'alice1'. But 'alice1' receives traffic
> -# from 172.16.1.1
> -NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that SNAT indeed happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=192.168.2.2,dst=172.16.1.2,id=<cleared>,type=8,code=0),reply=(src=172.16.1.2,dst=172.16.1.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- DNAT and SNAT on distributed router - E/W])
> -AT_KEYWORDS([ovnnat])
> -
> -CHECK_CONNTRACK()
> -CHECK_CONNTRACK_NAT()
> -ovn_start
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
> -# and alice (172.16.1.0/24) connected to it.  The port between R1 and
> -# alice is the router gateway port where the R1 NAT rules are applied.
> -#
> -#    foo -- R1 -- alice
> -#           |
> -#    bar ----
> -
> -ovn-nbctl lr-add R1
> -
> -ovn-nbctl ls-add foo
> -ovn-nbctl ls-add bar
> -ovn-nbctl ls-add alice
> -
> -ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> -ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> -ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
> -    -- set Logical_Router_Port alice options:redirect-chassis=hv1
> -
> -# Connect foo to R1
> -ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> -    type=router options:router-port=foo \
> -    -- lsp-set-addresses rp-foo router
> -
> -# Connect bar to R1
> -ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> -    type=router options:router-port=bar \
> -    -- lsp-set-addresses rp-bar router
> -
> -# Connect alice to R1
> -ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> -    type=router options:router-port=alice \
> -    -- lsp-set-addresses rp-alice router
> -
> -# Logical port 'foo1' in switch 'foo'.
> -ADD_NAMESPACES(foo1)
> -ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo1 \
> --- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> -
> -# Logical port 'foo2' in switch 'foo'.
> -ADD_NAMESPACES(foo2)
> -ADD_VETH(foo2, foo2, br-int, "192.168.1.3/24", "f0:00:00:01:02:06", \
> -         "192.168.1.1")
> -ovn-nbctl lsp-add foo foo2 \
> --- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
> -
> -# Logical port 'bar1' in switch 'bar'.
> -ADD_NAMESPACES(bar1)
> -ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:04", \
> -         "192.168.2.1")
> -ovn-nbctl lsp-add bar bar1 \
> --- lsp-set-addresses bar1 "f0:00:00:01:02:04 192.168.2.2"
> -
> -# Logical port 'alice1' in switch 'alice'.
> -ADD_NAMESPACES(alice1)
> -ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:05", \
> -         "172.16.1.1")
> -ovn-nbctl lsp-add alice alice1 \
> --- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
> -
> -# Add DNAT rules
> -AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.3 192.168.1.2
> foo1 00:00:02:02:03:04])
> -AT_CHECK([ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.4 192.168.2.2
> bar1 00:00:02:02:03:05])
> -
> -# Add a SNAT rule
> -AT_CHECK([ovn-nbctl lr-nat-add R1 snat 172.16.1.1 192.168.0.0/16])
> -
> -ovn-nbctl --wait=hv sync
> -OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep 'nat(src=172.16.1.1)'])
> -
> -echo "------ hv dump ------"
> -ovs-ofctl show br-int
> -ovs-ofctl dump-flows br-int
> -echo "---------------------"
> -
> -# East-West No NAT: 'foo1' pings 'bar1' using 192.168.2.2.
> -NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 192.168.2.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that no NAT happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(192.168.2.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/' | wc -l], [0], [0
> -])
> -
> -# East-West No NAT: 'foo2' pings 'bar1' using 192.168.2.2.
> -NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 192.168.2.2 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that no NAT happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(192.168.2.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/' | wc -l], [0], [0
> -])
> -
> -# East-West No NAT: 'bar1' pings 'foo2' using 192.168.1.3.
> -NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 192.168.1.3 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# We verify that no NAT happened via 'dump-conntrack' command.
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(192.168.2.2) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/' | wc -l], [0], [0
> -])
> -
> -# East-West NAT: 'foo1' pings 'bar1' using 172.16.1.4.
> -NS_CHECK_EXEC([foo1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.4 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# Check conntrack entries.  First SNAT of 'foo1' address happens.
> -# Then DNAT of 'bar1' address happens (listed first below).
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.3) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.3,dst=172.16.1.4,id=<cleared>,type=8,code=0),reply=(src=192.168.2.2,dst=172.16.1.3,id=<cleared>,type=0,code=0),zone=<cleared>
>
> -icmp,orig=(src=192.168.1.2,dst=172.16.1.4,id=<cleared>,type=8,code=0),reply=(src=172.16.1.4,dst=172.16.1.3,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -# East-West NAT: 'foo2' pings 'bar1' using 172.16.1.4.
> -NS_CHECK_EXEC([foo2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.4 |
> FORMAT_PING], \
> -[0], [dnl
> -3 packets transmitted, 3 received, 0% packet loss, time 0ms
> -])
> -
> -# Check conntrack entries.  First SNAT of 'foo2' address happens.
> -# Then DNAT of 'bar1' address happens (listed first below).
> -AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.1) | \
> -sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
>
> -icmp,orig=(src=172.16.1.1,dst=172.16.1.4,id=<cleared>,type=8,code=0),reply=(src=192.168.2.2,dst=172.16.1.1,id=<cleared>,type=0,code=0),zone=<cleared>
>
> -icmp,orig=(src=192.168.1.3,dst=172.16.1.4,id=<cleared>,type=8,code=0),reply=(src=172.16.1.4,dst=172.16.1.1,id=<cleared>,type=0,code=0),zone=<cleared>
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> -
> -AT_SETUP([ovn -- 2 LSs IGMP])
> -AT_KEYWORDS([ovnigmp])
> -
> -ovn_start
> -
> -OVS_TRAFFIC_VSWITCHD_START()
> -ADD_BR([br-int])
> -
> -# Set external-ids in br-int needed for ovn-controller
> -ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=hv1 \
> -        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -
> -# Start ovn-controller
> -start_daemon ovn-controller
> -
> -# Logical network:
> -# Two independent logical switches (sw1 and sw2).
> -# sw1:
> -#   - subnet 10.0.0.0/8
> -#   - 2 ports (sw1-p1 - sw1-p2)
> -# sw2:
> -#   - subnet 20.0.0.0/8
> -#   - 2 port (sw2-p1 - sw2-p2)
> -#   - IGMP Querier from 20.0.0.254
> -
> -ovn-nbctl ls-add sw1
> -ovn-nbctl ls-add sw2
> -
> -for i in `seq 1 2`
> -do
> -    ADD_NAMESPACES(sw1-p$i)
> -    ADD_VETH(sw1-p$i, sw1-p$i, br-int, "10.0.0.$i/24",
> "00:00:00:00:01:0$i", \
> -            "10.0.0.254")
> -    ovn-nbctl lsp-add sw1 sw1-p$i \
> -        -- lsp-set-addresses sw1-p$i "00:00:00:00:01:0$i 10.0.0.$i"
> -done
> -
> -for i in `seq 1 2`
> -do
> -    ADD_NAMESPACES(sw2-p$i)
> -    ADD_VETH(sw2-p$i, sw2-p$i, br-int, "20.0.0.$i/24",
> "00:00:00:00:02:0$i", \
> -            "20.0.0.254")
> -    ovn-nbctl lsp-add sw2 sw2-p$i \
> -        -- lsp-set-addresses sw2-p$i "00:00:00:00:02:0$i 20.0.0.$i"
> -done
> -
> -# Enable IGMP snooping on sw1.
> -ovn-nbctl set Logical_Switch sw1 other_config:mcast_querier="false"
> -ovn-nbctl set Logical_Switch sw1 other_config:mcast_snoop="true"
> -
> -# Inject IGMP Join for 239.0.1.68 on sw1-p1.
> -NS_CHECK_EXEC([sw1-p1], [ip addr add dev sw1-p1 239.0.1.68/32 autojoin],
> [0])
> -
> -# Inject IGMP Join for 239.0.1.68 on sw1-p2
> -NS_CHECK_EXEC([sw1-p2], [ip addr add dev sw1-p2 239.0.1.68/32 autojoin],
> [0])
> -
> -# Check that the IGMP Group is learned.
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    ports=`ovn-sbctl find IGMP_Group | grep ports | cut -f 2 -d ":" | wc
> -w`
> -    test "${total_entries}" = "1"
> -    test "${ports}" = "2"
> -])
> -
> -# Inject IGMP Leave for 239.0.1.68 on sw1-p2.
> -NS_CHECK_EXEC([sw1-p2], [ip addr del dev sw1-p2 239.0.1.68/32], [0])
> -
> -# Check that only one port is left in the group.
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    ports=`ovn-sbctl find IGMP_Group | grep ports | cut -f 2 -d ":" | wc
> -w`
> -    test "${total_entries}" = "1"
> -    test "${ports}" = "1"
> -])
> -
> -# Flush IGMP groups.
> -ovn-sbctl ip-multicast-flush sw1
> -ovn-nbctl --wait=hv -t 3 sync
> -OVS_WAIT_UNTIL([
> -    total_entries=`ovn-sbctl find IGMP_Group | grep "239.0.1.68" | wc -l`
> -    test "${total_entries}" = "0"
> -])
> -
> -# Enable IGMP snooping and querier on sw2 and set query interval to
> minimum.
> -ovn-nbctl set Logical_Switch sw2 \
> -    other_config:mcast_snoop="true" \
> -    other_config:mcast_querier="true" \
> -    other_config:mcast_query_interval=1 \
> -    other_config:mcast_eth_src="00:00:00:00:02:fe" \
> -    other_config:mcast_ip4_src="20.0.0.254"
> -
> -# Check that queries are generated.
> -NS_CHECK_EXEC([sw2-p1], [tcpdump -n -c 2 -i sw2-p1 igmp > sw2-p1.pcap &])
> -
> -OVS_WAIT_UNTIL([
> -    total_queries=`cat sw2-p1.pcap | grep "igmp query" | wc -l`
> -    test "${total_queries}" = "2"
> -])
> -
> -OVS_APP_EXIT_AND_WAIT([ovn-controller])
> -
> -as ovn-sb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as ovn-nb
> -OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> -
> -as northd
> -OVS_APP_EXIT_AND_WAIT([ovn-northd])
> -
> -as
> -OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> -/connection dropped.*/d"])
> -AT_CLEANUP
> diff --git a/tests/system-userspace-testsuite.at b/tests/
> system-userspace-testsuite.at
> index 4af36bef0..b40da9579 100644
> --- a/tests/system-userspace-testsuite.at
> +++ b/tests/system-userspace-testsuite.at
> @@ -19,12 +19,10 @@ m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
>  m4_include([tests/ovs-macros.at])
>  m4_include([tests/ovsdb-macros.at])
>  m4_include([tests/ofproto-macros.at])
> -m4_include([tests/ovn-macros.at])
>  m4_include([tests/system-userspace-macros.at])
>  m4_include([tests/system-common-macros.at])
>
>  m4_include([tests/system-traffic.at])
>  m4_include([tests/system-layer3-tunnels.at])
> -m4_include([tests/system-ovn.at])
>  m4_include([tests/system-interface.at])
>  m4_include([tests/system-userspace-packet-type-aware.at])
> diff --git a/tests/test-ovn.c b/tests/test-ovn.c
> deleted file mode 100644
> index 0b9e8246e..000000000
> --- a/tests/test-ovn.c
> +++ /dev/null
> @@ -1,1584 +0,0 @@
> -/*
> - * Copyright (c) 2015, 2016, 2017 Nicira, Inc.
> - *
> - * Licensed under the Apache License, Version 2.0 (the "License");
> - * you may not use this file except in compliance with the License.
> - * You may obtain a copy of the License at:
> - *
> - *     http://www.apache.org/licenses/LICENSE-2.0
> - *
> - * Unless required by applicable law or agreed to in writing, software
> - * distributed under the License is distributed on an "AS IS" BASIS,
> - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> - * See the License for the specific language governing permissions and
> - * limitations under the License.
> - */
> -
> -#include <config.h>
> -#include <errno.h>
> -#include <getopt.h>
> -#include <sys/wait.h>
> -
> -#include "command-line.h"
> -#include "dp-packet.h"
> -#include "fatal-signal.h"
> -#include "flow.h"
> -#include "openvswitch/dynamic-string.h"
> -#include "openvswitch/match.h"
> -#include "openvswitch/ofp-actions.h"
> -#include "openvswitch/ofpbuf.h"
> -#include "openvswitch/vlog.h"
> -#include "ovn/actions.h"
> -#include "ovn/expr.h"
> -#include "ovn/lex.h"
> -#include "ovn/logical-fields.h"
> -#include "ovn/lib/ovn-l7.h"
> -#include "ovn/lib/extend-table.h"
> -#include "ovs-thread.h"
> -#include "ovstest.h"
> -#include "openvswitch/shash.h"
> -#include "simap.h"
> -#include "util.h"
> -
> -/* --relops: Bitmap of the relational operators to test, in exhaustive
> test. */
> -static unsigned int test_relops;
> -
> -/* --nvars: Number of numeric variables to test, in exhaustive test. */
> -static int test_nvars = 2;
> -
> -/* --svars: Number of string variables to test, in exhaustive test. */
> -static int test_svars = 2;
> -
> -/* --bits: Number of bits per variable, in exhaustive test. */
> -static int test_bits = 3;
> -
> -/* --operation: The operation to test, in exhaustive test. */
> -static enum { OP_CONVERT, OP_SIMPLIFY, OP_NORMALIZE, OP_FLOW } operation
> -    = OP_FLOW;
> -
> -/* --parallel: Number of parallel processes to use in test. */
> -static int test_parallel = 1;
> -
> -/* -m, --more: Message verbosity */
> -static int verbosity;
> -
> -static void
> -compare_token(const struct lex_token *a, const struct lex_token *b)
> -{
> -    if (a->type != b->type) {
> -        fprintf(stderr, "type differs: %d -> %d\n", a->type, b->type);
> -        return;
> -    }
> -
> -    if (!((a->s && b->s && !strcmp(a->s, b->s))
> -          || (!a->s && !b->s))) {
> -        fprintf(stderr, "string differs: %s -> %s\n",
> -                a->s ? a->s : "(null)",
> -                b->s ? b->s : "(null)");
> -        return;
> -    }
> -
> -    if (a->type == LEX_T_INTEGER || a->type == LEX_T_MASKED_INTEGER) {
> -        if (memcmp(&a->value, &b->value, sizeof a->value)) {
> -            fprintf(stderr, "value differs\n");
> -            return;
> -        }
> -
> -        if (a->type == LEX_T_MASKED_INTEGER
> -            && memcmp(&a->mask, &b->mask, sizeof a->mask)) {
> -            fprintf(stderr, "mask differs\n");
> -            return;
> -        }
> -
> -        if (a->format != b->format
> -            && !(a->format == LEX_F_HEXADECIMAL
> -                 && b->format == LEX_F_DECIMAL
> -                 && a->value.integer == 0)) {
> -            fprintf(stderr, "format differs: %d -> %d\n",
> -                    a->format, b->format);
> -        }
> -    }
> -}
> -
> -static void
> -test_lex(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    struct ds input;
> -    struct ds output;
> -
> -    ds_init(&input);
> -    ds_init(&output);
> -    while (!ds_get_test_line(&input, stdin)) {
> -        struct lexer lexer;
> -
> -        lexer_init(&lexer, ds_cstr(&input));
> -        ds_clear(&output);
> -        while (lexer_get(&lexer) != LEX_T_END) {
> -            size_t len = output.length;
> -            lex_token_format(&lexer.token, &output);
> -
> -            /* Check that the formatted version can really be parsed back
> -             * losslessly. */
> -            if (lexer.token.type != LEX_T_ERROR) {
> -                const char *s = ds_cstr(&output) + len;
> -                struct lexer l2;
> -
> -                lexer_init(&l2, s);
> -                lexer_get(&l2);
> -                compare_token(&lexer.token, &l2.token);
> -                lexer_destroy(&l2);
> -            }
> -            ds_put_char(&output, ' ');
> -        }
> -        lexer_destroy(&lexer);
> -
> -        ds_chomp(&output, ' ');
> -        puts(ds_cstr(&output));
> -    }
> -    ds_destroy(&input);
> -    ds_destroy(&output);
> -}
> -
> -static void
> -create_symtab(struct shash *symtab)
> -{
> -    ovn_init_symtab(symtab);
> -
> -    /* For negative testing. */
> -    expr_symtab_add_field(symtab, "bad_prereq", MFF_XREG0, "xyzzy",
> false);
> -    expr_symtab_add_field(symtab, "self_recurse", MFF_XREG0,
> -                          "self_recurse != 0", false);
> -    expr_symtab_add_field(symtab, "mutual_recurse_1", MFF_XREG0,
> -                          "mutual_recurse_2 != 0", false);
> -    expr_symtab_add_field(symtab, "mutual_recurse_2", MFF_XREG0,
> -                          "mutual_recurse_1 != 0", false);
> -    expr_symtab_add_string(symtab, "big_string", MFF_XREG0, NULL);
> -}
> -
> -static void
> -create_gen_opts(struct hmap *dhcp_opts, struct hmap *dhcpv6_opts,
> -                struct hmap *nd_ra_opts,
> -                struct controller_event_options *event_opts)
> -{
> -    hmap_init(dhcp_opts);
> -    dhcp_opt_add(dhcp_opts, "offerip", 0, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "netmask", 1, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "router",  3, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "dns_server", 6, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "log_server", 7, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "lpr_server",  9, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "domain_name", 15, "str");
> -    dhcp_opt_add(dhcp_opts, "swap_server", 16, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "policy_filter", 21, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "router_solicitation",  32, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "nis_server", 41, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "ntp_server", 42, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "server_id",  54, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "tftp_server", 66, "ipv4");
> -    dhcp_opt_add(dhcp_opts, "classless_static_route", 121,
> "static_routes");
> -    dhcp_opt_add(dhcp_opts, "ip_forward_enable",  19, "bool");
> -    dhcp_opt_add(dhcp_opts, "router_discovery", 31, "bool");
> -    dhcp_opt_add(dhcp_opts, "ethernet_encap", 36, "bool");
> -    dhcp_opt_add(dhcp_opts, "default_ttl",  23, "uint8");
> -    dhcp_opt_add(dhcp_opts, "tcp_ttl", 37, "uint8");
> -    dhcp_opt_add(dhcp_opts, "mtu", 26, "uint16");
> -    dhcp_opt_add(dhcp_opts, "lease_time",  51, "uint32");
> -    dhcp_opt_add(dhcp_opts, "wpad", 252, "str");
> -    dhcp_opt_add(dhcp_opts, "bootfile_name", 67, "str");
> -    dhcp_opt_add(dhcp_opts, "path_prefix", 210, "str");
> -    dhcp_opt_add(dhcp_opts, "tftp_server_address", 150, "ipv4");
> -
> -    /* DHCPv6 options. */
> -    hmap_init(dhcpv6_opts);
> -    dhcp_opt_add(dhcpv6_opts, "server_id",  2, "mac");
> -    dhcp_opt_add(dhcpv6_opts, "ia_addr",  5, "ipv6");
> -    dhcp_opt_add(dhcpv6_opts, "dns_server",  23, "ipv6");
> -    dhcp_opt_add(dhcpv6_opts, "domain_search",  24, "str");
> -
> -    /* IPv6 ND RA options. */
> -    hmap_init(nd_ra_opts);
> -    nd_ra_opts_init(nd_ra_opts);
> -
> -    /* OVN controller events options. */
> -    controller_event_opts_init(event_opts);
> -}
> -
> -static void
> -create_addr_sets(struct shash *addr_sets)
> -{
> -    shash_init(addr_sets);
> -
> -    static const char *const addrs1[] = {
> -        "10.0.0.1", "10.0.0.2", "10.0.0.3",
> -    };
> -    static const char *const addrs2[] = {
> -        "::1", "::2", "::3",
> -    };
> -    static const char *const addrs3[] = {
> -        "00:00:00:00:00:01", "00:00:00:00:00:02", "00:00:00:00:00:03",
> -    };
> -    static const char *const addrs4[] = { NULL };
> -
> -    expr_const_sets_add(addr_sets, "set1", addrs1, 3, true);
> -    expr_const_sets_add(addr_sets, "set2", addrs2, 3, true);
> -    expr_const_sets_add(addr_sets, "set3", addrs3, 3, true);
> -    expr_const_sets_add(addr_sets, "set4", addrs4, 0, true);
> -}
> -
> -static void
> -create_port_groups(struct shash *port_groups)
> -{
> -    shash_init(port_groups);
> -
> -    static const char *const pg1[] = {
> -        "lsp1", "lsp2", "lsp3",
> -    };
> -    static const char *const pg2[] = { NULL };
> -
> -    expr_const_sets_add(port_groups, "pg1", pg1, 3, false);
> -    expr_const_sets_add(port_groups, "pg_empty", pg2, 0, false);
> -}
> -
> -static bool
> -lookup_port_cb(const void *ports_, const char *port_name, unsigned int
> *portp)
> -{
> -    const struct simap *ports = ports_;
> -    const struct simap_node *node = simap_find(ports, port_name);
> -    if (!node) {
> -        return false;
> -    }
> -    *portp = node->data;
> -    return true;
> -}
> -
> -static bool
> -is_chassis_resident_cb(const void *ports_, const char *port_name)
> -{
> -    const struct simap *ports = ports_;
> -    const struct simap_node *node = simap_find(ports, port_name);
> -    if (node) {
> -        return true;
> -    }
> -    return false;
> -}
> -
> -static void
> -test_parse_expr__(int steps)
> -{
> -    struct shash symtab;
> -    struct shash addr_sets;
> -    struct shash port_groups;
> -    struct simap ports;
> -    struct ds input;
> -
> -    create_symtab(&symtab);
> -    create_addr_sets(&addr_sets);
> -    create_port_groups(&port_groups);
> -
> -    simap_init(&ports);
> -    simap_put(&ports, "eth0", 5);
> -    simap_put(&ports, "eth1", 6);
> -    simap_put(&ports, "LOCAL", ofp_to_u16(OFPP_LOCAL));
> -    simap_put(&ports, "lsp1", 0x11);
> -    simap_put(&ports, "lsp2", 0x12);
> -    simap_put(&ports, "lsp3", 0x13);
> -
> -    ds_init(&input);
> -    while (!ds_get_test_line(&input, stdin)) {
> -        struct expr *expr;
> -        char *error;
> -
> -        expr = expr_parse_string(ds_cstr(&input), &symtab, &addr_sets,
> -                                 &port_groups, NULL, &error);
> -        if (!error && steps > 0) {
> -            expr = expr_annotate(expr, &symtab, &error);
> -        }
> -        if (!error) {
> -            if (steps > 1) {
> -                expr = expr_simplify(expr, is_chassis_resident_cb,
> &ports);
> -            }
> -            if (steps > 2) {
> -                expr = expr_normalize(expr);
> -                ovs_assert(expr_is_normalized(expr));
> -            }
> -        }
> -        if (!error) {
> -            if (steps > 3) {
> -                struct hmap matches;
> -
> -                expr_to_matches(expr, lookup_port_cb, &ports, &matches);
> -                expr_matches_print(&matches, stdout);
> -                expr_matches_destroy(&matches);
> -            } else {
> -                struct ds output = DS_EMPTY_INITIALIZER;
> -                expr_format(expr, &output);
> -                puts(ds_cstr(&output));
> -                ds_destroy(&output);
> -            }
> -        } else {
> -            puts(error);
> -            free(error);
> -        }
> -        expr_destroy(expr);
> -    }
> -    ds_destroy(&input);
> -
> -    simap_destroy(&ports);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -    expr_const_sets_destroy(&addr_sets);
> -    shash_destroy(&addr_sets);
> -    expr_const_sets_destroy(&port_groups);
> -    shash_destroy(&port_groups);
> -}
> -
> -static void
> -test_parse_expr(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    test_parse_expr__(0);
> -}
> -
> -static void
> -test_annotate_expr(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    test_parse_expr__(1);
> -}
> -
> -static void
> -test_simplify_expr(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    test_parse_expr__(2);
> -}
> -
> -static void
> -test_normalize_expr(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    test_parse_expr__(3);
> -}
> -
> -static void
> -test_expr_to_flows(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    test_parse_expr__(4);
> -}
> -
> -/* Print the symbol table. */
> -
> -static void
> -test_dump_symtab(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    struct shash symtab;
> -    create_symtab(&symtab);
> -
> -    const struct shash_node **nodes = shash_sort(&symtab);
> -    for (size_t i = 0; i < shash_count(&symtab); i++) {
> -        const struct expr_symbol *symbol = nodes[i]->data;
> -        struct ds s = DS_EMPTY_INITIALIZER;
> -        expr_symbol_format(symbol, &s);
> -        puts(ds_cstr(&s));
> -        ds_destroy(&s);
> -    }
> -
> -    free(nodes);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -/* Evaluate an expression. */
> -
> -static bool
> -lookup_atoi_cb(const void *aux OVS_UNUSED, const char *port_name,
> -               unsigned int *portp)
> -{
> -    *portp = atoi(port_name);
> -    return true;
> -}
> -
> -static void
> -test_evaluate_expr(struct ovs_cmdl_context *ctx)
> -{
> -    struct shash symtab;
> -    struct ds input;
> -
> -    ovn_init_symtab(&symtab);
> -
> -    struct flow uflow;
> -    char *error = expr_parse_microflow(ctx->argv[1], &symtab, NULL, NULL,
> -                                       lookup_atoi_cb, NULL, &uflow);
> -    if (error) {
> -        ovs_fatal(0, "%s", error);
> -    }
> -
> -    ds_init(&input);
> -    while (!ds_get_test_line(&input, stdin)) {
> -        struct expr *expr;
> -
> -        expr = expr_parse_string(ds_cstr(&input), &symtab, NULL, NULL,
> NULL,
> -                                 &error);
> -        if (!error) {
> -            expr = expr_annotate(expr, &symtab, &error);
> -        }
> -        if (!error) {
> -            printf("%d\n", expr_evaluate(expr, &uflow, lookup_atoi_cb,
> NULL));
> -        } else {
> -            puts(error);
> -            free(error);
> -        }
> -        expr_destroy(expr);
> -    }
> -    ds_destroy(&input);
> -
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -/* Compositions.
> - *
> - * The "compositions" of a positive integer N are all of the ways that
> one can
> - * add up positive integers to sum to N.  For example, the compositions
> of 3
> - * are 3, 2+1, 1+2, and 1+1+1.
> - *
> - * We use compositions to find all the ways to break up N terms of a
> Boolean
> - * expression into subexpressions.  Suppose we want to generate all
> expressions
> - * with 3 terms.  The compositions of 3 (ignoring 3 itself) provide the
> - * possibilities (x && x) || x, x || (x && x), and x || x || x.  (Of
> course one
> - * can exchange && for || in each case.)  One must recursively compose the
> - * sub-expressions whose values are 3 or greater; that is what the "tree
> shape"
> - * concept later covers.
> - *
> - * To iterate through all compositions of, e.g., 5:
> - *
> - *     unsigned int state;
> - *     int s[5];
> - *     int n;
> - *
> - *     for (n = first_composition(ARRAY_SIZE(s), &state, s); n > 0;
> - *          n = next_composition(&state, s, n)) {
> - *          // Do something with composition 's' with 'n' elements.
> - *     }
> - *
> - * Algorithm from D. E. Knuth, _The Art of Computer Programming, Vol. 4A:
> - * Combinatorial Algorithms, Part 1_, section 7.2.1.1, answer to exercise
> - * 12(a).
> - */
> -
> -/* Begins iteration through the compositions of 'n'.  Initializes 's' to
> the
> - * number of elements in the first composition of 'n' and returns that
> number
> - * of elements.  The first composition in fact is always 'n' itself, so
> the
> - * return value will be 1.
> - *
> - * Initializes '*state' to some internal state information.  The caller
> must
> - * maintain this state (and 's') for use by next_composition().
> - *
> - * 's' must have room for at least 'n' elements. */
> -static int
> -first_composition(int n, unsigned int *state, int s[])
> -{
> -    *state = 0;
> -    s[0] = n;
> -    return 1;
> -}
> -
> -/* Advances 's', with 'sn' elements, to the next composition and returns
> the
> - * number of elements in this new composition, or 0 if no compositions are
> - * left.  'state' is the same internal state passed to
> first_composition(). */
> -static int
> -next_composition(unsigned int *state, int s[], int sn)
> -{
> -    int j = sn - 1;
> -    if (++*state & 1) {
> -        if (s[j] > 1) {
> -            s[j]--;
> -            s[j + 1] = 1;
> -            j++;
> -        } else {
> -            j--;
> -            s[j]++;
> -        }
> -    } else {
> -        if (s[j - 1] > 1) {
> -            s[j - 1]--;
> -            s[j + 1] = s[j];
> -            s[j] = 1;
> -            j++;
> -        } else {
> -            j--;
> -            if (!j) {
> -                return 0;
> -            }
> -            s[j] = s[j + 1];
> -            s[j - 1]++;
> -        }
> -    }
> -    return j + 1;
> -}
> -
> -static void
> -test_composition(struct ovs_cmdl_context *ctx)
> -{
> -    int n = atoi(ctx->argv[1]);
> -    unsigned int state;
> -    int s[50];
> -
> -    for (int sn = first_composition(n, &state, s); sn;
> -         sn = next_composition(&state, s, sn)) {
> -        for (int i = 0; i < sn; i++) {
> -            printf("%d%c", s[i], i == sn - 1 ? '\n' : ' ');
> -        }
> -    }
> -}
> -
> -/* Tree shapes.
> - *
> - * This code generates all possible Boolean expressions with a specified
> number
> - * of terms N (equivalent to the number of external nodes in a tree).
> - *
> - * See test_tree_shape() for a simple example. */
> -
> -/* An array of these structures describes the shape of a tree.
> - *
> - * A single element of struct tree_shape describes a single node in the
> tree.
> - * The node has 'sn' direct children.  From left to right, for i in
> 0...sn-1,
> - * s[i] is 1 if the child is a leaf node, otherwise the child is a
> subtree and
> - * s[i] is the number of leaf nodes within that subtree.  In the latter
> case,
> - * the subtree is described by another struct tree_shape within the
> enclosing
> - * array.  The tree_shapes are ordered in the array in in-order.
> - */
> -struct tree_shape {
> -    unsigned int state;
> -    int s[50];
> -    int sn;
> -};
> -
> -static int
> -init_tree_shape__(struct tree_shape ts[], int n)
> -{
> -    if (n <= 2) {
> -        return 0;
> -    }
> -
> -    int n_tses = 1;
> -    /* Skip the first composition intentionally. */
> -    ts->sn = first_composition(n, &ts->state, ts->s);
> -    ts->sn = next_composition(&ts->state, ts->s, ts->sn);
> -    for (int i = 0; i < ts->sn; i++) {
> -        n_tses += init_tree_shape__(&ts[n_tses], ts->s[i]);
> -    }
> -    return n_tses;
> -}
> -
> -/* Initializes 'ts[]' as the first in the set of all of possible shapes of
> - * trees with 'n' leaves.  Returns the number of "struct tree_shape"s in
> the
> - * first tree shape. */
> -static int
> -init_tree_shape(struct tree_shape ts[], int n)
> -{
> -    switch (n) {
> -    case 1:
> -        ts->sn = 1;
> -        ts->s[0] = 1;
> -        return 1;
> -    case 2:
> -        ts->sn = 2;
> -        ts->s[0] = 1;
> -        ts->s[1] = 1;
> -        return 1;
> -    default:
> -        return init_tree_shape__(ts, n);
> -    }
> -}
> -
> -/* Advances 'ts', which currently has 'n_tses' elements, to the next
> possible
> - * tree shape with the number of leaves passed to init_tree_shape().
> Returns
> - * the number of "struct tree_shape"s in the next shape, or 0 if all tree
> - * shapes have been visited. */
> -static int
> -next_tree_shape(struct tree_shape ts[], int n_tses)
> -{
> -    if (n_tses == 1 && ts->sn == 2 && ts->s[0] == 1 && ts->s[1] == 1) {
> -        return 0;
> -    }
> -    while (n_tses > 0) {
> -        struct tree_shape *p = &ts[n_tses - 1];
> -        p->sn = p->sn > 1 ? next_composition(&p->state, p->s, p->sn) : 0;
> -        if (p->sn) {
> -            for (int i = 0; i < p->sn; i++) {
> -                n_tses += init_tree_shape__(&ts[n_tses], p->s[i]);
> -            }
> -            break;
> -        }
> -        n_tses--;
> -    }
> -    return n_tses;
> -}
> -
> -static void
> -print_tree_shape(const struct tree_shape ts[], int n_tses)
> -{
> -    for (int i = 0; i < n_tses; i++) {
> -        if (i) {
> -            printf(", ");
> -        }
> -        for (int j = 0; j < ts[i].sn; j++) {
> -            int k = ts[i].s[j];
> -            if (k > 9) {
> -                printf("(%d)", k);
> -            } else {
> -                printf("%d", k);
> -            }
> -        }
> -    }
> -}
> -
> -static void
> -test_tree_shape(struct ovs_cmdl_context *ctx)
> -{
> -    int n = atoi(ctx->argv[1]);
> -    struct tree_shape ts[50];
> -    int n_tses;
> -
> -    for (n_tses = init_tree_shape(ts, n); n_tses;
> -         n_tses = next_tree_shape(ts, n_tses)) {
> -        print_tree_shape(ts, n_tses);
> -        putchar('\n');
> -    }
> -}
> -
> -/* Iteration through all possible terminal expressions (e.g. EXPR_T_CMP
> and
> - * EXPR_T_BOOLEAN expressions).
> - *
> - * Given a tree shape, this allows the code to try all possible ways to
> plug in
> - * terms.
> - *
> - * Example use:
> - *
> - *     struct expr terminal;
> - *     const struct expr_symbol *vars = ...;
> - *     int n_vars = ...;
> - *     int n_bits = ...;
> - *
> - *     init_terminal(&terminal, vars[0]);
> - *     do {
> - *         // Something with 'terminal'.
> - *     } while (next_terminal(&terminal, vars, n_vars, n_bits));
> - */
> -
> -/* Sets 'expr' to the first possible terminal expression.  'var' should
> be the
> - * first variable in the ones to be tested. */
> -static void
> -init_terminal(struct expr *expr, int phase,
> -              const struct expr_symbol *nvars[], int n_nvars,
> -              const struct expr_symbol *svars[], int n_svars)
> -{
> -    if (phase < 1 && n_nvars) {
> -        expr->type = EXPR_T_CMP;
> -        expr->cmp.symbol = nvars[0];
> -        expr->cmp.relop = rightmost_1bit_idx(test_relops);
> -        memset(&expr->cmp.value, 0, sizeof expr->cmp.value);
> -        memset(&expr->cmp.mask, 0, sizeof expr->cmp.mask);
> -        expr->cmp.value.integer = htonll(0);
> -        expr->cmp.mask.integer = htonll(0);
> -        return;
> -    }
> -
> -    if (phase < 2 && n_svars) {
> -        expr->type = EXPR_T_CMP;
> -        expr->cmp.symbol = svars[0];
> -        expr->cmp.relop = EXPR_R_EQ;
> -        expr->cmp.string = xstrdup("0");
> -        return;
> -    }
> -
> -    expr->type = EXPR_T_BOOLEAN;
> -    expr->boolean = false;
> -}
> -
> -/* Returns 'x' with the rightmost contiguous string of 1s changed to 0s,
> - * e.g. 01011100 => 01000000.  See H. S. Warren, Jr., _Hacker's Delight_,
> 2nd
> - * ed., section 2-1. */
> -static unsigned int
> -turn_off_rightmost_1s(unsigned int x)
> -{
> -    return ((x & -x) + x) & x;
> -}
> -
> -static const struct expr_symbol *
> -next_var(const struct expr_symbol *symbol,
> -         const struct expr_symbol *vars[], int n_vars)
> -{
> -    for (int i = 0; i < n_vars; i++) {
> -        if (symbol == vars[i]) {
> -            return i + 1 >= n_vars ? NULL : vars[i + 1];
> -        }
> -    }
> -    OVS_NOT_REACHED();
> -}
> -
> -static enum expr_relop
> -next_relop(enum expr_relop relop)
> -{
> -    unsigned int remaining_relops = test_relops & ~((1u << (relop + 1)) -
> 1);
> -    return (remaining_relops
> -            ? rightmost_1bit_idx(remaining_relops)
> -            : rightmost_1bit_idx(test_relops));
> -}
> -
> -/* Advances 'expr' to the next possible terminal expression within the
> 'n_vars'
> - * variables of 'n_bits' bits each in 'vars[]'. */
> -static bool
> -next_terminal(struct expr *expr,
> -              const struct expr_symbol *nvars[], int n_nvars, int n_bits,
> -              const struct expr_symbol *svars[], int n_svars)
> -{
> -    if (expr->type == EXPR_T_BOOLEAN) {
> -        if (expr->boolean) {
> -            return false;
> -        } else {
> -            expr->boolean = true;
> -            return true;
> -        }
> -    }
> -
> -    if (!expr->cmp.symbol->width) {
> -        int next_value = atoi(expr->cmp.string) + 1;
> -        free(expr->cmp.string);
> -        if (next_value > 1) {
> -            expr->cmp.symbol = next_var(expr->cmp.symbol, svars, n_svars);
> -            if (!expr->cmp.symbol) {
> -                init_terminal(expr, 2, nvars, n_nvars, svars, n_svars);
> -                return true;
> -            }
> -            next_value = 0;
> -        }
> -        expr->cmp.string = xasprintf("%d", next_value);
> -        return true;
> -    }
> -
> -    unsigned int next;
> -
> -    next = (ntohll(expr->cmp.value.integer)
> -            + (ntohll(expr->cmp.mask.integer) << n_bits));
> -    for (;;) {
> -        next++;
> -        unsigned m = next >> n_bits;
> -        unsigned v = next & ((1u << n_bits) - 1);
> -        if (next >= (1u << (2 * n_bits))) {
> -            enum expr_relop old_relop = expr->cmp.relop;
> -            expr->cmp.relop = next_relop(old_relop);
> -            if (expr->cmp.relop <= old_relop) {
> -                expr->cmp.symbol = next_var(expr->cmp.symbol, nvars,
> n_nvars);
> -                if (!expr->cmp.symbol) {
> -                    init_terminal(expr, 1, nvars, n_nvars, svars,
> n_svars);
> -                    return true;
> -                }
> -            }
> -            next = UINT_MAX;
> -        } else if (v & ~m) {
> -            /* Skip: 1-bits in value correspond to 0-bits in mask. */
> -        } else if ((!m || turn_off_rightmost_1s(m))
> -                   && (expr->cmp.relop != EXPR_R_EQ &&
> -                       expr->cmp.relop != EXPR_R_NE)) {
> -            /* Skip: can't have discontiguous or all-0 mask for > >= <
> <=. */
> -        } else {
> -            expr->cmp.value.integer = htonll(v);
> -            expr->cmp.mask.integer = htonll(m);
> -            return true;
> -        }
> -    }
> -}
> -
> -static struct expr *
> -make_terminal(struct expr ***terminalp)
> -{
> -    struct expr *e = expr_create_boolean(true);
> -    **terminalp = e;
> -    (*terminalp)++;
> -    return e;
> -}
> -
> -static struct expr *
> -build_simple_tree(enum expr_type type, int n, struct expr ***terminalp)
> -{
> -    if (n == 2) {
> -        struct expr *e = expr_create_andor(type);
> -        for (int i = 0; i < 2; i++) {
> -            struct expr *sub = make_terminal(terminalp);
> -            ovs_list_push_back(&e->andor, &sub->node);
> -        }
> -        return e;
> -    } else if (n == 1) {
> -        return make_terminal(terminalp);
> -    } else {
> -        OVS_NOT_REACHED();
> -    }
> -}
> -
> -static struct expr *
> -build_tree_shape(enum expr_type type, const struct tree_shape **tsp,
> -                 struct expr ***terminalp)
> -{
> -    const struct tree_shape *ts = *tsp;
> -    (*tsp)++;
> -
> -    struct expr *e = expr_create_andor(type);
> -    enum expr_type t = type == EXPR_T_AND ? EXPR_T_OR : EXPR_T_AND;
> -    for (int i = 0; i < ts->sn; i++) {
> -        struct expr *sub = (ts->s[i] > 2
> -                            ? build_tree_shape(t, tsp, terminalp)
> -                            : build_simple_tree(t, ts->s[i], terminalp));
> -        ovs_list_push_back(&e->andor, &sub->node);
> -    }
> -    return e;
> -}
> -
> -struct test_rule {
> -    struct cls_rule cr;
> -};
> -
> -static void
> -free_rule(struct test_rule *test_rule)
> -{
> -    cls_rule_destroy(&test_rule->cr);
> -    free(test_rule);
> -}
> -
> -static bool
> -tree_shape_is_chassis_resident_cb(const void *c_aux OVS_UNUSED,
> -                                  const char *port_name OVS_UNUSED)
> -{
> -    return true;
> -}
> -
> -static int
> -test_tree_shape_exhaustively(struct expr *expr, struct shash *symtab,
> -                             struct expr *terminals[], int n_terminals,
> -                             const struct expr_symbol *nvars[], int
> n_nvars,
> -                             int n_bits,
> -                             const struct expr_symbol *svars[], int
> n_svars)
> -{
> -    int n_tested = 0;
> -
> -    const unsigned int var_mask = (1u << n_bits) - 1;
> -    for (int i = 0; i < n_terminals; i++) {
> -        init_terminal(terminals[i], 0, nvars, n_nvars, svars, n_svars);
> -    }
> -
> -    struct ds s = DS_EMPTY_INITIALIZER;
> -    struct flow f;
> -    memset(&f, 0, sizeof f);
> -    for (;;) {
> -        for (int i = n_terminals - 1; ; i--) {
> -            if (!i) {
> -                ds_destroy(&s);
> -                return n_tested;
> -            }
> -            if (next_terminal(terminals[i], nvars, n_nvars, n_bits,
> -                              svars, n_svars)) {
> -                break;
> -            }
> -            init_terminal(terminals[i], 0, nvars, n_nvars, svars,
> n_svars);
> -        }
> -        ovs_assert(expr_honors_invariants(expr));
> -
> -        n_tested++;
> -
> -        struct expr *modified;
> -        if (operation == OP_CONVERT) {
> -            ds_clear(&s);
> -            expr_format(expr, &s);
> -
> -            char *error;
> -            modified = expr_parse_string(ds_cstr(&s), symtab, NULL,
> -                                         NULL, NULL, &error);
> -            if (error) {
> -                fprintf(stderr, "%s fails to parse (%s)\n",
> -                        ds_cstr(&s), error);
> -                exit(EXIT_FAILURE);
> -            }
> -        } else if (operation >= OP_SIMPLIFY) {
> -            modified = expr_simplify(expr_clone(expr),
> -                                     tree_shape_is_chassis_resident_cb,
> -                                     NULL);
> -            ovs_assert(expr_honors_invariants(modified));
> -
> -            if (operation >= OP_NORMALIZE) {
> -                modified = expr_normalize(modified);
> -                ovs_assert(expr_honors_invariants(modified));
> -                ovs_assert(expr_is_normalized(modified));
> -            }
> -        }
> -
> -        struct hmap matches;
> -        struct classifier cls;
> -        if (operation >= OP_FLOW) {
> -            struct expr_match *m;
> -            struct test_rule *test_rule;
> -
> -            expr_to_matches(modified, lookup_atoi_cb, NULL, &matches);
> -
> -            classifier_init(&cls, NULL);
> -            HMAP_FOR_EACH (m, hmap_node, &matches) {
> -                test_rule = xmalloc(sizeof *test_rule);
> -                cls_rule_init(&test_rule->cr, &m->match, 0);
> -                classifier_insert(&cls, &test_rule->cr, OVS_VERSION_MIN,
> -                                  m->conjunctions, m->n);
> -            }
> -        }
> -        for (int subst = 0; subst < 1 << (n_bits * n_nvars + n_svars);
> -             subst++) {
> -            for (int i = 0; i < n_nvars; i++) {
> -                f.regs[i] = (subst >> (i * n_bits)) & var_mask;
> -            }
> -            for (int i = 0; i < n_svars; i++) {
> -                f.regs[n_nvars + i] = ((subst >> (n_nvars * n_bits + i))
> -                                       & 1);
> -            }
> -
> -            bool expected = expr_evaluate(expr, &f, lookup_atoi_cb, NULL);
> -            bool actual = expr_evaluate(modified, &f, lookup_atoi_cb,
> NULL);
> -            if (actual != expected) {
> -                struct ds expr_s, modified_s;
> -
> -                ds_init(&expr_s);
> -                expr_format(expr, &expr_s);
> -
> -                ds_init(&modified_s);
> -                expr_format(modified, &modified_s);
> -
> -                fprintf(stderr,
> -                        "%s evaluates to %d, but %s evaluates to %d, for",
> -                        ds_cstr(&expr_s), expected,
> -                        ds_cstr(&modified_s), actual);
> -                for (int i = 0; i < n_nvars; i++) {
> -                    if (i > 0) {
> -                        fputs(",", stderr);
> -                    }
> -                    fprintf(stderr, " n%d = 0x%x", i,
> -                            (subst >> (n_bits * i)) & var_mask);
> -                }
> -                for (int i = 0; i < n_svars; i++) {
> -                    fprintf(stderr, ", s%d = \"%d\"", i,
> -                            (subst >> (n_bits * n_nvars + i)) & 1);
> -                }
> -                putc('\n', stderr);
> -                exit(EXIT_FAILURE);
> -            }
> -
> -            if (operation >= OP_FLOW) {
> -                bool found = classifier_lookup(&cls, OVS_VERSION_MIN,
> -                                               &f, NULL) != NULL;
> -                if (expected != found) {
> -                    struct ds expr_s, modified_s;
> -
> -                    ds_init(&expr_s);
> -                    expr_format(expr, &expr_s);
> -
> -                    ds_init(&modified_s);
> -                    expr_format(modified, &modified_s);
> -
> -                    fprintf(stderr,
> -                            "%s and %s evaluate to %d, for",
> -                            ds_cstr(&expr_s), ds_cstr(&modified_s),
> expected);
> -                    for (int i = 0; i < n_nvars; i++) {
> -                        if (i > 0) {
> -                            fputs(",", stderr);
> -                        }
> -                        fprintf(stderr, " n%d = 0x%x", i,
> -                                (subst >> (n_bits * i)) & var_mask);
> -                    }
> -                    for (int i = 0; i < n_svars; i++) {
> -                        fprintf(stderr, ", s%d = \"%d\"", i,
> -                                (subst >> (n_bits * n_nvars + i)) & 1);
> -                    }
> -                    fputs(".\n", stderr);
> -
> -                    fprintf(stderr, "Converted to classifier:\n");
> -                    expr_matches_print(&matches, stderr);
> -                    fprintf(stderr,
> -                            "However, %s flow was found in the
> classifier.\n",
> -                            found ? "a" : "no");
> -                    exit(EXIT_FAILURE);
> -                }
> -            }
> -        }
> -        if (operation >= OP_FLOW) {
> -            struct test_rule *test_rule;
> -
> -            CLS_FOR_EACH (test_rule, cr, &cls) {
> -                classifier_remove_assert(&cls, &test_rule->cr);
> -                ovsrcu_postpone(free_rule, test_rule);
> -            }
> -            classifier_destroy(&cls);
> -            ovsrcu_quiesce();
> -
> -            expr_matches_destroy(&matches);
> -        }
> -        expr_destroy(modified);
> -    }
> -}
> -
> -#ifndef _WIN32
> -static void
> -wait_pid(pid_t *pids, int *n)
> -{
> -    int status;
> -    pid_t pid;
> -
> -    pid = waitpid(-1, &status, 0);
> -    if (pid < 0) {
> -        ovs_fatal(errno, "waitpid failed");
> -    } else if (WIFEXITED(status)) {
> -        if (WEXITSTATUS(status)) {
> -            exit(WEXITSTATUS(status));
> -        }
> -    } else if (WIFSIGNALED(status)) {
> -        raise(WTERMSIG(status));
> -        exit(1);
> -    } else {
> -        OVS_NOT_REACHED();
> -    }
> -
> -    for (int i = 0; i < *n; i++) {
> -        if (pids[i] == pid) {
> -            pids[i] = pids[--*n];
> -            return;
> -        }
> -    }
> -    ovs_fatal(0, "waitpid returned unknown child");
> -}
> -#endif
> -
> -static void
> -test_exhaustive(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    int n_terminals = atoi(ctx->argv[1]);
> -    struct tree_shape ts[50];
> -    int n_tses;
> -
> -    struct shash symtab;
> -    const struct expr_symbol *nvars[4];
> -    const struct expr_symbol *svars[4];
> -
> -    ovs_assert(test_nvars <= ARRAY_SIZE(nvars));
> -    ovs_assert(test_svars <= ARRAY_SIZE(svars));
> -    ovs_assert(test_nvars + test_svars <= FLOW_N_REGS);
> -
> -    shash_init(&symtab);
> -    for (int i = 0; i < test_nvars; i++) {
> -        char *name = xasprintf("n%d", i);
> -        nvars[i] = expr_symtab_add_field(&symtab, name, MFF_REG0 + i,
> NULL,
> -                                         false);
> -        free(name);
> -    }
> -    for (int i = 0; i < test_svars; i++) {
> -        char *name = xasprintf("s%d", i);
> -        svars[i] = expr_symtab_add_string(&symtab, name,
> -                                          MFF_REG0 + test_nvars + i,
> NULL);
> -        free(name);
> -    }
> -
> -#ifndef _WIN32
> -    pid_t *children = xmalloc(test_parallel * sizeof *children);
> -    int n_children = 0;
> -#endif
> -
> -    int n_tested = 0;
> -    for (int i = 0; i < 2; i++) {
> -        enum expr_type base_type = i ? EXPR_T_OR : EXPR_T_AND;
> -
> -        for (n_tses = init_tree_shape(ts, n_terminals); n_tses;
> -             n_tses = next_tree_shape(ts, n_tses)) {
> -            const struct tree_shape *tsp = ts;
> -            struct expr *terminals[50];
> -            struct expr **terminalp = terminals;
> -            struct expr *expr = build_tree_shape(base_type, &tsp,
> &terminalp);
> -            ovs_assert(terminalp == &terminals[n_terminals]);
> -
> -            if (verbosity > 0) {
> -                print_tree_shape(ts, n_tses);
> -                printf(": ");
> -                struct ds s = DS_EMPTY_INITIALIZER;
> -                expr_format(expr, &s);
> -                puts(ds_cstr(&s));
> -                ds_destroy(&s);
> -            }
> -
> -#ifndef _WIN32
> -            if (test_parallel > 1) {
> -                pid_t pid = xfork();
> -                if (!pid) {
> -                    test_tree_shape_exhaustively(expr, &symtab,
> -                                                 terminals, n_terminals,
> -                                                 nvars, test_nvars,
> test_bits,
> -                                                 svars, test_svars);
> -                    expr_destroy(expr);
> -                    exit(0);
> -                } else {
> -                    if (n_children >= test_parallel) {
> -                        wait_pid(children, &n_children);
> -                    }
> -                    children[n_children++] = pid;
> -                }
> -            } else
> -#endif
> -            {
> -                n_tested += test_tree_shape_exhaustively(
> -                    expr, &symtab, terminals, n_terminals,
> -                    nvars, test_nvars, test_bits,
> -                    svars, test_svars);
> -            }
> -            expr_destroy(expr);
> -        }
> -    }
> -#ifndef _WIN32
> -    while (n_children > 0) {
> -        wait_pid(children, &n_children);
> -    }
> -    free(children);
> -#endif
> -
> -    printf("Tested ");
> -    switch (operation) {
> -    case OP_CONVERT:
> -        printf("converting");
> -        break;
> -    case OP_SIMPLIFY:
> -        printf("simplifying");
> -        break;
> -    case OP_NORMALIZE:
> -        printf("normalizing");
> -        break;
> -    case OP_FLOW:
> -        printf("converting to flows");
> -        break;
> -    }
> -    if (n_tested) {
> -        printf(" %d expressions of %d terminals", n_tested, n_terminals);
> -    } else {
> -        printf(" all %d-terminal expressions", n_terminals);
> -    }
> -    if (test_nvars || test_svars) {
> -        printf(" with");
> -        if (test_nvars) {
> -            printf(" %d numeric vars (each %d bits) in terms of
> operators",
> -                   test_nvars, test_bits);
> -            for (unsigned int relops = test_relops; relops;
> -                 relops = zero_rightmost_1bit(relops)) {
> -                enum expr_relop r = rightmost_1bit_idx(relops);
> -                printf(" %s", expr_relop_to_string(r));
> -            }
> -        }
> -        if (test_nvars && test_svars) {
> -            printf (" and");
> -        }
> -        if (test_svars) {
> -            printf(" %d string vars", test_svars);
> -        }
> -    } else {
> -        printf(" in terms of Boolean constants only");
> -    }
> -    printf(".\n");
> -
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -static void
> -test_expr_to_packets(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    struct shash symtab;
> -    struct ds input;
> -
> -    create_symtab(&symtab);
> -
> -    ds_init(&input);
> -    while (!ds_get_test_line(&input, stdin)) {
> -        struct flow uflow;
> -        char *error = expr_parse_microflow(ds_cstr(&input), &symtab, NULL,
> -                                           NULL, lookup_atoi_cb, NULL,
> &uflow);
> -        if (error) {
> -            puts(error);
> -            free(error);
> -            continue;
> -        }
> -
> -        uint64_t packet_stub[128 / 8];
> -        struct dp_packet packet;
> -        dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub);
> -        flow_compose(&packet, &uflow, NULL, 64);
> -
> -        struct ds output = DS_EMPTY_INITIALIZER;
> -        const uint8_t *buf = dp_packet_data(&packet);
> -        for (int i = 0; i < dp_packet_size(&packet); i++) {
> -            uint8_t val = buf[i];
> -            ds_put_format(&output, "%02"PRIx8, val);
> -        }
> -        puts(ds_cstr(&output));
> -        ds_destroy(&output);
> -
> -        dp_packet_uninit(&packet);
> -    }
> -    ds_destroy(&input);
> -
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -}
> -
> -/* Actions. */
> -
> -static void
> -test_parse_actions(struct ovs_cmdl_context *ctx OVS_UNUSED)
> -{
> -    struct shash symtab;
> -    struct hmap dhcp_opts;
> -    struct hmap dhcpv6_opts;
> -    struct hmap nd_ra_opts;
> -    struct controller_event_options event_opts;
> -    struct simap ports;
> -    struct ds input;
> -    bool ok = true;
> -
> -    create_symtab(&symtab);
> -    create_gen_opts(&dhcp_opts, &dhcpv6_opts, &nd_ra_opts, &event_opts);
> -
> -    /* Initialize group ids. */
> -    struct ovn_extend_table group_table;
> -    ovn_extend_table_init(&group_table);
> -
> -    /* Initialize meter ids for QoS. */
> -    struct ovn_extend_table meter_table;
> -    ovn_extend_table_init(&meter_table);
> -
> -    simap_init(&ports);
> -    simap_put(&ports, "eth0", 5);
> -    simap_put(&ports, "eth1", 6);
> -    simap_put(&ports, "LOCAL", ofp_to_u16(OFPP_LOCAL));
> -
> -    ds_init(&input);
> -    while (!ds_get_test_line(&input, stdin)) {
> -        struct ofpbuf ovnacts;
> -        struct expr *prereqs;
> -        char *error;
> -
> -        puts(ds_cstr(&input));
> -
> -        ofpbuf_init(&ovnacts, 0);
> -
> -        const struct ovnact_parse_params pp = {
> -            .symtab = &symtab,
> -            .dhcp_opts = &dhcp_opts,
> -            .dhcpv6_opts = &dhcpv6_opts,
> -            .nd_ra_opts = &nd_ra_opts,
> -            .controller_event_opts = &event_opts,
> -            .n_tables = 24,
> -            .cur_ltable = 10,
> -        };
> -        error = ovnacts_parse_string(ds_cstr(&input), &pp, &ovnacts,
> &prereqs);
> -        if (!error) {
> -            /* Convert the parsed representation back to a string and
> print it,
> -             * if it's different from the input. */
> -            struct ds ovnacts_s = DS_EMPTY_INITIALIZER;
> -            ovnacts_format(ovnacts.data, ovnacts.size, &ovnacts_s);
> -            if (strcmp(ds_cstr(&input), ds_cstr(&ovnacts_s))) {
> -                printf("    formats as %s\n", ds_cstr(&ovnacts_s));
> -            }
> -
> -            /* Encode the actions into OpenFlow and print. */
> -            const struct ovnact_encode_params ep = {
> -                .lookup_port = lookup_port_cb,
> -                .aux = &ports,
> -                .is_switch = true,
> -                .group_table = &group_table,
> -                .meter_table = &meter_table,
> -
> -                .pipeline = OVNACT_P_INGRESS,
> -                .ingress_ptable = 8,
> -                .egress_ptable = 40,
> -                .output_ptable = 64,
> -                .mac_bind_ptable = 65,
> -            };
> -            struct ofpbuf ofpacts;
> -            ofpbuf_init(&ofpacts, 0);
> -            ovnacts_encode(ovnacts.data, ovnacts.size, &ep, &ofpacts);
> -            struct ds ofpacts_s = DS_EMPTY_INITIALIZER;
> -            struct ofpact_format_params fp = { .s = &ofpacts_s };
> -            ofpacts_format(ofpacts.data, ofpacts.size, &fp);
> -            printf("    encodes as %s\n", ds_cstr(&ofpacts_s));
> -            ds_destroy(&ofpacts_s);
> -            ofpbuf_uninit(&ofpacts);
> -
> -            /* Print prerequisites if any. */
> -            if (prereqs) {
> -                struct ds prereqs_s = DS_EMPTY_INITIALIZER;
> -                expr_format(prereqs, &prereqs_s);
> -                printf("    has prereqs %s\n", ds_cstr(&prereqs_s));
> -                ds_destroy(&prereqs_s);
> -            }
> -
> -            /* Now re-parse and re-format the string to verify that it's
> -             * round-trippable. */
> -            struct ofpbuf ovnacts2;
> -            struct expr *prereqs2;
> -            ofpbuf_init(&ovnacts2, 0);
> -            error = ovnacts_parse_string(ds_cstr(&ovnacts_s), &pp,
> &ovnacts2,
> -                                         &prereqs2);
> -            if (!error) {
> -                struct ds ovnacts2_s = DS_EMPTY_INITIALIZER;
> -                ovnacts_format(ovnacts2.data, ovnacts2.size, &ovnacts2_s);
> -                if (strcmp(ds_cstr(&ovnacts_s), ds_cstr(&ovnacts2_s))) {
> -                    printf("    bad reformat: %s\n",
> ds_cstr(&ovnacts2_s));
> -                    ok = false;
> -                }
> -                ds_destroy(&ovnacts2_s);
> -            } else {
> -                printf("    reparse error: %s\n", error);
> -                free(error);
> -                ok = false;
> -            }
> -            expr_destroy(prereqs2);
> -
> -            ovnacts_free(ovnacts2.data, ovnacts2.size);
> -            ofpbuf_uninit(&ovnacts2);
> -            ds_destroy(&ovnacts_s);
> -        } else {
> -            printf("    %s\n", error);
> -            free(error);
> -        }
> -
> -        expr_destroy(prereqs);
> -        ovnacts_free(ovnacts.data, ovnacts.size);
> -        ofpbuf_uninit(&ovnacts);
> -    }
> -    ds_destroy(&input);
> -
> -    simap_destroy(&ports);
> -    expr_symtab_destroy(&symtab);
> -    shash_destroy(&symtab);
> -    dhcp_opts_destroy(&dhcp_opts);
> -    dhcp_opts_destroy(&dhcpv6_opts);
> -    nd_ra_opts_destroy(&nd_ra_opts);
> -    controller_event_opts_destroy(&event_opts);
> -    ovn_extend_table_destroy(&group_table);
> -    ovn_extend_table_destroy(&meter_table);
> -    exit(ok ? EXIT_SUCCESS : EXIT_FAILURE);
> -}
> -
> -static unsigned int
> -parse_relops(const char *s)
> -{
> -    unsigned int relops = 0;
> -    struct lexer lexer;
> -
> -    lexer_init(&lexer, s);
> -    lexer_get(&lexer);
> -    do {
> -        enum expr_relop relop;
> -
> -        if (expr_relop_from_token(lexer.token.type, &relop)) {
> -            relops |= 1u << relop;
> -            lexer_get(&lexer);
> -        } else {
> -            ovs_fatal(0, "%s: relational operator expected at `%.*s'",
> -                      s, (int) (lexer.input - lexer.start), lexer.start);
> -        }
> -        lexer_match(&lexer, LEX_T_COMMA);
> -    } while (lexer.token.type != LEX_T_END);
> -    lexer_destroy(&lexer);
> -
> -    return relops;
> -}
> -
> -static void
> -usage(void)
> -{
> -    printf("\
> -%s: OVN test utility\n\
> -usage: test-ovn %s [OPTIONS] COMMAND [ARG...]\n\
> -\n\
> -lex\n\
> -  Lexically analyzes OVN input from stdin and print them back on
> stdout.\n\
> -\n\
> -parse-expr\n\
> -annotate-expr\n\
> -simplify-expr\n\
> -normalize-expr\n\
> -expr-to-flows\n\
> -  Parses OVN expressions from stdin and prints them back on stdout
> after\n\
> -  differing degrees of analysis.  Available fields are based on packet\n\
> -  headers.\n\
> -\n\
> -expr-to-packets\n\
> -  Parses OVN expressions from stdin and prints out matching packets in\n\
> -  hexadecimal on stdout.\n\
> -\n\
> -evaluate-expr MICROFLOW\n\
> -  Parses OVN expressions from stdin and evaluates them against the flow\n\
> -  specified in MICROFLOW, which must be an expression that constrains\n\
> -  the packet, e.g. \"ip4 && tcp.src == 80\" for a TCP packet with
> source\n\
> -  port 80, and prints the results on stdout, either 1 for true or 0 for\n\
> -  false.  Use quoted integers, e.g. \"123\", for string fields.\n\
> -\n\
> -  Example: for MICROFLOW of \"ip4 && tcp.src == 80\", \"eth.type ==
> 0x800\"\n\
> -  evaluates to true, \"udp\" evaluates to false, and \"udp || tcp\"\n\
> -  evaluates to true.\n\
> -\n\
> -composition N\n\
> -  Prints all the compositions of N on stdout.\n\
> -\n\
> -tree-shape N\n\
> -  Prints all the tree shapes with N terminals on stdout.\n\
> -\n\
> -exhaustive N\n\
> -  Tests that all possible Boolean expressions with N terminals are
> properly\n\
> -  simplified, normalized, and converted to flows.  Available options:\n\
> -   Overall options:\n\
> -    --operation=OPERATION  Operation to test, one of: convert,
> simplify,\n\
> -        normalize, flow.  Default: flow.  'normalize' includes
> 'simplify',\n\
> -        'flow' includes 'simplify' and 'normalize'.\n\
> -    --parallel=N  Number of processes to use in parallel, default 1.\n\
> -   Numeric vars:\n\
> -    --nvars=N  Number of numeric vars to test, in range 0...4, default
> 2.\n\
> -    --bits=N  Number of bits per variable, in range 1...3, default 3.\n\
> -    --relops=OPERATORS   Test only the specified Boolean operators.\n\
> -                         OPERATORS may include == != < <= > >=, space
> or\n\
> -                         comma separated.  Default is all operators.\n\
> -   String vars:\n\
> -    --svars=N  Number of string vars to test, in range 0...4, default
> 2.\n\
> -\n\
> -parse-actions\n\
> -  Parses OVN actions from stdin and prints the equivalent OpenFlow
> actions\n\
> -  on stdout.\n\
> -",
> -           program_name, program_name);
> -    exit(EXIT_SUCCESS);
> -}
> -
> -static void
> -test_ovn_main(int argc, char *argv[])
> -{
> -    enum {
> -        OPT_RELOPS = UCHAR_MAX + 1,
> -        OPT_NVARS,
> -        OPT_SVARS,
> -        OPT_BITS,
> -        OPT_OPERATION,
> -        OPT_PARALLEL
> -    };
> -    static const struct option long_options[] = {
> -        {"relops", required_argument, NULL, OPT_RELOPS},
> -        {"nvars", required_argument, NULL, OPT_NVARS},
> -        {"svars", required_argument, NULL, OPT_SVARS},
> -        {"bits", required_argument, NULL, OPT_BITS},
> -        {"operation", required_argument, NULL, OPT_OPERATION},
> -        {"parallel", required_argument, NULL, OPT_PARALLEL},
> -        {"more", no_argument, NULL, 'm'},
> -        {"help", no_argument, NULL, 'h'},
> -        {NULL, 0, NULL, 0},
> -    };
> -    char *short_options =
> ovs_cmdl_long_options_to_short_options(long_options);
> -
> -    set_program_name(argv[0]);
> -
> -    test_relops = parse_relops("== != < <= > >=");
> -    for (;;) {
> -        int option_index = 0;
> -        int c = getopt_long (argc, argv, short_options, long_options,
> -                             &option_index);
> -
> -        if (c == -1) {
> -            break;
> -        }
> -        switch (c) {
> -        case OPT_RELOPS:
> -            test_relops = parse_relops(optarg);
> -            break;
> -
> -        case OPT_NVARS:
> -            test_nvars = atoi(optarg);
> -            if (test_nvars < 0 || test_nvars > 4) {
> -                ovs_fatal(0, "number of numeric variables must be "
> -                          "between 0 and 4");
> -            }
> -            break;
> -
> -        case OPT_SVARS:
> -            test_svars = atoi(optarg);
> -            if (test_svars < 0 || test_svars > 4) {
> -                ovs_fatal(0, "number of string variables must be "
> -                          "between 0 and 4");
> -            }
> -            break;
> -
> -        case OPT_BITS:
> -            test_bits = atoi(optarg);
> -            if (test_bits < 1 || test_bits > 3) {
> -                ovs_fatal(0, "number of bits must be between 1 and 3");
> -            }
> -            break;
> -
> -        case OPT_OPERATION:
> -            if (!strcmp(optarg, "convert")) {
> -                operation = OP_CONVERT;
> -            } else if (!strcmp(optarg, "simplify")) {
> -                operation = OP_SIMPLIFY;
> -            } else if (!strcmp(optarg, "normalize")) {
> -                operation = OP_NORMALIZE;
> -            } else if (!strcmp(optarg, "flow")) {
> -                operation = OP_FLOW;
> -            } else {
> -                ovs_fatal(0, "%s: unknown operation", optarg);
> -            }
> -            break;
> -
> -        case OPT_PARALLEL:
> -            test_parallel = atoi(optarg);
> -            break;
> -
> -        case 'm':
> -            verbosity++;
> -            break;
> -
> -        case 'h':
> -            usage();
> -            /* fall through */
> -
> -        case '?':
> -            exit(1);
> -
> -        default:
> -            abort();
> -        }
> -    }
> -    free(short_options);
> -
> -    static const struct ovs_cmdl_command commands[] = {
> -        /* Lexer. */
> -        {"lex", NULL, 0, 0, test_lex, OVS_RO},
> -
> -        /* Symbol table. */
> -        {"dump-symtab", NULL, 0, 0, test_dump_symtab, OVS_RO},
> -
> -        /* Expressions. */
> -        {"parse-expr", NULL, 0, 0, test_parse_expr, OVS_RO},
> -        {"annotate-expr", NULL, 0, 0, test_annotate_expr, OVS_RO},
> -        {"simplify-expr", NULL, 0, 0, test_simplify_expr, OVS_RO},
> -        {"normalize-expr", NULL, 0, 0, test_normalize_expr, OVS_RO},
> -        {"expr-to-flows", NULL, 0, 0, test_expr_to_flows, OVS_RO},
> -        {"evaluate-expr", NULL, 1, 1, test_evaluate_expr, OVS_RO},
> -        {"composition", NULL, 1, 1, test_composition, OVS_RO},
> -        {"tree-shape", NULL, 1, 1, test_tree_shape, OVS_RO},
> -        {"exhaustive", NULL, 1, 1, test_exhaustive, OVS_RO},
> -        {"expr-to-packets", NULL, 0, 0, test_expr_to_packets, OVS_RO},
> -
> -        /* Actions. */
> -        {"parse-actions", NULL, 0, 0, test_parse_actions, OVS_RO},
> -
> -        {NULL, NULL, 0, 0, NULL, OVS_RO},
> -    };
> -    struct ovs_cmdl_context ctx;
> -    ctx.argc = argc - optind;
> -    ctx.argv = argv + optind;
> -    ovs_cmdl_run_command(&ctx, commands);
> -}
> -
> -OVSTEST_REGISTER("test-ovn", test_ovn_main);
> diff --git a/tests/testsuite.at b/tests/testsuite.at
> index 4d5e81618..e75912300 100644
> --- a/tests/testsuite.at
> +++ b/tests/testsuite.at
> @@ -19,7 +19,6 @@ m4_ifdef([AT_COLOR_TESTS], [AT_COLOR_TESTS])
>  m4_include([tests/ovs-macros.at])
>  m4_include([tests/ovsdb-macros.at])
>  m4_include([tests/ofproto-macros.at])
> -m4_include([tests/ovn-macros.at])
>
>  m4_include([tests/completion.at])
>  m4_include([tests/checkpatch.at])
> @@ -74,13 +73,6 @@ m4_include([tests/rstp.at])
>  m4_include([tests/vlog.at])
>  m4_include([tests/vtep-ctl.at])
>  m4_include([tests/auto-attach.at])
> -m4_include([tests/ovn.at])
> -m4_include([tests/ovn-northd.at])
> -m4_include([tests/ovn-nbctl.at])
> -m4_include([tests/ovn-sbctl.at])
> -m4_include([tests/ovn-controller.at])
> -m4_include([tests/ovn-controller-vtep.at])
>  m4_include([tests/mcast-snooping.at])
>  m4_include([tests/packet-type-aware.at])
>  m4_include([tests/nsh.at])
> -m4_include([tests/ovn-performance.at])
> diff --git a/tutorial/automake.mk b/tutorial/automake.mk
> index b7ea10c98..0f6b0fff5 100644
> --- a/tutorial/automake.mk
> +++ b/tutorial/automake.mk
> @@ -5,8 +5,7 @@ EXTRA_DIST += \
>         tutorial/t-stage1 \
>         tutorial/t-stage2 \
>         tutorial/t-stage3 \
> -       tutorial/t-stage4 \
> -       tutorial/ovn-setup.sh
> +       tutorial/t-stage4
>  sandbox: all
>         cd $(srcdir)/tutorial && MAKE=$(MAKE) HAVE_OPENSSL=$(HAVE_OPENSSL)
> \
>                 ./ovs-sandbox -b $(abs_builddir) $(SANDBOXFLAGS)
> diff --git a/tutorial/ovn-setup.sh b/tutorial/ovn-setup.sh
> deleted file mode 100755
> index 969b2330f..000000000
> --- a/tutorial/ovn-setup.sh
> +++ /dev/null
> @@ -1,37 +0,0 @@
> -#!/bin/bash
> -
> -# Create the first logical switch with one port
> -ovn-nbctl ls-add sw0
> -ovn-nbctl lsp-add sw0 sw0-port1
> -ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 192.168.0.2"
> -
> -# Create the second logical switch with one port
> -ovn-nbctl ls-add sw1
> -ovn-nbctl lsp-add sw1 sw1-port1
> -ovn-nbctl lsp-set-addresses sw1-port1 "50:54:00:00:00:03 11.0.0.2"
> -
> -# Create a logical router and attach both logical switches
> -ovn-nbctl lr-add lr0
> -ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:ff:01 192.168.0.1/24
> -ovn-nbctl lsp-add sw0 lrp0-attachment
> -ovn-nbctl lsp-set-type lrp0-attachment router
> -ovn-nbctl lsp-set-addresses lrp0-attachment 00:00:00:00:ff:01
> -ovn-nbctl lsp-set-options lrp0-attachment router-port=lrp0
> -ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:ff:02 11.0.0.1/24
> -ovn-nbctl lsp-add sw1 lrp1-attachment
> -ovn-nbctl lsp-set-type lrp1-attachment router
> -ovn-nbctl lsp-set-addresses lrp1-attachment 00:00:00:00:ff:02
> -ovn-nbctl lsp-set-options lrp1-attachment router-port=lrp1
> -
> -ovs-vsctl add-port br-int p1 -- \
> -    set Interface p1 external_ids:iface-id=sw0-port1
> -ovs-vsctl add-port br-int p2 -- \
> -    set Interface p2 external_ids:iface-id=sw1-port1
> -
> -# View a summary of the configuration
> -printf "\n=== ovn-nbctl show ===\n\n"
> -ovn-nbctl show
> -printf "\n=== ovn-nbctl show with wait hv ===\n\n"
> -ovn-nbctl --wait=hv show
> -printf "\n=== ovn-sbctl show ===\n\n"
> -ovn-sbctl show
> diff --git a/tutorial/ovs-sandbox b/tutorial/ovs-sandbox
> index 601d0381f..09e9773ce 100755
> --- a/tutorial/ovs-sandbox
> +++ b/tutorial/ovs-sandbox
> @@ -56,27 +56,11 @@ gdb_vswitchd=false
>  gdb_ovsdb=false
>  gdb_vswitchd_ex=false
>  gdb_ovsdb_ex=false
> -gdb_ovn_northd=false
> -gdb_ovn_northd_ex=false
> -gdb_ovn_controller=false
> -gdb_ovn_controller_ex=false
> -gdb_ovn_controller_vtep=false
> -gdb_ovn_controller_vtep_ex=false
>  builddir=
>  srcdir=
>  schema=
>  installed=false
>  built=false
> -ovn=false
> -ovnsb_schema=
> -ovnnb_schema=
> -ovn_rbac=true
> -n_northds=1
> -n_controllers=1
> -nbdb_model=standalone
> -nbdb_servers=3
> -sbdb_model=backup
> -sbdb_servers=3
>  dummy=override
>
>  for option; do
> @@ -120,23 +104,11 @@ These options force ovs-sandbox to use an installed
> Open vSwitch:
>  General options:
>    -g, --gdb-vswitchd   run ovs-vswitchd under gdb
>    -d, --gdb-ovsdb      run ovsdb-server under gdb
> -  --gdb-ovn-northd     run ovn-northd under gdb
> -  --gdb-ovn-controller run ovn-controller under gdb
> -  --gdb-ovn-controller-vtep run ovn-controller-vtep under gdb
>    --dummy=ARG          pass --enable-dummy=ARG to vswitchd (default:
> override)
>    -R, --gdb-run        automatically start running the daemon in gdb
>                         for any daemon set to run under gdb
>    -S, --schema=FILE    use FILE as vswitch.ovsschema
>
> -OVN options:
> -  -o, --ovn            enable OVN
> -  --no-ovn-rbac        disable role-based access control for OVN
> -  --n-northds=NUMBER   run NUMBER copies of northd (default: 1)
> -  --nbdb-model=standalone|backup|clustered    northbound database model
> -  --nbdb-servers=N     number of servers in nbdb cluster (default: 3)
> -  --sbdb-model=standalone|backup|clustered    southbound database model
> -  --sbdb-servers=N     number of servers in sbdb cluster (default: 3)
> -
>  Other options:
>    -h, --help           Print this usage message.
>  EOF
> @@ -192,67 +164,9 @@ EOF
>              gdb_ovsdb=true
>              gdb_ovsdb_ex=true
>              ;;
> -        --gdb-ovn-northd)
> -            gdb_ovn_northd=true
> -            ;;
> -        --gdb-ovn-controller)
> -            gdb_ovn_controller=true
> -            ;;
> -        --gdb-ovn-controller-vtep)
> -            gdb_ovn_controller_vtep=true
> -            ;;
> -        -o|--ovn)
> -            ovn=true
> -            ;;
> -        --no-ovn-rbac)
> -            ovn_rbac=false
> -            ;;
> -        --n-northd*=*)
> -            n_northds=$optarg
> -            ;;
> -        --n-northd*)
> -            prev=n_northds
> -            ;;
> -        --n-controller*=*)
> -            n_controllers=$optarg
> -            ;;
> -        --n-controller*)
> -            prev=n_controllers
> -            ;;
> -        --nbdb-s*=*)
> -            nbdb_servers=$optarg
> -            nbdb_model=clustered
> -            ;;
> -        --nbdb-s*)
> -            prev=nbdb_servers
> -            nbdb_model=clustered
> -            ;;
> -        --nbdb-m*=*)
> -            nbdb_model=$optarg
> -            ;;
> -        --nbdb-m*)
> -            prev=nbdb_model
> -            ;;
> -        --sbdb-s*=*)
> -            sbdb_servers=$optarg
> -            sbdb_model=clustered
> -            ;;
> -        --sbdb-s*)
> -            prev=sbdb_servers
> -            sbdb_model=clustered
> -            ;;
> -        --sbdb-m*=*)
> -            sbdb_model=$optarg
> -            ;;
> -        --sbdb-m*)
> -            prev=sbdb_model
> -            ;;
>          -R|--gdb-run)
>              gdb_vswitchd_ex=true
>              gdb_ovsdb_ex=true
> -            gdb_ovn_northd_ex=true
> -            gdb_ovn_controller_ex=true
> -            gdb_ovn_controller_vtep_ex=true
>              ;;
>          -*)
>              echo "unrecognized option $option (use --help for help)" >&2
> @@ -304,23 +218,6 @@ if $built; then
>          echo >&2 'source directory not found, please use --srcdir'
>          exit 1
>      fi
> -    if $ovn; then
> -        ovnsb_schema=$srcdir/ovn/ovn-sb.ovsschema
> -        if test ! -e "$ovnsb_schema"; then
> -            echo >&2 'source directory not found, please use --srcdir'
> -            exit 1
> -        fi
> -        ovnnb_schema=$srcdir/ovn/ovn-nb.ovsschema
> -        if test ! -e "$ovnnb_schema"; then
> -            echo >&2 'source directory not found, please use --srcdir'
> -            exit 1
> -        fi
> -        vtep_schema=$srcdir/vtep/vtep.ovsschema
> -        if test ! -e "$vtep_schema"; then
> -            echo >&2 'source directory not found, please use --srcdir'
> -            exit 1
> -        fi
> -    fi
>
>      # Put built tools early in $PATH.
>      if test ! -e $builddir/vswitchd/ovs-vswitchd; then
> @@ -328,9 +225,6 @@ if $built; then
>          exit 1
>      fi
>
>  PATH=$builddir/ovsdb:$builddir/vswitchd:$builddir/utilities:$builddir/vtep:$PATH
> -    if $ovn; then
> -
> PATH=$builddir/ovn/controller:$builddir/ovn/controller-vtep:$builddir/ovn/northd:$builddir/ovn/utilities:$PATH
> -    fi
>      export PATH
>  else
>      case $schema in
> @@ -351,10 +245,6 @@ else
>          echo "can't find vswitch.ovsschema, please specify --schema" >&2
>          exit 1
>      fi
> -    if $ovn; then
> -        echo "running with ovn is only supported from the build dir." >&2
> -        exit 1
> -    fi
>  fi
>
>  # Create sandbox.
> @@ -381,109 +271,10 @@ trap 'kill `cat "$sandbox"/*.pid`' 0 1 2 3 13 14 15
>  touch "$sandbox"/.conf.db.~lock~
>  run ovsdb-tool create conf.db "$schema"
>  ovsdb_server_args=
> -if $ovn; then
> -    touch "$sandbox"/.ovnnb.db.~lock~
> -    run ovsdb-tool create ovnnb.db "$ovnnb_schema"
> -    run ovsdb-tool create vtep.db "$vtep_schema"
> -    ovsdb_server_args="vtep.db conf.db"
> -    ovsdb_nb_server_args="ovnnb.db"
> -
> -    if [ "$HAVE_OPENSSL" = yes ]; then
> -        OVS_PKI="run ovs-pki --dir=$sandbox/pki
> --log=$sandbox/ovs-pki.log"
> -        $OVS_PKI init
> -        $OVS_PKI req+sign ovnsb switch
> -        $OVS_PKI req+sign ovnnb switch
> -        for i in $(seq $n_controllers); do
> -            $OVS_PKI -u req+sign chassis-$i switch
> -        done
> -    fi
> -fi
>  rungdb $gdb_ovsdb $gdb_ovsdb_ex ovsdb-server --detach --no-chdir
> --pidfile -vconsole:off --log-file -vsyslog:off \
>         --remote=punix:"$sandbox"/db.sock \
>         --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
>         $ovsdb_server_args
> -if $ovn; then
> -    ovn_start_db() {
> -        local db=$1 model=$2 servers=$3 schema=$4
> -        local DB=$(echo $db | tr a-z A-Z)
> -        local schema_name=$(ovsdb-tool schema-name $schema)
> -
> -        case $model in
> -            standalone | backup) ;;
> -            clustered)
> -                case $servers in
> -                    [1-9] | [1-9][0-9]) ;;
> -                    *) echo "${db}db servers must be between 1 and 99" >&2
> -                       exit 1
> -                       ;;
> -                esac
> -                ;;
> -            *)
> -                echo "unknown ${db}db model \"$model\"" >&2
> -                exit 1
> -                ;;
> -        esac
> -
> -        ovn_start_ovsdb_server() {
> -            local i=$1; shift
> -            rungdb $gdb_ovsdb $gdb_ovsdb_ex ovsdb-server --detach
> --no-chdir \
> -                   --pidfile=$db$i.pid -vconsole:off --log-file=$db$i.log
> \
> -                   -vsyslog:off \
> -                   --remote=db:$schema_name,${DB}_Global,connections \
> -                   --private-key=db:$schema_name,SSL,private_key \
> -                   --certificate=db:$schema_name,SSL,certificate \
> -                   --ca-cert=db:$schema_name,SSL,ca_cert \
> -                   --ssl-protocols=db:$schema_name,SSL,ssl_protocols \
> -                   --ssl-ciphers=db:$schema_name,SSL,ssl_ciphers \
> -                   --unixctl=${db}$i --remote=punix:$db$i.ovsdb
> ${db}$i.db "$@"
> -        }
> -
> -        case $model in
> -            standalone)
> -                run ovsdb-tool create ${db}1.db "$schema"
> -                ovn_start_ovsdb_server 1
> -                remote=unix:${db}1.ovsdb
> -                ;;
> -            backup)
> -                for i in 1 2; do
> -                    run ovsdb-tool create $db$i.db "$schema"
> -                done
> -                ovn_start_ovsdb_server 1
> -                ovn_start_ovsdb_server 2 --sync-from=unix:${db}1.ovsdb
> -                remote=unix:${db}1.ovsdb
> -                backup_note="$backup_note
> -The backup server of OVN $DB can be accessed by:
> -* ovn-${db}ctl --db=unix:`pwd`/sandbox/${db}2.ovsdb
> -* ovs-appctl -t `pwd`/sandbox/${db}2
> -The backup database file is sandbox/${db}2.db
> -"
> -                ;;
> -            clustered)
> -                for i in $(seq $servers); do
> -                    if test $i = 1; then
> -                        run ovsdb-tool create-cluster ${db}1.db "$schema"
> unix:${db}1.raft;
> -                    else
> -                        run ovsdb-tool join-cluster $db$i.db $schema_name
> unix:$db$i.raft unix:${db}1.raft
> -                    fi
> -                    ovn_start_ovsdb_server $i
> -                done
> -                remote=unix:${db}1.ovsdb
> -                for i in `seq 2 $servers`; do
> -                    remote=$remote,unix:$db$i.ovsdb
> -                done
> -                for i in $(seq $servers); do
> -                    run ovsdb-client wait unix:$db$i.ovsdb $schema_name
> connected
> -                done
> -                ;;
> -        esac
> -        eval OVN_${DB}_DB=\$remote
> -        eval export OVN_${DB}_DB
> -    }
> -
> -    backup_note=
> -    ovn_start_db nb "$nbdb_model" "$nbdb_servers" "$ovnnb_schema"
> -    ovn_start_db sb "$sbdb_model" "$sbdb_servers" "$ovnsb_schema"
> -fi
>
>  #Add a small delay to allow ovsdb-server to launch.
>  sleep 0.1
> @@ -504,50 +295,6 @@ run ovs-vsctl --no-wait -- init
>  rungdb $gdb_vswitchd $gdb_vswitchd_ex ovs-vswitchd --detach --no-chdir
> --pidfile -vconsole:off --log-file -vsyslog:off \
>      --enable-dummy=$dummy -vvconn -vnetdev_dummy
>
> -if $ovn; then
> -    ovn-nbctl init
> -    ovn-sbctl init
> -
> -    ovs-vsctl set open . external-ids:system-id=chassis-1
> -    ovs-vsctl set open . external-ids:hostname=sandbox
> -    ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> -    ovs-vsctl set open . external-ids:ovn-encap-ip=127.0.0.1
> -
> -    if [ "$HAVE_OPENSSL" = yes ]; then
> -        ovn-nbctl set-ssl $sandbox/ovnnb-privkey.pem
> $sandbox/ovnnb-cert.pem $sandbox/pki/switchca/cacert.pem
> -        ovn-nbctl set-connection pssl:6641
> -        ovn-sbctl set-ssl $sandbox/ovnsb-privkey.pem
> $sandbox/ovnsb-cert.pem $sandbox/pki/switchca/cacert.pem
> -        if $ovn_rbac; then
> -            ovn-sbctl set-connection role=ovn-controller pssl:6642
> -        else
> -            ovn-sbctl set-connection pssl:6642
> -        fi
> -        ovs-vsctl set open . external-ids:ovn-remote=ssl:127.0.0.1:6642
> -        OVN_CTRLR_PKI="-p $sandbox/chassis-1-privkey.pem -c
> $sandbox/chassis-1-cert.pem -C $sandbox/pki/switchca/cacert.pem"
> -    else
> -        ovs-vsctl set open . external-ids:ovn-remote=$OVN_SB_DB
> -        OVN_CTRLR_PKI=""
> -    fi
> -    for i in $(seq $n_northds); do
> -        if [ $i -eq 1 ]; then inst=""; else inst=$i; fi
> -        rungdb $gdb_ovn_northd $gdb_ovn_northd_ex ovn-northd --detach \
> -               --no-chdir --pidfile=ovn-northd${inst}.pid -vconsole:off \
> -               --log-file=ovn-northd${inst}.log -vsyslog:off \
> -               --ovnsb-db="$OVN_SB_DB" --ovnnb-db="$OVN_NB_DB"
> -    done
> -    for i in $(seq $n_controllers); do
> -        if [ $i -eq 1 ]; then inst=""; else inst=$i; fi
> -        rungdb $gdb_ovn_controller $gdb_ovn_controller_ex ovn-controller \
> -               $OVN_CTRLR_PKI --detach --no-chdir -vsyslog:off \
> -               --log-file=ovn-controller${inst}.log \
> -               --pidfile=ovn-controller${inst}.pid -vconsole:off
> -    done
> -    rungdb $gdb_ovn_controller_vtep $gdb_ovn_controller_vtep_ex \
> -        ovn-controller-vtep --detach --no-chdir --pidfile -vconsole:off \
> -        $OVN_CTRLR_PKI --log-file -vsyslog:off \
> -        --ovnsb-db=unix:"$sandbox"/ovnsb_db.sock
> -fi
> -
>  cat <<EOF
>
>
> @@ -557,14 +304,6 @@ You are running in a dummy Open vSwitch environment.
> You can use
>  ovs-vsctl, ovs-ofctl, ovs-appctl, and other tools to work with the
>  dummy switch.
>
> -EOF
> -if $ovn; then cat << EOF
> -This environment also has the OVN daemons and databases enabled.
> -You can use ovn-nbctl and ovn-sbctl to interact with the OVN databases.
> -$backup_note
> -EOF
> -fi
> -cat <<EOF
>  Log files, pidfiles, and the configuration database are in the
>  "sandbox" subdirectory.
>
> diff --git a/utilities/bugtool/automake.mk b/utilities/bugtool/automake.mk
> index 18fa3478e..40980b367 100644
> --- a/utilities/bugtool/automake.mk
> +++ b/utilities/bugtool/automake.mk
> @@ -32,8 +32,7 @@ bugtoolpluginsdir = $(pkgdatadir)/bugtool-plugins
>  INSTALL_DATA_LOCAL += bugtool-install-data-local
>  bugtool-install-data-local:
>         for plugin in $(bugtool_plugins); do \
> -         stem=`echo "$$plugin" | sed 's,ovn/,,'`; \
> -         stem=`echo "$$stem" | sed 's,utilities/bugtool/plugins/,,'`; \
> +         stem=`echo "$$plugin" | sed 's,utilities/bugtool/plugins/,,'`; \
>           dir=`expr "$$stem" : '\(.*\)/[^/]*$$'`; \
>           $(MKDIR_P) "$(DESTDIR)$(bugtoolpluginsdir)/$$dir"; \
>           $(INSTALL_DATA) "$(srcdir)/$$plugin"
> "$(DESTDIR)$(bugtoolpluginsdir)/$$stem"; \
> @@ -42,13 +41,11 @@ bugtool-install-data-local:
>  UNINSTALL_LOCAL += bugtool-uninstall-local
>  bugtool-uninstall-local:
>         for plugin in $(bugtool_plugins); do \
> -         stem=`echo "$$plugin" | sed 's,ovn/,,'`; \
> -         stem=`echo "$$stem" | sed 's,utilities/bugtool/plugins/,,'`; \
> +         stem=`echo "$$plugin" | sed 's,utilities/bugtool/plugins/,,'`; \
>           rm -f "$(DESTDIR)$(bugtoolpluginsdir)/$$stem"; \
>         done
>         for plugin in $(bugtool_plugins); do \
> -         stem=`echo "$$plugin" | sed 's,ovn/,,'`; \
> -         stem=`echo "$$stem" | sed 's,utilities/bugtool/plugins/,,'`; \
> +         stem=`echo "$$plugin" | sed 's,utilities/bugtool/plugins/,,'`; \
>           dir=`expr "$$stem" : '\(.*\)/[^/]*$$'`; \
>           if [ ! -z "$$dir" ]; then \
>             rm -rf "$(DESTDIR)$(bugtoolpluginsdir)/$$dir"; \
> diff --git a/utilities/ovs-sim.in b/utilities/ovs-sim.in
> index 47329da21..08957bdf4 100755
> --- a/utilities/ovs-sim.in
> +++ b/utilities/ovs-sim.in
> @@ -70,7 +70,6 @@ fi
>
>  # Put built tools early in $PATH.
>
>  PATH=$sim_builddir/ovsdb:$sim_builddir/vswitchd:$sim_builddir/utilities:$PATH
>
> -PATH=$sim_builddir/ovn/controller:$sim_builddir/ovn/northd:$sim_builddir/ovn/utilities:$PATH
>  export PATH
>
>  rm -rf sandbox
> @@ -101,8 +100,6 @@ sim_setvars() {
>  export -f sim_setvars
>
>  ovs-vsctl () { command ovs-vsctl -vsyslog:off "$@"; }; export -f ovs-vsctl
> -ovs-nbctl () { command ovs-nbctl -vsyslog:off "$@"; }; export -f ovs-nbctl
> -ovs-sbctl () { command ovs-sbctl -vsyslog:off "$@"; }; export -f ovs-sbctl
>  vtep-ctl () { command vtep-ctl -vsyslog:off "$@"; }; export -f vtep-ctl
>
>  as() {
> @@ -187,7 +184,7 @@ $FUNCNAME: create a new interconnection network
>  usage: $FUNCNAME NETWORK
>
>  where NETWORK is the name of the new network.  Interconnection networks
> -are used with net_attach and ovn_attach.
> +are used with net_attach.
>  EOF
>          return 0
>      fi
> @@ -235,234 +232,6 @@ EOF
>  }
>  export -f net_attach
>
> -ovn_start_db() {
> -    local db=$1 model=$2 servers=$3 schema=$4
> -    local DB=$(echo $db | tr a-z A-Z)
> -    local schema_name=$(ovsdb-tool schema-name $schema)
> -
> -    case $model in
> -        standalone | backup) ;;
> -        clustered)
> -            case $servers in
> -                [1-9] | [1-9][0-9]) ;;
> -                *) echo "${db}db servers must be between 1 and 99" >&2
> -                   exit 1
> -                   ;;
> -            esac
> -            ;;
> -        *)
> -            echo "unknown ${db}db model \"$model\"" >&2
> -            exit 1
> -            ;;
> -    esac
> -
> -    ovn_start_ovsdb_server() {
> -        local i=$1; shift
> -        as ${db}$i ovsdb-server --detach --no-chdir --pidfile=$db.pid \
> -           -vsyslog:off -vconsole:off
> --log-file="$sim_base"/$db$i/$db.log \
> -           --remote=db:$schema_name,${DB}_Global,connections \
> -           --private-key=db:$schema_name,SSL,private_key \
> -           --certificate=db:$schema_name,SSL,certificate \
> -           --ca-cert=db:$schema_name,SSL,ca_cert \
> -           --ssl-protocols=db:$schema_name,SSL,ssl_protocols \
> -           --ssl-ciphers=db:$schema_name,SSL,ssl_ciphers \
> -           --unixctl=${db} --remote=punix:$db.ovsdb \
> -           "$sim_base"/$db$i/$db.db "$@"
> -    }
> -
> -    ovn_prep_db() {
> -        local i=$1
> -        mkdir "$sim_base"/${db}$i
> -        touch "$sim_base"/${db}$i/.$db.db.~lock~
> -    }
> -
> -    local n_remotes=1
> -    case $model in
> -        standalone)
> -            ovn_prep_db 1
> -            ovsdb-tool create "$sim_base"/${db}1/$db.db "$schema"
> -            ovn_start_ovsdb_server 1
> -            ;;
> -        backup)
> -            for i in 1 2; do
> -                ovn_prep_db $i
> -                ovsdb-tool create "$sim_base"/$db$i/$db.db "$schema"
> -            done
> -            ovn_start_ovsdb_server 1
> -            ovn_start_ovsdb_server 2
> --sync-from=unix:"$sim_base"/${db}1/$db.ovsdb
> -            cat <<EOF
> -The backup server of OVN $DB can be accessed by:
> -* ovn-${db}ctl --db=unix:$sim_base/${db}2/$db.ovsdb
> -* ovs-appctl -t $sim_base/${db}2/${db}
> -The backup database file is $sim_base/${db}2/$db.db
> -EOF
> -            ;;
> -        clustered)
> -            n_remotes=$servers
> -            for i in $(seq $servers); do
> -                ovn_prep_db $i
> -                if test $i = 1; then
> -                    ovsdb-tool create-cluster "$sim_base"/$db$i/$db.db
> "$schema" unix:"$sim_base"/$db$i/db.raft
> -                else
> -                    ovsdb-tool join-cluster "$sim_base"/$db$i/$db.db
> $schema_name unix:"$sim_base"/$db$i/db.raft unix:"$sim_base"/${db}1/db.raft
> -                fi
> -                ovn_start_ovsdb_server $i
> -            done
> -            for i in $(seq $servers); do
> -                ovsdb-client wait unix:"$sim_base"/${db}$i/$db.ovsdb
> $schema_name connected
> -            done
> -            ;;
> -    esac
> -
> -    remote=unix:"$sim_base"/${db}1/$db.ovsdb
> -    for i in `seq 2 $n_remotes`; do
> -        remote=$remote,unix:"$sim_base"/${db}$i/$db.ovsdb
> -    done
> -    eval OVN_${DB}_DB=\$remote
> -    eval export OVN_${DB}_DB
> -}
> -export -f ovn_start_db
> -
> -ovn_start() {
> -    local nbdb_model=standalone
> -    local nbdb_servers=3
> -    local sbdb_model=standalone
> -    local sbdb_servers=3
> -    local prev=
> -    for option; do
> -        # This option-parsing mechanism borrowed from a Autoconf-generated
> -        # configure script under the following license:
> -
> -        # Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000,
> 2001,
> -        # 2002, 2003, 2004, 2005, 2006, 2009, 2013 Free Software
> Foundation, Inc.
> -        # This configure script is free software; the Free Software
> Foundation
> -        # gives unlimited permission to copy, distribute and modify it.
> -
> -        # If the previous option needs an argument, assign it.
> -        if test -n "$prev"; then
> -            eval $prev=\$option
> -            prev=
> -            continue
> -        fi
> -        case $option in
> -            *=*) optarg=`expr "X$option" : '[^=]*=\(.*\)'` ;;
> -            *) optarg=yes ;;
> -        esac
> -
> -        case $dashdash$option in
> -            --)
> -                dashdash=yes ;;
> -            -h|--help)
> -                cat <<EOF
> -$FUNCNAME: start OVN central databases and daemons
> -usage: $FUNCNAME [OPTION...]
> -
> -This creates and initializes the central OVN databases (northbound and
> -southbound), starts their ovsdb-server daemons, and starts the ovn-northd
> -daemon.
> -
> -Options:
> -  --nbdb-model=standalone|backup|clustered    northbound database model
> -  --nbdb-servers=N     number of servers in nbdb cluster (default: 3)
> -  --sbdb-model=standalone|backup|clustered    southbound database model
> -  --sbdb-servers=N     number of servers in sbdb cluster (default: 3)
> -  -h, --help           Print this usage message.
> -EOF
> -                return
> -                ;;
> -
> -            --nbdb-s*=*)
> -                nbdb_servers=$optarg
> -                nbdb_model=clustered
> -                ;;
> -            --nbdb-s*)
> -                prev=nbdb_servers
> -                nbdb_model=clustered
> -                ;;
> -            --nbdb-m*=*)
> -                nbdb_model=$optarg
> -                ;;
> -            --nbdb-m*)
> -                prev=nbdb_model
> -                ;;
> -            --sbdb-s*=*)
> -                sbdb_servers=$optarg
> -                sbdb_model=clustered
> -                ;;
> -            --sbdb-s*)
> -                prev=sbdb_servers
> -                sbdb_model=clustered
> -                ;;
> -            --sbdb-m*=*)
> -                sbdb_model=$optarg
> -                ;;
> -            --sbdb-m*)
> -                prev=sbdb_model
> -                ;;
> -            -*)
> -                echo "unrecognized option $option (use --help for help)"
> >&2
> -                return 1
> -                ;;
> -            *)
> -                echo "$option: non-option arguments not supported (use
> --help for help)" >&2
> -                return 1
> -                ;;
> -        esac
> -        shift
> -    done
> -
> -    if test -d ovn-sb || test -d ovn-nb; then
> -        echo >&2 "OVN already started"
> -        return 1
> -    fi
> -
> -    ovn_start_db nb "$nbdb_model" "$nbdb_servers"
> "$sim_srcdir"/ovn/ovn-nb.ovsschema
> -    ovn_start_db sb "$sbdb_model" "$sbdb_servers"
> "$sim_srcdir"/ovn/ovn-sb.ovsschema
> -
> -    ovn-nbctl init
> -    ovn-sbctl init
> -
> -    mkdir "$sim_base"/northd
> -    as northd ovn-northd --ovnnb-db="$OVN_NB_DB" --ovnsb-db="$OVN_SB_DB" \
> -       $daemon_opts
> -}
> -export -f ovn_start
> -
> -ovn_attach() {
> -    if test "$1" == --help; then
> -        cat <<EOF
> -$FUNCNAME: attach default sandbox to an interconnection network for OVN
> -usage: $FUNCNAME NETWORK BRIDGE IP [MASKLEN]
> -
> -This starts by doing everything that net_attach does.  Then it configures
> the
> -specified IP and MASKLEN (e.g. 192.168.0.1 and 24) on BRIDGE and starts
> -and configures ovn-controller.
> -
> -MASKLEN defaults to 24 if it is not specified.
> -EOF
> -        return 0
> -    fi
> -    if test $# != 3 && test $# != 4; then
> -        echo >&2 "$FUNCNAME: wrong number of arguments (use --help for
> help)"
> -        return 1
> -    fi
> -
> -    local net=$1 bridge=$2 ip=$3 masklen=${4-24}
> -    net_attach $net $bridge || return $?
> -
> -    ovs-appctl netdev-dummy/ip4addr $bridge $ip/$masklen >/dev/null
> -    ovs-appctl ovs/route/add $ip/$masklen $bridge > /dev/null
> -    ovs-vsctl \
> -        -- set Open_vSwitch . external-ids:system-id=$sandbox \
> -        -- set Open_vSwitch . external-ids:ovn-remote=$OVN_SB_DB \
> -        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> -        -- set Open_vSwitch . external-ids:ovn-encap-ip=$ip\
> -        -- add-br br-int \
> -        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> -    ovn-controller --detach --no-chdir --pidfile -vconsole:off
> -vsyslog:off --log-file
> -}
> -export -f ovn_attach
> -
>  # Easy access to OVS manpages.
>  mkdir $sim_base/man
>  mandir=`cd $sim_base/man && pwd`
> @@ -494,8 +263,8 @@ rc='
>   ______________________________________________________________________
>  |
>  | You are running in a nested shell environment meant for Open vSwitch
> -| and OVN testing in simulation.   The OVS manpages are available via
> -| "man".  Please see ovs-sim(1) for more information.
> +| testing in simulation.   The OVS manpages are available via "man".
> +| Please see ovs-sim(1) for more information.
>  |
>  | Exit the shell to kill the running daemons and leave the simulation
>  | environment.
> diff --git a/xenserver/openvswitch-xen.spec.in b/xenserver/
> openvswitch-xen.spec.in
> index ba3580836..cdc341dcc 100644
> --- a/xenserver/openvswitch-xen.spec.in
> +++ b/xenserver/openvswitch-xen.spec.in
> @@ -456,7 +456,6 @@ exit 0
>  /usr/share/openvswitch/scripts/ovs-ctl
>  /usr/share/openvswitch/scripts/ovs-lib
>  /usr/share/openvswitch/scripts/ovs-vtep
> -/usr/share/openvswitch/scripts/ovndb-servers.ocf
>  /usr/share/openvswitch/vswitch.ovsschema
>  /usr/share/openvswitch/vtep.ovsschema
>  /usr/sbin/ovs-bugtool
> @@ -507,12 +506,6 @@ exit 0
>  %exclude /usr/share/openvswitch/python/*.py[co]
>  %exclude /usr/share/openvswitch/python/ovs/*.py[co]
>  %exclude /usr/share/openvswitch/python/ovs/db/*.py[co]
> -%exclude /usr/bin/ovn-*
> -%exclude /usr/share/man/man5/ovn-*
> -%exclude /usr/share/man/man7/ovn-*
> -%exclude /usr/share/man/man8/ovn-*
> -%exclude /usr/share/openvswitch/ovn-*
> -%exclude /usr/share/openvswitch/scripts/ovn-*
>
>  %files %{module_package}
>  /lib/modules/%{xen_version}/extra/openvswitch/openvswitch.ko
> --
> 2.14.5
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>


More information about the dev mailing list