[ovs-dev] [PATCH] [Patch V2] OVN: Logical-Physical Separation and L2 SW Gateways

Darrell Ball dlu998 at gmail.com
Mon Mar 21 15:47:37 UTC 2016


The following patch series implements physical-logical separation
to be used presently by gateways and localnet.
The patch series also includes L2 SW Gateway support which depends on the
physical-logical separation changes.

The physical logical separation changes allow the physical network to be
managed more easily by a dedicated CMS or CMS function and also allow
the northbound schema to remain purely logical.
Another benefit to allow more easily for additional encapsulations to be
used when interacting with physical provider networks.

The gateway changes in this patch series are for SW gateways only, as
HW gateways will be updated later.

Physical endpoints are defined here as endpoints at the border of a physical network.
This presently applies to localnet and gateways.

For localnet physical endpoints, chassis name and chassis port can be supplied as
dummy arguments or omitted. This is useful if the same localnet logical port is
distributed across multiple chassis. If chassis and port are omitted, the same
encapsulation on all HVs of a given localnet is implied and a single logical port
should be used. Then the physical endpoint is bound to the localnet logical port.
If multiple encapsulations are needed, multiple physical endpoints would be defined
and bound to separate localnet logical ports.
If no physical endpt is bound to a localnet, the encap is assumed empty.
Note that physical location information for localnet ports can also serve just
for administrative purposes as well.

For gateways, a single physical endpoint must be bound to each logical port.
The chassis name must match the chassis system-id. All 6 arguments should
be supplied for gateway physical endpoints configuration.

The specific changes are:
ovn-sb.ovsschema: Add physical endpoint table and phys_endpts to logical ports
ovn-sb.xml: Update documentation regarding physical endpoints and their
binding to logical ports. Also describe possible future encapsulation type usages.
Document the new "gw" logical port type

ovn-sbctl.c: Add configuration for physcial endpoints and binding to logical ports.

ovn-sbctl.8.in: Update the ovn-sbctl man page for physical endpoints and binding
to logical ports.

The second part of the patch series supporting SW gateways also uses physical
endpoints.
The SW gateway runs in the context of ovn-controller as other HVs.
The gateway node uses a single bridge (call it br-int) that is actively controlled by
OVN. This bridge also houses the tunnels connecting other HVs.
Additional physical bridges are created for each physical port supported
by the gateway. These bridges enforce normal action only by default.
A pair of patch ports are created to connect each LS to br-int.

A new logical port type is added for SW gateways called "gw". This is needed
to differentiate logic from HW gateway support. Changes to HW gateway
support are coming in a subsequent series.

patch.c: Physical bridge and patch port creation

physical.c: Add SW gateway flow generation support, including physical endpoint
support and update localnet flow generation to use physical endpoints.
Support gateway br-int patch ports as "physical" ports

binding.c support the use of physical endpoint for gateway (gw) logical ports

ovn-nb.xml: document the new "gw" logical port type
                    remove tag field for localnet under container support

ovn-controller.c; patch.h: Add a chassis name parameter needed for gateways
Particular logical port types are not presently specified/enforced in the NB and SB
schemas themselves. This may be to allow flexibility and ease of adding new types.

Test case updates:

ovn.at: A new test is added to exercise the SW gateway for L2 switching and also
            using physical endpoints.
            The localnet test case is updated to use physical endpoints.

v1->v2: Fix physical bridge creation logic and update based on RFC patch
series review

Signed-off-by: Darrell Ball <dball at vmware.com>
---
 ovn/controller/binding.c        |  15 +-
 ovn/controller/ovn-controller.c |   2 +-
 ovn/controller/patch.c          | 135 +++++++++++++++++-
 ovn/controller/patch.h          |   2 +-
 ovn/controller/physical.c       |  47 ++++++-
 ovn/ovn-nb.xml                  |  11 +-
 ovn/ovn-sb.ovsschema            |  23 +++-
 ovn/ovn-sb.xml                  |  92 ++++++++++++-
 ovn/utilities/ovn-sbctl.8.in    |  48 +++++++
 ovn/utilities/ovn-sbctl.c       | 295 +++++++++++++++++++++++++++++++++++++++-
 tests/ovn.at                    | 135 ++++++++++++++++++
 11 files changed, 774 insertions(+), 31 deletions(-)

diff --git a/ovn/controller/binding.c b/ovn/controller/binding.c
index d3ca9c9..d947329 100644
--- a/ovn/controller/binding.c
+++ b/ovn/controller/binding.c
@@ -181,7 +181,9 @@ binding_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
             = shash_find_and_delete(&lports, binding_rec->logical_port);
         if (iface_rec
             || (binding_rec->parent_port && binding_rec->parent_port[0] &&
-                sset_contains(&all_lports, binding_rec->parent_port))) {
+                sset_contains(&all_lports, binding_rec->parent_port))
+            || ((!strcmp(binding_rec->type, "gw")) &&
+                (binding_rec->chassis == chassis_rec))) {
             if (binding_rec->parent_port && binding_rec->parent_port[0]) {
                 /* Add child logical port to the set of all local ports. */
                 sset_add(&all_lports, binding_rec->logical_port);
@@ -207,9 +209,14 @@ binding_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
             }
         } else if (binding_rec->chassis == chassis_rec) {
             if (ctx->ovnsb_idl_txn) {
-                VLOG_INFO("Releasing lport %s from this chassis.",
-                          binding_rec->logical_port);
-                sbrec_port_binding_set_chassis(binding_rec, NULL);
+
+                if (strcmp(binding_rec->type, "gw")) {
+                    /* gw logical port bindings are not based
+                     * on external_ids mappings */
+                    sbrec_port_binding_set_chassis(binding_rec, NULL);
+                    VLOG_INFO("Releasing lport %s from this chassis.",
+                               binding_rec->logical_port);
+                }
             }
         } else if (!binding_rec->chassis
                    && !strcmp(binding_rec->type, "localnet")) {
diff --git a/ovn/controller/ovn-controller.c b/ovn/controller/ovn-controller.c
index 5852e35..ff3cdac 100644
--- a/ovn/controller/ovn-controller.c
+++ b/ovn/controller/ovn-controller.c
@@ -293,7 +293,7 @@ main(int argc, char *argv[])
         }
 
         if (br_int) {
-            patch_run(&ctx, br_int, &local_datapaths);
+            patch_run(&ctx, br_int, &local_datapaths, chassis_id);
 
             struct lport_index lports;
             struct mcgroup_index mcgroups;
diff --git a/ovn/controller/patch.c b/ovn/controller/patch.c
index 753ce3e..1cc7d6c 100644
--- a/ovn/controller/patch.c
+++ b/ovn/controller/patch.c
@@ -276,9 +276,137 @@ add_logical_patch_ports(struct controller_ctx *ctx,
     }
 }
 
+static const struct ovsrec_bridge *
+create_br_physical_for_gateway(struct controller_ctx *ctx,
+                               const struct ovsrec_open_vswitch *cfg,
+                               const char *bridge_name)
+{
+    if (!ctx->ovs_idl_txn) {
+        return NULL;
+    }
+
+    ovsdb_idl_txn_add_comment(ctx->ovs_idl_txn,
+            "ovn-controller: creating gateway physical bridge '%s'",
+            bridge_name);
+
+    struct ovsrec_interface *iface;
+    iface = ovsrec_interface_insert(ctx->ovs_idl_txn);
+    ovsrec_interface_set_name(iface, bridge_name);
+    ovsrec_interface_set_type(iface, "internal");
+
+    struct ovsrec_port *port;
+    port = ovsrec_port_insert(ctx->ovs_idl_txn);
+    ovsrec_port_set_name(port, bridge_name);
+    ovsrec_port_set_interfaces(port, &iface, 1);
+
+    struct ovsrec_bridge *bridge;
+    bridge = ovsrec_bridge_insert(ctx->ovs_idl_txn);
+    ovsrec_bridge_set_name(bridge, bridge_name);
+    ovsrec_bridge_set_fail_mode(bridge, "standalone");
+    ovsrec_bridge_set_ports(bridge, &port, 1);
+
+    struct ovsrec_bridge **bridges;
+    size_t bytes = sizeof *bridges * cfg->n_bridges;
+    bridges = xmalloc(bytes + sizeof *bridges);
+    memcpy(bridges, cfg->bridges, bytes);
+    bridges[cfg->n_bridges] = bridge;
+    ovsrec_open_vswitch_verify_bridges(cfg);
+    ovsrec_open_vswitch_set_bridges(cfg, bridges, cfg->n_bridges + 1);
+
+    return bridge;
+}
+
+static const struct ovsrec_bridge *
+get_and_create_br_as_needed(struct controller_ctx *ctx, const char *br_name)
+{
+    const struct ovsrec_open_vswitch *cfg;
+
+    cfg = ovsrec_open_vswitch_first(ctx->ovs_idl);
+    if (!cfg) {
+        return NULL;
+    }
+
+    const struct ovsrec_bridge *br;
+    br = get_bridge(ctx->ovs_idl, br_name);
+    if (!br) {
+        return create_br_physical_for_gateway(ctx, cfg, br_name);
+    }
+
+    return br;
+}
+
+/* Note that create_patch_port checks/avoids redundant creates */
+static void
+add_gateway_ls_br_and_patch_ports(struct controller_ctx *ctx,
+                                  const struct ovsrec_bridge *br_int,
+                                  struct shash *existing_ports,
+                                  const char *chassis_id)
+{
+    const struct ovsrec_bridge *br_gateway_physical = NULL;
+
+    if (!br_int || !chassis_id) {
+        return;
+    }
+
+    const struct sbrec_port_binding *binding;
+    const struct sbrec_physical_endpoint * phys_endpt_rec;
+    const struct sbrec_chassis *chassis_rec;
+
+    SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
+
+        if (!binding || !binding->phys_endpts ||
+            strcmp(binding->type, "gw")) {
+            /* Not a binding for a gw port. */
+            continue;
+        }
+
+        if(!binding->phys_endpts[0]) {
+            VLOG_ERR("No physical endpt configured for gw port '%s' ",
+                     binding->logical_port);
+            return;
+        }
+
+        phys_endpt_rec = binding->phys_endpts[0];
+        chassis_rec = phys_endpt_rec->chassis;
+
+        if(!chassis_rec || !chassis_rec->name) {
+            VLOG_ERR("No chassis configured "
+                     "for gw port '%s' in phys endpt ",
+                     binding->logical_port);
+            return;
+        }
+
+        /* The logical port is not on this chassis */
+        if(strcmp(chassis_rec->name, chassis_id)) {
+           continue;
+        }
+
+        /* Sync phys endpt and port binding chassis record */
+        if (ctx->ovnsb_idl_txn) {
+            sbrec_port_binding_set_chassis(binding, chassis_rec);
+        }
+
+        br_gateway_physical =
+            get_and_create_br_as_needed(ctx, phys_endpt_rec->chassis_port);
+
+        char *name1 = patch_port_name(br_int->name, binding->logical_port);
+        char *name2 = patch_port_name(binding->logical_port, br_int->name);
+
+        create_patch_port(ctx, "ovn-gateway-patch-port", binding->logical_port,
+                          br_int, name1, br_gateway_physical, name2, existing_ports);
+        create_patch_port(ctx, "ovn-gateway-patch-port", binding->logical_port,
+                          br_gateway_physical, name2, br_int, name1, existing_ports);
+
+        free(name1);
+        free(name2);
+
+    }
+
+}
+
 void
 patch_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
-          struct hmap *local_datapaths)
+          struct hmap *local_datapaths, const char *chassis_id)
 {
     if (!ctx->ovs_idl_txn) {
         return;
@@ -289,7 +417,8 @@ patch_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
     const struct ovsrec_port *port;
     OVSREC_PORT_FOR_EACH (port, ctx->ovs_idl) {
         if (smap_get(&port->external_ids, "ovn-localnet-port") ||
-            smap_get(&port->external_ids, "ovn-logical-patch-port")) {
+            smap_get(&port->external_ids, "ovn-logical-patch-port") ||
+            smap_get(&port->external_ids, "ovn-gateway-patch-port")) {
             shash_add(&existing_ports, port->name, port);
         }
     }
@@ -298,6 +427,8 @@ patch_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
      * 'existing_ports' any patch ports that do exist in the database and
      * should be there. */
     add_bridge_mappings(ctx, br_int, &existing_ports, local_datapaths);
+    add_gateway_ls_br_and_patch_ports(ctx, br_int,
+                                      &existing_ports, chassis_id);
     add_logical_patch_ports(ctx, br_int, &existing_ports);
 
     /* Now 'existing_ports' only still contains patch ports that exist in the
diff --git a/ovn/controller/patch.h b/ovn/controller/patch.h
index 38ee7a8..f040b25 100644
--- a/ovn/controller/patch.h
+++ b/ovn/controller/patch.h
@@ -27,6 +27,6 @@ struct hmap;
 struct ovsrec_bridge;
 
 void patch_run(struct controller_ctx *, const struct ovsrec_bridge *br_int,
-               struct hmap *local_datapaths);
+               struct hmap *local_datapaths, const char *chassis_id);
 
 #endif /* ovn/patch.h */
diff --git a/ovn/controller/physical.c b/ovn/controller/physical.c
index 657c3e2..5ed5f8e 100644
--- a/ovn/controller/physical.c
+++ b/ovn/controller/physical.c
@@ -169,6 +169,8 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
                                         "ovn-localnet-port");
         const char *logpatch = smap_get(&port_rec->external_ids,
                                         "ovn-logical-patch-port");
+        const char *gateway_patch = smap_get(&port_rec->external_ids,
+                                        "ovn-gateway-patch-port");
 
         for (int j = 0; j < port_rec->n_interfaces; j++) {
             const struct ovsrec_interface *iface_rec = port_rec->interfaces[j];
@@ -189,6 +191,9 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
                 /* localnet patch ports can be handled just like VIFs. */
                 simap_put(&localvif_to_ofport, localnet, ofport);
                 break;
+            } else if (is_patch && gateway_patch) {
+                /* gateway patch ports can be handled just like VIFs. */
+                simap_put(&localvif_to_ofport, gateway_patch, ofport);
             } else if (is_patch && logpatch) {
                 /* Logical patch ports can be handled just like VIFs. */
                 simap_put(&localvif_to_ofport, logpatch, ofport);
@@ -231,6 +236,7 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
     /* Set up flows in table 0 for physical-to-logical translation and in table
      * 64 for logical-to-physical translation. */
     const struct sbrec_port_binding *binding;
+    const struct sbrec_physical_endpoint * phys_endpt_rec;
     SBREC_PORT_BINDING_FOR_EACH (binding, ctx->ovnsb_idl) {
         /* Find the OpenFlow port for the logical port, as 'ofport'.  This is
          * one of:
@@ -267,8 +273,28 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
         } else {
             ofport = u16_to_ofp(simap_get(&localvif_to_ofport,
                                           binding->logical_port));
-            if (!strcmp(binding->type, "localnet") && ofport && binding->tag) {
-                tag = *binding->tag;
+
+            /* For localnet and gw logical ports without phys_endpt
+             * binding, a tag of 0 is the default */
+
+            /* gw logical port type added for illustrative purposes
+             * in this patch  */
+
+            if ((!strcmp(binding->type, "localnet") ||
+                (!strcmp(binding->type, "gw"))) && ofport &&
+                 binding->phys_endpts) {
+
+                /* Use any phys_endpt for localnet if shared port
+                 * name; if localnet port is unique name, then there is a
+                 * single phys_endpt.
+                 * gw logical ports have a single phys_endpt */
+                phys_endpt_rec = binding->phys_endpts[0];
+
+                /* only single vlan encap is supported initially */
+                if (phys_endpt_rec && (!strcmp(phys_endpt_rec->type, "vlan"))) {
+                    /* valid values verified on configuration */
+                    (void) str_to_int(phys_endpt_rec->ingress_encap, 10, &tag);
+                }
             }
         }
 
@@ -326,7 +352,13 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
             /* Match a VLAN tag and strip it, including stripping priority tags
              * (e.g. VLAN ID 0).  In the latter case we'll add a second flow
              * for frames that lack any 802.1Q header later. */
-            if (tag || !strcmp(binding->type, "localnet")) {
+
+            /* gw logical port type added for illustrative purposes
+             * in this patch */
+            if (tag ||
+                (!strcmp(binding->type, "localnet")) ||
+                (!strcmp(binding->type, "gw"))) {
+
                 match_set_dl_vlan(&match, htons(tag));
                 ofpact_put_STRIP_VLAN(&ofpacts);
             }
@@ -350,7 +382,13 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
             ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG,
                             tag ? 150 : 100, &match, &ofpacts);
 
-            if (!tag && !strcmp(binding->type, "localnet")) {
+            /* gw logical port type added for illustrative purposes
+             * in this patch
+             */
+            if (!tag &&
+                    ((!strcmp(binding->type, "localnet")) ||
+                     (!strcmp(binding->type, "gw")))) {
+
                 /* Add a second flow for frames that lack any 802.1Q
                  * header.  For these, drop the OFPACT_STRIP_VLAN
                  * action. */
@@ -648,6 +686,7 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
      * xxx Due to resubmitting to table 16, broadcasts will be re-sent to
      * xxx all logical ports, including non-local ones which could cause
      * xxx duplicate packets to be received by multiply-connected gateways. */
+
     HMAP_FOR_EACH (tun, hmap_node, &tunnels) {
         if (tun->type != VXLAN) {
             continue;
diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml
index 5c8e942..d902288 100644
--- a/ovn/ovn-nb.xml
+++ b/ovn/ovn-nb.xml
@@ -138,6 +138,10 @@
           <dd>
             A port to a logical switch on a VTEP gateway.
           </dd>
+          <dt><code>gw</code></dt>
+          <dd>
+            A port to a logical switch on a software gateway.
+          </dd>
         </dl>
       </column>
     </group>
@@ -242,13 +246,6 @@
           The VLAN tag in the network traffic associated with a container's
           network interface.
         </p>
-
-        <p>
-          When <ref column="type"/> is set to <code>localnet</code>, this can
-          be set to indicate that the port represents a connection to a
-          specific VLAN on a locally accessible network. The VLAN ID is used to
-          match incoming traffic and is also added to outgoing traffic.
-        </p>
       </column>
     </group>
 
diff --git a/ovn/ovn-sb.ovsschema b/ovn/ovn-sb.ovsschema
index ead733b..45d0f0e 100644
--- a/ovn/ovn-sb.ovsschema
+++ b/ovn/ovn-sb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_Southbound",
-    "version": "1.1.0",
-    "cksum": "1223981720 5320",
+    "version": "1.1.1",
+    "cksum": "18101380 6346",
     "tables": {
         "Chassis": {
             "columns": {
@@ -70,6 +70,21 @@
                              "min": 0, "max": "unlimited"}}},
             "indexes": [["tunnel_key"]],
             "isRoot": true},
+        "Physical_Endpoint": {
+            "columns": {
+                "name": {"type": "string"},
+                "chassis": {"type": {"key": {"type": "uuid",
+                                             "refTable": "Chassis",
+                                             "refType": "weak"},
+                                     "min": 0, "max": 1}},
+                "chassis_port": {"type": {"key": "string", "min": 0, "max": 1}},
+                "type": {"type": {"key": {
+                           "type": "string",
+                           "enum": ["set", ["vlan"]]}}},
+                "ingress_encap": {"type": "string"},
+                "egress_encap": {"type": "string"}},
+            "indexes": [["name"]],
+            "isRoot": true},
         "Port_Binding": {
             "columns": {
                 "logical_port": {"type": "string"},
@@ -95,6 +110,10 @@
                                              "refTable": "Chassis",
                                              "refType": "weak"},
                                      "min": 0, "max": 1}},
+                "phys_endpts": {"type": {"key": {"type": "uuid",
+                                             "refTable": "Physical_Endpoint",
+                                             "refType": "weak"},
+                                     "min": 0, "max": "unlimited"}},
                 "mac": {"type": {"key": "string",
                                  "min": 0,
                                  "max": "unlimited"}}},
diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml
index d62f1e4..5e7a468 100644
--- a/ovn/ovn-sb.xml
+++ b/ovn/ovn-sb.xml
@@ -1168,6 +1168,81 @@ tcp.flags = RST;
     </group>
   </table>
 
+  <table name="Physical_Endpoint" title="Endpoints on physical networks ">
+   <p>
+      Each row in this table identifies an endpoint on a physical access
+      network.  The <ref table="Port_Binding"/> table references physical
+      endpoints for localnet and gateways. A physical endpoint comprises
+      both a location on a chassis port and encapsulation for access.
+      A given chassis port location can have several encapsulations
+      and therefore physical endpoints.
+    </p>
+
+    <p>
+      For localnet physical endpoints, chassis name and chassis port
+      can be supplied as dummy arguments or omitted. If chassis and
+      port are omitted, the same encapsulation on all HVs part of a
+      given localnet is implied and a single logical port could be used.
+      Then, the physical endpoint is bound to the localnet logical port.
+      If multiple encapsulations are needed, multiple physical endpoints
+      would be defined and bound to separate localnet logical ports.
+      If no physical endpt is bound to a localnet, encap is assumed empty.
+    </p>
+
+    <p>
+      For gateways, a single physical endpoint must be bound to each
+      logical port. The chassis name of the physical endpoint must
+      match the chassis system-id.
+    </p>
+
+    <column name="name">
+      Unique name to reference the physical endpoint.
+    </column>
+
+    <column name="chassis">
+      A transport node bordering on a physical provider network;
+      gateways are an example and HVs that have a localnet logical
+      port are another example. 
+    </column>
+
+    <column name="chassis_port">
+      A port, such as eth0, on a transport node that borders
+      on a physical provider network. Two use cases are gateways
+      and localnet.
+    </column>
+
+    <column name="type">
+      The encapsulation type of the physical endpoint. Types include
+      a single vlan. This could later be extended to include mpls
+      labels, IP tunnel outer header and dual vlans. Note that
+      encapsulations are often directionally different, meaning
+      the encapsulation expected on ingress is different from the
+      encapsulation for egressing packets; mpls labels and IP tunnels
+      being two examples.
+      Single mpls label support would be later defined with default
+      values of 0 for exp, 1 for BOS and 64 for TTL. Default would
+      also imply downstream label assigned with ethertype of 0x8847.
+      An example would be:
+      type: mpls_label_def
+      egress_encap: 1023
+      ingress_encap: 667
+      which means a mpls label with 667 is expected on ingress and
+      a label with 1023 is used on egress. mpls_label_def implies
+      no vlan encapsulation in addition to the mpls. This behavior
+      can be further extended with a vlan being present in addition
+      to an mpls label.  
+    </column>
+
+    <column name="ingress_encap">
+      The ingress encapsulation type of the physical endpoint.
+    </column>
+
+    <column name="egress_encap">
+      The egress encapsulation type of the physical endpoint.
+    </column>
+
+  </table>
+
   <table name="Port_Binding" title="Physical-Logical Port Bindings">
     <p>
       Most rows in this table identify the physical location of a logical port.
@@ -1222,6 +1297,13 @@ tcp.flags = RST;
         <code>ovn-controller</code>/<code>ovn-controller-vtep</code>.
       </column>
 
+      <column name="phys_endpts">
+        The physical endpoints that the logical port resides on.
+        To successfully identify phys_endpts, this column must be
+        <ref table="Physical_Endpoint"/> records.  This is populated by a
+        <code>CMS</code> physical management component.
+      </column>
+
       <column name="tunnel_key">
         <p>
           A number that represents the logical port in the key (e.g. STT key or
@@ -1285,6 +1367,10 @@ tcp.flags = RST;
             table="Port_Binding"/>:<code>vtep-logical-switch</code> must also
             be defined.
           </dd>
+          <dt><code>gw</code></dt>
+          <dd>
+            A port to a logical switch on a software gateway chassis.
+          </dd>
         </dl>
       </column>
     </group>
@@ -1329,12 +1415,6 @@ tcp.flags = RST;
           another chassis.
         </p>
       </column>
-
-      <column name="tag">
-        If set, indicates that the port represents a connection to a specific
-        VLAN on a locally accessible network. The VLAN ID is used to match
-        incoming traffic and is also added to outgoing traffic.
-      </column>
     </group>
 
     <group title="VTEP Options">
diff --git a/ovn/utilities/ovn-sbctl.8.in b/ovn/utilities/ovn-sbctl.8.in
index e4e4431..67c78ee 100644
--- a/ovn/utilities/ovn-sbctl.8.in
+++ b/ovn/utilities/ovn-sbctl.8.in
@@ -131,6 +131,30 @@ Without \fB\-\-if\-exists\fR, attempting to delete a chassis that does
 not exist is an error.  With \fB\-\-if\-exists\fR, attempting to
 delete a chassis that does not exist has no effect.
 .
+.SS "Physical Endpoint Commands"
+These commands manipulate \fBOVN_Southbound\fR physical endpoint.
+.
+.IP "[\fB\-\-may\-exist\fR] \fBphys\-endpt\-add \fIphys\-endpt\fR \fIchassis\fR \fIchassis\-port\fR \fIencap\-type\fR \fIingress\-encap\fR \fIegress\-encap\fR"
+Creates a new physical endpoint named \fIphys\-endpt\fR.  \fIchassis\fR is a
+chassis name on a physical network edge, such as localnet or gateway.
+\fIchassis\-port\fR is a chassis port on a physical edge.
+\fIencap\-type\fR is a type of encap such as vlan, double vlan, mpls or one
+of several IP tunnel encaps. \fIingress\-encap\fR is the
+ingress encap value. \fIegress\-encap\fR is the egress encap value.
+Both encap values may be the same in some cases.
+Chassis and chassis-port are optional arguments.
+.IP
+Without \fB\-\-may\-exist\fR, attempting to create a phys-endpt
+that exists is an error.  With \fB\-\-may\-exist\fR, this command does
+nothing if phys-endpt already exists.
+.
+.IP "[\fB\-\-if\-exists\fR] \fBphys\-endpt\-del \fIphys\-endpt\fR"
+Deletes \fIphys\-endpt\fR.
+.IP
+Without \fB\-\-if\-exists\fR, attempting to delete a phys-endpt that does
+not exist is an error.  With \fB\-\-if\-exists\fR, attempting to
+delete a phys-endpt that does not exist has no effect.
+.
 .SS "Port binding Commands"
 .
 These commands manipulate \fBOVN_Southbound\fR port bindings.
@@ -150,6 +174,30 @@ Without \fB\-\-if\-exists\fR, attempting to unbind a logical port
 that is not bound is an error.  With \fB\-\-if\-exists\fR, attempting
 to unbind logical port that is not bound has no effect.
 .
+.SS "Port binding to physical endpoint Commands"
+.
+These commands manipulate \fBOVN_Southbound\fR port binding to physical endpoints.
+.
+.IP "[\fB\-\-may\-exist\fR] \fBlport\-bind\-phys\-endpt \fIlogical\-port\fR \fIphys\-endpt\fR"
+Binds the logical port named \fIlogical\-port\fR to \fIphys\-endpts\fR, which
+is a common separated list of physical endpoints. Localnet logical ports
+can be bound to zero, one or more than one physical endpoint. If no
+physical endpoint is bound for localnet, the encap is empty. A single
+name for a localnet port implies a single encap; multiple names allow
+different encaps. Gateway logical ports should be bound to one physical
+endpoint for the gateway to function properly.
+.IP
+Without \fB\-\-may\-exist\fR, attempting to bind a logical port that
+has already been bound is an error.  With \fB\-\-may\-exist\fR, this
+command does nothing if \fIlogical\-port\fR has already been bound.
+.
+.IP "[\fB\-\-if\-exists\fR] \fBlport\-unbind\-phys\-endpt\fR \fIlogical\-port\fR"
+Resets the binding of \fIlogical\-port\fR to \fINULL\fR.
+.IP
+Without \fB\-\-if\-exists\fR, attempting to unbind a logical port
+that is not bound is an error.  With \fB\-\-if\-exists\fR, attempting
+to unbind logical port that is not bound has no effect.
+.
 .SS "Logical Flow Commands"
 .
 .IP "\fBlflow\-list\fR [\fIlogical\-datapath\fR]"
diff --git a/ovn/utilities/ovn-sbctl.c b/ovn/utilities/ovn-sbctl.c
index 0f402cd..382128b 100644
--- a/ovn/utilities/ovn-sbctl.c
+++ b/ovn/utilities/ovn-sbctl.c
@@ -311,11 +311,25 @@ Chassis commands:\n\
                                            and ENCAP-IP\n\
   chassis-del CHASSIS         delete CHASSIS and all of its encaps\n\
                               and gateway_ports\n\
+Physical Endpoint commands:\n\
+  phys-endpt-add PHYS-ENDPT CHASSIS PORT TYPE ING-ENC EGR-ENC\n\
+                                 create a new phys endpt named\n\
+                                 PHYS-ENDPT on CHASSIS/PORT\n\
+                                 with TYPE encap\n\
+                                 and encap values ING-ENC EGR-ENC\n\
+                                 chassis and port may be omitted \n\
+  phys-endpt-del PHYS-ENDPT      delete PHYS-ENDPT \n\
 \n\
 Port binding commands:\n\
   lport-bind LPORT CHASSIS    bind logical port LPORT to CHASSIS\n\
   lport-unbind LPORT          reset the port binding of logical port LPORT\n\
 \n\
+Port binding Phys Endpt commands:\n\
+  lport-bind-phys-endpt LPORT PHYS-ENDPT\n\
+                             bind logical port LPORT to PHYS-ENDPT\n\
+  lport-unbind-phys-endpt LPORT\n\
+                          reset the binding of LPORT to PHYS-ENDPT\n\
+\n\
 Logical flow commands:\n\
   lflow-list [DATAPATH]       List logical flows for all or a single datapath\n\
   dump-flows [DATAPATH]       Alias for lflow-list\n\
@@ -357,6 +371,9 @@ struct sbctl_context {
     struct shash chassis;
     /* Maps from lport name to struct sbctl_port_binding. */
     struct shash port_bindings;
+
+    /* Maps from phys_endpt name to struct sbctl_physical_endpoint. */
+    struct shash phys_endpts_name_to_rec;
 };
 
 /* Casts 'base' into 'struct sbctl_context'. */
@@ -374,6 +391,10 @@ struct sbctl_port_binding {
     const struct sbrec_port_binding *bd_cfg;
 };
 
+struct sbctl_physical_endpoint {
+    const struct sbrec_physical_endpoint *phys_endpt_db_rec;
+};
+
 static void
 sbctl_context_invalidate_cache(struct ctl_context *ctx)
 {
@@ -382,9 +403,10 @@ sbctl_context_invalidate_cache(struct ctl_context *ctx)
     if (!sbctl_ctx->cache_valid) {
         return;
     }
-    sbctl_ctx->cache_valid = false;
     shash_destroy_free_data(&sbctl_ctx->chassis);
     shash_destroy_free_data(&sbctl_ctx->port_bindings);
+    shash_destroy_free_data(&sbctl_ctx->phys_endpts_name_to_rec);
+    sbctl_ctx->cache_valid = false;
 }
 
 static void
@@ -393,7 +415,8 @@ sbctl_context_populate_cache(struct ctl_context *ctx)
     struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
     const struct sbrec_chassis *chassis_rec;
     const struct sbrec_port_binding *port_binding_rec;
-    struct sset chassis, port_bindings;
+    const struct sbrec_physical_endpoint *phys_endpt_db_rec;
+    struct sset chassis, port_bindings, physical_endpoints_set;
 
     if (sbctl_ctx->cache_valid) {
         /* Cache is already populated. */
@@ -402,6 +425,8 @@ sbctl_context_populate_cache(struct ctl_context *ctx)
     sbctl_ctx->cache_valid = true;
     shash_init(&sbctl_ctx->chassis);
     shash_init(&sbctl_ctx->port_bindings);
+    shash_init(&sbctl_ctx->phys_endpts_name_to_rec);
+
     sset_init(&chassis);
     SBREC_CHASSIS_FOR_EACH(chassis_rec, ctx->idl) {
         struct sbctl_chassis *ch;
@@ -435,10 +460,30 @@ sbctl_context_populate_cache(struct ctl_context *ctx)
                   bd);
     }
     sset_destroy(&port_bindings);
+
+    sset_init(&physical_endpoints_set);
+    SBREC_PHYSICAL_ENDPOINT_FOR_EACH(phys_endpt_db_rec, ctx->idl) {
+        struct sbctl_physical_endpoint *phys_endpt_lrec;
+
+        if (!sset_add(&physical_endpoints_set, phys_endpt_db_rec->name)) {
+            VLOG_WARN("database contains duplicate physical endpoint record "
+                      "for name (%s)",
+					  phys_endpt_db_rec->name);
+            continue;
+        }
+
+        phys_endpt_lrec
+		     = xmalloc(sizeof *phys_endpt_lrec);
+        phys_endpt_lrec->phys_endpt_db_rec = phys_endpt_db_rec;
+        shash_add(&sbctl_ctx->phys_endpts_name_to_rec,
+                  phys_endpt_db_rec->name, phys_endpt_lrec);
+    }
+    sset_destroy(&physical_endpoints_set);
+
 }
 
 static void
-check_conflicts(struct sbctl_context *sbctl_ctx, const char *name,
+chassis_check_conflict(struct sbctl_context *sbctl_ctx, const char *name,
                 char *msg)
 {
     if (shash_find(&sbctl_ctx->chassis, name)) {
@@ -448,6 +493,18 @@ check_conflicts(struct sbctl_context *sbctl_ctx, const char *name,
     free(msg);
 }
 
+static void
+physical_endpoint_check_conflict(
+          struct sbctl_context *sbctl_ctx, const char *name,
+          char *msg)
+{
+    if (shash_find(&sbctl_ctx->phys_endpts_name_to_rec, name)) {
+        ctl_fatal("%s because a physical endpoint named %s already exists",
+                    msg, name);
+    }
+    free(msg);
+}
+
 static struct sbctl_chassis *
 find_chassis(struct sbctl_context *sbctl_ctx, const char *name,
              bool must_exist)
@@ -480,6 +537,23 @@ find_port_binding(struct sbctl_context *sbctl_ctx, const char *name,
     return bd;
 }
 
+static struct sbctl_physical_endpoint *
+find_phys_endpt(struct sbctl_context *sbctl_ctx, const char *name,
+                bool must_exist)
+{
+    struct sbctl_physical_endpoint *phys_endpt_lrec;
+
+    ovs_assert(sbctl_ctx->cache_valid);
+
+    phys_endpt_lrec =
+        shash_find_data(&sbctl_ctx->phys_endpts_name_to_rec, name);
+    if (must_exist && !phys_endpt_lrec) {
+        ctl_fatal("no physical endpoint named %s", name);
+    }
+
+    return phys_endpt_lrec;
+}
+
 static void
 pre_get_info(struct ctl_context *ctx)
 {
@@ -489,8 +563,16 @@ pre_get_info(struct ctl_context *ctx)
     ovsdb_idl_add_column(ctx->idl, &sbrec_encap_col_type);
     ovsdb_idl_add_column(ctx->idl, &sbrec_encap_col_ip);
 
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_name);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_chassis);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_chassis_port);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_type);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_ingress_encap);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_physical_endpoint_col_egress_encap);
+
     ovsdb_idl_add_column(ctx->idl, &sbrec_port_binding_col_logical_port);
     ovsdb_idl_add_column(ctx->idl, &sbrec_port_binding_col_chassis);
+    ovsdb_idl_add_column(ctx->idl, &sbrec_port_binding_col_phys_endpts);
 
     ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_logical_datapath);
     ovsdb_idl_add_column(ctx->idl, &sbrec_logical_flow_col_pipeline);
@@ -518,6 +600,13 @@ static struct cmd_show_table cmd_show_tables[] = {
       NULL},
      {NULL, NULL, NULL}},
 
+     {&sbrec_table_physical_endpoint,
+	  &sbrec_physical_endpoint_col_name,
+	  {&sbrec_physical_endpoint_col_chassis,
+	   &sbrec_physical_endpoint_col_chassis_port,
+	   &sbrec_physical_endpoint_col_ingress_encap},
+	  {NULL, NULL, NULL}},
+
     {NULL, NULL, {NULL, NULL, NULL}, {NULL, NULL, NULL}},
 };
 
@@ -541,7 +630,7 @@ cmd_chassis_add(struct ctl_context *ctx)
             return;
         }
     }
-    check_conflicts(sbctl_ctx, ch_name,
+    chassis_check_conflict(sbctl_ctx, ch_name,
                     xasprintf("cannot create a chassis named %s", ch_name));
 
     char *tokstr = xstrdup(encap_types);
@@ -598,6 +687,113 @@ cmd_chassis_del(struct ctl_context *ctx)
 }
 
 static void
+cmd_phys_endpt_add(struct ctl_context *ctx)
+{
+    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
+    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+    const char *phys_endpt_names, *chassis_name, *chassis_port_name;
+    const char *type_name, *ing_encap, *egr_encap;
+    struct sbctl_chassis *sbctl_ch;
+    int v;
+
+    if (ctx->argc == 7) {
+        phys_endpt_names = ctx->argv[1];
+        chassis_name = ctx->argv[2];
+        chassis_port_name = ctx->argv[3];
+        type_name = ctx->argv[4];
+        ing_encap = ctx->argv[5];
+        egr_encap = ctx->argv[6];
+    } else if (ctx->argc == 5) {
+        phys_endpt_names = ctx->argv[1];
+        chassis_name = NULL;
+        chassis_port_name = NULL;
+        type_name = ctx->argv[2];
+        ing_encap = ctx->argv[3];
+        egr_encap = ctx->argv[4];
+    } else {
+        ctl_fatal("phys endpt config must have 6 args "
+                  "or 4 args if no chassis and chassis port "
+                  "is supplied ");
+    }
+
+    sbctl_context_populate_cache(ctx);
+    if (may_exist) {
+        struct sbctl_physical_endpoint
+            *phys_endpt_lrec =
+               find_phys_endpt(
+                    sbctl_ctx, phys_endpt_names, false);
+        if (phys_endpt_lrec) {
+            return;
+        }
+    }
+    physical_endpoint_check_conflict(
+        sbctl_ctx, phys_endpt_names,
+        xasprintf("cannot create physical endpoint %s",
+                  phys_endpt_names));
+
+    /* Reminder: Splice out a encap verify function.
+       Presently only supporting single vlan with same
+       value for ingress and egress */
+    if ( (strcmp(type_name, "vlan")) ||
+        (!str_to_int(ing_encap, 10, &v) || v < 0 || v > 4095) ||
+        (!str_to_int(egr_encap, 10, &v) || v < 0 || v > 4095) ||
+        (strcmp(ing_encap, egr_encap))) {
+        ctl_fatal("phys endpt (%s) unsupported encap ",
+                  phys_endpt_names);
+    }
+
+    struct sbrec_physical_endpoint *phys_endpt_db_rec =
+       sbrec_physical_endpoint_insert(ctx->txn);
+
+    sbrec_physical_endpoint_set_name(
+            phys_endpt_db_rec,
+            phys_endpt_names);
+
+    sbctl_ch = find_chassis(sbctl_ctx, chassis_name, true);
+    sbrec_physical_endpoint_set_chassis(
+            phys_endpt_db_rec,
+			sbctl_ch->ch_cfg);
+
+    sbrec_physical_endpoint_set_chassis_port(
+            phys_endpt_db_rec,
+			chassis_port_name);
+    sbrec_physical_endpoint_set_type(
+            phys_endpt_db_rec,
+            type_name);
+    sbrec_physical_endpoint_set_ingress_encap(
+            phys_endpt_db_rec,
+			ing_encap);
+    sbrec_physical_endpoint_set_egress_encap(
+            phys_endpt_db_rec,
+            egr_encap);
+
+    sbctl_context_invalidate_cache(ctx);
+}
+
+static void
+cmd_phys_endpt_del(struct ctl_context *ctx)
+{
+    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
+    bool must_exist = !shash_find(&ctx->options, "--if-exists");
+    struct sbctl_physical_endpoint *phys_endpt_lrec;
+
+    sbctl_context_populate_cache(ctx);
+    phys_endpt_lrec = find_phys_endpt(
+            sbctl_ctx, ctx->argv[1], must_exist);
+    if (phys_endpt_lrec) {
+        if (phys_endpt_lrec->phys_endpt_db_rec) {
+
+            sbrec_physical_endpoint_delete(
+                phys_endpt_lrec->phys_endpt_db_rec);
+        }
+        shash_find_and_delete(
+                     &sbctl_ctx->phys_endpts_name_to_rec,
+                     ctx->argv[1]);
+        free(phys_endpt_lrec);
+    }
+}
+
+static void
 cmd_lport_bind(struct ctl_context *ctx)
 {
     struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
@@ -642,6 +838,79 @@ cmd_lport_unbind(struct ctl_context *ctx)
     }
 }
 
+static void
+cmd_lport_bind_phys_endpts(struct ctl_context *ctx)
+{
+    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
+    bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+    struct sbctl_port_binding *sbctl_bd;
+    struct sbctl_physical_endpoint *phys_endpt_lrec;
+    char *lport_name, *phys_endpt_names;
+
+
+    /* port_binding must exist, chassis must exist! */
+    lport_name = ctx->argv[1];
+    phys_endpt_names = ctx->argv[2];
+
+    sbctl_context_populate_cache(ctx);
+    sbctl_bd = find_port_binding(sbctl_ctx, lport_name, true);
+
+    if (sbctl_bd->bd_cfg->phys_endpts) {
+        if (may_exist) {
+            return;
+        } else {
+            ctl_fatal("lport (%s) already bound to phys_endpts ",
+                      lport_name);
+        }
+    }
+
+    /* Parse the individual endpoints from the endpoints string
+     * and dump them into a set
+     */
+    char *tokstr = xstrdup(phys_endpt_names);
+    char *token, *save_ptr = NULL;
+    struct sset phys_endpt_names_set = SSET_INITIALIZER(&phys_endpt_names_set);
+    for (token = strtok_r(tokstr, ",", &save_ptr); token != NULL;
+         token = strtok_r(NULL, ",", &save_ptr)) {
+        sset_add(&phys_endpt_names_set, token);
+    }
+    free(tokstr);
+
+    size_t num_phys_endpts = sset_count(&phys_endpt_names_set);
+    struct sbrec_physical_endpoint **phys_endpts =
+        xmalloc(num_phys_endpts * sizeof *phys_endpts);
+    const char *phys_endpt_name;
+    int i = 0;
+    SSET_FOR_EACH (phys_endpt_name, &phys_endpt_names_set){
+        phys_endpt_lrec = find_phys_endpt(sbctl_ctx, phys_endpt_name, true);
+        if (phys_endpt_lrec) {
+            phys_endpts[i] = CONST_CAST(struct sbrec_physical_endpoint *,
+                                        phys_endpt_lrec->phys_endpt_db_rec);
+            i++;
+        }
+    }
+    sset_destroy(&phys_endpt_names_set);
+
+    sbrec_port_binding_set_phys_endpts(sbctl_bd->bd_cfg, phys_endpts, i);
+    sbctl_context_invalidate_cache(ctx);
+}
+
+static void
+cmd_lport_unbind_phys_endpts(struct ctl_context *ctx)
+{
+    struct sbctl_context *sbctl_ctx = sbctl_context_cast(ctx);
+    bool must_exist = !shash_find(&ctx->options, "--if-exists");
+    struct sbctl_port_binding *sbctl_bd;
+    char *lport_name;
+
+    lport_name = ctx->argv[1];
+    sbctl_context_populate_cache(ctx);
+    sbctl_bd = find_port_binding(sbctl_ctx, lport_name, must_exist);
+    if (sbctl_bd) {
+        sbrec_port_binding_set_phys_endpts(sbctl_bd->bd_cfg, NULL, 0);
+    }
+}
+
 enum {
     PL_INGRESS,
     PL_EGRESS,
@@ -775,6 +1044,11 @@ static const struct ctl_table_class tables[] = {
      {{&sbrec_table_mac_binding, &sbrec_mac_binding_col_logical_port, NULL},
       {NULL, NULL, NULL}}},
 
+    {&sbrec_table_physical_endpoint,
+     {{&sbrec_table_physical_endpoint,
+       &sbrec_physical_endpoint_col_name, NULL},
+      {NULL, NULL, NULL}}},
+
     {NULL, {{NULL, NULL, NULL}, {NULL, NULL, NULL}}}
 };
 
@@ -1011,17 +1285,30 @@ sbctl_exit(int status)
 
 static const struct ctl_command_syntax sbctl_commands[] = {
     /* Chassis commands. */
+
     {"chassis-add", 3, 3, "CHASSIS ENCAP-TYPE ENCAP-IP", pre_get_info,
      cmd_chassis_add, NULL, "--may-exist", RW},
     {"chassis-del", 1, 1, "CHASSIS", pre_get_info, cmd_chassis_del, NULL,
      "--if-exists", RW},
 
+	/* Physical Endpoint commands */
+	{"phys-endpt-add", 4, 6, "PHYS-ENDPT CHASSIS PORT TYPE ING-ENC EGR-ENC",
+     pre_get_info, cmd_phys_endpt_add, NULL, "--may-exist", RW},
+	{"phys-endpt-del", 1, 1, "PHYS-ENDPT",
+     pre_get_info, cmd_phys_endpt_del, NULL, "--if-exists", RW},
+
     /* Port binding commands. */
     {"lport-bind", 2, 2, "LPORT CHASSIS", pre_get_info, cmd_lport_bind, NULL,
      "--may-exist", RW},
     {"lport-unbind", 1, 1, "LPORT", pre_get_info, cmd_lport_unbind, NULL,
      "--if-exists", RW},
 
+	/* Port to physical endpoint binding */
+    {"lport-bind-phys-endpt", 2, 2, "LPORT PHYS-ENDPT", pre_get_info,
+     cmd_lport_bind_phys_endpts, NULL, "--may-exist", RW},
+    {"lport-unbind-phys-endpt", 1, 1, "LPORT", pre_get_info,
+     cmd_lport_unbind_phys_endpts, NULL, "--if-exists", RW},
+
     /* Logical flow commands */
     {"lflow-list", 0, 1, "[DATAPATH]", pre_get_info, cmd_lflow_list, NULL,
      "", RO},
diff --git a/tests/ovn.at b/tests/ovn.at
index 689e544..711db34 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -840,12 +840,15 @@ for i in 1 2; do
 
         if test $j -le 2; then
             ovn-nbctl lport-add $lswitch_name $ln_port_name
+            ovn-sbctl phys-endpt-add pe$i$j hv$i vif$i$j vlan 0 0
         else
             ovn-nbctl lport-add $lswitch_name $ln_port_name "" 101
+            ovn-sbctl phys-endpt-add pe$i$j hv$i vif$i$j vlan 101 101
         fi
         ovn-nbctl lport-set-addresses $ln_port_name unknown
         ovn-nbctl lport-set-type $ln_port_name localnet
         ovn-nbctl lport-set-options $ln_port_name network_name=phys
+        ovn-sbctl lport-bind-phys-endpt $ln_port_name pe$i$j
 
         ovn-nbctl lport-add $lswitch_name $lport_name
         ovn-nbctl lport-set-addresses $lport_name f0:00:00:00:00:$i$j
@@ -1118,6 +1121,138 @@ for daemon in ovs-vtep ovn-controller-vtep ovn-controller ovn-northd ovsdb-serve
 done
 AT_CLEANUP
 
+# Similar test to "hardware GW"
+AT_SETUP([ovn -- 3 HVs, 1 VIFs/HV, 1 software GW, 1 LS])
+AT_SKIP_IF([test $HAVE_PYTHON = no])
+ovn_start
+
+# Configure the Northbound database
+ovn-nbctl lswitch-add lsw0
+
+ovn-nbctl lport-add lsw0 lp1
+ovn-nbctl lport-set-addresses lp1 f0:00:00:00:00:01
+
+ovn-nbctl lport-add lsw0 lp2
+ovn-nbctl lport-set-addresses lp2 f0:00:00:00:00:02
+
+ovn-nbctl lport-add lsw0 lp-gw
+ovn-nbctl lport-set-type lp-gw gw
+ovn-nbctl lport-set-addresses lp-gw unknown f0:00:00:00:00:03
+
+net_add n1               # Network to connect hv1, hv2, and gw
+net_add n2               # Network to connect gw and hv3
+
+# Create hypervisor hv1 connected to n1
+sim_add hv1
+as hv1
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+ovs-vsctl add-port br-int vif1 -- set Interface vif1 external-ids:iface-id=lp1 options:tx_pcap=hv1/vif1-tx.pcap options:rxq_pcap=hv1/vif1-rx.pcap ofport-request=1
+
+# Create hypervisor hv2 connected to n1
+sim_add hv2
+as hv2
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.2
+ovs-vsctl add-port br-int vif2 -- set Interface vif2 external-ids:iface-id=lp2 options:tx_pcap=hv2/vif2-tx.pcap options:rxq_pcap=hv2/vif2-rx.pcap ofport-request=1
+
+# Create hypervisor hv_gw connected to n1 and n2
+# connect br-phys bridge to n1; connect hv-gw bridge to n2
+sim_add hv_gw
+as hv_gw
+ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.3
+
+ovn-sbctl phys-endpt-add pe1 hv_gw port_1 vlan 0 0
+ovn-sbctl lport-bind-phys-endpt lp-gw pe1
+net_attach n2 port_1
+
+as hv_gw ovs-vsctl show
+
+# Add hv3 on the other side of the GW
+sim_add hv3
+as hv3
+ovs-vsctl add-br br-phys
+net_attach n2 br-phys
+ovs-vsctl add-port br-phys vif3 -- set Interface vif3 options:tx_pcap=hv3/vif3-tx.pcap options:rxq_pcap=hv3/vif3-rx.pcap ofport-request=1
+
+
+# Pre-populate the hypervisors' ARP tables so that we don't lose any
+# packets for ARP resolution (native tunneling doesn't queue packets
+# for ARP resolution).
+ovn_populate_arp
+
+# Allow some time for ovn-northd and ovn-controller to catch up.
+# XXX This should be more systematic.
+sleep 1
+
+# test_packet INPORT DST SRC ETHTYPE OUTPORT...
+#
+# This shell function causes a packet to be received on INPORT.  The packet's
+# content has Ethernet destination DST and source SRC (each exactly 12 hex
+# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
+# more) list the VIFs on which the packet should be received.  INPORT and the
+# OUTPORTs are specified as lport numbers, e.g. 1 for vif1.
+trim_zeros() {
+    sed 's/\(00\)\{1,\}$//'
+}
+for i in 1 2 3; do
+    : > $i.expected
+done
+test_packet() {
+    local inport=$1 packet=$2$3$4; shift; shift; shift; shift
+    #hv=hv`echo $inport | sed 's/^\(.\).*/\1/'`
+    hv=hv$inport
+    vif=vif$inport
+    as $hv ovs-appctl netdev-dummy/receive $vif $packet
+    for outport; do
+        echo $packet | trim_zeros >> $outport.expected
+    done
+}
+
+# Send packets between all pairs of source and destination ports:
+#
+# 1. Unicast packets are delivered to exactly one lport (except that packets
+#    destined to their input ports are dropped).
+#
+# 2. Broadcast and multicast are delivered to all lports except the input port.
+#
+# 3. The lswitch delivers packets with an unknown destination to lports with
+#    "unknown" among their MAC addresses (and port security disabled).
+for s in 1 2 3; do
+    bcast=
+    unknown=
+    for d in 1 2 3; do
+        if test $d != $s; then unicast=$d; else unicast=; fi
+        test_packet $s f0000000000$d f0000000000$s 00$s$d $unicast       #1
+
+        # The vtep (vif3) is the only one configured for "unknown"
+        if test $d != $s && test $d = 3; then
+            unknown="$unknown $d"
+        fi
+        bcast="$bcast $unicast"
+    done
+
+    test_packet $s ffffffffffff f0000000000$s 0${s}ff $bcast             #2
+    test_packet $s 010000000000 f0000000000$s 0${s}ff $bcast             #3
+    test_packet $s f0000000ffff f0000000000$s 0${s}66 $unknown           #4
+done
+
+# Allow some time for packet forwarding.
+# XXX This can be improved.
+sleep 3
+
+# Now check the packets actually received against the ones expected.
+for i in 1 2 3; do
+    file=hv$i/vif$i-tx.pcap
+    echo $file
+    $PYTHON "$top_srcdir/utilities/ovs-pcap.in" $file | trim_zeros > $i.packets
+    sort $i.expected > expout
+    AT_CHECK([sort $i.packets], [0], [expout])
+    echo
+done
+AT_CLEANUP
+
 # 3 hypervisors, 3 logical switches with 3 logical ports each, 1 logical router
 AT_SETUP([ovn -- 3 HVs, 3 LS, 3 lports/LS, 1 LR])
 AT_SKIP_IF([test $HAVE_PYTHON = no])
-- 
1.9.1




More information about the dev mailing list