[ovs-dev] [ovs-dev, RFC] ovn: support for service function chaining

jmcdowall at paloaltonetworks.com jmcdowall at paloaltonetworks.com
Thu Feb 2 23:22:56 UTC 2017


From: John McDowall <jmcdowall at paloaltonetworks.com>

This patchset is the first round at having Service Function Chaining
functionality through OVN. The implementation is done entirely
on the northbound side of OVN. It is a bump on the wire implementation,
so no attempt is being made in keeping state while packets visit each
hop of the chain. ACLs are used as the classifiers, with the augmentation
of action SFC, as well as option column.

The current implementation of traffic redirection to the service chain
is implemented by adding an additional action 'sfc' to the ACL stage. This
overloads the ACL stage and this might not be the best long term approach.
Guidence on whether this is "good enough" for now would be appreciated.

How to leverage load balancing is also an open issue. The current LB solution
in OVN is L3 based. Suggestions on how to implement LB at L2 for SFC would
also be appreciated.

This is not yet ready to be merged, as it lacks unit tests and a rigorous
code review. Nevertheless, it works fine if you take into account a
number of limitations that include:

* missing load balancer integration;
* no ipv6 support;
* chain spanning logical switches (not supported);
* bidirectional chains (not implemented);
* no test cases.
* other suggestions?

This is the code that was used for SFC demo and talk at OVSCon 2016.

Changes:

 * ovn-northd.xml: Added documentation for SFC ACL Action
 * ovn-northd.c: Added new stage for SFC and modified ACL stage to include sfc action
 * ovn-architecture.xml Included architecture of SFC in documentation
 * ovn-nb.ovsschema: Extended schema to include port-chain, port-pair-groups, port-pairs
    and added ACL SFC action
 * ovn-nb.xml: Added documentation for extensions to  ovn-nbctl for port-chain,
    port-pair-groups, port-pairs and ACL SFC action
 * ovn-nbctl.c: Added code to extend ovn-nbctl for port-chain,
    port-pair-groups, port-pairs and ACL SFC action

Current Limitations

This is not yet ready to be merged, as it lacks unit tests and a rigorous
code review. Nevertheless, it works fine if you take into account a
few limitations:

* missing load balancer integration.
* no ipv6 support.
* chain spanning logical switches (not supported).
* bidirectional chains (not implemented).
* no test cases.
* documentation needs rework as there have been several changes as the code has progressed.

Before I work on the limitations and start adding test cases I would like to make sure I am on the right track to get this approved for submission. Once it is approved for OVN/OVS I can add it to Openstack and also planning on using it for container service chaining.

Questions:

* Is the basic approach aligned with the direction of OVN/OVS.
* Is the approach to overloading the current ACL stage best approach for added a classifier action.
* How to approach load balancing - current approach in OVN is L# based any ideas on how to LB L2 virtual services.

Co-authored-by: Flavio Fernandes <flavio at flaviof.com>
Reported at: https://mail.openvswitch.org/pipermail/ovs-discuss/2016-March/040381.html
Reported at: https://mail.openvswitch.org/pipermail/ovs-discuss/2016-May/041359.html

Signed-off-by: John McDowall <jmcdowall at paloaltonetworks.com>
---
 ovn/northd/ovn-northd.8.xml |   5 +
 ovn/northd/ovn-northd.c     | 315 +++++++++++++++--
 ovn/ovn-architecture.7.xml  | 184 ++++++++++
 ovn/ovn-nb.ovsschema        |  64 +++-
 ovn/ovn-nb.xml              | 150 +++++++-
 ovn/utilities/ovn-nbctl.c   | 825 +++++++++++++++++++++++++++++++++++++++++++-
 6 files changed, 1515 insertions(+), 28 deletions(-)

diff --git ovn/northd/ovn-northd.8.xml ovn/northd/ovn-northd.8.xml
index ab8fd88..e68d1de 100644
--- ovn/northd/ovn-northd.8.xml
+++ ovn/northd/ovn-northd.8.xml
@@ -304,6 +304,11 @@
         connections.
       </li>
       <li>
+        <code>sfc</code> ACLs work as entry points for service function
+        chaining, also known as SFC classifiers. Further attributes such
+        as what chain to be used are provided via the options column.
+      </li>
+      <li>
         Other ACLs translate to <code>drop;</code> for new or untracked
         connections and <code>ct_commit(ct_label=1/1);</code> for known
         connections.  Setting <code>ct_label</code> marks a connection
diff --git ovn/northd/ovn-northd.c ovn/northd/ovn-northd.c
index a4f76a9..8a0982e 100644
--- ovn/northd/ovn-northd.c
+++ ovn/northd/ovn-northd.c
@@ -106,13 +106,14 @@ enum ovn_stage {
     PIPELINE_STAGE(SWITCH, IN,  PRE_LB,         4, "ls_in_pre_lb")        \
     PIPELINE_STAGE(SWITCH, IN,  PRE_STATEFUL,   5, "ls_in_pre_stateful")  \
     PIPELINE_STAGE(SWITCH, IN,  ACL,            6, "ls_in_acl")           \
-    PIPELINE_STAGE(SWITCH, IN,  QOS_MARK,       7, "ls_in_qos_mark")      \
-    PIPELINE_STAGE(SWITCH, IN,  LB,             8, "ls_in_lb")            \
-    PIPELINE_STAGE(SWITCH, IN,  STATEFUL,       9, "ls_in_stateful")      \
-    PIPELINE_STAGE(SWITCH, IN,  ARP_ND_RSP,    10, "ls_in_arp_rsp")       \
-    PIPELINE_STAGE(SWITCH, IN,  DHCP_OPTIONS,  11, "ls_in_dhcp_options")  \
-    PIPELINE_STAGE(SWITCH, IN,  DHCP_RESPONSE, 12, "ls_in_dhcp_response") \
-    PIPELINE_STAGE(SWITCH, IN,  L2_LKUP,       13, "ls_in_l2_lkup")       \
+    PIPELINE_STAGE(SWITCH, IN,  CHAIN,          7, "ls_in_chain")        \
+    PIPELINE_STAGE(SWITCH, IN,  QOS_MARK,       8, "ls_in_qos_mark")    \
+    PIPELINE_STAGE(SWITCH, IN,  LB,             9, "ls_in_lb")            \
+    PIPELINE_STAGE(SWITCH, IN,  STATEFUL,      10, "ls_in_stateful")      \
+    PIPELINE_STAGE(SWITCH, IN,  ARP_ND_RSP,    11, "ls_in_arp_rsp")       \
+    PIPELINE_STAGE(SWITCH, IN,  DHCP_OPTIONS,  12, "ls_in_dhcp_options")  \
+    PIPELINE_STAGE(SWITCH, IN,  DHCP_RESPONSE, 13, "ls_in_dhcp_response") \
+    PIPELINE_STAGE(SWITCH, IN,  L2_LKUP,       14, "ls_in_l2_lkup")       \
                                                                       \
     /* Logical switch egress stages. */                               \
     PIPELINE_STAGE(SWITCH, OUT, PRE_LB,       0, "ls_out_pre_lb")     \
@@ -2480,8 +2481,12 @@ build_pre_stateful(struct ovn_datapath *od, struct hmap *lflows)
                   REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;");
 }
 
+static bool
+build_chain_classifier_entry(struct ovn_datapath *od, struct hmap *ports,
+                             const struct nbrec_acl *acl, struct ds *ds_action);
+
 static void
-build_acls(struct ovn_datapath *od, struct hmap *lflows)
+build_acls(struct ovn_datapath *od, struct hmap *lflows, struct hmap *ports)
 {
     bool has_stateful = has_stateful_acl(od);
 
@@ -2637,6 +2642,18 @@ build_acls(struct ovn_datapath *od, struct hmap *lflows)
 
                 ds_destroy(&match);
             }
+        } else if (!strcmp(acl->action, "sfc")) {
+            struct ds ds_action = DS_EMPTY_INITIALIZER;
+
+            if (!build_chain_classifier_entry(od, ports, acl, &ds_action)) {
+                ds_put_format(&ds_action, "drop;");
+            }
+
+            ovn_lflow_add(lflows, od, stage,
+                          acl->priority + OVN_ACL_PRI_OFFSET,
+                          acl->match, ds_cstr_ro(&ds_action));
+
+            ds_destroy(&ds_action);
         } else if (!strcmp(acl->action, "drop")
                    || !strcmp(acl->action, "reject")) {
             struct ds match = DS_EMPTY_INITIALIZER;
@@ -2747,6 +2764,262 @@ build_acls(struct ovn_datapath *od, struct hmap *lflows)
     }
 }
 
+static int
+cmp_port_pair_groups(const void *ppg1_, const void *ppg2_)
+{
+    const struct nbrec_logical_port_pair_group *const *ppg1p = ppg1_;
+    const struct nbrec_logical_port_pair_group *const *ppg2p = ppg2_;
+    const struct nbrec_logical_port_pair_group *ppg1 = *ppg1p;
+    const struct nbrec_logical_port_pair_group *ppg2 = *ppg2p;
+
+    if (ppg1->n_sortkey == 0 || ppg2->n_sortkey == 0) {
+        return 0;
+    }
+
+    const int64_t key1 = ppg1->sortkey[0];
+    const int64_t key2 = ppg2->sortkey[0];
+
+    if (key1 < key2) {
+        return -1;
+    } else if (key1 > key2) {
+        return 1;
+    } else {
+        return 0;
+    }
+}
+
+/*
+ * Build the rules to insert service chains
+ */
+static bool
+build_chain_classifier_entry(struct ovn_datapath *od, struct hmap *ports,
+                             const struct nbrec_acl *acl, struct ds *ds_action /*output*/)
+{
+    /* TODO: match needs to explicitly include inport. That is needed until sfc
+     *       has safeguards to tell apart initial packet from a packet that is
+     *       already in the chain.
+     */
+    if (!strstr(acl->match, "inport")) {
+        VLOG_INFO("Acl match clause for sfc action needs to provide inport -- %s\n", acl->match);
+        return false;
+    }
+
+    const char *const chain_id = smap_get(&acl->options, "sfc-port-chain");
+    if (!chain_id) {
+        VLOG_INFO("Acl match %s with sfc action requires sfc-port-chain option\n", acl->match);
+        return false;
+    }
+
+    struct uuid lsp_chain_uuid;
+    const bool is_uuid = uuid_from_string(&lsp_chain_uuid, chain_id);
+    bool chain_found = false;
+    const struct nbrec_logical_port_chain *lsp_chain = NULL;
+
+    /* Find port chain referred by acl option */
+    for (size_t i = 0; i < od->nbs->n_port_chains; i++) {
+        lsp_chain = od->nbs->port_chains[i];
+        if (is_uuid) {
+            if (uuid_equals(&lsp_chain->header_.uuid, &lsp_chain_uuid)) {
+                chain_found = true;
+                break;
+            }
+        } else {
+            if (!strcmp(chain_id, lsp_chain->name)) {
+                chain_found = true;
+                break;
+            }
+        }
+    }
+
+    if (!chain_found) {
+        VLOG_INFO("Acl match for sfc action could not locate chain %s\n", chain_id);
+        return false;
+    }
+
+    /* extract first logical port of chain, by looking at the initial port pair group */
+    const struct nbrec_logical_port_pair_group *lppg = NULL;
+    const struct nbrec_logical_port_pair *lpp = NULL;
+    struct ovn_port *first_ovn_port = NULL;
+
+    /* Identify initial port pair group to be used, according to their sortkey */
+    if (lsp_chain->n_port_pair_groups > 0) {
+        struct nbrec_logical_port_pair_group **port_pair_groups
+            = xmalloc(sizeof *port_pair_groups * lsp_chain->n_port_pair_groups);
+        memcpy(port_pair_groups, lsp_chain->port_pair_groups,
+               sizeof *port_pair_groups * lsp_chain->n_port_pair_groups);
+        qsort(port_pair_groups, lsp_chain->n_port_pair_groups, sizeof *port_pair_groups,
+              cmp_port_pair_groups);
+        lppg = port_pair_groups[0];
+        free(port_pair_groups);
+    }
+
+    // TODO: any port pair off of the intial pair group of chain could be used
+    lpp = (lppg && lppg->n_port_pairs > 0) ? lppg->port_pairs[0] : NULL;
+    first_ovn_port = (lpp && lpp->inport) ? ovn_port_find(ports, lpp->inport->name) : NULL;
+
+    if (!first_ovn_port) {
+        VLOG_INFO("Acl match for sfc action could not locate logical port from chain %s" \
+                  " port-group %p port-chain %p\n", chain_id, lppg, lpp);
+        return false;
+    }
+
+    ds_put_format(ds_action, "outport = %s;"" output;", first_ovn_port->json_key);
+    return true;
+}
+
+static void
+build_chain(struct ovn_datapath *od, struct hmap *lflows, struct hmap *ports)
+{
+   /*
+    * TODO Items
+    *     * IPV6 support
+    *     * Load balancing support
+    *     * Bidirectional parameter support
+    *     * Support modes of VNF (BitW, L2, L3)
+    *     * Remove port-security on VNF Ports (if set by CMS)
+    *     * Add some state to allow match that does not require 'inport'
+    *     * Support visiting the same VNF more than once
+    *     * Unit tests!
+    */
+    const uint16_t ingress_inner_priority = 150;
+    //const uint16_t egress_inner_priority = 150;
+
+    struct ovn_port **input_port_array = NULL;
+    struct ovn_port **output_port_array = NULL;
+
+    struct ovn_port *dst_port = NULL;
+    struct ovn_port *src_port = NULL;
+
+    struct nbrec_logical_port_chain *lpc;
+    struct nbrec_logical_port_pair_group *lppg;
+    struct nbrec_logical_port_pair *lpp;
+
+    VLOG_INFO("beginning port-chain insertion\n");
+    /* Ingress table ls_in_chain: default to passing through to the next table
+     * (priority 0)
+     */
+    ovn_lflow_add(lflows, od, S_SWITCH_IN_CHAIN, 0, "1", "next;");
+
+    if (!od->nbs->port_chains) {
+        return;  // no shoes, no shirt, no chains: no further work to do here :)
+    }
+
+    struct ds ds_match = DS_EMPTY_INITIALIZER;
+    struct ds ds_action = DS_EMPTY_INITIALIZER;
+
+    VLOG_INFO("Iterating through %"PRIuSIZE" port-chains\n", od->nbs->n_port_chains);
+    /*
+     * Iterate through all the port-chains defined for this datapath.
+     */
+    for (size_t i = 0; i < od->nbs->n_port_chains; i++) {
+        bool obtainedAllNeededInfo = true;
+
+        lpc = od->nbs->port_chains[i];
+        VLOG_INFO("Iterating through %"PRIuSIZE" port-pair-groups for %s\n", lpc->n_port_pair_groups, lpc->name);
+        /*
+         * Allocate space for port-pairs + 1. The Extra 1 represents the final hop to reach desired destination.
+         * TODO: We are going to allocate enough space to hold all the hops: 1 x portGroups + 1. This needs
+         *       to enhanced to: SUM(port pairs of each port group) + 1
+         */
+        input_port_array = xmalloc(sizeof *src_port * lpc->n_port_pair_groups + 1);
+        output_port_array = xmalloc(sizeof *dst_port * (lpc->n_port_pair_groups + 1));
+
+        /* Copy port groups from chain and sort them according to their sortkey */
+        struct nbrec_logical_port_pair_group **port_pair_groups
+            = xmalloc(sizeof *port_pair_groups * lpc->n_port_pair_groups);
+        memcpy(port_pair_groups, lpc->port_pair_groups,
+               sizeof *port_pair_groups * lpc->n_port_pair_groups);
+        qsort(port_pair_groups, lpc->n_port_pair_groups, sizeof *port_pair_groups,
+              cmp_port_pair_groups);
+
+        /*
+         * For each port-pair-group in a port chain pull out the port-pairs
+         */
+        for (size_t j = 0; j < lpc->n_port_pair_groups; j++) {
+            lppg = port_pair_groups[j];
+            VLOG_INFO("Iterating through %"PRIuSIZE" port-pair-group %s for chain %s\n", j, lppg->name, lpc->name);
+            for (size_t k = 0; k < lppg->n_port_pairs; k++){
+                /*
+                 * TODO: Need to add load balancing logic when LB becomes available.
+                 *       Until LB is available just take the first PP in the PPG.
+                 */
+                if (k > 0) {
+                    VLOG_INFO("Warning: Currently lacking support for more than one port-pair %"PRIuSIZE"\n", \
+                              lppg->n_port_pairs);
+                    break;
+                }
+                lpp = lppg->port_pairs[k];
+
+                input_port_array[j] = lpp->inport ? ovn_port_find(ports, lpp->inport->name) : NULL;
+                VLOG_INFO("In port for port-pair-group %s : %s  %p\n", lppg->name,
+                          (lpp->inport ? lpp->inport->name : ""), input_port_array[j]);
+                output_port_array[j] = lpp->outport ? ovn_port_find(ports, lpp->outport->name) : NULL;
+                VLOG_INFO("Out port for port-pair-group %s : %s  %p \n", lppg->name,
+                          (lpp->outport ? lpp->outport->name : ""), output_port_array[j]);
+
+                if (!input_port_array[j] || !output_port_array[j]) obtainedAllNeededInfo = false;
+            }
+        }
+
+        /*
+         * Set last entry in port_array as the last port in chain. Note that for reverse
+         * output_port_array[lpc->n_port_pair_groups] would become the first inport.
+         */
+        dst_port = lpc->last_hop_port ? ovn_port_find(ports, lpc->last_hop_port->name) : NULL;
+        input_port_array[lpc->n_port_pair_groups] = dst_port;
+        VLOG_INFO("In port for last port-pair-group %s  %p\n",
+                  (lpc->last_hop_port ? lpc->last_hop_port->name : ""),
+                  input_port_array[lpc->n_port_pair_groups]);
+
+        struct eth_addr dst_logical_port_ea;
+        ovs_be32 dst_logical_port_ip;
+        if (dst_port && dst_port->nbsp) {
+            if (!ovs_scan(dst_port->nbsp->addresses[0],  ETH_ADDR_SCAN_FMT" "IP_SCAN_FMT,
+                          ETH_ADDR_SCAN_ARGS(dst_logical_port_ea), IP_SCAN_ARGS(&dst_logical_port_ip))) {
+                obtainedAllNeededInfo = false;
+            }
+        } else {
+            obtainedAllNeededInfo = false;
+        }
+
+        output_port_array[lpc->n_port_pair_groups] = NULL;
+
+        /*
+         * Steer traffic through the port-chain (FORWARD)
+         */
+        if (obtainedAllNeededInfo) {
+            for (size_t j = 0; j < lpc->n_port_pair_groups; j++){
+#ifdef _SFC_CHAIN_IS_IP_BASED
+                ds_put_format(&ds_match, "inport == %s && ip4.dst == " IP_FMT,
+                              output_port_array[j]->json_key,
+                              IP_ARGS(dst_logical_port_ip));
+#else // #ifdef _SFC_CHAIN_IS_IP_BASED
+                ds_put_format(&ds_match, "inport == %s && eth.dst == " ETH_ADDR_FMT,
+                              output_port_array[j]->json_key,
+                              ETH_ADDR_ARGS(dst_logical_port_ea));
+#endif // #ifdef _SFC_CHAIN_IS_IP_BASED
+
+                ds_put_format(&ds_action, "outport = %s;"" output;",
+                              input_port_array[j+1]->json_key);
+
+                VLOG_INFO("Port chain action %s\n", ds_cstr_ro(&ds_action));
+                ovn_lflow_add(lflows, od, S_SWITCH_IN_CHAIN, ingress_inner_priority,
+                              ds_cstr_ro(&ds_match), ds_cstr_ro(&ds_action));
+
+                ds_clear(&ds_match);
+                ds_clear(&ds_action);
+            }
+        }
+
+        free(input_port_array);
+        free(output_port_array);
+        free(port_pair_groups);
+    }
+
+    ds_destroy(&ds_match);
+    ds_destroy(&ds_action);
+}
+
 static void
 build_qos(struct ovn_datapath *od, struct hmap *lflows) {
     ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;");
@@ -2875,7 +3148,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
     struct ds actions = DS_EMPTY_INITIALIZER;
 
     /* Build pre-ACL and ACL tables for both ingress and egress.
-     * Ingress tables 3 through 9.  Egress tables 0 through 6. */
+     * Ingress tables 3 through 10.  Egress tables 0 through 6. */
     struct ovn_datapath *od;
     HMAP_FOR_EACH (od, key_node, datapaths) {
         if (!od->nbs) {
@@ -2885,7 +3158,8 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         build_pre_acls(od, lflows);
         build_pre_lb(od, lflows);
         build_pre_stateful(od, lflows);
-        build_acls(od, lflows);
+        build_acls(od, lflows, ports);
+        build_chain(od, lflows, ports);
         build_qos(od, lflows);
         build_lb(od, lflows);
         build_stateful(od, lflows);
@@ -2958,9 +3232,9 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_IP, 0, "1", "next;");
     }
 
-    /* Ingress table 10: ARP/ND responder, skip requests coming from localnet
-     * and vtep ports. (priority 100); see ovn-northd.8.xml for the
-     * rationale. */
+    /* Ingress table 11: ARP/ND responder, skip requests coming from localnet
+     * ports. (priority 100). */
+
     HMAP_FOR_EACH (op, key_node, ports) {
         if (!op->nbsp) {
             continue;
@@ -2975,7 +3249,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         }
     }
 
-    /* Ingress table 10: ARP/ND responder, reply for known IPs.
+    /* Ingress table 11: ARP/ND responder, reply for known IPs.
      * (priority 50). */
     HMAP_FOR_EACH (op, key_node, ports) {
         if (!op->nbsp) {
@@ -3068,7 +3342,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         }
     }
 
-    /* Ingress table 10: ARP/ND responder, by default goto next.
+    /* Ingress table 11: ARP/ND responder, by default goto next.
      * (priority 0)*/
     HMAP_FOR_EACH (od, key_node, datapaths) {
         if (!od->nbs) {
@@ -3078,7 +3352,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;");
     }
 
-    /* Logical switch ingress table 11 and 12: DHCP options and response
+    /* Logical switch ingress table 12 and 13: DHCP options and response
          * priority 100 flows. */
     HMAP_FOR_EACH (op, key_node, ports) {
         if (!op->nbsp) {
@@ -3182,7 +3456,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         }
     }
 
-    /* Ingress table 11 and 12: DHCP options and response, by default goto next.
+    /* Ingress table 12 and 13: DHCP options and response, by default goto next.
      * (priority 0). */
 
     HMAP_FOR_EACH (od, key_node, datapaths) {
@@ -3194,7 +3468,8 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;");
     }
 
-    /* Ingress table 13: Destination lookup, broadcast and multicast handling
+
+    /* Ingress table 14: Destination lookup, broadcast and multicast handling
      * (priority 100). */
     HMAP_FOR_EACH (op, key_node, ports) {
         if (!op->nbsp) {
@@ -3214,7 +3489,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
                       "outport = \""MC_FLOOD"\"; output;");
     }
 
-    /* Ingress table 13: Destination lookup, unicast handling (priority 50), */
+    /* Ingress table 14: Destination lookup, unicast handling (priority 50), */
     HMAP_FOR_EACH (op, key_node, ports) {
         if (!op->nbsp) {
             continue;
@@ -3314,7 +3589,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports,
         }
     }
 
-    /* Ingress table 13: Destination lookup for unknown MACs (priority 0). */
+    /* Ingress table 14: Destination lookup for unknown MACs (priority 0). */
     HMAP_FOR_EACH (od, key_node, datapaths) {
         if (!od->nbs) {
             continue;
diff --git ovn/ovn-architecture.7.xml ovn/ovn-architecture.7.xml
index d8114f1..ee69fed 100644
--- ovn/ovn-architecture.7.xml
+++ ovn/ovn-architecture.7.xml
@@ -383,6 +383,12 @@
     </li>
 
     <li>
+      <dfn>Logical services</dfn> are logical references to virtual network functions
+      (VNF). Adding a logical service requires adding steering rules to the OVN Northbound
+      database. These are the only rules necessary to implement traffic steering for VNFs.
+      The section below "<code>Life Cycle of an inserted VNF</code>" provides more details.
+    </li>
+    <li>
       <p>
         <dfn>Logical ports</dfn> represent the points of connectivity in and
         out of logical switches and logical routers.  Some common types of
@@ -566,6 +572,184 @@
     </li>
   </ol>
 
+ <h2>Life Cycle of an inserted Virtual Network Function (VNF)</h2>
+
+ <p>
+   OVN provides an abstraction to enable the insertion of an arbitrary virtual network
+   function (VNF) into the path of traffic to and from an application. A VNF is different
+   from an application VM in that it acts on traffic between applications, and im most
+   cases does not terminiate a flow. Proxy functions are an exception as they terminate the
+   flow from the src and create a new flow to the dst. Examples of VNF's are security functions,
+   load balancing, and traffic enhancement services, this is not a complete list.
+ </p>
+ <p>
+   The requirements on the VNF to be inserted are minimal, it must
+   act as a <code>bump in the wire (BITW)</code> and can have one or two virtual network
+   ports for traffic. If it has two network ports traffic is directed into one and out the other,
+   if it has only one port, then traffic is bi-directional. The requirement for the VNF to act as
+   a BITW removes the need for the VNF to participate in L3/L2 networking which provides
+   improved agility and reduces the coupling between OVN and the VNF.
+ </p>
+ <p>
+   The service insertion is implemented by adding 4 new flow rules into the ovn-nb database for
+   each VNF inserted. The rules are added into the last table in the ingress path (L2_LKUP).
+   The service insertion rules have a higher priority than the standard forwarding rules.
+   This means that they override the existing forwarding rules. There are four
+   new rules added for each insertion. Two ingress and two egress, The first ingress
+   rule sends all traffic destined for the application into the VNF ingress port and the
+   second rule takes all traffic destined to the application from the VNF egress port
+   to the application, the priorities are such that the second rule is always checked
+   first. In the egress direction the rules are similar if the traffic is from the
+   application it is sent to the VNF egress port and if if is from the application
+   and is from the VNF ingress port it is delivered to the destination.
+   <!-- Should this be a new table or is it a naturally part of the L2 lookup table ? -->
+ </p>
+ <p>
+   The steps in this example refer to the details of the OVN Northbound database schema.
+   There is a new table in the OVN Northbound database to support service insertion
+   called <code>Services</code> this contains the required information for each new
+   service inserted. The same service can be used for multiple applications, as
+   there is typically a n:1 relationship between applications and VNFs. A
+   single VNF may be part of several service insertions, but each one is logically
+   separate.
+ </p>
+ <p>
+   The steps in this example refer often to details of the OVN and OVN
+   Northbound database schemas.  Please see <code>ovn-sb</code>(5) and
+   <code>ovn-nb</code>(5), respectively, for the full story on these
+   databases. The ovn-nb database has specific schema enhancements for the service
+   insertion function. The ovn-sb database has no schema changes for the
+   service insertion function.
+ </p>
+ <p>
+   The following steps are an overview to inserting a new VNF into the traffic path.
+   The sections below go into each step in more detail.
+ </p>
+ <ol>
+   <li>
+     The service insertion lifecycle begins when a CMS administrator creates a new
+     virtual network function <code>(VNF)</code> using the CMS user
+     interface or API. The CMS administrator creates the logical ports (ingress and egress)
+     for the VNF. If the CMS is Openstack this will create a reusable port-pair defining the
+     interface to the VNF. There is also typically a seperate management port for the VNF,
+     but that is not relevant to the service insertion workflow. A single VNF can
+     participate with several applications, either as a security VM, protecting multiple
+     applications or as a load balancer VM, distributing load across multiple applications.
+   </li>
+
+   <li>
+     The next step in the life cycle occurs when a CMS administrator creates a new application
+     with a VIF using the CMS user interface or API and adds it to a switch (one
+     implemented by OVN as a logical switch). Alternatively an already running application could
+     be used.
+
+     The CMS can now attach the port pair to the VIF by defining the logical port in the
+     service function classifier. This will direct traffic to the VIF to go through
+     the VNF, applying the rules discussed earlier.
+   </li>
+
+   <li>
+     While within CMS the service insertion may be broken down into multiple reusable steps
+     (as is the case with Openstack). Within OVN the creating of a new service insertion
+     is treated as an atomic operation. This enables the easy atomic insertion and deletion of
+     service insertions. This is important as it is envisioned that the number of serivce
+     insertions can easily number in the hundreds, all with separate lifecycles. For each new
+     service insertion operation a new row is added to the OVN Northbound Database. The new row is
+     only added to the ovn-nb when the VNF, application and network are enabled by the CMS.
+
+     Once the serivce insertion is applied to the ovn-nb database, the ovn-nb controller will be
+     notified of a change and the rules will be pushed down to the specific OVS instances, using
+     the existing OVN mechanisms. It is important to note here that the logical abstraction of
+     OVN enables service insertion with minimal changes.
+   </li>
+
+   <li>
+     When the application VM shuts down the classification rule should be removed, however as
+     no traffic will be destinated to the application there would be no issues. If the VM is
+     being moved and the new application VM port is used to update the service then the change
+     would be detected and the rules pushed down.
+   </li>
+   <li>
+     A VNF can be removed at any time and traffic flows will revert to the pre-VNF traffic
+     paths if it is removed from the ovn-nb database. OVN must detect that the VNF has been
+     shut-off as it must remove all the rules that are attached to that VNF. Otherwise
+     traffic loss will occur, if a failure in a VNF occurs that is not detected.
+  </li>
+
+  <li>
+    On every hypervisor, <code>ovn-controller</code> receives the
+    <code>Logical_Service</code> table updates that <code>ovn-northd</code> made
+    in the previous step.  As long as the VM that owns the VIF is powered
+    off, <code>ovn-controller</code> cannot do much; it cannot, for example,
+    arrange to send packets to or receive packets from the VIF, because the
+    VIF does not actually exist anywhere. In addition the VNF cannot be inserted
+    into the traffic path as it will have no source to forward traffic too.
+    <!-- If there is no VM then traffic cannot be sent to it therefore having the
+    rules inserted is probably not an issue? -->
+  </li>
+
+  <li>
+    Some CMS systems, including OpenStack, fully start a VM only when its
+    networking is ready.  To support this, <code>ovn-northd</code> notices
+    the <code>chassis</code> column updated for the row in
+    <code>Binding</code> table and pushes this upward by updating the
+    <ref column="up" table="Logical_Port" db="OVN_NB"/> column in the OVN
+    Northbound database's <ref table="Logical_Port" db="OVN_NB"/> table to
+    indicate that the VIF is now up.  The CMS, if it uses this feature, can
+    then react by allowing the VM's execution to proceed.
+
+    Traffic now flows into and out of the VM that has a VNF inserted in its
+    datapath. The rules are applied to direct traffic to the VNF on the way
+    into the VM and on the way out of the VM.
+  </li>
+
+  <!-- Need a section on having multiple VM's using the same VNF
+    ( physcially or logically ). Different rule sets -->
+  <!-- Need a section on distributed cases - one section for each -->
+  <li>
+    On every hypervisor but the one where the VIF resides,
+    <code>ovn-controller</code> notices the completely populated row in the
+    <code>Binding</code> table.  This provides <code>ovn-controller</code>
+    the physical location of the logical port, so each instance updates the
+    OpenFlow tables of its switch (based on logical datapath flows in the OVN
+    DB <code>Logical_Flow</code> table) so that packets to and from the VIF
+    can be properly handled via tunnels.
+  </li>
+  <!-- Current implementation is open on delete, i.e. when the VNF is removed
+    the datapath behaviour reverts to the pre-existing paths. Does this make sense?
+    - could argue that close on delete should be an option ? -->
+
+  <li>
+    Eventually, a user removes the inserted service function from the ovn nb
+    database. The rules are then updated in the southbound database and pushed
+    down to the relevant ovs. No other SIF is impacted as the row in the ovn nb
+    is independant of all the other SIF.
+    <!-- This is really important in the case where many SIF are being added
+      and removed. Without both the independance of the enteries confusion would
+      exist. Also for debugging - possible to remove/add individual VNF's to
+      determine potentail problems. -->
+  </li>
+
+  <li>
+    The CMS plugin removes the SIF from the OVN Northbound database,
+    by deleting its row in the <code>Logical_Service</code> table.
+  </li>
+
+  <li>
+    <code>ovn-northd</code> receives the OVN Northbound update and in turn
+    updates the OVN Southbound database accordingly, by removing or updating
+    the rows from the OVN Southbound database <code>Logical_Service</code> table.
+  </li>
+
+  <li>
+    On every hypervisor, <code>ovn-controller</code> receives the
+    <code>Logical_Service</code> table updates that <code>ovn-northd</code> made
+    in the previous step.  <code>ovn-controller</code> updates OpenFlow
+    tables to reflect the update. The datapath for the VM will now revert to
+    paths that existed before the VNF was inserted into the data path.
+  </li>
+</ol>
+
   <h2>Life Cycle of a Container Interface Inside a VM</h2>
 
   <p>
diff --git ovn/ovn-nb.ovsschema ovn/ovn-nb.ovsschema
index dd0ac3d..6d3474f 100644
--- ovn/ovn-nb.ovsschema
+++ ovn/ovn-nb.ovsschema
@@ -1,7 +1,7 @@
 {
     "name": "OVN_Northbound",
-    "version": "5.5.0",
-    "cksum": "2099428463 14236",
+    "version": "5.5.1",
+    "cksum": "3538062772 17547",
     "tables": {
         "NB_Global": {
             "columns": {
@@ -30,6 +30,16 @@
                                            "refType": "strong"},
                                    "min": 0,
                                    "max": "unlimited"}},
+                "port_chains":  {"type": {"key": {"type": "uuid",
+                                                  "refTable": "Logical_Port_Chain",
+                                                  "refType": "strong"},
+                                          "min": 0,
+                                          "max": "unlimited"}},
+                "port_pairs":  {"type": {"key": {"type": "uuid",
+                                                 "refTable": "Logical_Port_Pair",
+                                                 "refType": "strong"},
+                                         "min": 0,
+                                         "max": "unlimited"}},
                 "acls": {"type": {"key": {"type": "uuid",
                                           "refTable": "ACL",
                                           "refType": "strong"},
@@ -98,6 +108,50 @@
                              "min": 0, "max": "unlimited"}}},
             "indexes": [["name"]],
             "isRoot": false},
+        "Logical_Port_Chain": {
+            "columns": {
+                "name": {"type": "string"},
+                "port_pair_groups":  {"type": {"key": {"type": "uuid",
+                                                       "refTable": "Logical_Port_Pair_Group",
+                                                       "refType": "strong"},
+                                               "min": 0, "max": "unlimited"}},
+                "last_hop_port": {"type": {"key": {"type": "uuid",
+                                                   "refTable": "Logical_Switch_Port",
+                                                   "refType": "strong"},
+                                           "min": 0, "max": 1}},
+                "external_ids": {
+                    "type": {"key": "string", "value": "string",
+                             "min": 0, "max": "unlimited"}}},
+            "isRoot": false},
+        "Logical_Port_Pair_Group": {
+            "columns": {
+                "name": {"type": "string"},
+                "port_pairs":  {"type": {"key": {"type": "uuid",
+                                                 "refTable": "Logical_Port_Pair",
+                                                 "refType": "strong"},
+                                         "min": 0, "max": "unlimited"}},
+                "external_ids": {
+                    "type": {"key": "string", "value": "string",
+                             "min": 0, "max": "unlimited"}},
+                "sortkey": {"type": {"key": {"type": "integer"},
+                                     "min": 0, "max": 1}}
+            },
+            "isRoot": false},
+        "Logical_Port_Pair": {
+            "columns": {
+                "name": {"type": "string"},
+                "outport": {"type": {"key": {"type": "uuid",
+                                             "refTable": "Logical_Switch_Port",
+                                             "refType": "strong"},
+                                     "min": 0, "max": 1}},
+                "inport": {"type": {"key": {"type": "uuid",
+                                            "refTable": "Logical_Switch_Port",
+                                            "refType": "strong"},
+                                    "min": 0, "max": 1}},
+                "external_ids": {
+                    "type": {"key": "string", "value": "string",
+                             "min": 0, "max": "unlimited"}}},
+            "isRoot": false},
         "Address_Set": {
             "columns": {
                 "name": {"type": "string"},
@@ -132,8 +186,12 @@
                                             "enum": ["set", ["from-lport", "to-lport"]]}}},
                 "match": {"type": "string"},
                 "action": {"type": {"key": {"type": "string",
-                                            "enum": ["set", ["allow", "allow-related", "drop", "reject"]]}}},
+                                            "enum": ["set", ["allow", "allow-related", "drop", "reject", "sfc"]]}}},
                 "log": {"type": "boolean"},
+                "options": {
+                    "type": {"key": "string",
+                             "value": "string",
+                             "min": 0, "max": "unlimited"}},
                 "external_ids": {
                     "type": {"key": "string", "value": "string",
                              "min": 0, "max": "unlimited"}}},
diff --git ovn/ovn-nb.xml ovn/ovn-nb.xml
index c5ebbea..e45c670 100644
--- ovn/ovn-nb.xml
+++ ovn/ovn-nb.xml
@@ -120,7 +120,23 @@
         logical port.
       </p>
     </column>
-
+    <column name="port_chains">
+      <p>
+        The logical port-chains connected to the logical switch.
+      </p>
+      <p>
+        It is an error for multiple logical switches to include the same
+        logical port.
+      </p>
+    </column>
+    <column name="port_pairs">
+      <p>
+        The logical chains that define the service path.
+      </p>
+      <p>
+        Logical chains cannot currently cross logical switch boundaries.
+      </p>
+    </column>
     <column name="load_balancer">
       Load balance a virtual ipv4 address to a set of logical port endpoint
       ipv4 addresses.
@@ -155,7 +171,98 @@
       </column>
     </group>
   </table>
+  <table name="Logical_Port_Chain" title="Logical port chain">
+    <p>
+      Each row represents one logical port chain
+    </p>
+
+    <column name="name">
+      <p>
+        A name for the logical chain.  This name has no special meaning or purpose
+        other than to provide convenience for human interaction with the ovn-nb
+        database.  There is no requirement for the name to be unique.  The
+        logical chains's UUID should be used as the unique identifier.
+      </p>
+    </column>
+
+    <column name="port_pair_groups">
+      <p>
+        The logical list of port pairs that the flow goes through.
+      </p>
+
+      <p>
+        It is an error for a port pair group to be empty.
+      </p>
+    </column>
+    <column name="last_hop_port">
+      The <ref table="Logical_Switch_Port"/> to be used once packet reaches the end
+      of the chain.
+    </column>
+    <group title="Common Columns">
+      <column name="external_ids">
+        See <em>External IDs</em> at the beginning of this document.
+      </column>
+    </group>
+  </table>
+  <table name="Logical_Port_Pair_Group" title="logical port pair groups">
+    <p>
+      An ordered port pair list
+    </p>
+
+    <column name="name">
+      <p>
+        Logical port pair group name
+      </p>
+    </column>
+
+    <column name="port_pairs">
+      <p>
+        port pair for this group
+      </p>
+    </column>
+
+    <column name="sortkey">
+      <p>
+        An integer used for ordering instances of <ref table="Logical_Port_Pair_Group"/>
+        in the <ref column="port_pairs" table="Logical_Port_Chain"/> column
+        of <ref table="Logical_Port_Chain"/>.
+      </p>
+    </column>
+    <group title="Common Columns">
+      <column name="external_ids">
+        See <em>External IDs</em> at the beginning of this document.
+      </column>
+    </group>
+  </table>
+
+  <table name="Logical_Port_Pair" title="logical port pairs">
+    <p>
+      Ports pairs defining the service
+    </p>
 
+    <column name="name">
+      <p>
+        Logical port pair
+      </p>
+    </column>
+
+    <column name="outport">
+      <p>
+        Out logical port for this port pair. Can be the same value as inport.
+      </p>
+    </column>
+
+    <column name="inport">
+      <p>
+        In logical port for this port pair.
+      </p>
+    </column>
+    <group title="Common Columns">
+      <column name="external_ids">
+        See <em>External IDs</em> at the beginning of this document.
+      </column>
+    </group>
+  </table>
   <table name="Logical_Switch_Port" title="L2 logical switch port">
     <p>
       A port within an L2 logical switch.
@@ -884,6 +991,13 @@
           ICMP unreachable message for other IP-based protocols.
           <code>Not implemented--currently treated as drop</code>
         </li>
+
+        <li>
+          <code>sfc</code>: Forward the packet into a logical port chain.
+          The chain to be used -- as well as any other attributes that determine
+          the behavior of the packet while in the chain -- are provided
+          via <ref column="options"/>.
+        </li>
       </ul>
     </column>
 
@@ -899,6 +1013,40 @@
       </p>
     </column>
 
+    <group title="Options">
+      <column name = "options">
+        This column provides key/value settings specific to the ACL
+        <ref column="action"/>. The type-specific options are described
+        individually below.
+      </column>
+
+      <group title="Options for action sfc">
+        <p>
+          These options apply when <ref column="action"/> is <code>sfc</code>.
+        </p>
+
+        <column name="options" key="sfc-port-chain">
+          Required when <ref column="action"/> is <code>sfc</code>.
+          The uuid (or name) of the <ref table="Logical_Port_Chain"/> to be used.
+        </column>
+
+        <column name="options" key="sfc-bidirectional">
+          Optional and only applicable when <ref column="action"/> is <code>sfc</code>.
+          When set with value <code>true</code>, the implementation will also add rules to make packets
+          go through the chain in reverse direction. A restriction on making bidirectional chains is
+          that the inport parameter must be present in <ref column="match"/>, as it will be used as the
+          <ref table="Logical_Port_Chain" column="last_hop_port"/>. As expected, all <code>src*</code>
+          fields in <ref column="match"/> will be converted to <code>dst*</code> in order to derive the
+          reverse ACL.
+
+          <p>
+            sfc-bidirectional option is not yet implemented.
+          </p>
+        </column>
+      </group>
+
+    </group>
+
     <group title="Common Columns">
       <column name="external_ids">
         See <em>External IDs</em> at the beginning of this document.
diff --git ovn/utilities/ovn-nbctl.c ovn/utilities/ovn-nbctl.c
index 900b088..2fed89d 100644
--- ovn/utilities/ovn-nbctl.c
+++ ovn/utilities/ovn-nbctl.c
@@ -330,7 +330,7 @@ Logical switch commands:\n\
   ls-list                   print the names of all logical switches\n\
 \n\
 ACL commands:\n\
-  acl-add SWITCH DIRECTION PRIORITY MATCH ACTION [log]\n\
+  acl-add SWITCH DIRECTION PRIORITY MATCH ACTION [ACL-OPTIONS] [log]\n\
                             add an ACL to SWITCH\n\
   acl-del SWITCH [DIRECTION [PRIORITY MATCH]]\n\
                             remove ACLs from SWITCH\n\
@@ -366,6 +366,28 @@ Logical switch port commands:\n\
                             set dhcpv4 options for PORT\n\
   lsp-get-dhcpv4-options PORT  get the dhcpv4 options for PORT\n\
 \n\
+Logical port-chain commands:\n\
+  lsp-chain-add SWITCH [CHAIN] LAST_PORT create a logical port-chain [named LSP-CHAIN]\n\
+                                         that has LAST_PORT as last hop at the end of chain\n\
+  lsp-chain-del CHAIN                  delete LSP-CHAIN\n\
+  lsp-chain-list [SWITCH]              print the names of all logical port-chains [on SWITCH]\n\
+  lsp-chain-show SWITCH [CHAIN]        print details on port-chains on SWITCH\n\
+\n\
+Logical port-pair-groups commands:\n\
+  lsp-pair-group-add CHAIN [PAIR-GROUP [OFFSET]]\n\
+                    create a logical port-pair-group. Optionally, indicate the order it\n\
+                    should be in chain.\n\
+  lsp-pair-group-del PAIR-GROUP    delete a port-pair-group, does not delete port-pairs\n\
+  lsp-pair-group-list CHAIN    print port-pair-groups for a givan chain\n\
+  lsp-pair-group-add-port-pair PAIR-GROUP LSP-PAIR add a port pair to a port-pair-group\n\
+  lsp-pair-group-del-port-pair PAIR-GROUP LSP-PAIR del a port pair from a port-pair-group\n\
+\n\
+Logical port-pair commands:\n\
+  lsp-pair-add SWITCH PORT-IN PORT-OUT [LSP-PAIR]\n\
+                                     create a logical port-pair\n\
+  lsp-pair-del LSP-PAIR              delete a port-pair, does not delete ports\n\
+  lsp-pair-list [SWITCH [LSP-PAIR]]  print the names of all logical port-pairs\n\
+\n\
 Logical router commands:\n\
   lr-add [ROUTER]           create a logical router named ROUTER\n\
   lr-del ROUTER             delete ROUTER and all its ports\n\
@@ -764,6 +786,691 @@ lsp_by_name_or_uuid(struct ctl_context *ctx, const char *id,
 
     return lsp;
 }
+
+/*
+ * Port chain CLI Functions
+ */
+static const struct nbrec_logical_port_chain *
+lsp_chain_by_name_or_uuid(struct ctl_context *ctx, const char *id, const bool must_exist)
+{
+    const struct nbrec_logical_port_chain *lsp_chain = NULL;
+    bool is_uuid = false;
+    struct uuid lsp_chain_uuid;
+
+    if (uuid_from_string(&lsp_chain_uuid, id)) {
+        is_uuid = true;
+        lsp_chain = nbrec_logical_port_chain_get_for_uuid(ctx->idl,
+                                                          &lsp_chain_uuid);
+    }
+
+    if (!lsp_chain) {
+        NBREC_LOGICAL_PORT_CHAIN_FOR_EACH(lsp_chain, ctx->idl) {
+            if (!strcmp(lsp_chain->name, id)) {
+                break;
+            }
+        }
+    }
+    if (!lsp_chain && must_exist) {
+        ctl_fatal("lsp_chain not found for %s: '%s'",
+                  is_uuid ? "UUID" : "name", id);
+    }
+
+    return lsp_chain;
+}
+static const struct nbrec_logical_port_pair_group *
+lsp_pair_group_by_name_or_uuid(struct ctl_context *ctx, const char *id, const bool must_exist)
+{
+    const struct nbrec_logical_port_pair_group *lsp_pair_group = NULL;
+    bool is_uuid = false;
+    struct uuid lsp_pair_group_uuid;
+
+    if (uuid_from_string(&lsp_pair_group_uuid, id)) {
+        is_uuid = true;
+        lsp_pair_group = nbrec_logical_port_pair_group_get_for_uuid(ctx->idl,
+                                                                    &lsp_pair_group_uuid);
+    }
+
+    if (!lsp_pair_group) {
+        NBREC_LOGICAL_PORT_PAIR_GROUP_FOR_EACH(lsp_pair_group, ctx->idl) {
+            if (!strcmp(lsp_pair_group->name, id)) {
+                break;
+            }
+        }
+    }
+    if (!lsp_pair_group && must_exist) {
+        ctl_fatal("lsp_pair_group not found for %s: '%s'",
+                  is_uuid ? "UUID" : "name", id);
+    }
+
+    return lsp_pair_group;
+}
+
+static const struct nbrec_logical_port_pair *
+lsp_pair_by_name_or_uuid(struct ctl_context *ctx, const char *id, const bool must_exist)
+{
+    const struct nbrec_logical_port_pair *lsp_pair = NULL;
+    bool is_uuid = false;
+    struct uuid lsp_pair_uuid;
+
+    if (uuid_from_string(&lsp_pair_uuid, id)) {
+        is_uuid = true;
+        lsp_pair = nbrec_logical_port_pair_get_for_uuid(ctx->idl,
+                                                        &lsp_pair_uuid);
+    }
+
+    if (!lsp_pair) {
+        NBREC_LOGICAL_PORT_PAIR_FOR_EACH(lsp_pair, ctx->idl) {
+            if (!strcmp(lsp_pair->name, id)) {
+                break;
+            }
+        }
+    }
+    if (!lsp_pair && must_exist) {
+        ctl_fatal("lsp_pair not found for %s: '%s'",
+                  is_uuid ? "UUID" : "name", id);
+    }
+
+    return lsp_pair;
+}
+
+
+static void
+nbctl_lsp_chain_add(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_switch *lswitch;
+    const struct nbrec_logical_switch_port *last_hop_lsp;
+
+    lswitch = ls_by_name_or_uuid(ctx, ctx->argv[1], true /*must_exist*/);
+    const char *lsp_chain_name = ctx->argc > 3 ? ctx->argv[2] : NULL;
+    const char *last_hop_lsp_name = lsp_chain_name ? ctx->argv[3] : ctx->argv[2];
+
+    const bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+    const bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") != NULL;
+    if (may_exist && add_duplicate) {
+        ctl_fatal("--may-exist and --add-duplicate may not be used together");
+    }
+
+    last_hop_lsp = lsp_by_name_or_uuid(ctx, last_hop_lsp_name, true);
+
+    if (lsp_chain_name) {
+        if (!add_duplicate) {
+            const struct nbrec_logical_port_chain *lsp_chain;
+            NBREC_LOGICAL_PORT_CHAIN_FOR_EACH(lsp_chain, ctx->idl) {
+                if (!strcmp(lsp_chain->name, lsp_chain_name)) {
+                    if (may_exist) {
+                        return;
+                    }
+                    ctl_fatal("%s: an lsp_chain with this name already exists",
+                              lsp_chain_name);
+                }
+            }
+        }
+    } else if (may_exist) {
+        ctl_fatal("--may-exist requires specifying a name");
+    } else if (add_duplicate) {
+        ctl_fatal("--add-duplicate requires specifying a name");
+    }
+
+    struct nbrec_logical_port_chain *lsp_chain;
+    lsp_chain = nbrec_logical_port_chain_insert(ctx->txn);
+    if (lsp_chain_name) {
+        nbrec_logical_port_chain_set_name(lsp_chain, lsp_chain_name);
+    }
+    nbrec_logical_port_chain_set_last_hop_port(lsp_chain, last_hop_lsp);
+
+    /* Insert the logical port-chain into the logical switch. */
+
+    nbrec_logical_switch_verify_port_chains(lswitch);
+    struct nbrec_logical_port_chain  **new_port_chain = xmalloc(sizeof *new_port_chain *
+                                                                (lswitch->n_port_chains + 1));
+    memcpy(new_port_chain, lswitch->port_chains, sizeof *new_port_chain * lswitch->n_port_chains);
+    new_port_chain[lswitch->n_port_chains] = CONST_CAST(struct nbrec_logical_port_chain *, lsp_chain);
+    nbrec_logical_switch_set_port_chains(lswitch, new_port_chain, lswitch->n_port_chains + 1);
+    free(new_port_chain);
+}
+
+/* Removes lswitch->pair_chain[idx]'. */
+static void
+remove_lsp_chain(const struct nbrec_logical_switch *lswitch, size_t idx)
+{
+    const struct nbrec_logical_port_chain *lsp_chain = lswitch->port_chains[idx];
+
+    /* First remove 'lsp-chain' from the array of port-chains.  This is what will
+     * actually cause the logical port-chain to be deleted when the transaction is
+     * sent to the database server (due to garbage collection). */
+    struct nbrec_logical_port_chain **new_port_chain
+        = xmemdup(lswitch->port_chains, sizeof *new_port_chain * lswitch->n_port_chains);
+    new_port_chain[idx] = new_port_chain[lswitch->n_port_chains - 1];
+    nbrec_logical_switch_verify_port_chains(lswitch);
+    nbrec_logical_switch_set_port_chains(lswitch, new_port_chain, lswitch->n_port_chains - 1);
+    free(new_port_chain);
+
+    /* Delete 'lsp-chain' from the IDL.  This won't have a real effect on the
+     * database server (the IDL will suppress it in fact) but it means that it
+     * won't show up when we iterate with NBREC_LOGICAL_PORT_CHAIN_FOR_EACH later. */
+    nbrec_logical_port_chain_delete(lsp_chain);
+}
+
+static void
+nbctl_lsp_chain_del(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_chain *lsp_chain;
+    const bool must_exist = !shash_find(&ctx->options, "--if-exists");
+
+    lsp_chain = lsp_chain_by_name_or_uuid(ctx, ctx->argv[1], must_exist);
+    if (!lsp_chain) {
+        return;
+    }
+
+    /* Find the lswitch that contains 'port-chain', then delete it. */
+    const struct nbrec_logical_switch *lswitch;
+    NBREC_LOGICAL_SWITCH_FOR_EACH (lswitch, ctx->idl) {
+        for (size_t i = 0; i < lswitch->n_port_chains; i++) {
+            if (lswitch->port_chains[i] == lsp_chain) {
+                remove_lsp_chain(lswitch,i);
+                return;
+            }
+        }
+    }
+}
+
+static void
+print_lsp_chain_entry(struct ctl_context *ctx,
+                      const struct nbrec_logical_switch *lswitch,
+                      const char *chain_name_filter,
+                      const bool show_switch_name)
+{
+    struct smap lsp_chains;
+    size_t i;
+
+    smap_init(&lsp_chains);
+    for (i = 0; i < lswitch->n_port_chains; i++) {
+        const struct nbrec_logical_port_chain *lsp_chain = lswitch->port_chains[i];
+        if (chain_name_filter && strcmp(chain_name_filter, lsp_chain->name)) {
+            continue;
+        }
+        if (show_switch_name) {
+            smap_add_format(&lsp_chains, lsp_chain->name, UUID_FMT " (%s:%s)",
+                            UUID_ARGS(&lsp_chain->header_.uuid),
+                            lswitch->name, lsp_chain->name);
+        } else {
+            smap_add_format(&lsp_chains, lsp_chain->name, UUID_FMT " (%s)",
+                            UUID_ARGS(&lsp_chain->header_.uuid), lsp_chain->name);
+        }
+    }
+
+    const struct smap_node **nodes = smap_sort(&lsp_chains);
+    for (i = 0; i < smap_count(&lsp_chains); i++) {
+        const struct smap_node *node = nodes[i];
+        ds_put_format(&ctx->output, "%s\n", node->value);
+    }
+    smap_destroy(&lsp_chains);
+    free(nodes);
+}
+
+static void
+nbctl_lsp_chain_list(struct ctl_context *ctx)
+{
+    const char *id = ctx->argc > 1 ? ctx->argv[1] : NULL;
+    const char *chain_name_filter = ctx->argc > 2 ? ctx->argv[2] : NULL;
+    const struct nbrec_logical_switch *lswitch;
+
+    if (id) {
+        lswitch = ls_by_name_or_uuid(ctx, id, true);
+        print_lsp_chain_entry(ctx, lswitch, chain_name_filter, false);
+    } else {
+        NBREC_LOGICAL_SWITCH_FOR_EACH(lswitch, ctx->idl) {
+            if (lswitch->n_port_chains == 0) {
+                continue;
+            }
+            print_lsp_chain_entry(ctx, lswitch, chain_name_filter, true);
+        }
+    }
+}
+
+static void
+print_lsp_chain(const struct nbrec_logical_port_chain *lsp_chain,
+                  struct ctl_context *ctx)
+{
+    ds_put_format(&ctx->output, "lsp-chain "UUID_FMT" (%s)\n",
+                  UUID_ARGS(&lsp_chain->header_.uuid), lsp_chain->name);
+
+    for (size_t i = 0; i < lsp_chain->n_port_pair_groups; i++) {
+        const struct nbrec_logical_port_pair_group *lsp_pair_group
+            = lsp_chain->port_pair_groups[i];
+        ds_put_format(&ctx->output, "    lsp-pair-group %s\n", lsp_pair_group->name);
+        for (size_t j = 0; j < lsp_pair_group->n_port_pairs; j++){
+            const struct nbrec_logical_port_pair *lsp_pair = lsp_pair_group->port_pairs[j];
+            ds_put_format(&ctx->output, "        lsp-pair %s\n", lsp_pair->name);
+
+            const struct nbrec_logical_switch_port *linport  = lsp_pair->inport;
+            if (linport) {
+                ds_put_format(&ctx->output, "            lsp-pair inport "UUID_FMT" (%s)\n",
+                              UUID_ARGS(&linport->header_.uuid), linport->name);
+            }
+
+            const struct nbrec_logical_switch_port *loutport = lsp_pair->outport;
+            if (loutport) {
+                ds_put_format(&ctx->output, "            lsp-pair outport "UUID_FMT" (%s)\n",
+                              UUID_ARGS(&loutport->header_.uuid), loutport->name);
+            }
+        }
+    }
+
+    // TODO: iterate ACLs and display the ones that have action 'sfc' and use this lsp_chain
+}
+
+static void
+nbctl_lsp_chain_show(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_chain *lsp_chain;
+
+    if (ctx->argc == 2) {
+        lsp_chain = lsp_chain_by_name_or_uuid(ctx, ctx->argv[1], false);
+        if (lsp_chain) {
+            print_lsp_chain(lsp_chain, ctx);
+        }
+    } else {
+        NBREC_LOGICAL_PORT_CHAIN_FOR_EACH(lsp_chain, ctx->idl) {
+            print_lsp_chain(lsp_chain, ctx);
+        }
+    }
+}
+/* End of port-chain operations */
+
+/*
+ * Port Pair Groups CLI Functions
+ */
+static void
+nbctl_lsp_pair_group_add(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_pair_group *lsp_pair_group;
+    const char *ppg_name = ctx->argc >= 3 ? ctx->argv[2] : NULL;
+
+    const bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+    const bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") != NULL;
+    if (may_exist && add_duplicate) {
+        ctl_fatal("--may-exist and --add-duplicate may not be used together");
+    }
+
+    if (ppg_name) {
+        if (!add_duplicate) {
+            NBREC_LOGICAL_PORT_PAIR_GROUP_FOR_EACH(lsp_pair_group, ctx->idl) {
+                if (!strcmp(lsp_pair_group->name, ppg_name)) {
+                    if (may_exist) {
+                        return;
+                    }
+                    ctl_fatal("%s: an lsp_port_pair_group with this name already exists",
+                              ppg_name);
+                }
+            }
+        }
+    } else if (may_exist) {
+        ctl_fatal("--may-exist requires specifying a name");
+    } else if (add_duplicate) {
+        ctl_fatal("--add-duplicate requires specifying a name");
+    }
+
+    /* check lsp_chain exists */
+    const struct nbrec_logical_port_chain *lsp_chain;
+    lsp_chain = lsp_chain_by_name_or_uuid(ctx, ctx->argv[1], true);
+    if (!lsp_chain) {
+        return;
+    }
+
+    /* create the logical port-pair-group. */
+    lsp_pair_group = nbrec_logical_port_pair_group_insert(ctx->txn);
+    if (ppg_name) {
+        nbrec_logical_port_pair_group_set_name(lsp_pair_group, ctx->argv[2]);
+    }
+
+    int64_t sortkey = (int64_t) lsp_chain->n_port_pair_groups + 1;
+    if (ctx->argc >= 4) {
+        sortkey = (int64_t) atoi(ctx->argv[3]);
+    }
+    nbrec_logical_port_pair_group_set_sortkey(lsp_pair_group, &sortkey, 1);
+
+    /* Insert the logical port-pair-group into the logical switch. */
+    nbrec_logical_port_chain_verify_port_pair_groups(lsp_chain);
+    struct nbrec_logical_port_pair_group  **new_port_pair_group = xmalloc(sizeof *new_port_pair_group *
+                                                                          (lsp_chain->n_port_pair_groups + 1));
+    memcpy(new_port_pair_group, lsp_chain->port_pair_groups, sizeof *new_port_pair_group * lsp_chain->n_port_pair_groups);
+    new_port_pair_group[lsp_chain->n_port_pair_groups] =
+        CONST_CAST(struct nbrec_logical_port_pair_group *,lsp_pair_group);
+    nbrec_logical_port_chain_set_port_pair_groups(lsp_chain, new_port_pair_group, lsp_chain->n_port_pair_groups + 1);
+    free(new_port_pair_group);
+}
+
+/* Removes lsp-pair-group 'lsp_chain->port_pair_group[idx]'. */
+static void
+remove_lsp_pair_group(const struct nbrec_logical_port_chain *lsp_chain, size_t idx)
+{
+    const struct nbrec_logical_port_pair_group *lsp_pair_group = lsp_chain->port_pair_groups[idx];
+
+    /* First remove 'lsp-pair-group' from the array of port-pair-groups.  This is what will
+     * actually cause the logical port-pair-group to be deleted when the transaction is
+     * sent to the database server (due to garbage collection). */
+    struct nbrec_logical_port_pair_group **new_port_pair_group
+        = xmemdup(lsp_chain->port_pair_groups, sizeof *new_port_pair_group * lsp_chain->n_port_pair_groups);
+    new_port_pair_group[idx] = new_port_pair_group[lsp_chain->n_port_pair_groups - 1];
+    nbrec_logical_port_chain_verify_port_pair_groups(lsp_chain);
+    nbrec_logical_port_chain_set_port_pair_groups(lsp_chain, new_port_pair_group, lsp_chain->n_port_pair_groups - 1);
+    free(new_port_pair_group);
+
+    /* Delete 'lsp-pair-group' from the IDL.  This won't have a real effect on the
+     * database server (the IDL will suppress it in fact) but it means that it
+     * won't show up when we iterate with NBREC_LOGICAL_PORT_PAIR_GROUP_FOR_EACH later. */
+    nbrec_logical_port_pair_group_delete(lsp_pair_group);
+}
+
+static void
+nbctl_lsp_pair_group_del(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_pair_group *lsp_pair_group;
+    const bool must_exist = !shash_find(&ctx->options, "--if-exists");
+
+    lsp_pair_group = lsp_pair_group_by_name_or_uuid(ctx, ctx->argv[1], must_exist);
+    if (!lsp_pair_group) {
+        return;
+    }
+
+    /* Find the port-chain that contains 'port-pair-group', then delete it. */
+    const struct nbrec_logical_port_chain *lsp_chain;
+    NBREC_LOGICAL_PORT_CHAIN_FOR_EACH (lsp_chain, ctx->idl) {
+        for (size_t i = 0; i < lsp_chain->n_port_pair_groups; i++) {
+            if (lsp_chain->port_pair_groups[i] == lsp_pair_group) {
+                remove_lsp_pair_group(lsp_chain,i);
+                return;
+            }
+        }
+    }
+    if (must_exist) {
+        ctl_fatal("logical port-pair-group %s is not part of any logical port-chain",
+                  ctx->argv[1]);
+    }
+}
+
+static void
+nbctl_lsp_pair_group_list(struct ctl_context *ctx)
+{
+    const char *id = ctx->argv[1];
+    const struct nbrec_logical_port_chain *lsp_chain;
+    struct smap lsp_pair_groups;
+    size_t i;
+
+    lsp_chain = lsp_chain_by_name_or_uuid(ctx, id, true);
+    if (!lsp_chain) {
+        return;
+    }
+
+    smap_init(&lsp_pair_groups);
+    for (i = 0; i < lsp_chain->n_port_pair_groups; i++) {
+        const struct nbrec_logical_port_pair_group *lsp_pair_group = lsp_chain->port_pair_groups[i];
+        smap_add_format(&lsp_pair_groups, lsp_pair_group->name, UUID_FMT " (%s)",
+                        UUID_ARGS(&lsp_pair_group->header_.uuid), lsp_pair_group->name);
+    }
+    const struct smap_node **nodes = smap_sort(&lsp_pair_groups);
+    for (i = 0; i < smap_count(&lsp_pair_groups); i++) {
+        const struct smap_node *node = nodes[i];
+        ds_put_format(&ctx->output, "%s\n", node->value);
+    }
+    smap_destroy(&lsp_pair_groups);
+    free(nodes);
+}
+
+static void
+nbctl_lsp_pair_group_add_port_pair(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_pair_group *lsp_pair_group;
+    const struct nbrec_logical_port_pair *lsp_pair;
+    const bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+
+    lsp_pair_group = lsp_pair_group_by_name_or_uuid(ctx, ctx->argv[1], true);
+    if (!lsp_pair_group) {
+        return;
+    }
+
+    /* Check that port-pair exists  */
+    lsp_pair = lsp_pair_by_name_or_uuid(ctx, ctx->argv[2], true);
+    if (!lsp_pair){
+        return;
+    }
+
+    /* Do not add port pair more than once in a given port-pair-group */
+    for (size_t i = 0; i < lsp_pair_group->n_port_pairs; i++) {
+        if (lsp_pair_group->port_pairs[i] == lsp_pair) {
+            if (!may_exist) {
+                ctl_fatal("lsp_pair: %s is already added to port-pair-group %s\n", ctx->argv[2], ctx->argv[1]);
+            }
+            return;
+        }
+    }
+
+    /* Insert the logical port-pair into the logical port-pair-group. */
+    nbrec_logical_port_pair_group_verify_port_pairs(lsp_pair_group);
+    struct nbrec_logical_port_pair  **new_port_pair = xmalloc(sizeof *new_port_pair *
+                                                              (lsp_pair_group->n_port_pairs + 1));
+    memcpy(new_port_pair, lsp_pair_group->port_pairs, sizeof *new_port_pair * lsp_pair_group->n_port_pairs);
+    new_port_pair[lsp_pair_group->n_port_pairs] = CONST_CAST(struct nbrec_logical_port_pair *, lsp_pair);
+    nbrec_logical_port_pair_group_set_port_pairs(lsp_pair_group, new_port_pair, lsp_pair_group->n_port_pairs + 1);
+    free(new_port_pair);
+}
+
+/* Removes port-pair from port-pair-groiup but does not delete it'. */
+static void
+remove_lsp_pair_from_port_pair_group(const struct nbrec_logical_port_pair_group *lsp_pair_group, size_t idx)
+{
+    //TODO Check const struct nbrec_logical_port_pair *lsp_pair = lsp_pair_group->port_pairs[idx];
+
+    /* First remove 'lsp-pair' from the array of port-pairs.  This is what will
+     * actually cause the logical port-pair to be deleted when the transaction is
+     * sent to the database server (due to garbage collection). */
+    struct nbrec_logical_port_pair **new_port_pair
+        = xmemdup(lsp_pair_group->port_pairs, sizeof *new_port_pair * lsp_pair_group->n_port_pairs);
+    new_port_pair[idx] = new_port_pair[lsp_pair_group->n_port_pairs - 1];
+    nbrec_logical_port_pair_group_verify_port_pairs(lsp_pair_group);
+    nbrec_logical_port_pair_group_set_port_pairs(lsp_pair_group, new_port_pair, lsp_pair_group->n_port_pairs - 1);
+    free(new_port_pair);
+
+    /* Do not delete actual port-pair as they are owned by a lswitch and can be reused. */
+    //nbrec_logical_port_pair_delete(lsp_pair);
+}
+
+static void
+nbctl_lsp_pair_group_del_port_pair(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_pair *lsp_pair;
+    const bool must_exist = !shash_find(&ctx->options, "--if-exists");
+
+    lsp_pair = lsp_pair_by_name_or_uuid(ctx, ctx->argv[1], must_exist);
+    if (!lsp_pair) {
+        return;
+    }
+
+    /* Find the port-pair_group that contains 'port-pair', then delete it. */
+    const struct nbrec_logical_port_pair_group *lsp_pair_group;
+    NBREC_LOGICAL_PORT_PAIR_GROUP_FOR_EACH (lsp_pair_group, ctx->idl) {
+        for (size_t i = 0; i < lsp_pair_group->n_port_pairs; i++) {
+            if (lsp_pair_group->port_pairs[i] == lsp_pair) {
+                remove_lsp_pair_from_port_pair_group(lsp_pair_group,i);
+                return;
+            }
+        }
+    }
+    if (must_exist) {
+        ctl_fatal("logical port-pair %s is not part of any logical switch",
+                  ctx->argv[1]);
+    }
+}
+/* End of port-pair-group operations */
+
+/*
+ * port-pair operations
+ */
+static void
+nbctl_lsp_pair_add(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_switch *lswitch;
+    const struct nbrec_logical_switch_port *lsp_in,*lsp_out;
+    const struct nbrec_logical_port_pair *lsp_pair;
+
+    const bool may_exist = shash_find(&ctx->options, "--may-exist") != NULL;
+    const bool add_duplicate = shash_find(&ctx->options, "--add-duplicate") != NULL;
+
+    lswitch = ls_by_name_or_uuid(ctx, ctx->argv[1], true);
+    lsp_in = lsp_by_name_or_uuid(ctx, ctx->argv[2], true);
+    lsp_out = lsp_by_name_or_uuid(ctx, ctx->argv[3], true);
+
+    const char *lsp_pair_name = ctx->argc >= 5 ? ctx->argv[4] : NULL;
+    if (may_exist && add_duplicate) {
+        ctl_fatal("--may-exist and --add-duplicate may not be used together");
+    }
+
+    if (lsp_pair_name) {
+        if (!add_duplicate) {
+            NBREC_LOGICAL_PORT_PAIR_FOR_EACH(lsp_pair, ctx->idl) {
+                if (!strcmp(lsp_pair->name, lsp_pair_name)) {
+                    if (may_exist) {
+                        return;
+                    }
+                    ctl_fatal("%s: an lsp_pair with this name already exists",
+                              lsp_pair_name);
+                }
+            }
+        }
+    } else if (may_exist) {
+        ctl_fatal("--may-exist requires specifying a name");
+    } else if (add_duplicate) {
+        ctl_fatal("--add-duplicate requires specifying a name");
+    }
+
+    /* create the logical port-pair. */
+    lsp_pair = nbrec_logical_port_pair_insert(ctx->txn);
+    nbrec_logical_port_pair_set_inport(lsp_pair, lsp_in);
+    nbrec_logical_port_pair_set_outport(lsp_pair, lsp_out);
+    if (lsp_pair_name) {
+        nbrec_logical_port_pair_set_name(lsp_pair, lsp_pair_name);
+    }
+
+    /* Insert the logical port-pair into the logical port-pair-group. */
+    nbrec_logical_switch_verify_port_pairs(lswitch);
+    struct nbrec_logical_port_pair  **new_port_pair = xmalloc(sizeof *new_port_pair *
+                                                              (lswitch->n_port_pairs + 1));
+    memcpy(new_port_pair, lswitch->port_pairs, sizeof *new_port_pair * lswitch->n_port_pairs);
+    new_port_pair[lswitch->n_port_pairs] = CONST_CAST(struct nbrec_logical_port_pair *, lsp_pair);
+    nbrec_logical_switch_set_port_pairs(lswitch, new_port_pair, lswitch->n_port_pairs + 1);
+    free(new_port_pair);
+}
+/* Removes lswitch->pair_pair[idx]'. */
+static void
+remove_lsp_pair(const struct nbrec_logical_switch *lswitch, size_t idx)
+{
+    const struct nbrec_logical_port_pair *lsp_pair = lswitch->port_pairs[idx];
+
+    /* First remove 'lsp-pair' from the array of port-pairs.  This is what will
+     * actually cause the logical port-pair to be deleted when the transaction is
+     * sent to the database server (due to garbage collection). */
+    struct nbrec_logical_port_pair **new_port_pair
+        = xmemdup(lswitch->port_pairs, sizeof *new_port_pair * lswitch->n_port_pairs);
+    new_port_pair[idx] = new_port_pair[lswitch->n_port_pairs - 1];
+    nbrec_logical_switch_verify_port_pairs(lswitch);
+    nbrec_logical_switch_set_port_pairs(lswitch, new_port_pair, lswitch->n_port_pairs - 1);
+    free(new_port_pair);
+
+    /* Delete 'lsp-pair' from the IDL.  This won't have a real effect on the
+     * database server (the IDL will suppress it in fact) but it means that it
+     * won't show up when we iterate with NBREC_LOGICAL_PORT_PAIR_FOR_EACH later. */
+    nbrec_logical_port_pair_delete(lsp_pair);
+}
+
+static void
+nbctl_lsp_pair_del(struct ctl_context *ctx)
+{
+    const struct nbrec_logical_port_pair *lsp_pair;
+    const bool must_exist = !shash_find(&ctx->options, "--if-exists");
+
+    lsp_pair = lsp_pair_by_name_or_uuid(ctx, ctx->argv[1], must_exist);
+    if (!lsp_pair) {
+        if (must_exist) {
+            ctl_fatal("Cannot find lsp_pair: %s\n", ctx->argv[1]);
+        }
+    }
+
+    /* Find the port-pair_group that contains 'port-pair', then delete it. */
+    const struct nbrec_logical_switch *lswitch;
+    NBREC_LOGICAL_SWITCH_FOR_EACH(lswitch, ctx->idl) {
+        for (size_t i = 0; i < lswitch->n_port_pairs; i++) {
+            if (lswitch->port_pairs[i] == lsp_pair) {
+                remove_lsp_pair(lswitch,i);
+                return;
+            }
+        }
+    }
+    if (must_exist) {
+        ctl_fatal("logical port-pair %s is not part of any logical switch",
+                  ctx->argv[1]);
+    }
+}
+
+static void
+print_lsp_pairs_for_switch(struct ctl_context *ctx,
+                           const struct nbrec_logical_switch *lswitch,
+                           const char *ppair_name_filter,
+                           const bool show_switch_name)
+{
+    struct smap lsp_pairs;
+    size_t i;
+
+    smap_init(&lsp_pairs);
+    for (i = 0; i < lswitch->n_port_pairs; i++) {
+        const struct nbrec_logical_port_pair *lsp_pair = lswitch->port_pairs[i];
+        if (ppair_name_filter && strcmp(ppair_name_filter, lsp_pair->name)) {
+            continue;
+        }
+        const struct nbrec_logical_switch_port *linport  = lsp_pair->inport;
+        const struct nbrec_logical_switch_port *loutport = lsp_pair->outport;
+        const char *linport_name = linport ? linport->name : "<not_set>";
+        const char *loutport_name = loutport ? loutport->name : "<not_set>";
+
+        if (show_switch_name) {
+            smap_add_format(&lsp_pairs, lsp_pair->name, UUID_FMT " (%s:%s) in:%s out:%s",
+                            UUID_ARGS(&lsp_pair->header_.uuid), lswitch->name,
+                            lsp_pair->name, linport_name, loutport_name);
+        } else {
+            smap_add_format(&lsp_pairs, lsp_pair->name, UUID_FMT " (%s) in:%s out:%s",
+                            UUID_ARGS(&lsp_pair->header_.uuid),
+                            lsp_pair->name, linport_name, loutport_name);
+        }
+    }
+    const struct smap_node **nodes = smap_sort(&lsp_pairs);
+    for (i = 0; i < smap_count(&lsp_pairs); i++) {
+        const struct smap_node *node = nodes[i];
+        ds_put_format(&ctx->output, "%s\n", node->value);
+    }
+    smap_destroy(&lsp_pairs);
+    free(nodes);
+}
+
+static void
+nbctl_lsp_pair_list(struct ctl_context *ctx)
+{
+    const char *id = ctx->argc > 1 ? ctx->argv[1] : NULL;
+    const char *pair_name_filter = ctx->argc > 2 ? ctx->argv[2] : NULL;
+    const struct nbrec_logical_switch *lswitch;
+
+    if (id) {
+        lswitch = ls_by_name_or_uuid(ctx, id, true);
+        print_lsp_pairs_for_switch(ctx, lswitch, pair_name_filter, false);
+    } else {
+        NBREC_LOGICAL_SWITCH_FOR_EACH(lswitch, ctx->idl) {
+            if (lswitch->n_port_pairs == 0) {
+                continue;
+            }
+            print_lsp_pairs_for_switch(ctx, lswitch, pair_name_filter, true);
+        }
+    }
+}
+/* End of port-pair operations */
 
 /* Returns the logical switch that contains 'lsp'. */
 static const struct nbrec_logical_switch *
@@ -1304,12 +2011,33 @@ nbctl_acl_add(struct ctl_context *ctx)
 
     /* Validate action. */
     if (strcmp(action, "allow") && strcmp(action, "allow-related")
-        && strcmp(action, "drop") && strcmp(action, "reject")) {
+        && strcmp(action, "drop") && strcmp(action, "reject")
+        && strcmp(action, "sfc")) {
         ctl_fatal("%s: action must be one of \"allow\", \"allow-related\", "
-                  "\"drop\", and \"reject\"", action);
+                  "\"drop\", \"reject\" and \"sfc\"", action);
         return;
     }
 
+    /* Validate ACL Options, if there were any provided. */
+    struct smap acl_options = SMAP_INITIALIZER(&acl_options);
+    if (ctx->argc >= 7) {
+        struct sset acl_options_set;
+        sset_from_delimited_string(&acl_options_set, ctx->argv[6], " ");
+
+        const char *acl_option_tuple;
+        SSET_FOR_EACH (acl_option_tuple, &acl_options_set) {
+            char *key, *value;
+            value = xstrdup(acl_option_tuple);
+            key = strsep(&value, "=");
+            if (value) {
+                smap_add(&acl_options, key, value);
+            }
+            free(key);
+        }
+
+        sset_destroy(&acl_options_set);
+    }
+
     /* Create the acl. */
     struct nbrec_acl *acl = nbrec_acl_insert(ctx->txn);
     nbrec_acl_set_priority(acl, priority);
@@ -1319,6 +2047,9 @@ nbctl_acl_add(struct ctl_context *ctx)
     if (shash_find(&ctx->options, "--log") != NULL) {
         nbrec_acl_set_log(acl, true);
     }
+    if (! smap_is_empty(&acl_options)) {
+        nbrec_acl_set_options(acl, &acl_options);
+    }
 
     /* Check if same acl already exists for the ls */
     for (size_t i = 0; i < ls->n_acls; i++) {
@@ -1339,6 +2070,8 @@ nbctl_acl_add(struct ctl_context *ctx)
     new_acls[ls->n_acls] = acl;
     nbrec_logical_switch_set_acls(ls, new_acls, ls->n_acls + 1);
     free(new_acls);
+
+    smap_destroy(&acl_options);
 }
 
 static void
@@ -3027,7 +3760,65 @@ cmd_set_ssl(struct ctl_context *ctx)
 
     nbrec_nb_global_set_ssl(nb_global, ssl);
 }
+#ifdef JED
+static const struct ctl_table_class tables[] = {
+    {&nbrec_table_nb_global,
+     {{&nbrec_table_nb_global, NULL, NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_logical_switch,
+     {{&nbrec_table_logical_switch, &nbrec_logical_switch_col_name, NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_logical_switch_port,
+     {{&nbrec_table_logical_switch_port, &nbrec_logical_switch_port_col_name,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_acl,
+     {{NULL, NULL, NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_load_balancer,
+     {{NULL, NULL, NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_logical_router,
+     {{&nbrec_table_logical_router, &nbrec_logical_router_col_name, NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_logical_router_port,
+     {{&nbrec_table_logical_router_port, &nbrec_logical_router_port_col_name,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_logical_router_static_route,
+     {{&nbrec_table_logical_router_static_route, NULL,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_nat,
+     {{&nbrec_table_nat, NULL,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_address_set,
+     {{&nbrec_table_address_set, &nbrec_address_set_col_name, NULL},
+      {NULL, NULL, NULL}}},
 
+    {&nbrec_table_dhcp_options,
+     {{&nbrec_table_dhcp_options, NULL,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {&nbrec_table_qos,
+     {{&nbrec_table_qos, NULL,
+       NULL},
+      {NULL, NULL, NULL}}},
+
+    {NULL, {{NULL, NULL, NULL}, {NULL, NULL, NULL}}}
+};
+#endif
 static const struct ctl_table_class tables[NBREC_N_TABLES] = {
     [NBREC_TABLE_LOGICAL_SWITCH].row_ids[0]
     = {&nbrec_table_logical_switch, &nbrec_logical_switch_col_name, NULL},
@@ -3293,6 +4084,32 @@ static const struct ctl_command_syntax nbctl_commands[] = {
     { "sync", 0, 0, "", nbctl_pre_sync, nbctl_sync, NULL, "", RO },
     { "show", 0, 1, "[SWITCH]", NULL, nbctl_show, NULL, "", RO },
 
+    /* lsp-chain commands. */
+    { "lsp-chain-add", 2, 3, "SWITCH [CHAIN] LAST_PORT", NULL, nbctl_lsp_chain_add,
+      NULL, "--may-exist,--add-duplicate", RW },
+    { "lsp-chain-del", 1, 1, "CHAIN", NULL, nbctl_lsp_chain_del,
+      NULL, "--if-exists", RW },
+    { "lsp-chain-list", 0, 2, "[SWITCH [CHAIN]]", NULL, nbctl_lsp_chain_list, NULL, "", RO },
+    { "lsp-chain-show", 0, 1, "[CHAIN]", NULL, nbctl_lsp_chain_show, NULL, "", RO },
+
+    /* lsp-pair-group commands. */
+    { "lsp-pair-group-add", 1, 3, "CHAIN [PAIR-GROUP [OFFSET]]",
+      NULL, nbctl_lsp_pair_group_add, NULL, "--may-exist,--add-duplicate", RW },
+    { "lsp-pair-group-del", 1, 1, "PAIR-GROUP", NULL, nbctl_lsp_pair_group_del,
+      NULL, "--if-exists", RW },
+    { "lsp-pair-group-list", 1, 1, "CHAIN", NULL, nbctl_lsp_pair_group_list, NULL, "", RO },
+    { "lsp-pair-group-add-port-pair", 2, 2, "PAIR-GROUP LSP-PAIR",
+      NULL, nbctl_lsp_pair_group_add_port_pair, NULL, "--may-exist", RW },
+    { "lsp-pair-group-del-port-pair", 2, 2, "PAIR-GROUP LSP-PAIR",
+      NULL, nbctl_lsp_pair_group_del_port_pair, NULL, "--if-exists", RW },
+
+    /* lsp-pair commands. */
+    { "lsp-pair-add", 3, 4, "SWITCH, PORT-IN, PORT-OUT [LSP-PAIR]", NULL, nbctl_lsp_pair_add,
+      NULL, "--may-exist,--add-duplicate", RW },
+    { "lsp-pair-del", 1, 1, "LSP-PAIR", NULL, nbctl_lsp_pair_del,
+      NULL, "--if-exists", RW },
+    { "lsp-pair-list", 0, 2, "[SWITCH [LSP-PAIR]]", NULL, nbctl_lsp_pair_list, NULL, "", RO },
+
     /* logical switch commands. */
     { "ls-add", 0, 1, "[SWITCH]", NULL, nbctl_ls_add, NULL,
       "--may-exist,--add-duplicate", RW },
@@ -3300,7 +4117,7 @@ static const struct ctl_command_syntax nbctl_commands[] = {
     { "ls-list", 0, 0, "", NULL, nbctl_ls_list, NULL, "", RO },
 
     /* acl commands. */
-    { "acl-add", 5, 5, "SWITCH DIRECTION PRIORITY MATCH ACTION", NULL,
+    { "acl-add", 5, 6, "SWITCH DIRECTION PRIORITY MATCH ACTION [ACL-OPTIONS]", NULL,
       nbctl_acl_add, NULL, "--log,--may-exist", RW },
     { "acl-del", 1, 4, "SWITCH [DIRECTION [PRIORITY MATCH]]", NULL,
       nbctl_acl_del, NULL, "", RW },


More information about the dev mailing list