[ovs-dev] [RFC ovn] RFC: controller: Replace ofctl flow tables with a new dp flow manager module.

Numan Siddique numans at ovn.org
Wed Mar 3 14:12:13 UTC 2021


On Wed, Mar 3, 2021 at 3:19 AM Mark Michelson <mmichels at redhat.com> wrote:
>
> Hi Numan,
>
> I took a quick look through the patch. One question I have is what the
> reason is for having separate physical and logical flow hmaps in
> dp_flow_table. It looks like there is not any distinct difference in
> handling logical vs. physical flows. The only benefit this seems to
> provide is having separate "namespaces" for OVN flows. But this idea
> doesn't appear to work once it comes time to translate the OVN flows
> into openflow, since all of the flows are added to the same bridge.
> Separating the different flow types just seems to add unnecessary
> overhead. Am I missing something?
>
> Note that I have not read through your review request for separating
> physical and logical flow handling. With the below patch in isolation,
> making a distinction between physical and logical flows does not have a
> clear benefit to me.

Hi Mark,

Thanks for the comments.

The reason why it is separated is because it is possible that a full recompute
of physical flows can happen without the need to have a full recompute of
logical flows.

With the present master, we call physical_run() in the handler -
flow_output_physical_flow_changes_handler().
It is not a good thing to call physical_run() in a handler as it can
have side effects like some of the OF flows
are not cleared properly etc which we saw recently and hence Dumitru
addressed it [2].

The other patch which I have proposed here [1] separates flow output
engine into lflow_output and pflow_output
engine nodes so that a full recompute of physical flows can be done
without clearing the flows related to logical flows.
The patch [1] maintains 2 separate flow tables.

This RFC patch does the same.  Having separate flow tables would help
and I think it's better.
Let me know if you have other comments or a better way to handle this scenario.

[1] - https://patchwork.ozlabs.org/project/ovn/patch/20210218155445.1973089-1-numans@ovn.org/
[2] - https://github.com/ovn-org/ovn/commit/a2bf85296b2d0abe807d1cadf2ed29482318df11
Thanks
Numan




>
> On 2/25/21 2:19 PM, numans at ovn.org wrote:
> > From: Numan Siddique <numans at ovn.org>
> >
> > This patch attempts to refactor the ofctrl flow table management with
> > a different approch.  The existing ofctrl module maintains a desired
> > flow table and an installed flow table.  ofctrl_put() syncs the desired
> > flow table with the installed flow table after pushing the flows to
> > ovs-vswitchd.  The existing ofctrl code has become complicated over
> > a period of time.  This patch tries to refactor the flow table
> > management by taking a different approach.
> >
> > In this approach
> >    - We maintain a flow table for each datapath - datapath flow table.
> >    - A separate flow table with in the datapath table
> >      is maintained for logical flows and physical flows.
> >    - Adding a flow with a given match-action multiple times is allowed.
> >      i.e if a flow with a given match-action is already present, then
> >      attempting to add again would be ignored.
> >
> > TODO:
> >    - If the approach seems fine, then add unit tests.
> >    - Share the scale testing report comparing with the master.
> >    - 2 Load balancer hairpin test cases are failing which needs to be
> >      fixed.
> >
> > Signed-off-by: Numan Siddique <numans at ovn.org>
> > ---
> >
> > The same patch can be found here in case if this patch goes out of sync.
> >    - https://github.com/numansiddique/ovn/tree/dp_flow_manager/rfc/v1
> >
> >   controller/automake.mk      |    5 +-
> >   controller/dp-flow-mgr.c    | 1317 +++++++++++++++++++++++++++++++++++
> >   controller/dp-flow-mgr.h    |   64 ++
> >   controller/lflow.c          |  255 ++++---
> >   controller/lflow.h          |    8 +-
> >   controller/ofctrl.c         | 1233 +-------------------------------
> >   controller/ofctrl.h         |   64 +-
> >   controller/ovn-controller.c |   69 +-
> >   controller/physical.c       |  262 +++----
> >   controller/physical.h       |   17 +-
> >   10 files changed, 1697 insertions(+), 1597 deletions(-)
> >   create mode 100644 controller/dp-flow-mgr.c
> >   create mode 100644 controller/dp-flow-mgr.h
> >
> > diff --git a/controller/automake.mk b/controller/automake.mk
> > index e664f19800..28272a1cee 100644
> > --- a/controller/automake.mk
> > +++ b/controller/automake.mk
> > @@ -31,8 +31,9 @@ controller_ovn_controller_SOURCES = \
> >       controller/physical.c \
> >       controller/physical.h \
> >       controller/mac-learn.c \
> > -     controller/mac-learn.h
> > -
> > +     controller/mac-learn.h \
> > +     controller/dp-flow-mgr.c \
> > +     controller/dp-flow-mgr.h
> >   controller_ovn_controller_LDADD = lib/libovn.la $(OVS_LIBDIR)/libopenvswitch.la
> >   man_MANS += controller/ovn-controller.8
> >   EXTRA_DIST += controller/ovn-controller.8.xml
> > diff --git a/controller/dp-flow-mgr.c b/controller/dp-flow-mgr.c
> > new file mode 100644
> > index 0000000000..c83f371bc1
> > --- /dev/null
> > +++ b/controller/dp-flow-mgr.c
> > @@ -0,0 +1,1317 @@
> > +/* Copyright (c) 2021 Red Hat, Inc.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + *     http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#include <config.h>
> > +
> > +#include "dp-flow-mgr.h"
> > +#include "ofctrl.h"
> > +#include "lflow.h"
> > +
> > +/* ovs includes. */
> > +#include "lib/byte-order.h"
> > +#include "lib/uuid.h"
> > +#include "lib/hash.h"
> > +#include "lib/hmapx.h"
> > +#include "lib/ovs-atomic.h"
> > +#include "openvswitch/ofp-actions.h"
> > +#include "openvswitch/ofp-flow.h"
> > +#include "openvswitch/ofp-group.h"
> > +#include "openvswitch/ofp-match.h"
> > +#include "openvswitch/ofp-msgs.h"
> > +#include "openvswitch/ofp-meter.h"
> > +#include "openvswitch/ofp-packet.h"
> > +#include "openvswitch/ofp-print.h"
> > +#include "openvswitch/ofp-util.h"
> > +#include "openvswitch/ofpbuf.h"
> > +#include "openvswitch/vlog.h"
> > +
> > +VLOG_DEFINE_THIS_MODULE(dpflowmgr);
> > +
> > +static void dp_flow_table_destroy__(struct dp_flow_table *dp_table);
> > +
> > +static struct hmap dp_flow_tables = HMAP_INITIALIZER(&dp_flow_tables);
> > +
> > +#define CONJ_ACT_COOKIE UINT64_MAX
> > +
> > +/* Flow handling. */
> > +struct ovn_flow;
> > +
> > +/* An ovn_flow action. */
> > +struct ovn_flow_action {
> > +    struct ovs_list flow_uuid_action_node;
> > +
> > +    struct ofpact *ofpacts;
> > +    size_t ofpacts_len;
> > +    uint64_t cookie;
> > +    struct uuid flow_uuid;
> > +    bool stale;
> > +
> > +    struct ovs_list list_node;
> > +    struct ovn_flow *flow;
> > +};
> > +
> > +/* An OpenFlow flow. */
> > +struct ovn_flow {
> > +    struct hmap_node hmap_node;
> > +    struct hmap *match_flow_table; /* Points to the match flow table hmap for
> > +                                    * easy access. */
> > +    struct hmap *uuid_action_flow_table; /* Points to the uuid action flow
> > +                                          * table hmap for easy access. */
> > +    struct ovs_list flow_list_node; /* List node in dp_flow_table.lflows_list[]
> > +                                     * or dp_flow_table.pflows_list[]. */
> > +
> > +    /* Flow list in which this 'ovn_flow.flow_list_node' is present
> > +     * (for easy access). */
> > +    struct ovs_list *flow_list;
> > +
> > +    /* Key. */
> > +    uint8_t table_id;
> > +    uint16_t priority;
> > +    struct minimatch match;
> > +
> > +    /* Hash. */
> > +    uint32_t hash;
> > +
> > +    /* Actions associated with the flow.
> > +     * An ovn_flow can have
> > +     *   - A normal action - has precendence over all the other below actions
> > +     *                       if set.
> > +     *   - a list of allow actions - has precedence over drop and conjuctive
> > +     *                               actions.
> > +     *   - a list of drop actions  - has precedence over conjuctive actions.
> > +     *   - a list of conjuctive actions.
> > +     */
> > +    struct ovn_flow_action *normal_act;
> > +    struct ovs_list allow_act_list;
> > +    struct ovs_list drop_act_list;
> > +    struct ovs_list conj_act_list;
> > +
> > +    /* Presently installed ofpacts. */
> > +    struct ofpact *installed_ofpacts;
> > +    size_t installed_ofpacts_len;
> > +
> > +    /* Cookie of the ovn_action which is presently active. */
> > +    uint64_t active_cookie;
> > +};
> > +
> > +/* Represents a list of 'ovn flow actions' associated with
> > + * a flow uuid. */
> > +struct flow_uuid_to_acts {
> > +    struct hmap_node hmap_node; /* Node in
> > +                                 *  ovn_flow_table.uuid_flow_table. */
> > +    struct uuid flow_uuid;
> > +    struct ovs_list acts; /* A list of struct ovn_flow_action nodes that
> > +                           * are referenced by the sb_uuid. */
> > +};
> > +
> > +/* Represents a flow table. */
> > +struct ovn_flow_table {
> > +    /* Hash map of flow table using flow match conditions as hash key.*/
> > +    struct hmap match_flow_table;
> > +
> > +    /* Hash map of ovn_flow_action list table using uuid as hash key.*/
> > +    struct hmap uuid_action_flow_table;
> > +};
> > +
> > +/* Represents a datapath flow table.
> > + *
> > + * We maintain 2 lists for logical flows list. One represents an active flow
> > + * list and the other represents an old flow list.  When a flow is added to
> > + * the dp flow table, it is always added to the active flow list. If that flow
> > + * is present in the old flow list, it is removed from the old one.
> > + *
> > + * The function dp_flow_populate_oflow_msgs() clears all the
> > + * flows present in the old flow list.
> > + *
> > + * User can swap the lists by calling dp_flow_switch_logical_oflow_tables().
> > + *
> > + * The same goes for the physical flow lists.
> > + */
> > +struct dp_flow_table {
> > +    struct hmap_node hmap_node;
> > +    uint32_t dp_key;
> > +
> > +    struct ovn_flow_table lflow_table; /* For logical flows. */
> > +    struct ovn_flow_table pflow_table; /* For physical flows. */
> > +
> > +    struct ovs_list lflows_list[2];
> > +    struct ovs_list pflows_list[2];
> > +
> > +    struct ovs_list *active_lflows; /* Points to one of the element in
> > +                                     * lflows_list. */
> > +    struct ovs_list *old_lflows; /* Points to other element in
> > +                                  * lflows_list. */
> > +
> > +    struct ovs_list *active_pflows;
> > +    struct ovs_list *old_pflows;
> > +
> > +    struct hmapx modified_lflows;
> > +    struct hmapx modified_pflows;
> > +};
> > +
> > +/* Static functions. */
> > +static void dp_flow_switch_lflow_table__(struct dp_flow_table *);
> > +static void dp_flow_switch_pflow_table__(struct dp_flow_table *);
> > +static void dp_flow_table_destroy__(struct dp_flow_table *);
> > +
> > +static void dp_flow_add_openflow__(struct dp_flow_table *, bool lflow_table,
> > +                                   uint8_t table_id, uint16_t priority,
> > +                                   uint64_t cookie, const struct match *match,
> > +                                   const struct ofpbuf *actions,
> > +                                   const struct uuid *flow_uuid);
> > +static void dp_flows_remove(struct dp_flow_table *, bool lflow_table,
> > +                            const struct uuid *flow_uuid);
> > +
> > +static void ovn_flow_table_init(struct ovn_flow_table *);
> > +static void ovn_flow_table_clear(struct ovn_flow_table *);
> > +static void ovn_flow_table_destroy(struct ovn_flow_table *);
> > +
> > +static uint32_t ovn_flow_match_hash__(uint8_t table_id, uint16_t priority,
> > +                                      const struct minimatch *);
> > +static uint32_t ovn_flow_match_hash(const struct ovn_flow *);
> > +
> > +static void ovn_flow_init(struct ovn_flow *, uint8_t table_id,
> > +                          uint16_t priority, const struct match *);
> > +static void ovn_flow_uninit(struct ovn_flow *);
> > +static struct ovn_flow *ovn_flow_alloc(uint8_t table_id, uint16_t priority,
> > +                                       const struct match *);
> > +static void ovn_flow_destroy(struct ovn_flow *);
> > +
> > +static struct ovn_flow_action *ovn_flow_action_alloc(const struct ofpbuf *,
> > +                                                     uint64_t cookie,
> > +                                                     const struct uuid *);
> > +static void ovn_flow_action_init(struct ovn_flow_action *,
> > +                                 const struct ofpbuf *, uint64_t cookie,
> > +                                 const struct uuid *);
> > +static void ovn_flow_action_destroy(struct ovn_flow_action *);
> > +static struct ovn_flow *ovn_flow_lookup(struct ovn_flow_table *,
> > +                                        uint8_t table_id, uint16_t priority,
> > +                                        const struct minimatch *);
> > +static void ovn_flow_invalidate_all_actions(struct ovn_flow *f);
> > +static void ovn_flow_clear_actions_from_list(struct ovs_list *act_list);
> > +static void ovn_flow_invalidate_actions_from_list(struct ovs_list *act_list);
> > +static void ovn_flow_clear_actions(struct ovn_flow *);
> > +static bool ovn_flow_has_active_actions(struct ovn_flow *);
> > +
> > +static bool ovn_flow_action_is_normal(const struct ovn_flow_action *);
> > +static bool ovn_flow_action_is_allow(const struct ovn_flow_action *);
> > +static bool ovn_flow_action_is_drop(const struct ovn_flow_action *);
> > +static bool ovn_flow_action_is_conj(const struct ovn_flow_action *);
> > +
> > +static bool ovn_flow_has_action(struct ovn_flow *, struct ovn_flow_action *);
> > +static struct flow_uuid_to_acts *flow_uuid_to_acts_lookup(
> > +    struct hmap *uuid_action_table, const struct uuid *flow_uuid);
> > +static void ovn_flow_action_link_to_flow_uuid(
> > +    struct hmap *uuid_action_table, struct ovn_flow_action *);
> > +static void ovn_flow_unref_action__(struct ovn_flow *,
> > +                                    struct ovn_flow_action *);
> > +static struct ovn_flow_action * ovn_flow_action_get_matching_in_list(
> > +    struct ovs_list *act_list, struct ovn_flow_action *a);
> > +static void ovn_flow_unref_action(struct ovn_flow *, struct ovn_flow_action *);
> > +static struct ovn_flow_action *ovn_flow_get_matching_action(
> > +    struct ovn_flow *, struct ovn_flow_action *);
> > +
> > +static bool ovn_flow_update_actions(struct ovn_flow_table *,
> > +                                    struct ovn_flow *,
> > +                                    struct ovn_flow_action *);
> > +
> > +static void ovn_flow_prepare_ofmsg(struct ovn_flow *, struct ovs_list *msgs);
> > +static void dp_flow_populate_oflow_msgs__(struct dp_flow_table *,
> > +                                          struct ovs_list *msgs);
> > +
> > +static struct ofpbuf *encode_flow_mod(struct ofputil_flow_mod *);
> > +static void add_flow_mod(struct ofputil_flow_mod *, struct ovs_list *msgs);
> > +static void ovn_flow_add_oflow(struct ovn_flow *, struct ovn_flow_action *,
> > +                               struct ovs_list *msgs);
> > +static void ovn_flow_mod_oflow(struct ovn_flow *, struct ovn_flow_action *,
> > +                               struct ovs_list *msgs);
> > +static void ovn_flow_del_oflow(struct ovn_flow *, struct ovs_list *msgs);
> > +static void ovn_flow_log(const struct ovn_flow *);
> > +static char * ovn_flow_to_string(const struct ovn_flow *);
> > +
> > +void
> > +dp_flow_tables_init(void)
> > +{
> > +
> > +}
> > +
> > +void
> > +dp_flow_tables_destroy(void)
> > +{
> > +    struct dp_flow_table *dp_table;
> > +    HMAP_FOR_EACH_POP (dp_table, hmap_node, &dp_flow_tables) {
> > +        dp_flow_table_destroy__(dp_table);
> > +    }
> > +
> > +    hmap_destroy(&dp_flow_tables);
> > +}
> > +
> > +struct dp_flow_table *
> > +dp_flow_table_get(uint32_t dp_key)
> > +{
> > +    struct dp_flow_table *dp_table;
> > +    HMAP_FOR_EACH_WITH_HASH (dp_table, hmap_node, dp_key, &dp_flow_tables) {
> > +        if (dp_table->dp_key == dp_key) {
> > +            return dp_table;
> > +        }
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> > +struct dp_flow_table *
> > +dp_flow_table_alloc(uint32_t dp_key)
> > +{
> > +    struct dp_flow_table *dp_table = xzalloc(sizeof *dp_table);
> > +    ovn_flow_table_init(&dp_table->lflow_table);
> > +    ovn_flow_table_init(&dp_table->pflow_table);
> > +
> > +    ovs_list_init(&dp_table->lflows_list[0]);
> > +    ovs_list_init(&dp_table->lflows_list[1]);
> > +    ovs_list_init(&dp_table->pflows_list[0]);
> > +    ovs_list_init(&dp_table->pflows_list[1]);
> > +
> > +    hmapx_init(&dp_table->modified_lflows);
> > +    hmapx_init(&dp_table->modified_pflows);
> > +
> > +    dp_table->active_lflows = &dp_table->lflows_list[0];
> > +    dp_table->old_lflows = &dp_table->lflows_list[1];
> > +    dp_table->active_pflows = &dp_table->pflows_list[0];
> > +    dp_table->old_pflows = &dp_table->pflows_list[1];
> > +
> > +    dp_table->dp_key = dp_key;
> > +    hmap_insert(&dp_flow_tables, &dp_table->hmap_node, dp_key);
> > +    return dp_table;
> > +}
> > +
> > +void dp_flow_table_destroy(uint32_t dp_key)
> > +{
> > +    struct dp_flow_table *dp_table = dp_flow_table_get(dp_key);
> > +    if (dp_table) {
> > +        hmap_remove(&dp_flow_tables, &dp_table->hmap_node);
> > +        dp_flow_table_destroy__(dp_table);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_switch_logical_oflow_tables(void)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    HMAP_FOR_EACH (dp_ftable, hmap_node, &dp_flow_tables) {
> > +        dp_flow_switch_lflow_table__(dp_ftable);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_switch_logical_oflow_table(uint32_t dp_key)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    dp_ftable = dp_flow_table_get(dp_key);
> > +    if (dp_ftable) {
> > +        dp_flow_switch_lflow_table__(dp_ftable);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_switch_physical_oflow_tables(void)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    HMAP_FOR_EACH (dp_ftable, hmap_node, &dp_flow_tables) {
> > +        dp_flow_switch_pflow_table__(dp_ftable);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_switch_physical_oflow_table(uint32_t dp_key)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    dp_ftable = dp_flow_table_get(dp_key);
> > +    if (dp_ftable) {
> > +        dp_flow_switch_pflow_table__(dp_ftable);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_add_logical_oflow(uint32_t dp_key, uint8_t table_id,
> > +                          uint16_t priority, uint64_t cookie,
> > +                          const struct match *match,
> > +                          const struct ofpbuf *actions,
> > +                          const struct uuid *flow_uuid)
> > +{
> > +    struct dp_flow_table *dp_ftable = dp_flow_table_get(dp_key);
> > +    if (!dp_ftable) {
> > +        dp_ftable = dp_flow_table_alloc(dp_key);
> > +    }
> > +
> > +    dp_flow_add_openflow__(dp_ftable, true, table_id, priority, cookie, match,
> > +                           actions, flow_uuid);
> > +}
> > +
> > +void
> > +dp_flow_add_physical_oflow(uint32_t dp_key, uint8_t table_id,
> > +                           uint16_t priority, uint64_t cookie,
> > +                           const struct match *match,
> > +                           const struct ofpbuf *actions,
> > +                           const struct uuid *flow_uuid)
> > +{
> > +    struct dp_flow_table *dp_ftable = dp_flow_table_get(dp_key);
> > +    if (!dp_ftable) {
> > +        dp_ftable = dp_flow_table_alloc(dp_key);
> > +    }
> > +
> > +    dp_flow_add_openflow__(dp_ftable, false, table_id, priority, cookie, match,
> > +                           actions, flow_uuid);
> > +}
> > +
> > +void
> > +dp_flow_remove_logical_oflows_all(const struct uuid *flow_uuid)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    HMAP_FOR_EACH (dp_ftable, hmap_node, &dp_flow_tables) {
> > +        dp_flows_remove(dp_ftable, true, flow_uuid);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_remove_logical_oflows(uint32_t dp_key, const struct uuid *flow_uuid)
> > +{
> > +    struct dp_flow_table *dp_ftable = dp_flow_table_get(dp_key);
> > +    if (dp_ftable) {
> > +        dp_flows_remove(dp_ftable, true, flow_uuid);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_remove_physical_oflows(uint32_t dp_key, const struct uuid *flow_uuid)
> > +{
> > +    struct dp_flow_table *dp_ftable = dp_flow_table_get(dp_key);
> > +    if (dp_ftable) {
> > +        dp_flows_remove(dp_ftable, false, flow_uuid);
> > +    }
> > +}
> > +
> > +void
> > +dp_flow_flush_all_oflows(void)
> > +{
> > +    struct dp_flow_table *dp_table;
> > +    HMAP_FOR_EACH_POP (dp_table, hmap_node, &dp_flow_tables) {
> > +        dp_flow_table_destroy__(dp_table);
> > +    }
> > +
> > +    hmap_destroy(&dp_flow_tables);
> > +    hmap_init(&dp_flow_tables);
> > +}
> > +
> > +void
> > +dp_flow_populate_oflow_msgs(struct ovs_list *msgs)
> > +{
> > +    struct dp_flow_table *dp_ftable;
> > +    HMAP_FOR_EACH (dp_ftable, hmap_node, &dp_flow_tables) {
> > +        dp_flow_populate_oflow_msgs__(dp_ftable, msgs);
> > +    }
> > +}
> > +
> > +/* Static functions. */
> > +
> > +static void
> > +dp_flow_switch_lflow_table__(struct dp_flow_table *dp_ftable)
> > +{
> > +    struct ovs_list *t = dp_ftable->active_lflows;
> > +    dp_ftable->active_lflows = dp_ftable->old_lflows;
> > +    dp_ftable->old_lflows = t;
> > +
> > +    hmapx_clear(&dp_ftable->modified_lflows);
> > +}
> > +
> > +static void
> > +dp_flow_switch_pflow_table__(struct dp_flow_table *dp_ftable)
> > +{
> > +    struct ovs_list *t = dp_ftable->active_pflows;
> > +    dp_ftable->active_pflows = dp_ftable->old_pflows;
> > +    dp_ftable->old_pflows = t;
> > +
> > +    hmapx_clear(&dp_ftable->modified_pflows);
> > +}
> > +
> > +static void
> > +dp_flow_table_destroy__(struct dp_flow_table *dp_ftable)
> > +{
> > +    hmapx_clear(&dp_ftable->modified_lflows);
> > +    hmapx_destroy(&dp_ftable->modified_lflows);
> > +    hmapx_clear(&dp_ftable->modified_pflows);
> > +    hmapx_destroy(&dp_ftable->modified_pflows);
> > +
> > +    ovn_flow_table_destroy(&dp_ftable->lflow_table);
> > +    ovn_flow_table_destroy(&dp_ftable->pflow_table);
> > +
> > +    free(dp_ftable);
> > +}
> > +
> > +/* This function
> > + *   - Looks up in the datapath flow table if the flow F
> > + *     with the provided (match, table_id, priority) is already
> > + *     present or not.
> > + *
> > + *   - If not present it creates an ovn_flow 'F' with the provided
> > + *     (match, table_id, priority) and inserts into the flow table.
> > + *
> > + *   - Allocates ovn_flow_action 'A' with the given -
> > + *     (actions, cookie, flow_uuid).
> > + *
> > + *   - If 'F' already has 'A' it does nothing and returns. Otherwise
> > + *     it associates 'A' to the 'F' actions.
> > + *
> > + *   - Adds 'F' to the active flow list.
> > + */
> > +static void
> > +dp_flow_add_openflow__(struct dp_flow_table *dp_ftable, bool lflow_table,
> > +                       uint8_t table_id, uint16_t priority,
> > +                       uint64_t cookie, const struct match *match,
> > +                       const struct ofpbuf *actions,
> > +                       const struct uuid *flow_uuid)
> > +{
> > +    struct ovn_flow_table *ftable;
> > +    struct hmapx *modified_flows;
> > +    struct ovs_list *active_flows;
> > +
> > +    if (lflow_table) {
> > +        ftable = &dp_ftable->lflow_table;
> > +        modified_flows = &dp_ftable->modified_lflows;
> > +        active_flows = dp_ftable->active_lflows;
> > +    } else {
> > +        ftable = &dp_ftable->pflow_table;
> > +        modified_flows = &dp_ftable->modified_pflows;
> > +        active_flows = dp_ftable->active_pflows;
> > +    }
> > +
> > +    struct minimatch minimatch;
> > +    minimatch_init(&minimatch, match);
> > +
> > +    struct ovn_flow *f = ovn_flow_lookup(ftable, table_id, priority,
> > +                                         &minimatch);
> > +
> > +    minimatch_destroy(&minimatch);
> > +
> > +    bool active_flow_list_changed = false;
> > +    bool flow_exists = true;
> > +
> > +    if (!f) {
> > +        f = ovn_flow_alloc(table_id, priority, match);
> > +        f->match_flow_table = &ftable->match_flow_table;
> > +        f->uuid_action_flow_table = &ftable->uuid_action_flow_table;
> > +        ovs_list_init(&f->flow_list_node);
> > +        flow_exists = false;
> > +        hmap_insert(&ftable->match_flow_table, &f->hmap_node,
> > +                    f->hash);
> > +    } else {
> > +        ovs_list_remove(&f->flow_list_node);
> > +        active_flow_list_changed = (active_flows != f->flow_list);
> > +    }
> > +
> > +    f->flow_list = active_flows;
> > +    ovs_list_push_back(active_flows, &f->flow_list_node);
> > +
> > +    struct ovn_flow_action *a = ovn_flow_action_alloc(actions, cookie,
> > +                                                      flow_uuid);
> > +
> > +    if (active_flow_list_changed) {
> > +        ovn_flow_invalidate_all_actions(f);
> > +    }
> > +
> > +    bool push_flow_to_switch = true;
> > +    if (flow_exists && ovn_flow_has_action(f, a)) {
> > +        struct ovn_flow_action *existing_a =
> > +            ovn_flow_get_matching_action(f, a);
> > +        if (uuid_equals(&existing_a->flow_uuid, flow_uuid)) {
> > +            existing_a->stale = false;
> > +        }
> > +        if (!active_flow_list_changed) {
> > +            /* The flow-action pair already exists. Nothing to be done. */
> > +            push_flow_to_switch = false;
> > +        }
> > +        ovn_flow_action_destroy(a);
> > +    } else {
> > +        ovn_flow_update_actions(ftable, f, a);
> > +    }
> > +
> > +    ovn_flow_log(f);
> > +    if (push_flow_to_switch) {
> > +        hmapx_add(modified_flows, (void *) f);
> > +    }
> > +}
> > +
> > +static void
> > +dp_flows_remove(struct dp_flow_table *dp_ftable, bool lflow_table,
> > +                const struct uuid *flow_uuid)
> > +{
> > +    struct ovn_flow_table *flow_table;
> > +    struct hmapx *modified_flows;
> > +
> > +    if (lflow_table) {
> > +        flow_table = &dp_ftable->lflow_table;
> > +        modified_flows = &dp_ftable->modified_lflows;
> > +    } else {
> > +        flow_table = &dp_ftable->pflow_table;
> > +        modified_flows = &dp_ftable->modified_pflows;
> > +    }
> > +
> > +    struct flow_uuid_to_acts *f_uuid_to_acts =
> > +        flow_uuid_to_acts_lookup(&flow_table->uuid_action_flow_table,
> > +                                 flow_uuid);
> > +
> > +    if (!f_uuid_to_acts) {
> > +        return;
> > +    }
> > +
> > +    struct ovn_flow_action *a;
> > +    LIST_FOR_EACH_POP (a, flow_uuid_action_node, &f_uuid_to_acts->acts) {
> > +        struct ovn_flow *f = a->flow;
> > +        ovn_flow_unref_action__(a->flow, a);
> > +        ovn_flow_action_destroy(a);
> > +        hmapx_add(modified_flows, (void *) f);
> > +    }
> > +
> > +    hmap_remove(&flow_table->uuid_action_flow_table,
> > +                &f_uuid_to_acts->hmap_node);
> > +}
> > +
> > +/* ovn flow table functions. */
> > +static void
> > +ovn_flow_table_init(struct ovn_flow_table *flow_table)
> > +{
> > +    hmap_init(&flow_table->match_flow_table);
> > +    hmap_init(&flow_table->uuid_action_flow_table);
> > +}
> > +
> > +static void
> > +ovn_flow_table_clear(struct ovn_flow_table *flow_table)
> > +{
> > +    struct flow_uuid_to_acts *f_uuid_to_acts;
> > +    HMAP_FOR_EACH_POP (f_uuid_to_acts, hmap_node,
> > +                       &flow_table->uuid_action_flow_table) {
> > +        struct ovn_flow_action *a;
> > +        LIST_FOR_EACH_POP (a, flow_uuid_action_node,
> > +                           &f_uuid_to_acts->acts) {
> > +            ovn_flow_unref_action__(a->flow, a);
> > +            ovn_flow_action_destroy(a);
> > +        }
> > +
> > +        free(f_uuid_to_acts);
> > +    }
> > +
> > +    struct ovn_flow *f;
> > +    HMAP_FOR_EACH_POP (f, hmap_node, &flow_table->match_flow_table) {
> > +            ovn_flow_destroy(f);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_table_destroy(struct ovn_flow_table *flow_table)
> > +{
> > +    ovn_flow_table_clear(flow_table);
> > +    hmap_destroy(&flow_table->match_flow_table);
> > +    hmap_destroy(&flow_table->uuid_action_flow_table);
> > +}
> > +
> > +static uint32_t
> > +ovn_flow_match_hash__(uint8_t table_id, uint16_t priority,
> > +                       const struct minimatch *match)
> > +{
> > +    return hash_2words((table_id << 16) | priority,
> > +                       minimatch_hash(match, 0));
> > +}
> > +
> > +/* Returns a hash of the match key in 'f'. */
> > +static uint32_t
> > +ovn_flow_match_hash(const struct ovn_flow *f)
> > +{
> > +    return ovn_flow_match_hash__(f->table_id, f->priority, &f->match);
> > +}
> > +
> > +static void
> > +ovn_flow_action_init(struct ovn_flow_action *a, const struct ofpbuf *actions,
> > +                     uint64_t cookie, const struct uuid *flow_uuid)
> > +{
> > +    a->ofpacts = xmemdup(actions->data, actions->size);
> > +    a->ofpacts_len = actions->size;
> > +    a->cookie = cookie;
> > +    a->flow_uuid = *flow_uuid;
> > +    a->stale = false;
> > +    ovs_list_init(&a->list_node);
> > +}
> > +
> > +static void
> > +ovn_flow_action_destroy(struct ovn_flow_action *a)
> > +{
> > +    free(a->ofpacts);
> > +    free(a);
> > +}
> > +
> > +static void
> > +ovn_flow_init(struct ovn_flow *f, uint8_t table_id, uint16_t priority,
> > +              const struct match *match)
> > +{
> > +    f->table_id = table_id;
> > +    f->priority = priority;
> > +    minimatch_init(&f->match, match);
> > +
> > +    f->hash = ovn_flow_match_hash(f);
> > +
> > +    ovs_list_init(&f->allow_act_list);
> > +    ovs_list_init(&f->drop_act_list);
> > +    ovs_list_init(&f->conj_act_list);
> > +
> > +    f->installed_ofpacts = NULL;
> > +    f->installed_ofpacts_len = 0;
> > +
> > +    f->active_cookie = 0;
> > +}
> > +
> > +static struct ovn_flow *
> > +ovn_flow_alloc(uint8_t table_id, uint16_t priority,
> > +               const struct match *match)
> > +{
> > +    struct ovn_flow *f = xzalloc(sizeof *f);
> > +    ovn_flow_init(f, table_id, priority, match);
> > +
> > +    return f;
> > +}
> > +
> > +static struct ovn_flow_action *
> > +ovn_flow_action_alloc(const struct ofpbuf *actions,
> > +                      uint64_t cookie, const struct uuid *flow_uuid)
> > +{
> > +    struct ovn_flow_action *a = xzalloc(sizeof *a);
> > +    ovn_flow_action_init(a, actions, cookie, flow_uuid);
> > +
> > +    return a;
> > +}
> > +
> > +/* Finds and returns a ovn_flow in 'flow_table' whose key is identical to
> > + * 'target''s key, or NULL if there is none.
> > + */
> > +static struct ovn_flow *
> > +ovn_flow_lookup(struct ovn_flow_table *flow_table,
> > +                uint8_t table_id, uint16_t priority,
> > +                const struct minimatch *match)
> > +{
> > +    size_t hash = ovn_flow_match_hash__(table_id, priority, match);
> > +
> > +    struct ovn_flow *f;
> > +    HMAP_FOR_EACH_WITH_HASH (f, hmap_node, hash,
> > +                             &flow_table->match_flow_table) {
> > +        if (f->priority == priority
> > +            && minimatch_equal(&f->match, match)) {
> > +
> > +            return f;
> > +        }
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> > +static void
> > +ovn_flow_clear_actions_from_list(struct ovs_list *act_list)
> > +{
> > +    struct ovn_flow_action *a;
> > +    LIST_FOR_EACH_POP (a, list_node, act_list) {
> > +        ovs_list_remove(&a->flow_uuid_action_node);
> > +        ovn_flow_action_destroy(a);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_invalidate_actions_from_list(struct ovs_list *act_list)
> > +{
> > +    struct ovn_flow_action *a;
> > +    LIST_FOR_EACH (a, list_node, act_list) {
> > +        a->stale = true;
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_clear_actions(struct ovn_flow *f)
> > +{
> > +    if (f->normal_act) {
> > +        ovs_list_remove(&f->normal_act->flow_uuid_action_node);
> > +        ovn_flow_action_destroy(f->normal_act);
> > +        f->normal_act = NULL;
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        ovn_flow_clear_actions_from_list(&f->allow_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        ovn_flow_clear_actions_from_list(&f->drop_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->conj_act_list)) {
> > +        ovn_flow_clear_actions_from_list(&f->conj_act_list);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_invalidate_all_actions(struct ovn_flow *f)
> > +{
> > +    if (f->normal_act) {
> > +        f->normal_act->stale = true;
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        ovn_flow_invalidate_actions_from_list(&f->allow_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        ovn_flow_invalidate_actions_from_list(&f->drop_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->conj_act_list)) {
> > +        ovn_flow_invalidate_actions_from_list(&f->conj_act_list);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_delete_stale_actions_from_list(struct ovs_list *act_list)
> > +{
> > +    struct ovn_flow_action *a, *next;
> > +    LIST_FOR_EACH_SAFE (a, next, list_node, act_list) {
> > +        if (a->stale) {
> > +            ovn_flow_unref_action(a->flow, a);
> > +        }
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_delete_stale_actions(struct ovn_flow *f)
> > +{
> > +    if (f->normal_act && f->normal_act->stale) {
> > +        ovn_flow_unref_action(f, f->normal_act);
> > +        f->normal_act = NULL;
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        ovn_flow_delete_stale_actions_from_list(&f->allow_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        ovn_flow_delete_stale_actions_from_list(&f->drop_act_list);
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->conj_act_list)) {
> > +        ovn_flow_delete_stale_actions_from_list(&f->conj_act_list);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_uninit(struct ovn_flow *f)
> > +{
> > +    minimatch_destroy(&f->match);
> > +    ovn_flow_clear_actions(f);
> > +}
> > +
> > +static void
> > +ovn_flow_destroy(struct ovn_flow *f)
> > +{
> > +    if (f) {
> > +        ovn_flow_uninit(f);
> > +        free(f->installed_ofpacts);
> > +        free(f);
> > +    }
> > +}
> > +
> > +static bool
> > +ovn_flow_action_is_drop(const struct ovn_flow_action *f)
> > +{
> > +    return f->ofpacts_len == 0;
> > +}
> > +
> > +static bool
> > +ovn_flow_action_is_conj(const struct ovn_flow_action *f)
> > +{
> > +    const struct ofpact *a = NULL;
> > +
> > +    OFPACT_FOR_EACH (a, f->ofpacts, f->ofpacts_len) {
> > +        if (a->type == OFPACT_CONJUNCTION) {
> > +            return true;
> > +        }
> > +    }
> > +    return false;
> > +}
> > +
> > +static bool
> > +ovn_flow_action_is_allow(const struct ovn_flow_action *f)
> > +{
> > +    if (ovn_flow_action_is_drop(f)) {
> > +        return false;
> > +    }
> > +
> > +    const struct ofpact *a = f->ofpacts;
> > +    return (a->type == OFPACT_RESUBMIT &&
> > +            ofpact_last(a, f->ofpacts, f->ofpacts_len));
> > +}
> > +
> > +static bool
> > +ovn_flow_action_is_normal(const struct ovn_flow_action *f)
> > +{
> > +    return (!ovn_flow_action_is_allow(f) && !ovn_flow_action_is_drop(f) &&
> > +            !ovn_flow_action_is_conj(f));
> > +}
> > +
> > +static struct ovn_flow_action *
> > +ovn_flow_action_get_matching_in_list(struct ovs_list *act_list,
> > +                                     struct ovn_flow_action *a)
> > +{
> > +    struct ovn_flow_action *act_in_list;
> > +    LIST_FOR_EACH (act_in_list, list_node, act_list) {
> > +        if (ofpacts_equal(act_in_list->ofpacts,
> > +                          act_in_list->ofpacts_len,
> > +                          a->ofpacts,
> > +                          a->ofpacts_len)) {
> > +            return act_in_list;
> > +        }
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> > +static struct ovn_flow_action *
> > +ovn_flow_get_matching_action(struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    if (f->normal_act && ovn_flow_action_is_normal(a)) {
> > +        if (ofpacts_equal(f->normal_act->ofpacts,
> > +                         f->normal_act->ofpacts_len,
> > +                         a->ofpacts,
> > +                         a->ofpacts_len)) {
> > +            return f->normal_act;
> > +        }
> > +    }
> > +
> > +    if (ovn_flow_action_is_allow(a)) {
> > +        return ovn_flow_action_get_matching_in_list(&f->allow_act_list, a);
> > +    }
> > +
> > +    if (ovn_flow_action_is_drop(a)) {
> > +        return ovn_flow_action_get_matching_in_list(&f->drop_act_list, a);
> > +    }
> > +
> > +    if (ovn_flow_action_is_conj(a)) {
> > +        return ovn_flow_action_get_matching_in_list(&f->conj_act_list, a);
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> > +static bool
> > +ovn_flow_has_action(struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    struct ovn_flow_action *ma = ovn_flow_get_matching_action(f, a);
> > +
> > +    if (ovn_flow_action_is_normal(a)) {
> > +        return ma ? true: false;
> > +    }
> > +
> > +    return ma ? uuid_equals(&ma->flow_uuid, &a->flow_uuid) : false;
> > +}
> > +
> > +static bool
> > +ovn_flow_has_active_actions(struct ovn_flow *f)
> > +{
> > +    if (f->normal_act) {
> > +        return true;
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        return true;
> > +    }
> > +
> > +    if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        return true;
> > +    }
> > +
> > +    return !ovs_list_is_empty(&f->conj_act_list);
> > +}
> > +
> > +static struct flow_uuid_to_acts *
> > +flow_uuid_to_acts_lookup(struct hmap *uuid_action_table,
> > +                         const struct uuid *flow_uuid)
> > +{
> > +    struct flow_uuid_to_acts *f_uuid_to_acts;
> > +    HMAP_FOR_EACH_WITH_HASH (f_uuid_to_acts, hmap_node, uuid_hash(flow_uuid),
> > +                             uuid_action_table) {
> > +        if (uuid_equals(flow_uuid, &f_uuid_to_acts->flow_uuid)) {
> > +            return f_uuid_to_acts;
> > +        }
> > +    }
> > +    return NULL;
> > +}
> > +
> > +static void
> > +ovn_flow_action_link_to_flow_uuid(struct hmap *uuid_action_table,
> > +                                  struct ovn_flow_action *a)
> > +{
> > +    struct flow_uuid_to_acts *f_uuid_to_acts;
> > +    f_uuid_to_acts = flow_uuid_to_acts_lookup(uuid_action_table, &a->flow_uuid);
> > +    if (!f_uuid_to_acts) {
> > +        f_uuid_to_acts = xzalloc(sizeof *f_uuid_to_acts);
> > +        f_uuid_to_acts->flow_uuid = a->flow_uuid;
> > +        ovs_list_init(&f_uuid_to_acts->acts);
> > +        hmap_insert(uuid_action_table, &f_uuid_to_acts->hmap_node,
> > +                    uuid_hash(&a->flow_uuid));
> > +    }
> > +
> > +    ovs_list_push_back(&f_uuid_to_acts->acts, &a->flow_uuid_action_node);
> > +}
> > +
> > +static void
> > +ovn_flow_unref_action__(struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    if (f->normal_act == a) {
> > +        f->normal_act = NULL;
> > +    } else if (!ovs_list_is_empty(&a->list_node)) {
> > +        ovs_list_remove(&a->list_node);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_unref_action(struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    ovs_list_remove(&a->flow_uuid_action_node);
> > +    ovn_flow_unref_action__(f, a);
> > +    ovn_flow_action_destroy(a);
> > +}
> > +
> > +static bool
> > +ovn_flow_update_actions(struct ovn_flow_table *ftable,
> > +                        struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    bool flow_updated = true;
> > +    if (ovn_flow_action_is_normal(a)) {
> > +        if (f->normal_act) {
> > +            ovn_flow_unref_action(f, f->normal_act);
> > +        }
> > +        f->normal_act = a;
> > +    } else if (ovn_flow_action_is_allow(a)) {
> > +        flow_updated = ovs_list_is_empty(&f->allow_act_list);
> > +        ovs_list_push_back(&f->allow_act_list, &a->list_node);
> > +    } else if (ovn_flow_action_is_drop(a)) {
> > +        flow_updated = ovs_list_is_empty(&f->drop_act_list);
> > +        ovs_list_push_back(&f->drop_act_list, &a->list_node);
> > +    } else {
> > +        /* conj action. */
> > +        ovs_list_push_back(&f->conj_act_list, &a->list_node);
> > +    }
> > +
> > +    a->flow = f;
> > +    a->stale = false;
> > +
> > +    ovn_flow_action_link_to_flow_uuid(&ftable->uuid_action_flow_table, a);
> > +
> > +    return flow_updated;
> > +}
> > +
> > +static bool
> > +ovn_flow_needs_flow_mod(struct ovn_flow *f, struct ovn_flow_action *a)
> > +{
> > +    if (f->installed_ofpacts_len != a->ofpacts_len) {
> > +        return true;
> > +    }
> > +
> > +    if (f->installed_ofpacts_len && a->ofpacts_len) {
> > +        return !ofpacts_equal(f->installed_ofpacts,
> > +                              f->installed_ofpacts_len,
> > +                              a->ofpacts,
> > +                              a->ofpacts_len);
> > +    }
> > +
> > +    return f->active_cookie ? false : true;
> > +}
> > +
> > +static void
> > +ovn_flow_prepare_ofmsg(struct ovn_flow *f, struct ovs_list *msgs)
> > +{
> > +    struct ovn_flow_action *a = NULL;
> > +    struct ovn_flow_action *conj_act = NULL;
> > +
> > +    if (f->normal_act) {
> > +        a = f->normal_act;
> > +    } else if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        a = CONTAINER_OF(ovs_list_front(&f->allow_act_list),
> > +                         struct ovn_flow_action,
> > +                         list_node);
> > +    } else if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        a = CONTAINER_OF(ovs_list_front(&f->drop_act_list),
> > +                         struct ovn_flow_action,
> > +                         list_node);
> > +    } else if (!ovs_list_is_empty(&f->conj_act_list)) {
> > +        struct ovn_flow_action *c;
> > +        uint64_t ofpacts_stub[1024 / 8];
> > +        struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> > +
> > +        LIST_FOR_EACH (c, list_node, &f->conj_act_list) {
> > +            ofpbuf_put(&ofpacts, c->ofpacts, c->ofpacts_len);
> > +        }
> > +        conj_act = xzalloc(sizeof *conj_act);
> > +        conj_act->cookie = CONJ_ACT_COOKIE;
> > +        conj_act->ofpacts = xmemdup(ofpacts.data, ofpacts.size);
> > +        conj_act->ofpacts_len = ofpacts.size;
> > +        ofpbuf_uninit(&ofpacts);
> > +        a = conj_act;
> > +    }
> > +
> > +    if (a) {
> > +        /* check if there is really a need to install or not. */
> > +        if (ovn_flow_needs_flow_mod(f, a)) {
> > +            if (f->active_cookie == 0) {
> > +                ovn_flow_add_oflow(f, a, msgs);
> > +            } else {
> > +                ovn_flow_mod_oflow(f, a, msgs);
> > +            }
> > +            free(f->installed_ofpacts);
> > +            if (a->ofpacts_len) {
> > +                f->installed_ofpacts = xmemdup(a->ofpacts, a->ofpacts_len);
> > +                f->installed_ofpacts_len = a->ofpacts_len;
> > +            } else {
> > +                f->installed_ofpacts = NULL;
> > +                f->installed_ofpacts_len = 0;
> > +            }
> > +        }
> > +    } else {
> > +        ovn_flow_del_oflow(f, msgs);
> > +        free(f->installed_ofpacts);
> > +        f->installed_ofpacts = NULL;
> > +        f->installed_ofpacts_len = 0;
> > +        f->active_cookie = 0;
> > +    }
> > +
> > +    if (conj_act) {
> > +        free(conj_act->ofpacts);
> > +        free(conj_act);
> > +    }
> > +}
> > +
> > +static void
> > +ovn_flow_handle_modified(struct ovn_flow *f, struct ovs_list *msgs)
> > +{
> > +    ovn_flow_delete_stale_actions(f);
> > +    ovn_flow_prepare_ofmsg(f, msgs);
> > +    if (!ovn_flow_has_active_actions(f)) {
> > +        hmap_remove(f->match_flow_table, &f->hmap_node);
> > +        ovs_list_remove(&f->flow_list_node);
> > +        ovn_flow_destroy(f);
> > +    }
> > +}
> > +
> > +static void
> > +dp_flow_populate_oflow_msgs__(struct dp_flow_table *dp_ftable,
> > +                              struct ovs_list *msgs)
> > +{
> > +    struct ovn_flow *f;
> > +
> > +    LIST_FOR_EACH_POP (f, flow_list_node, dp_ftable->old_lflows) {
> > +        ovn_flow_clear_actions(f);
> > +        ovn_flow_prepare_ofmsg(f, msgs);
> > +        hmap_remove(f->match_flow_table, &f->hmap_node);
> > +        hmapx_find_and_delete(&dp_ftable->modified_lflows, f);
> > +        ovn_flow_destroy(f);
> > +    }
> > +
> > +    LIST_FOR_EACH_POP (f, flow_list_node, dp_ftable->old_pflows) {
> > +        ovn_flow_clear_actions(f);
> > +        ovn_flow_prepare_ofmsg(f, msgs);
> > +        hmap_remove(f->match_flow_table, &f->hmap_node);
> > +        hmapx_find_and_delete(&dp_ftable->modified_pflows, f);
> > +        ovn_flow_destroy(f);
> > +    }
> > +
> > +    struct hmapx_node *node;
> > +    HMAPX_FOR_EACH (node, &dp_ftable->modified_lflows) {
> > +        f = node->data;
> > +        ovn_flow_handle_modified(f, msgs);
> > +    }
> > +
> > +    hmapx_clear(&dp_ftable->modified_lflows);
> > +
> > +    HMAPX_FOR_EACH (node, &dp_ftable->modified_pflows) {
> > +        f = node->data;
> > +        ovn_flow_handle_modified(f, msgs);
> > +    }
> > +
> > +    hmapx_clear(&dp_ftable->modified_pflows);
> > +}
> > +
> > +/* Flow table update. */
> > +
> > +static struct ofpbuf *
> > +encode_flow_mod(struct ofputil_flow_mod *fm)
> > +{
> > +    fm->buffer_id = UINT32_MAX;
> > +    fm->out_port = OFPP_ANY;
> > +    fm->out_group = OFPG_ANY;
> > +    return ofputil_encode_flow_mod(fm, OFPUTIL_P_OF15_OXM);
> > +}
> > +
> > +static void
> > +add_flow_mod(struct ofputil_flow_mod *fm, struct ovs_list *msgs)
> > +{
> > +    struct ofpbuf *msg = encode_flow_mod(fm);
> > +    ovs_list_push_back(msgs, &msg->list_node);
> > +}
> > +
> > +static void
> > +ovn_flow_add_oflow(struct ovn_flow *f, struct ovn_flow_action *a,
> > +                   struct ovs_list *msgs)
> > +{
> > +    /* Send flow_mod to add flow. */
> > +    struct ofputil_flow_mod fm = {
> > +        .match = f->match,
> > +        .priority = f->priority,
> > +        .table_id = f->table_id,
> > +        .ofpacts = a->ofpacts,
> > +        .ofpacts_len = a->ofpacts_len,
> > +        .new_cookie = htonll(a->cookie),
> > +        .command = OFPFC_ADD,
> > +    };
> > +    add_flow_mod(&fm, msgs);
> > +    f->active_cookie = a->cookie;
> > +}
> > +
> > +static void
> > +ovn_flow_mod_oflow(struct ovn_flow *f, struct ovn_flow_action *a,
> > +                   struct ovs_list *msgs)
> > +{
> > +    /* Update actions in installed flow. */
> > +    struct ofputil_flow_mod fm = {
> > +        .match = f->match,
> > +        .priority = f->priority,
> > +        .table_id = f->table_id,
> > +        .ofpacts = a->ofpacts,
> > +        .ofpacts_len = a->ofpacts_len,
> > +        .command = OFPFC_MODIFY_STRICT,
> > +    };
> > +    /* Update cookie if it is changed. */
> > +    if (f->active_cookie != a->cookie) {
> > +        fm.modify_cookie = true;
> > +        fm.new_cookie = htonll(a->cookie);
> > +        /* Use OFPFC_ADD so that cookie can be updated. */
> > +        fm.command = OFPFC_ADD;
> > +    }
> > +    add_flow_mod(&fm, msgs);
> > +    f->active_cookie = a->cookie;
> > +}
> > +
> > +static void
> > +ovn_flow_del_oflow(struct ovn_flow *f, struct ovs_list *msgs)
> > +{
> > +    struct ofputil_flow_mod fm = {
> > +        .match = f->match,
> > +        .priority = f->priority,
> > +        .table_id = f->table_id,
> > +        .command = OFPFC_DELETE_STRICT,
> > +    };
> > +    add_flow_mod(&fm, msgs);
> > +    f->active_cookie = 0;
> > +}
> > +
> > +static char *
> > +ovn_flow_to_string(const struct ovn_flow *f)
> > +{
> > +    struct ds s = DS_EMPTY_INITIALIZER;
> > +
> > +    struct ovn_flow_action *a = NULL;
> > +    struct ovn_flow_action *conj_act = NULL;
> > +
> > +    if (f->normal_act) {
> > +        a = f->normal_act;
> > +    } else if (!ovs_list_is_empty(&f->allow_act_list)) {
> > +        a = CONTAINER_OF(ovs_list_front(&f->allow_act_list),
> > +                         struct ovn_flow_action,
> > +                         list_node);
> > +    } else if (!ovs_list_is_empty(&f->drop_act_list)) {
> > +        a = CONTAINER_OF(ovs_list_front(&f->drop_act_list),
> > +                         struct ovn_flow_action,
> > +                         list_node);
> > +    } else if (!ovs_list_is_empty(&f->conj_act_list)) {
> > +        struct ovn_flow_action *c;
> > +        uint64_t ofpacts_stub[1024 / 8];
> > +        struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub);
> > +
> > +        LIST_FOR_EACH (c, list_node, &f->conj_act_list) {
> > +            ofpbuf_put(&ofpacts, c->ofpacts, c->ofpacts_len);
> > +        }
> > +        conj_act = xzalloc(sizeof *conj_act);
> > +        conj_act->cookie = CONJ_ACT_COOKIE;
> > +        conj_act->ofpacts = xmemdup(ofpacts.data, ofpacts.size);
> > +        conj_act->ofpacts_len = ofpacts.size;
> > +        ofpbuf_uninit(&ofpacts);
> > +        a = conj_act;
> > +    }
> > +
> > +    if (a) {
> > +        ds_put_format(&s, "cookie=%"PRIx64", ", a->cookie);
> > +    }
> > +    ds_put_format(&s, "table_id=%"PRIu8", ", f->table_id);
> > +    ds_put_format(&s, "priority=%"PRIu16", ", f->priority);
> > +    minimatch_format(&f->match, NULL, NULL, &s, OFP_DEFAULT_PRIORITY);
> > +    ds_put_cstr(&s, ", actions=");
> > +    struct ofpact_format_params fp = { .s = &s };
> > +    if (a) {
> > +        ofpacts_format(a->ofpacts, a->ofpacts_len, &fp);
> > +    } else {
> > +        ds_put_cstr(&s, "<EMPTY>");
> > +    }
> > +
> > +    if (conj_act) {
> > +        free(conj_act->ofpacts);
> > +        free(conj_act);
> > +    }
> > +    return ds_steal_cstr(&s);
> > +}
> > +
> > +static void
> > +ovn_flow_log(const struct ovn_flow *f)
> > +{
> > +    if (VLOG_IS_DBG_ENABLED()) {
> > +        char *s = ovn_flow_to_string(f);
> > +        VLOG_DBG("flow: %s", s);
> > +        free(s);
> > +    }
> > +}
> > +
> > +#if 0
> > +static void
> > +dp_flow_print_flow_table__(struct ovn_flow_table *ftable, FILE *stream)
> > +{
> > +    bool is_empty = true;
> > +    struct ovn_flow *f;
> > +    for (uint8_t i = 0 ; i < OFTABLE_MAX_TABLE_IDS; i++) {
> > +        is_empty = (is_empty && hmap_is_empty(&ftable->match_flow_table[i]));
> > +
> > +        HMAP_FOR_EACH (f, hmap_node, &ftable->match_flow_table[i]) {
> > +            char *s = ovn_flow_to_string(f);
> > +            if (s) {
> > +                fputs(s, stream);
> > +                free(s);
> > +                fputc('\n', stream);
> > +            }
> > +        }
> > +    }
> > +
> > +    if (is_empty) {
> > +        fputs("empty\n", stream);
> > +    }
> > +}
> > +
> > +#endif
> > diff --git a/controller/dp-flow-mgr.h b/controller/dp-flow-mgr.h
> > new file mode 100644
> > index 0000000000..75ac11d3fa
> > --- /dev/null
> > +++ b/controller/dp-flow-mgr.h
> > @@ -0,0 +1,64 @@
> > +/* Copyright (c) 2021 Red Hat, Inc.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + *     http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +
> > +#ifndef OVN_DATAPATH_H
> > +#define OVN_DATAPATH_H 1
> > +
> > +#include <stdio.h>
> > +
> > +/* ovn includes. */
> > +
> > +/* ovs includes. */
> > +#include "openvswitch/hmap.h"
> > +#include "openvswitch/list.h"
> > +
> > +struct match;
> > +struct ofpbuf;
> > +struct uuid;
> > +
> > +#define DP_FLOW_TABLE_GLOBAL_KEY 0
> > +
> > +void dp_flow_tables_init(void);
> > +void dp_flow_tables_destroy(void);
> > +
> > +struct dp_flow_table *dp_flow_table_alloc(uint32_t dp_key);
> > +struct dp_flow_table * dp_flow_table_get(uint32_t dp_key);
> > +void dp_flow_table_destroy(uint32_t dp_key);
> > +
> > +void dp_flow_switch_logical_oflow_tables(void);
> > +void dp_flow_switch_logical_oflow_table(uint32_t dp_key);
> > +void dp_flow_switch_physical_oflow_tables(void);
> > +void dp_flow_switch_physical_oflow_table(uint32_t dp_key);
> > +
> > +void dp_flow_add_logical_oflow(uint32_t dp_key, uint8_t table_id,
> > +                               uint16_t priority, uint64_t cookie,
> > +                               const struct match *match,
> > +                               const struct ofpbuf *actions,
> > +                               const struct uuid *flow_uuid);
> > +void dp_flow_add_physical_oflow(uint32_t dp_key, uint8_t table_id,
> > +                                uint16_t priority, uint64_t cookie,
> > +                                const struct match *match,
> > +                                const struct ofpbuf *actions,
> > +                                const struct uuid *flow_uuid);
> > +void dp_flow_remove_logical_oflows_all(const struct uuid *flow_uuid);
> > +void dp_flow_remove_logical_oflows(uint32_t dp_key,
> > +                                   const struct uuid *flow_uuid);
> > +void dp_flow_remove_physical_oflows(uint32_t dp_key,
> > +                                    const struct uuid *flow_uuid);
> > +void dp_flow_flush_all_oflows(void);
> > +
> > +void dp_flow_populate_oflow_msgs(struct ovs_list *msgs);
> > +#endif /* OVN_DATAPATH_H */
> > diff --git a/controller/lflow.c b/controller/lflow.c
> > index a3d84aff4e..e3c3e1ca72 100644
> > --- a/controller/lflow.c
> > +++ b/controller/lflow.c
> > @@ -14,6 +14,7 @@
> >    */
> >
> >   #include <config.h>
> > +#include "dp-flow-mgr.h"
> >   #include "lflow.h"
> >   #include "coverage.h"
> >   #include "ha-chassis.h"
> > @@ -165,6 +166,28 @@ is_chassis_resident_cb(const void *c_aux_, const char *port_name)
> >       }
> >   }
> >
> > +static void
> > +lflow_remove_dp_flows(const struct sbrec_logical_flow *lflow)
> > +{
> > +    const struct sbrec_logical_dp_group *dp_group = lflow->logical_dp_group;
> > +    const struct sbrec_datapath_binding *dp = lflow->logical_datapath;
> > +
> > +    if (!dp_group && !dp) {
> > +        return;
> > +    }
> > +
> > +    ovs_assert(!dp_group || !dp);
> > +
> > +    if (dp) {
> > +        dp_flow_remove_logical_oflows(dp->tunnel_key, &lflow->header_.uuid);
> > +    }
> > +
> > +    for (size_t i = 0; dp_group && i < dp_group->n_datapaths; i++) {
> > +        dp_flow_remove_logical_oflows(dp_group->datapaths[i]->tunnel_key,
> > +                                      &lflow->header_.uuid);
> > +    }
> > +}
> > +
> >   void
> >   lflow_resource_init(struct lflow_resource_ref *lfrr)
> >   {
> > @@ -389,48 +412,67 @@ lflow_handle_changed_flows(struct lflow_ctx_in *l_ctx_in,
> >       struct controller_event_options controller_event_opts;
> >       controller_event_opts_init(&controller_event_opts);
> >
> > -    /* Flood remove the flows for all the tracked lflows.  Its possible that
> > -     * lflow_add_flows_for_datapath() may have been called before calling
> > -     * this function. */
> > -    struct hmap flood_remove_nodes = HMAP_INITIALIZER(&flood_remove_nodes);
> > -    struct ofctrl_flood_remove_node *ofrn, *next;
> > +    /* Handle flow removing first (for deleted or updated lflows), and then
> > +     * handle reprocessing or adding flows, so that when the flows being
> > +     * removed and added with same match conditions can be processed in the
> > +     * proper order */
> > +
> >       SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow,
> >                                                  l_ctx_in->logical_flow_table) {
> > -        VLOG_DBG("delete lflow "UUID_FMT, UUID_ARGS(&lflow->header_.uuid));
> > -        ofrn = xmalloc(sizeof *ofrn);
> > -        ofrn->sb_uuid = lflow->header_.uuid;
> > -        hmap_insert(&flood_remove_nodes, &ofrn->hmap_node,
> > -                    uuid_hash(&ofrn->sb_uuid));
> >           if (!sbrec_logical_flow_is_new(lflow)) {
> > +            VLOG_DBG("delete lflow "UUID_FMT,
> > +                     UUID_ARGS(&lflow->header_.uuid));
> > +            lflow_remove_dp_flows(lflow);
> > +            ovn_extend_table_remove_desired(l_ctx_out->group_table,
> > +                                            &lflow->header_.uuid);
> > +            ovn_extend_table_remove_desired(l_ctx_out->meter_table,
> > +                                            &lflow->header_.uuid);
> >               if (lflow_cache_is_enabled(l_ctx_out->lflow_cache)) {
> >                   lflow_cache_delete(l_ctx_out->lflow_cache,
> >                                      &lflow->header_.uuid);
> >               }
> >           }
> >       }
> > -    ofctrl_flood_remove_flows(l_ctx_out->flow_table, &flood_remove_nodes);
> > -    HMAP_FOR_EACH (ofrn, hmap_node, &flood_remove_nodes) {
> > -        /* Delete entries from lflow resource reference. */
> > -        lflow_resource_destroy_lflow(l_ctx_out->lfrr, &ofrn->sb_uuid);
> > -        /* Reprocessing the lflow if the sb record is not deleted. */
> > -        lflow = sbrec_logical_flow_table_get_for_uuid(
> > -            l_ctx_in->logical_flow_table, &ofrn->sb_uuid);
> > -        if (lflow) {
> > -            VLOG_DBG("re-add lflow "UUID_FMT,
> > +
> > +    SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow,
> > +                                               l_ctx_in->logical_flow_table) {
> > +        if (!sbrec_logical_flow_is_new(lflow)) {
> > +            /* Delete entries from lflow resource reference. */
> > +            lflow_resource_destroy_lflow(l_ctx_out->lfrr,
> > +                                         &lflow->header_.uuid);
> > +            /* Reprocessing the lflow if the sb record is not deleted. */
> > +            const struct sbrec_logical_flow *lflow_;
> > +            lflow_ = sbrec_logical_flow_table_get_for_uuid(
> > +                l_ctx_in->logical_flow_table, &lflow->header_.uuid);
> > +            if (lflow_) {
> > +                VLOG_DBG("re-add lflow "UUID_FMT,
> > +                        UUID_ARGS(&lflow_->header_.uuid));
> > +                if (!consider_logical_flow(lflow_, &dhcp_opts, &dhcpv6_opts,
> > +                                           &nd_ra_opts, &controller_event_opts,
> > +                                           l_ctx_in, l_ctx_out)) {
> > +                    ret = false;
> > +                    break;
> > +                }
> > +            }
> > +        }
> > +    }
> > +
> > +    /* Now handle new lflows only. */
> > +    SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_TRACKED (lflow,
> > +                                               l_ctx_in->logical_flow_table) {
> > +        if (sbrec_logical_flow_is_new(lflow)) {
> > +            VLOG_DBG("add lflow "UUID_FMT,
> >                        UUID_ARGS(&lflow->header_.uuid));
> >               if (!consider_logical_flow(lflow, &dhcp_opts, &dhcpv6_opts,
> >                                          &nd_ra_opts, &controller_event_opts,
> >                                          l_ctx_in, l_ctx_out)) {
> >                   ret = false;
> > +                l_ctx_out->conj_id_overflow = true;
> >                   break;
> >               }
> >           }
> >       }
> > -    HMAP_FOR_EACH_SAFE (ofrn, next, hmap_node, &flood_remove_nodes) {
> > -        hmap_remove(&flood_remove_nodes, &ofrn->hmap_node);
> > -        free(ofrn);
> > -    }
> > -    hmap_destroy(&flood_remove_nodes);
> > +
> >
> >       dhcp_opts_destroy(&dhcp_opts);
> >       dhcp_opts_destroy(&dhcpv6_opts);
> > @@ -491,29 +533,25 @@ lflow_handle_changed_ref(enum ref_type ref_type, const char *ref_name,
> >       controller_event_opts_init(&controller_event_opts);
> >
> >       /* Re-parse the related lflows. */
> > -    /* Firstly, flood remove the flows from desired flow table. */
> > -    struct hmap flood_remove_nodes = HMAP_INITIALIZER(&flood_remove_nodes);
> > -    struct ofctrl_flood_remove_node *ofrn, *ofrn_next;
> >       HMAP_FOR_EACH (lrln, hmap_node, &rlfn->lflow_uuids) {
> > -        VLOG_DBG("Reprocess lflow "UUID_FMT" for resource type: %d,"
> > -                 " name: %s.",
> > -                 UUID_ARGS(&lrln->lflow_uuid),
> > -                 ref_type, ref_name);
> > -        ofctrl_flood_remove_add_node(&flood_remove_nodes, &lrln->lflow_uuid);
> > +        ovn_extend_table_remove_desired(l_ctx_out->group_table,
> > +                                        &lrln->lflow_uuid);
> > +        ovn_extend_table_remove_desired(l_ctx_out->meter_table,
> > +                                        &lrln->lflow_uuid);
> > +        dp_flow_remove_logical_oflows_all(&lrln->lflow_uuid);
> >       }
> > -    ofctrl_flood_remove_flows(l_ctx_out->flow_table, &flood_remove_nodes);
> >
> >       /* Secondly, for each lflow that is actually removed, reprocessing it. */
> > -    HMAP_FOR_EACH (ofrn, hmap_node, &flood_remove_nodes) {
> > -        lflow_resource_destroy_lflow(l_ctx_out->lfrr, &ofrn->sb_uuid);
> > +    HMAP_FOR_EACH (lrln, hmap_node, &rlfn->lflow_uuids) {
> > +        lflow_resource_destroy_lflow(l_ctx_out->lfrr, &lrln->lflow_uuid);
> >
> >           const struct sbrec_logical_flow *lflow =
> >               sbrec_logical_flow_table_get_for_uuid(l_ctx_in->logical_flow_table,
> > -                                                  &ofrn->sb_uuid);
> > +                                                  &lrln->lflow_uuid);
> >           if (!lflow) {
> >               VLOG_DBG("lflow "UUID_FMT" not found while reprocessing for"
> >                        " resource type: %d, name: %s.",
> > -                     UUID_ARGS(&ofrn->sb_uuid),
> > +                     UUID_ARGS(&lrln->lflow_uuid),
> >                        ref_type, ref_name);
> >               continue;
> >           }
> > @@ -527,11 +565,7 @@ lflow_handle_changed_ref(enum ref_type ref_type, const char *ref_name,
> >           }
> >           *changed = true;
> >       }
> > -    HMAP_FOR_EACH_SAFE (ofrn, ofrn_next, hmap_node, &flood_remove_nodes) {
> > -        hmap_remove(&flood_remove_nodes, &ofrn->hmap_node);
> > -        free(ofrn);
> > -    }
> > -    hmap_destroy(&flood_remove_nodes);
> > +
> >
> >       HMAP_FOR_EACH_SAFE (lrln, next, hmap_node, &rlfn->lflow_uuids) {
> >           hmap_remove(&rlfn->lflow_uuids, &lrln->hmap_node);
> > @@ -621,9 +655,11 @@ add_matches_to_flow_table(const struct sbrec_logical_flow *lflow,
> >               }
> >           }
> >           if (!m->n) {
> > -            ofctrl_add_flow(l_ctx_out->flow_table, ptable, lflow->priority,
> > -                            lflow->header_.uuid.parts[0], &m->match, &ofpacts,
> > -                            &lflow->header_.uuid);
> > +            dp_flow_add_logical_oflow(dp->tunnel_key, ptable,
> > +                                      lflow->priority,
> > +                                      lflow->header_.uuid.parts[0],
> > +                                      &m->match, &ofpacts,
> > +                                      &lflow->header_.uuid);
> >           } else {
> >               uint64_t conj_stubs[64 / 8];
> >               struct ofpbuf conj;
> > @@ -639,9 +675,11 @@ add_matches_to_flow_table(const struct sbrec_logical_flow *lflow,
> >                   dst->n_clauses = src->n_clauses;
> >               }
> >
> > -            ofctrl_add_or_append_flow(l_ctx_out->flow_table, ptable,
> > -                                      lflow->priority, 0,
> > -                                      &m->match, &conj, &lflow->header_.uuid);
> > +            dp_flow_add_logical_oflow(dp->tunnel_key, ptable,
> > +                                      lflow->priority,
> > +                                      lflow->header_.uuid.parts[0],
> > +                                      &m->match, &conj,
> > +                                      &lflow->header_.uuid);
> >               ofpbuf_uninit(&conj);
> >           }
> >       }
> > @@ -966,8 +1004,7 @@ put_load64(uint64_t value, enum mf_field_id dst, int ofs, int n_bits,
> >   static void
> >   consider_neighbor_flow(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                          const struct hmap *local_datapaths,
> > -                       const struct sbrec_mac_binding *b,
> > -                       struct ovn_desired_flow_table *flow_table)
> > +                       const struct sbrec_mac_binding *b)
> >   {
> >       const struct sbrec_port_binding *pb
> >           = lport_lookup_by_name(sbrec_port_binding_by_name, b->logical_port);
> > @@ -1026,17 +1063,20 @@ consider_neighbor_flow(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >       put_load(mac.ea, sizeof mac.ea, MFF_ETH_DST, 0, 48, &ofpacts);
> >       put_load(&value, sizeof value, MFF_LOG_FLAGS, MLF_LOOKUP_MAC_BIT, 1,
> >                &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_MAC_BINDING, 100,
> > -                    b->header_.uuid.parts[0], &get_arp_match,
> > -                    &ofpacts, &b->header_.uuid);
> > +    dp_flow_add_logical_oflow(pb->datapath->tunnel_key,
> > +                              OFTABLE_MAC_BINDING, 100,
> > +                              b->header_.uuid.parts[0],
> > +                              &get_arp_match, &ofpacts,
> > +                              &b->header_.uuid);
> >
> >       ofpbuf_clear(&ofpacts);
> >       put_load(&value, sizeof value, MFF_LOG_FLAGS, MLF_LOOKUP_MAC_BIT, 1,
> >                &ofpacts);
> >       match_set_dl_src(&lookup_arp_match, mac);
> > -    ofctrl_add_flow(flow_table, OFTABLE_MAC_LOOKUP, 100,
> > -                    b->header_.uuid.parts[0], &lookup_arp_match,
> > -                    &ofpacts, &b->header_.uuid);
> > +    dp_flow_add_logical_oflow(pb->datapath->tunnel_key,
> > +                              OFTABLE_MAC_LOOKUP, 100,
> > +                              b->header_.uuid.parts[0], &lookup_arp_match,
> > +                              &ofpacts, &b->header_.uuid);
> >
> >       ofpbuf_uninit(&ofpacts);
> >   }
> > @@ -1046,13 +1086,11 @@ consider_neighbor_flow(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >   static void
> >   add_neighbor_flows(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                      const struct sbrec_mac_binding_table *mac_binding_table,
> > -                   const struct hmap *local_datapaths,
> > -                   struct ovn_desired_flow_table *flow_table)
> > +                   const struct hmap *local_datapaths)
> >   {
> >       const struct sbrec_mac_binding *b;
> >       SBREC_MAC_BINDING_TABLE_FOR_EACH (b, mac_binding_table) {
> > -        consider_neighbor_flow(sbrec_port_binding_by_name, local_datapaths,
> > -                               b, flow_table);
> > +        consider_neighbor_flow(sbrec_port_binding_by_name, local_datapaths, b);
> >       }
> >   }
> >
> > @@ -1238,8 +1276,7 @@ static void
> >   add_lb_vip_hairpin_flows(struct ovn_controller_lb *lb,
> >                            struct ovn_lb_vip *lb_vip,
> >                            struct ovn_lb_backend *lb_backend,
> > -                         uint8_t lb_proto,
> > -                         struct ovn_desired_flow_table *flow_table)
> > +                         uint8_t lb_proto)
> >   {
> >       uint64_t stub[1024 / 8];
> >       struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(stub);
> > @@ -1335,9 +1372,9 @@ add_lb_vip_hairpin_flows(struct ovn_controller_lb *lb,
> >       };
> >       match_set_ct_label_masked(&hairpin_match, lb_ct_label, lb_ct_label);
> >
> > -    ofctrl_add_flow(flow_table, OFTABLE_CHK_LB_HAIRPIN, 100,
> > -                    lb->slb->header_.uuid.parts[0], &hairpin_match,
> > -                    &ofpacts, &lb->slb->header_.uuid);
> > +    dp_flow_add_logical_oflow(0, OFTABLE_CHK_LB_HAIRPIN, 100,
> > +                              lb->slb->header_.uuid.parts[0], &hairpin_match,
> > +                              &ofpacts, &lb->slb->header_.uuid);
> >       ofpbuf_uninit(&ofpacts);
> >   }
> >
> > @@ -1350,8 +1387,7 @@ add_lb_vip_hairpin_flows(struct ovn_controller_lb *lb,
> >   static void
> >   add_lb_ct_snat_vip_flows(struct ovn_controller_lb *lb,
> >                            struct ovn_lb_vip *lb_vip,
> > -                         uint8_t lb_proto,
> > -                         struct ovn_desired_flow_table *flow_table)
> > +                         uint8_t lb_proto)
> >   {
> >       uint64_t stub[1024 / 8];
> >       struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(stub);
> > @@ -1438,9 +1474,10 @@ add_lb_ct_snat_vip_flows(struct ovn_controller_lb *lb,
> >           match_set_metadata(&match,
> >                              htonll(lb->slb->datapaths[i]->tunnel_key));
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_CT_SNAT_FOR_VIP, 100,
> > -                        lb->slb->header_.uuid.parts[0],
> > -                        &match, &ofpacts, &lb->slb->header_.uuid);
> > +        dp_flow_add_logical_oflow(lb->slb->datapaths[i]->tunnel_key,
> > +                                  OFTABLE_CT_SNAT_FOR_VIP, 100,
> > +                                  lb->slb->header_.uuid.parts[0],
> > +                                  &match, &ofpacts, &lb->slb->header_.uuid);
> >       }
> >
> >       ofpbuf_uninit(&ofpacts);
> > @@ -1448,8 +1485,7 @@ add_lb_ct_snat_vip_flows(struct ovn_controller_lb *lb,
> >
> >   static void
> >   consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb,
> > -                          const struct hmap *local_datapaths,
> > -                          struct ovn_desired_flow_table *flow_table)
> > +                          const struct hmap *local_datapaths)
> >   {
> >       /* Check if we need to add flows or not.  If there is one datapath
> >        * in the local_datapaths, it means all the datapaths of the lb
> > @@ -1482,11 +1518,10 @@ consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb,
> >           for (size_t j = 0; j < lb_vip->n_backends; j++) {
> >               struct ovn_lb_backend *lb_backend = &lb_vip->backends[j];
> >
> > -            add_lb_vip_hairpin_flows(lb, lb_vip, lb_backend, lb_proto,
> > -                                     flow_table);
> > +            add_lb_vip_hairpin_flows(lb, lb_vip, lb_backend, lb_proto);
> >           }
> >
> > -        add_lb_ct_snat_vip_flows(lb, lb_vip, lb_proto, flow_table);
> > +        add_lb_ct_snat_vip_flows(lb, lb_vip, lb_proto);
> >       }
> >
> >       ovn_controller_lb_destroy(lb);
> > @@ -1496,12 +1531,11 @@ consider_lb_hairpin_flows(const struct sbrec_load_balancer *sbrec_lb,
> >    * backends to handle the load balanced hairpin traffic. */
> >   static void
> >   add_lb_hairpin_flows(const struct sbrec_load_balancer_table *lb_table,
> > -                     const struct hmap *local_datapaths,
> > -                     struct ovn_desired_flow_table *flow_table)
> > +                     const struct hmap *local_datapaths)
> >   {
> >       const struct sbrec_load_balancer *lb;
> >       SBREC_LOAD_BALANCER_TABLE_FOR_EACH (lb, lb_table) {
> > -        consider_lb_hairpin_flows(lb, local_datapaths, flow_table);
> > +        consider_lb_hairpin_flows(lb, local_datapaths);
> >       }
> >   }
> >
> > @@ -1510,8 +1544,7 @@ void
> >   lflow_handle_changed_neighbors(
> >       struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >       const struct sbrec_mac_binding_table *mac_binding_table,
> > -    const struct hmap *local_datapaths,
> > -    struct ovn_desired_flow_table *flow_table)
> > +    const struct hmap *local_datapaths)
> >   {
> >       const struct sbrec_mac_binding *mb;
> >       /* Handle deleted mac_bindings first, to avoid *duplicated flow* problem
> > @@ -1521,7 +1554,8 @@ lflow_handle_changed_neighbors(
> >           if (sbrec_mac_binding_is_deleted(mb)) {
> >               VLOG_DBG("handle deleted mac_binding "UUID_FMT,
> >                        UUID_ARGS(&mb->header_.uuid));
> > -            ofctrl_remove_flows(flow_table, &mb->header_.uuid);
> > +            dp_flow_remove_logical_oflows(mb->datapath->tunnel_key,
> > +                                          &mb->header_.uuid);
> >           }
> >       }
> >       SBREC_MAC_BINDING_TABLE_FOR_EACH_TRACKED (mb, mac_binding_table) {
> > @@ -1529,20 +1563,18 @@ lflow_handle_changed_neighbors(
> >               if (!sbrec_mac_binding_is_new(mb)) {
> >                   VLOG_DBG("handle updated mac_binding "UUID_FMT,
> >                            UUID_ARGS(&mb->header_.uuid));
> > -                ofctrl_remove_flows(flow_table, &mb->header_.uuid);
> >               }
> >               VLOG_DBG("handle new mac_binding "UUID_FMT,
> >                        UUID_ARGS(&mb->header_.uuid));
> > -            consider_neighbor_flow(sbrec_port_binding_by_name, local_datapaths,
> > -                                   mb, flow_table);
> > +            consider_neighbor_flow(sbrec_port_binding_by_name,
> > +                                   local_datapaths, mb);
> >           }
> >       }
> >   }
> >
> >   static void
> >   consider_fdb_flows(const struct sbrec_fdb *fdb,
> > -                   const struct hmap *local_datapaths,
> > -                   struct ovn_desired_flow_table *flow_table)
> > +                   const struct hmap *local_datapaths)
> >   {
> >       if (!get_local_datapath(local_datapaths, fdb->dp_key)) {
> >           return;
> > @@ -1562,7 +1594,7 @@ consider_fdb_flows(const struct sbrec_fdb *fdb,
> >       uint64_t stub[1024 / 8];
> >       struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(stub);
> >       put_load64(fdb->port_key, MFF_LOG_OUTPORT, 0, 32, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_GET_FDB, 100,
> > +    dp_flow_add_logical_oflow(fdb->dp_key, OFTABLE_GET_FDB, 100,
> >                       fdb->header_.uuid.parts[0], &match, &ofpacts,
> >                       &fdb->header_.uuid);
> >       ofpbuf_clear(&ofpacts);
> > @@ -1575,7 +1607,7 @@ consider_fdb_flows(const struct sbrec_fdb *fdb,
> >       match_set_metadata(&lookup_match, htonll(fdb->dp_key));
> >       match_set_dl_src(&lookup_match, mac);
> >       match_set_reg(&lookup_match, MFF_LOG_INPORT - MFF_REG0, fdb->port_key);
> > -    ofctrl_add_flow(flow_table, OFTABLE_LOOKUP_FDB, 100,
> > +    dp_flow_add_logical_oflow(fdb->dp_key, OFTABLE_LOOKUP_FDB, 100,
> >                       fdb->header_.uuid.parts[0], &lookup_match, &ofpacts,
> >                       &fdb->header_.uuid);
> >       ofpbuf_uninit(&ofpacts);
> > @@ -1585,12 +1617,11 @@ consider_fdb_flows(const struct sbrec_fdb *fdb,
> >    * southbound database. */
> >   static void
> >   add_fdb_flows(const struct sbrec_fdb_table *fdb_table,
> > -              const struct hmap *local_datapaths,
> > -              struct ovn_desired_flow_table *flow_table)
> > +              const struct hmap *local_datapaths)
> >   {
> >       const struct sbrec_fdb *fdb;
> >       SBREC_FDB_TABLE_FOR_EACH (fdb, fdb_table) {
> > -        consider_fdb_flows(fdb, local_datapaths, flow_table);
> > +        consider_fdb_flows(fdb, local_datapaths);
> >       }
> >   }
> >
> > @@ -1604,12 +1635,9 @@ lflow_run(struct lflow_ctx_in *l_ctx_in, struct lflow_ctx_out *l_ctx_out)
> >
> >       add_logical_flows(l_ctx_in, l_ctx_out);
> >       add_neighbor_flows(l_ctx_in->sbrec_port_binding_by_name,
> > -                       l_ctx_in->mac_binding_table, l_ctx_in->local_datapaths,
> > -                       l_ctx_out->flow_table);
> > -    add_lb_hairpin_flows(l_ctx_in->lb_table, l_ctx_in->local_datapaths,
> > -                         l_ctx_out->flow_table);
> > -    add_fdb_flows(l_ctx_in->fdb_table, l_ctx_in->local_datapaths,
> > -                  l_ctx_out->flow_table);
> > +                       l_ctx_in->mac_binding_table, l_ctx_in->local_datapaths);
> > +    add_lb_hairpin_flows(l_ctx_in->lb_table, l_ctx_in->local_datapaths);
> > +    add_fdb_flows(l_ctx_in->fdb_table, l_ctx_in->local_datapaths);
> >   }
> >
> >   /* Should be called at every ovn-controller iteration before IDL tracked
> > @@ -1723,8 +1751,7 @@ lflow_processing_end:
> >        * associated. */
> >       for (size_t i = 0; i < dp->n_load_balancers; i++) {
> >           consider_lb_hairpin_flows(dp->load_balancers[i],
> > -                                  l_ctx_in->local_datapaths,
> > -                                  l_ctx_out->flow_table);
> > +                                  l_ctx_in->local_datapaths);
> >       }
> >
> >       return handled;
> > @@ -1746,8 +1773,7 @@ lflow_handle_flows_for_lport(const struct sbrec_port_binding *pb,
> >   }
> >
> >   bool
> > -lflow_handle_changed_lbs(struct lflow_ctx_in *l_ctx_in,
> > -                         struct lflow_ctx_out *l_ctx_out)
> > +lflow_handle_changed_lbs(struct lflow_ctx_in *l_ctx_in)
> >   {
> >       const struct sbrec_load_balancer *lb;
> >
> > @@ -1755,7 +1781,13 @@ lflow_handle_changed_lbs(struct lflow_ctx_in *l_ctx_in,
> >           if (sbrec_load_balancer_is_deleted(lb)) {
> >               VLOG_DBG("Remove hairpin flows for deleted load balancer "UUID_FMT,
> >                        UUID_ARGS(&lb->header_.uuid));
> > -            ofctrl_remove_flows(l_ctx_out->flow_table, &lb->header_.uuid);
> > +
> > +            for (size_t i = 0; i < lb->n_datapaths; i++) {
> > +                dp_flow_remove_logical_oflows(lb->datapaths[i]->tunnel_key,
> > +                                              &lb->header_.uuid);
> > +            }
> > +            dp_flow_remove_logical_oflows(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                          &lb->header_.uuid);
> >           }
> >       }
> >
> > @@ -1764,24 +1796,16 @@ lflow_handle_changed_lbs(struct lflow_ctx_in *l_ctx_in,
> >               continue;
> >           }
> >
> > -        if (!sbrec_load_balancer_is_new(lb)) {
> > -            VLOG_DBG("Remove hairpin flows for updated load balancer "UUID_FMT,
> > -                     UUID_ARGS(&lb->header_.uuid));
> > -            ofctrl_remove_flows(l_ctx_out->flow_table, &lb->header_.uuid);
> > -        }
> > -
> >           VLOG_DBG("Add load balancer hairpin flows for "UUID_FMT,
> >                    UUID_ARGS(&lb->header_.uuid));
> > -        consider_lb_hairpin_flows(lb, l_ctx_in->local_datapaths,
> > -                                  l_ctx_out->flow_table);
> > +        consider_lb_hairpin_flows(lb, l_ctx_in->local_datapaths);
> >       }
> >
> >       return true;
> >   }
> >
> >   bool
> > -lflow_handle_changed_fdbs(struct lflow_ctx_in *l_ctx_in,
> > -                         struct lflow_ctx_out *l_ctx_out)
> > +lflow_handle_changed_fdbs(struct lflow_ctx_in *l_ctx_in)
> >   {
> >       const struct sbrec_fdb *fdb;
> >
> > @@ -1789,7 +1813,7 @@ lflow_handle_changed_fdbs(struct lflow_ctx_in *l_ctx_in,
> >           if (sbrec_fdb_is_deleted(fdb)) {
> >               VLOG_DBG("Remove fdb flows for deleted fdb "UUID_FMT,
> >                        UUID_ARGS(&fdb->header_.uuid));
> > -            ofctrl_remove_flows(l_ctx_out->flow_table, &fdb->header_.uuid);
> > +            dp_flow_remove_logical_oflows(fdb->dp_key, &fdb->header_.uuid);
> >           }
> >       }
> >
> > @@ -1801,13 +1825,12 @@ lflow_handle_changed_fdbs(struct lflow_ctx_in *l_ctx_in,
> >           if (!sbrec_fdb_is_new(fdb)) {
> >               VLOG_DBG("Remove fdb flows for updated fdb "UUID_FMT,
> >                        UUID_ARGS(&fdb->header_.uuid));
> > -            ofctrl_remove_flows(l_ctx_out->flow_table, &fdb->header_.uuid);
> > +            dp_flow_remove_logical_oflows(fdb->dp_key, &fdb->header_.uuid);
> >           }
> >
> >           VLOG_DBG("Add fdb flows for fdb "UUID_FMT,
> >                    UUID_ARGS(&fdb->header_.uuid));
> > -        consider_fdb_flows(fdb, l_ctx_in->local_datapaths,
> > -                           l_ctx_out->flow_table);
> > +        consider_fdb_flows(fdb, l_ctx_in->local_datapaths);
> >       }
> >
> >       return true;
> > diff --git a/controller/lflow.h b/controller/lflow.h
> > index 3c929d8a6a..8054fbba92 100644
> > --- a/controller/lflow.h
> > +++ b/controller/lflow.h
> > @@ -147,7 +147,6 @@ struct lflow_ctx_in {
> >   };
> >
> >   struct lflow_ctx_out {
> > -    struct ovn_desired_flow_table *flow_table;
> >       struct ovn_extend_table *group_table;
> >       struct ovn_extend_table *meter_table;
> >       struct lflow_resource_ref *lfrr;
> > @@ -167,10 +166,9 @@ bool lflow_handle_changed_ref(enum ref_type, const char *ref_name,
> >   void lflow_handle_changed_neighbors(
> >       struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >       const struct sbrec_mac_binding_table *,
> > -    const struct hmap *local_datapaths,
> > -    struct ovn_desired_flow_table *);
> > -bool lflow_handle_changed_lbs(struct lflow_ctx_in *, struct lflow_ctx_out *);
> > -bool lflow_handle_changed_fdbs(struct lflow_ctx_in *, struct lflow_ctx_out *);
> > +    const struct hmap *local_datapaths);
> > +bool lflow_handle_changed_lbs(struct lflow_ctx_in *);
> > +bool lflow_handle_changed_fdbs(struct lflow_ctx_in *);
> >   void lflow_destroy(void);
> >
> >   bool lflow_add_flows_for_datapath(const struct sbrec_datapath_binding *,
> > diff --git a/controller/ofctrl.c b/controller/ofctrl.c
> > index 415d9b7e16..1f4bc70fff 100644
> > --- a/controller/ofctrl.c
> > +++ b/controller/ofctrl.c
> > @@ -17,6 +17,7 @@
> >   #include "bitmap.h"
> >   #include "byte-order.h"
> >   #include "dirs.h"
> > +#include "dp-flow-mgr.h"
> >   #include "dp-packet.h"
> >   #include "flow.h"
> >   #include "hash.h"
> > @@ -51,197 +52,6 @@
> >
> >   VLOG_DEFINE_THIS_MODULE(ofctrl);
> >
> > -/* An OpenFlow flow. */
> > -struct ovn_flow {
> > -    /* Key. */
> > -    uint8_t table_id;
> > -    uint16_t priority;
> > -    struct minimatch match;
> > -
> > -    /* Hash. */
> > -    uint32_t hash;
> > -
> > -    /* Data. */
> > -    struct ofpact *ofpacts;
> > -    size_t ofpacts_len;
> > -    uint64_t cookie;
> > -};
> > -
> > -/* A desired flow, in struct ovn_desired_flow_table, calculated by the
> > - * incremental processing engine.
> > - * - They are added/removed incrementally when I-P engine is able to process
> > - *   the changes incrementally, or
> > - * - Completely cleared and recomputed by I-P engine when recompute happens.
> > - *
> > - * Links are maintained between desired flows and SB data. The relationship
> > - * is M to N. The struct sb_flow_ref is used to link a pair of desired flow
> > - * and SB UUID. The below diagram depicts the data structure.
> > - *
> > - *                   SB UUIDs
> > - *                 +-----+-----+-----+-----+-----+-----+-----+
> > - *                 |     |     |     |     |     |     |     |
> > - *                 +--+--+--+--+--+--+-----+--+--+--+--+--+--+
> > - *                    |     |     |           |     |     |
> > - *  Desired Flows     |     |     |           |     |     |
> > - *     +----+       +-+-+   |   +-+-+         |   +-+-+   |
> > - *     |    +-------+   +-------+   +-------------+   |   |
> > - *     +----+       +---+   |   +-+-+         |   +---+   |
> > - *     |    |               |     |           |           |
> > - *     +----+               |     |         +-+-+         |
> > - *     |    +-------------------------------+   |         |
> > - *     +----+             +---+   |         +---+         |
> > - *     |    +-------------+   |   |                       |
> > - *     +----+             +---+   |                       |
> > - *     |    |                     |                       |
> > - *     +----+                   +-+-+                   +-+-+
> > - *     |    +-------------------+   +-------------------+   |
> > - *     +----+                   +---+                   +---+
> > - *     |    |
> > - *     +----+
> > - *
> > - * The links are updated whenever there is a change in desired flows, which is
> > - * usually triggered by a SB data change in I-P engine.
> > - *
> > - * ** Tracking **
> > - *
> > - * A desired flow can be tracked - listed in ovn_desired_flow_table's
> > - * tracked_flows.
> > - *
> > - * Tracked flows is initially empty, and stays empty after the first run of I-P
> > - * engine when installed flows are initially populated. After that, flow
> > - * changes are tracked when I-P engine incrementally computes flow changes.
> > - * Tracked flows are then processed and removed completely in ofctrl_put.
> > - * ("processed" means OpenFlow change messages are composed and sent/queued to
> > - * OVS, which ensures flows in OVS is always in sync (eventually) with the
> > - * installed flows table).
> > - *
> > - * In case of full recompute of I-P engine, tracked flows are not
> > - * added/removed, and ofctrl_put will not rely on tracked flows. (It is I-P
> > - * engine's responsibility to ensure the tracked flows are cleared before
> > - * recompute).
> > - *
> > - * Tracked flows can be preserved across multiple I-P engine runs - if in some
> > - * iterations ofctrl_put() is skipped. Tracked flows are cleared only when it
> > - * is consumed or when flow recompute happens.
> > - *
> > - * The "change_tracked" member of desired flow table maintains the status of
> > - * whether flow changes are tracked or not. It is always set to true when
> > - * ofctrl_put is completed, and transition to false whenever
> > - * ovn_desired_flow_table_clear is called.
> > - *
> > - * NOTE: A tracked flow is just a reference to a desired flow, instead of a new
> > - * copy. When a desired flow is removed and tracked, it is removed from the
> > - * match_flow_table and uuid_flow_table indexes, and added to the tracked_flows
> > - * list, marking is_deleted = true, but not immediately destroyed. It is
> > - * destroyed when the tracking is processed for installed flow updates.
> > - */
> > -struct desired_flow {
> > -    struct ovn_flow flow;
> > -    struct hmap_node match_hmap_node; /* For match based hashing. */
> > -    struct ovs_list list_node; /* For handling lists of flows. */
> > -
> > -    /* A list of struct sb_flow_ref nodes, which references this flow. (There
> > -     * are cases that multiple SB entities share the same desired OpenFlow
> > -     * flow, e.g. when conjunction is used.) */
> > -    struct ovs_list references;
> > -
> > -    /* The corresponding flow in installed table. */
> > -    struct installed_flow *installed_flow;
> > -
> > -    /* Node in installed_flow.desired_refs list. */
> > -    struct ovs_list installed_ref_list_node;
> > -
> > -    /* For tracking. */
> > -    struct ovs_list track_list_node; /* node in ovn_desired_flow_table's
> > -                                      * tracked_flows list. */
> > -    bool is_deleted; /* If the tracked flow is deleted. */
> > -};
> > -
> > -struct sb_to_flow {
> > -    struct hmap_node hmap_node; /* Node in
> > -                                   ovn_desired_flow_table.uuid_flow_table. */
> > -    struct uuid sb_uuid;
> > -    struct ovs_list flows; /* A list of struct sb_flow_ref nodes that
> > -                                      are referenced by the sb_uuid. */
> > -};
> > -
> > -struct sb_flow_ref {
> > -    struct ovs_list sb_list; /* List node in desired_flow.references. */
> > -    struct ovs_list flow_list; /* List node in sb_to_flow.desired_flows. */
> > -    struct desired_flow *flow;
> > -    struct uuid sb_uuid;
> > -};
> > -
> > -/* A installed flow, in static variable installed_flows.
> > - *
> > - * Installed flows are updated in ofctrl_put for maintaining the flow
> > - * installation to OVS. They are updated according to desired flows: either by
> > - * processing the tracked desired flow changes, or by comparing desired flows
> > - * with currently installed flows when tracked desired flows changes are not
> > - * available.
> > - *
> > - * In addition, when ofctrl state machine enters S_CLEAR, the installed flows
> > - * will be cleared. (This happens in initialization phase and also when
> > - * ovs-vswitchd is disconnected/reconnected).
> > - *
> > - * Links are maintained between installed flows and desired flows. The
> > - * relationship is 1 to N. A link is added when a flow addition is processed.
> > - * A link is removed when a flow deletion is processed, the desired flow
> > - * table is cleared, or the installed flow table is cleared.
> > - *
> > - * To ensure predictable behavior, the list of desired flows is maintained
> > - * partially sorted in the following way (from least restrictive to most
> > - * restrictive wrt. match):
> > - * - allow flows without action conjunction.
> > - * - drop flows without action conjunction.
> > - * - a single flow with action conjunction.
> > - *
> > - * The first desired_flow in the list is the active one, the one that is
> > - * actually installed.
> > - */
> > -struct installed_flow {
> > -    struct ovn_flow flow;
> > -    struct hmap_node match_hmap_node; /* For match based hashing. */
> > -
> > -    /* A list of desired ovn_flow nodes (linked by
> > -     * desired_flow.installed_ref_list_node), which reference this installed
> > -     * flow.  (There are cases that multiple desired flows reference the same
> > -     * installed flow, e.g. when there are conflict/duplicated ACLs that
> > -     * generates same match conditions). */
> > -    struct ovs_list desired_refs;
> > -};
> > -
> > -typedef bool
> > -(*desired_flow_match_cb)(const struct desired_flow *candidate,
> > -                         const void *arg);
> > -static struct desired_flow *desired_flow_alloc(
> > -    uint8_t table_id,
> > -    uint16_t priority,
> > -    uint64_t cookie,
> > -    const struct match *match,
> > -    const struct ofpbuf *actions);
> > -static struct desired_flow *desired_flow_lookup(
> > -    struct ovn_desired_flow_table *,
> > -    const struct ovn_flow *target);
> > -static struct desired_flow *desired_flow_lookup_check_uuid(
> > -    struct ovn_desired_flow_table *,
> > -    const struct ovn_flow *target,
> > -    const struct uuid *);
> > -static struct desired_flow *desired_flow_lookup_conjunctive(
> > -    struct ovn_desired_flow_table *,
> > -    const struct ovn_flow *target);
> > -static void desired_flow_destroy(struct desired_flow *);
> > -
> > -static struct installed_flow *installed_flow_lookup(
> > -    const struct ovn_flow *target);
> > -static void installed_flow_destroy(struct installed_flow *);
> > -static struct installed_flow *installed_flow_dup(struct desired_flow *);
> > -static struct desired_flow *installed_flow_get_active(struct installed_flow *);
> > -
> > -static uint32_t ovn_flow_match_hash(const struct ovn_flow *);
> > -static char *ovn_flow_to_string(const struct ovn_flow *);
> > -static void ovn_flow_log(const struct ovn_flow *, const char *action);
> > -
> >   /* OpenFlow connection to the switch. */
> >   static struct rconn *swconn;
> >
> > @@ -301,10 +111,6 @@ static ovs_be32 xid, xid2;
> >    * zero, to avoid unbounded buffering. */
> >   static struct rconn_packet_counter *tx_counter;
> >
> > -/* Flow table of "struct ovn_flow"s, that holds the flow table currently
> > - * installed in the switch. */
> > -static struct hmap installed_flows;
> > -
> >   /* A reference to the group_table. */
> >   static struct ovn_extend_table *groups;
> >
> > @@ -328,10 +134,6 @@ static struct ofpbuf *encode_group_mod(const struct ofputil_group_mod *);
> >
> >   static struct ofpbuf *encode_meter_mod(const struct ofputil_meter_mod *);
> >
> > -static void ovn_installed_flow_table_clear(void);
> > -static void ovn_installed_flow_table_destroy(void);
> > -
> > -
> >   static void ofctrl_recv(const struct ofp_header *, enum ofptype);
> >
> >   void
> > @@ -342,7 +144,6 @@ ofctrl_init(struct ovn_extend_table *group_table,
> >       swconn = rconn_create(inactivity_probe_interval, 0,
> >                             DSCP_DEFAULT, 1 << OFP15_VERSION);
> >       tx_counter = rconn_packet_counter_create();
> > -    hmap_init(&installed_flows);
> >       ovs_list_init(&flow_updates);
> >       ovn_init_symtab(&symtab);
> >       groups = group_table;
> > @@ -572,8 +373,7 @@ run_S_CLEAR_FLOWS(void)
> >       queue_msg(encode_group_mod(&gm));
> >       ofputil_uninit_group_mod(&gm);
> >
> > -    /* Clear installed_flows, to match the state of the switch. */
> > -    ovn_installed_flow_table_clear();
> > +    dp_flow_flush_all_oflows();
> >
> >       /* Clear existing groups, to match the state of the switch. */
> >       if (groups) {
> > @@ -758,7 +558,6 @@ void
> >   ofctrl_destroy(void)
> >   {
> >       rconn_destroy(swconn);
> > -    ovn_installed_flow_table_destroy();
> >       rconn_packet_counter_destroy(tx_counter);
> >       expr_symtab_destroy(&symtab);
> >       shash_destroy(&symtab);
> > @@ -804,759 +603,7 @@ ofctrl_recv(const struct ofp_header *oh, enum ofptype type)
> >       }
> >   }
> >
> > -static bool
> > -flow_action_has_drop(const struct ovn_flow *f)
> > -{
> > -    return f->ofpacts_len == 0;
> > -}
> > -
> > -static bool
> > -flow_action_has_conj(const struct ovn_flow *f)
> > -{
> > -    const struct ofpact *a = NULL;
> > -
> > -    OFPACT_FOR_EACH (a, f->ofpacts, f->ofpacts_len) {
> > -        if (a->type == OFPACT_CONJUNCTION) {
> > -            return true;
> > -        }
> > -    }
> > -    return false;
> > -}
> > -
> > -static bool
> > -flow_action_has_allow(const struct ovn_flow *f)
> > -{
> > -    return !flow_action_has_drop(f) && !flow_action_has_conj(f);
> > -}
> > -
> > -/* Returns true if flow 'a' is preferred over flow 'b'. */
> > -static bool
> > -flow_is_preferred(const struct ovn_flow *a, const struct ovn_flow *b)
> > -{
> > -    if (flow_action_has_allow(b)) {
> > -        return false;
> > -    }
> > -    if (flow_action_has_allow(a)) {
> > -        return true;
> > -    }
> > -    if (flow_action_has_drop(b)) {
> > -        return false;
> > -    }
> > -    if (flow_action_has_drop(a)) {
> > -        return true;
> > -    }
> > -
> > -    /* Flows 'a' and 'b' should never both have action conjunction. */
> > -    OVS_NOT_REACHED();
> > -}
> > -
> > -/* Adds the desired flow to the list of desired flows that have same match
> > - * conditions as the installed flow.
> > - *
> > - * It is caller's responsibility to make sure the link between the pair didn't
> > - * exist before.
> > - *
> > - * Returns true if the newly added desired flow is selected to be the active
> > - * one.
> > - */
> > -static bool
> > -link_installed_to_desired(struct installed_flow *i, struct desired_flow *d)
> > -{
> > -    struct desired_flow *f;
> > -
> > -    /* Find first 'f' such that 'd' is preferred over 'f'.  If no such desired
> > -     * flow exists then 'f' will point after the last element of the list.
> > -     */
> > -    LIST_FOR_EACH (f, installed_ref_list_node, &i->desired_refs) {
> > -        if (flow_is_preferred(&d->flow, &f->flow)) {
> > -            break;
> > -        }
> > -    }
> > -    ovs_list_insert(&f->installed_ref_list_node, &d->installed_ref_list_node);
> > -    d->installed_flow = i;
> > -    return installed_flow_get_active(i) == d;
> > -}
> > -
> > -/* Replaces 'old_desired' with 'new_desired' in the list of desired flows
> > - * that have same match conditions as the installed flow.
> > - */
> > -static void
> > -replace_installed_to_desired(struct installed_flow *i,
> > -                             struct desired_flow *old_desired,
> > -                             struct desired_flow *new_desired)
> > -{
> > -    ovs_assert(old_desired->installed_flow == i);
> > -    ovs_list_replace(&new_desired->installed_ref_list_node,
> > -                     &old_desired->installed_ref_list_node);
> > -    old_desired->installed_flow = NULL;
> > -    new_desired->installed_flow = i;
> > -}
> > -
> > -/* Removes the desired flow from the list of desired flows that have the same
> > - * match conditions as the installed flow.
> > - *
> > - * Returns true if the desired flow was the previously active flow.
> > - */
> > -static bool
> > -unlink_installed_to_desired(struct installed_flow *i, struct desired_flow *d)
> > -{
> > -    struct desired_flow *old_active = installed_flow_get_active(i);
> > -
> > -    ovs_assert(d && d->installed_flow == i);
> > -    ovs_list_remove(&d->installed_ref_list_node);
> > -    d->installed_flow = NULL;
> > -    return old_active == d;
> > -}
> > -
> > -static void
> > -unlink_all_refs_for_installed_flow(struct installed_flow *i)
> > -{
> > -    struct desired_flow *d, *next;
> > -    LIST_FOR_EACH_SAFE (d, next, installed_ref_list_node, &i->desired_refs) {
> > -        unlink_installed_to_desired(i, d);
> > -    }
> > -}
> > -
> > -static void
> > -track_flow_add_or_modify(struct ovn_desired_flow_table *flow_table,
> > -                         struct desired_flow *f)
> > -{
> > -    if (!flow_table->change_tracked) {
> > -        return;
> > -    }
> > -
> > -    /* If same node (flow adding/modifying) was tracked, remove it from
> > -     * tracking first. */
> > -    if (!ovs_list_is_empty(&f->track_list_node)) {
> > -        ovs_list_remove(&f->track_list_node);
> > -    }
> > -    f->is_deleted = false;
> > -    ovs_list_push_back(&flow_table->tracked_flows, &f->track_list_node);
> > -
> > -}
> > -
> > -static void
> > -track_flow_del(struct ovn_desired_flow_table *flow_table,
> > -               struct desired_flow *f)
> > -{
> > -    if (!flow_table->change_tracked) {
> > -        return;
> > -    }
> > -    /* If same node (flow adding/modifying) was tracked, remove it from
> > -     * tracking first. */
> > -    if (!ovs_list_is_empty(&f->track_list_node)) {
> > -        ovs_list_remove(&f->track_list_node);
> > -        if (!f->installed_flow) {
> > -            /* If it is not installed yet, simply destroy it. */
> > -            desired_flow_destroy(f);
> > -            return;
> > -        }
> > -    }
> > -    f->is_deleted = true;
> > -    ovs_list_push_back(&flow_table->tracked_flows, &f->track_list_node);
> > -}
> > -
> > -/* When a desired flow is being removed, depending on "change_tracked", this
> > - * function either unlinks a desired flow from installed flow and destroy it,
> > - * or do nothing but track it. */
> > -static void
> > -track_or_destroy_for_flow_del(struct ovn_desired_flow_table *flow_table,
> > -                              struct desired_flow *f)
> > -{
> > -    if (flow_table->change_tracked) {
> > -        track_flow_del(flow_table, f);
> > -    } else {
> > -        if (f->installed_flow) {
> > -            unlink_installed_to_desired(f->installed_flow, f);
> > -        }
> > -        desired_flow_destroy(f);
> > -    }
> > -}
> > -
> > -static struct sb_to_flow *
> > -sb_to_flow_find(struct hmap *uuid_flow_table, const struct uuid *sb_uuid)
> > -{
> > -    struct sb_to_flow *stf;
> > -    HMAP_FOR_EACH_WITH_HASH (stf, hmap_node, uuid_hash(sb_uuid),
> > -                            uuid_flow_table) {
> > -        if (uuid_equals(sb_uuid, &stf->sb_uuid)) {
> > -            return stf;
> > -        }
> > -    }
> > -    return NULL;
> > -}
> > -
> > -static void
> > -link_flow_to_sb(struct ovn_desired_flow_table *flow_table,
> > -                struct desired_flow *f, const struct uuid *sb_uuid)
> > -{
> > -    struct sb_flow_ref *sfr = xmalloc(sizeof *sfr);
> > -    sfr->flow = f;
> > -    sfr->sb_uuid = *sb_uuid;
> > -    ovs_list_insert(&f->references, &sfr->sb_list);
> > -    struct sb_to_flow *stf = sb_to_flow_find(&flow_table->uuid_flow_table,
> > -                                             sb_uuid);
> > -    if (!stf) {
> > -        stf = xmalloc(sizeof *stf);
> > -        stf->sb_uuid = *sb_uuid;
> > -        ovs_list_init(&stf->flows);
> > -        hmap_insert(&flow_table->uuid_flow_table, &stf->hmap_node,
> > -                    uuid_hash(sb_uuid));
> > -    }
> > -    ovs_list_insert(&stf->flows, &sfr->flow_list);
> > -}
> > -
> > -/* Flow table interfaces to the rest of ovn-controller. */
> > -
> > -/* Adds a flow to 'desired_flows' with the specified 'match' and 'actions' to
> > - * the OpenFlow table numbered 'table_id' with the given 'priority' and
> > - * OpenFlow 'cookie'.  The caller retains ownership of 'match' and 'actions'.
> > - *
> > - * The flow is also linked to the sb_uuid that generates it.
> > - *
> > - * This just assembles the desired flow table in memory.  Nothing is actually
> > - * sent to the switch until a later call to ofctrl_put().
> > - *
> > - * The caller should initialize its own hmap to hold the flows. */
> > -void
> > -ofctrl_check_and_add_flow(struct ovn_desired_flow_table *flow_table,
> > -                          uint8_t table_id, uint16_t priority,
> > -                          uint64_t cookie, const struct match *match,
> > -                          const struct ofpbuf *actions,
> > -                          const struct uuid *sb_uuid,
> > -                          bool log_duplicate_flow)
> > -{
> > -    struct desired_flow *f = desired_flow_alloc(table_id, priority, cookie,
> > -                                                match, actions);
> > -
> > -    if (desired_flow_lookup_check_uuid(flow_table, &f->flow, sb_uuid)) {
> > -        if (log_duplicate_flow) {
> > -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5);
> > -            if (!VLOG_DROP_DBG(&rl)) {
> > -                char *s = ovn_flow_to_string(&f->flow);
> > -                VLOG_DBG("dropping duplicate flow: %s", s);
> > -                free(s);
> > -            }
> > -        }
> > -        desired_flow_destroy(f);
> > -        return;
> > -    }
> > -
> > -    hmap_insert(&flow_table->match_flow_table, &f->match_hmap_node,
> > -                f->flow.hash);
> > -    link_flow_to_sb(flow_table, f, sb_uuid);
> > -    track_flow_add_or_modify(flow_table, f);
> > -    ovn_flow_log(&f->flow, "ofctrl_add_flow");
> > -}
> > -
> > -void
> > -ofctrl_add_flow(struct ovn_desired_flow_table *desired_flows,
> > -                uint8_t table_id, uint16_t priority, uint64_t cookie,
> > -                const struct match *match, const struct ofpbuf *actions,
> > -                const struct uuid *sb_uuid)
> > -{
> > -    ofctrl_check_and_add_flow(desired_flows, table_id, priority, cookie,
> > -                              match, actions, sb_uuid, true);
> > -}
> > -
> > -/* Either add a new flow, or append actions on an existing flow. If the
> > - * flow existed, a new link will also be created between the new sb_uuid
> > - * and the existing flow. */
> > -void
> > -ofctrl_add_or_append_flow(struct ovn_desired_flow_table *desired_flows,
> > -                          uint8_t table_id, uint16_t priority, uint64_t cookie,
> > -                          const struct match *match,
> > -                          const struct ofpbuf *actions,
> > -                          const struct uuid *sb_uuid)
> > -{
> > -    struct desired_flow *existing;
> > -    struct desired_flow *f;
> > -
> > -    f = desired_flow_alloc(table_id, priority, cookie, match, actions);
> > -    existing = desired_flow_lookup_conjunctive(desired_flows, &f->flow);
> > -    if (existing) {
> > -        /* There's already a flow with this particular match and action
> > -         * 'conjunction'. Append the action to that flow rather than
> > -         * adding a new flow.
> > -         */
> > -        uint64_t compound_stub[64 / 8];
> > -        struct ofpbuf compound;
> > -        ofpbuf_use_stub(&compound, compound_stub, sizeof(compound_stub));
> > -        ofpbuf_put(&compound, existing->flow.ofpacts,
> > -                   existing->flow.ofpacts_len);
> > -        ofpbuf_put(&compound, f->flow.ofpacts, f->flow.ofpacts_len);
> > -
> > -        free(existing->flow.ofpacts);
> > -        existing->flow.ofpacts = xmemdup(compound.data, compound.size);
> > -        existing->flow.ofpacts_len = compound.size;
> > -
> > -        ofpbuf_uninit(&compound);
> > -        desired_flow_destroy(f);
> > -        f = existing;
> > -    } else {
> > -        hmap_insert(&desired_flows->match_flow_table, &f->match_hmap_node,
> > -                    f->flow.hash);
> > -    }
> > -    link_flow_to_sb(desired_flows, f, sb_uuid);
> > -    track_flow_add_or_modify(desired_flows, f);
> > -
> > -    if (existing) {
> > -        ovn_flow_log(&f->flow, "ofctrl_add_or_append_flow (append)");
> > -    } else {
> > -        ovn_flow_log(&f->flow, "ofctrl_add_or_append_flow (add)");
> > -    }
> > -}
> > -
> > -/* Remove ovn_flows for the given "sb_to_flow" node in the uuid_flow_table.
> > - * Optionally log the message for each flow that is acturally removed, if
> > - * log_msg is not NULL. */
> > -static void
> > -remove_flows_from_sb_to_flow(struct ovn_desired_flow_table *flow_table,
> > -                             struct sb_to_flow *stf,
> > -                             const char *log_msg)
> > -{
> > -    struct sb_flow_ref *sfr, *next;
> > -    LIST_FOR_EACH_SAFE (sfr, next, flow_list, &stf->flows) {
> > -        ovs_list_remove(&sfr->sb_list);
> > -        ovs_list_remove(&sfr->flow_list);
> > -        struct desired_flow *f = sfr->flow;
> > -        free(sfr);
> > -
> > -        if (ovs_list_is_empty(&f->references)) {
> > -            if (log_msg) {
> > -                ovn_flow_log(&f->flow, log_msg);
> > -            }
> > -            hmap_remove(&flow_table->match_flow_table,
> > -                        &f->match_hmap_node);
> > -            track_or_destroy_for_flow_del(flow_table, f);
> > -        }
> > -    }
> > -    hmap_remove(&flow_table->uuid_flow_table, &stf->hmap_node);
> > -    free(stf);
> > -}
> > -
> > -void
> > -ofctrl_remove_flows(struct ovn_desired_flow_table *flow_table,
> > -                    const struct uuid *sb_uuid)
> > -{
> > -    struct sb_to_flow *stf = sb_to_flow_find(&flow_table->uuid_flow_table,
> > -                                             sb_uuid);
> > -    if (stf) {
> > -        remove_flows_from_sb_to_flow(flow_table, stf, "ofctrl_remove_flow");
> > -    }
> > -
> > -    /* remove any related group and meter info */
> > -    ovn_extend_table_remove_desired(groups, sb_uuid);
> > -    ovn_extend_table_remove_desired(meters, sb_uuid);
> > -}
> > -
> > -static struct ofctrl_flood_remove_node *
> > -flood_remove_find_node(struct hmap *flood_remove_nodes, struct uuid *sb_uuid)
> > -{
> > -    struct ofctrl_flood_remove_node *ofrn;
> > -    HMAP_FOR_EACH_WITH_HASH (ofrn, hmap_node, uuid_hash(sb_uuid),
> > -                             flood_remove_nodes) {
> > -        if (uuid_equals(&ofrn->sb_uuid, sb_uuid)) {
> > -            return ofrn;
> > -        }
> > -    }
> > -    return NULL;
> > -}
> > -
> > -void
> > -ofctrl_flood_remove_add_node(struct hmap *flood_remove_nodes,
> > -                             const struct uuid *sb_uuid)
> > -{
> > -    struct ofctrl_flood_remove_node *ofrn = xmalloc(sizeof *ofrn);
> > -    ofrn->sb_uuid = *sb_uuid;
> > -    hmap_insert(flood_remove_nodes, &ofrn->hmap_node, uuid_hash(sb_uuid));
> > -}
> > -
> > -static void
> > -flood_remove_flows_for_sb_uuid(struct ovn_desired_flow_table *flow_table,
> > -                               const struct uuid *sb_uuid,
> > -                               struct hmap *flood_remove_nodes)
> > -{
> > -    struct sb_to_flow *stf = sb_to_flow_find(&flow_table->uuid_flow_table,
> > -                                             sb_uuid);
> > -    if (!stf) {
> > -        return;
> > -    }
> > -
> > -    /* ovn_flows that have other references and waiting to be removed. */
> > -    struct ovs_list to_be_removed = OVS_LIST_INITIALIZER(&to_be_removed);
> > -
> > -    /* Traverse all flows for the given sb_uuid. */
> > -    struct sb_flow_ref *sfr, *next;
> > -    LIST_FOR_EACH_SAFE (sfr, next, flow_list, &stf->flows) {
> > -        struct desired_flow *f = sfr->flow;
> > -        ovn_flow_log(&f->flow, "flood remove");
> > -
> > -        ovs_list_remove(&sfr->sb_list);
> > -        ovs_list_remove(&sfr->flow_list);
> > -        free(sfr);
> > -
> > -        ovs_assert(ovs_list_is_empty(&f->list_node));
> > -        if (ovs_list_is_empty(&f->references)) {
> > -            /* This is to optimize the case when most flows have only
> > -             * one referencing sb_uuid, so to_be_removed list should
> > -             * be empty in most cases. */
> > -            hmap_remove(&flow_table->match_flow_table,
> > -                        &f->match_hmap_node);
> > -            track_or_destroy_for_flow_del(flow_table, f);
> > -        } else {
> > -            ovs_list_insert(&to_be_removed, &f->list_node);
> > -        }
> > -    }
> > -    hmap_remove(&flow_table->uuid_flow_table, &stf->hmap_node);
> > -    free(stf);
> > -
> > -    /* Traverse other referencing sb_uuids for the flows in the to_be_removed
> > -     * list. */
> > -
> > -    /* Detach the items in f->references from the sfr.flow_list lists,
> > -     * so that recursive calls will not mess up the sfr.sb_list list. */
> > -    struct desired_flow *f, *f_next;
> > -    LIST_FOR_EACH (f, list_node, &to_be_removed) {
> > -        ovs_assert(!ovs_list_is_empty(&f->references));
> > -        LIST_FOR_EACH (sfr, sb_list, &f->references) {
> > -            ovs_list_remove(&sfr->flow_list);
> > -        }
> > -    }
> > -    LIST_FOR_EACH_SAFE (f, f_next, list_node, &to_be_removed) {
> > -        LIST_FOR_EACH_SAFE (sfr, next, sb_list, &f->references) {
> > -            if (!flood_remove_find_node(flood_remove_nodes, &sfr->sb_uuid)) {
> > -                ofctrl_flood_remove_add_node(flood_remove_nodes,
> > -                                             &sfr->sb_uuid);
> > -                flood_remove_flows_for_sb_uuid(flow_table, &sfr->sb_uuid,
> > -                                               flood_remove_nodes);
> > -            }
> > -            ovs_list_remove(&sfr->sb_list);
> > -            free(sfr);
> > -        }
> > -        ovs_list_remove(&f->list_node);
> > -        hmap_remove(&flow_table->match_flow_table,
> > -                    &f->match_hmap_node);
> > -        track_or_destroy_for_flow_del(flow_table, f);
> > -    }
> > -
> > -}
> > -
> > -void
> > -ofctrl_flood_remove_flows(struct ovn_desired_flow_table *flow_table,
> > -                          struct hmap *flood_remove_nodes)
> > -{
> > -    struct ofctrl_flood_remove_node *ofrn;
> > -    int i, n = 0;
> > -
> > -    /* flood_remove_flows_for_sb_uuid() will modify the 'flood_remove_nodes'
> > -     * hash map by inserting new items, so we can't use it for iteration.
> > -     * Copying the sb_uuids into an array. */
> > -    struct uuid *sb_uuids;
> > -    sb_uuids = xmalloc(hmap_count(flood_remove_nodes) * sizeof *sb_uuids);
> > -    struct hmap flood_remove_uuids = HMAP_INITIALIZER(&flood_remove_uuids);
> > -    HMAP_FOR_EACH (ofrn, hmap_node, flood_remove_nodes) {
> > -        sb_uuids[n++] = ofrn->sb_uuid;
> > -    }
> > -
> > -    for (i = 0; i < n; i++) {
> > -        flood_remove_flows_for_sb_uuid(flow_table, &sb_uuids[i],
> > -                                       flood_remove_nodes);
> > -    }
> > -    free(sb_uuids);
> > -
> > -    /* remove any related group and meter info */
> > -    HMAP_FOR_EACH (ofrn, hmap_node, flood_remove_nodes) {
> > -        ovn_extend_table_remove_desired(groups, &ofrn->sb_uuid);
> > -        ovn_extend_table_remove_desired(meters, &ofrn->sb_uuid);
> > -    }
> > -}
> > -
> > -/* flow operations. */
> > -
> > -static void
> > -ovn_flow_init(struct ovn_flow *f, uint8_t table_id, uint16_t priority,
> > -              uint64_t cookie, const struct match *match,
> > -              const struct ofpbuf *actions)
> > -{
> > -    f->table_id = table_id;
> > -    f->priority = priority;
> > -    minimatch_init(&f->match, match);
> > -    f->ofpacts = xmemdup(actions->data, actions->size);
> > -    f->ofpacts_len = actions->size;
> > -    f->hash = ovn_flow_match_hash(f);
> > -    f->cookie = cookie;
> > -}
> > -
> > -static struct desired_flow *
> > -desired_flow_alloc(uint8_t table_id, uint16_t priority, uint64_t cookie,
> > -                   const struct match *match, const struct ofpbuf *actions)
> > -{
> > -    struct desired_flow *f = xmalloc(sizeof *f);
> > -    ovs_list_init(&f->references);
> > -    ovs_list_init(&f->list_node);
> > -    ovs_list_init(&f->installed_ref_list_node);
> > -    ovs_list_init(&f->track_list_node);
> > -    f->installed_flow = NULL;
> > -    f->is_deleted = false;
> > -    ovn_flow_init(&f->flow, table_id, priority, cookie, match, actions);
> > -
> > -    return f;
> > -}
> > -
> > -/* Returns a hash of the match key in 'f'. */
> > -static uint32_t
> > -ovn_flow_match_hash(const struct ovn_flow *f)
> > -{
> > -    return hash_2words((f->table_id << 16) | f->priority,
> > -                       minimatch_hash(&f->match, 0));
> > -}
> > -
> > -/* Duplicate a desired flow to an installed flow. */
> > -static struct installed_flow *
> > -installed_flow_dup(struct desired_flow *src)
> > -{
> > -    struct installed_flow *dst = xmalloc(sizeof *dst);
> > -    ovs_list_init(&dst->desired_refs);
> > -    dst->flow.table_id = src->flow.table_id;
> > -    dst->flow.priority = src->flow.priority;
> > -    minimatch_clone(&dst->flow.match, &src->flow.match);
> > -    dst->flow.ofpacts = xmemdup(src->flow.ofpacts, src->flow.ofpacts_len);
> > -    dst->flow.ofpacts_len = src->flow.ofpacts_len;
> > -    dst->flow.hash = src->flow.hash;
> > -    dst->flow.cookie = src->flow.cookie;
> > -    return dst;
> > -}
> > -
> > -static struct desired_flow *
> > -installed_flow_get_active(struct installed_flow *f)
> > -{
> > -    if (!ovs_list_is_empty(&f->desired_refs)) {
> > -        return CONTAINER_OF(ovs_list_front(&f->desired_refs),
> > -                            struct desired_flow,
> > -                            installed_ref_list_node);
> > -    }
> > -    return NULL;
> > -}
> > -
> > -static struct desired_flow *
> > -desired_flow_lookup__(struct ovn_desired_flow_table *flow_table,
> > -                      const struct ovn_flow *target,
> > -                      desired_flow_match_cb match_cb,
> > -                      const void *arg)
> > -{
> > -    struct desired_flow *d;
> > -    HMAP_FOR_EACH_WITH_HASH (d, match_hmap_node, target->hash,
> > -                             &flow_table->match_flow_table) {
> > -        struct ovn_flow *f = &d->flow;
> > -        if (f->table_id == target->table_id
> > -            && f->priority == target->priority
> > -            && minimatch_equal(&f->match, &target->match)) {
> > -
> > -            if (!match_cb || match_cb(d, arg)) {
> > -                return d;
> > -            }
> > -        }
> > -    }
> > -    return NULL;
> > -}
> > -
> > -/* Finds and returns a desired_flow in 'flow_table' whose key is identical to
> > - * 'target''s key, or NULL if there is none.
> > - */
> > -static struct desired_flow *
> > -desired_flow_lookup(struct ovn_desired_flow_table *flow_table,
> > -                    const struct ovn_flow *target)
> > -{
> > -    return desired_flow_lookup__(flow_table, target, NULL, NULL);
> > -}
> > -
> > -static bool
> > -flow_lookup_match_uuid_cb(const struct desired_flow *candidate,
> > -                          const void *arg)
> > -{
> > -    const struct uuid *sb_uuid = arg;
> > -    struct sb_flow_ref *sfr;
> > -
> > -    LIST_FOR_EACH (sfr, sb_list, &candidate->references) {
> > -        if (uuid_equals(sb_uuid, &sfr->sb_uuid)) {
> > -            return true;
> > -        }
> > -    }
> > -    return false;
> > -}
> > -
> > -/* Finds and returns a desired_flow in 'flow_table' whose key is identical to
> > - * 'target''s key, or NULL if there is none.
> > - *
> > - * The function will also check if the found flow is referenced by the
> > - * 'sb_uuid'.
> > - */
> > -static struct desired_flow *
> > -desired_flow_lookup_check_uuid(struct ovn_desired_flow_table *flow_table,
> > -                            const struct ovn_flow *target,
> > -                            const struct uuid *sb_uuid)
> > -{
> > -    return desired_flow_lookup__(flow_table, target, flow_lookup_match_uuid_cb,
> > -                                 sb_uuid);
> > -}
> > -
> > -static bool
> > -flow_lookup_match_conj_cb(const struct desired_flow *candidate,
> > -                          const void *arg OVS_UNUSED)
> > -{
> > -    return flow_action_has_conj(&candidate->flow);
> > -}
> > -
> > -/* Finds and returns a desired_flow in 'flow_table' whose key is identical to
> > - * 'target''s key, or NULL if there is none.
> > - *
> > - * The function will only return a matching flow if it contains action
> > - * 'conjunction'.
> > - */
> > -static struct desired_flow *
> > -desired_flow_lookup_conjunctive(struct ovn_desired_flow_table *flow_table,
> > -                                const struct ovn_flow *target)
> > -{
> > -    return desired_flow_lookup__(flow_table, target, flow_lookup_match_conj_cb,
> > -                                 NULL);
> > -}
> > -
> > -/* Finds and returns an installed_flow in installed_flows whose key is
> > - * identical to 'target''s key, or NULL if there is none. */
> > -static struct installed_flow *
> > -installed_flow_lookup(const struct ovn_flow *target)
> > -{
> > -    struct installed_flow *i;
> > -    HMAP_FOR_EACH_WITH_HASH (i, match_hmap_node, target->hash,
> > -                             &installed_flows) {
> > -        struct ovn_flow *f = &i->flow;
> > -        if (f->table_id == target->table_id
> > -            && f->priority == target->priority
> > -            && minimatch_equal(&f->match, &target->match)) {
> > -            return i;
> > -        }
> > -    }
> > -    return NULL;
> > -}
> > -
> > -static char *
> > -ovn_flow_to_string(const struct ovn_flow *f)
> > -{
> > -    struct ds s = DS_EMPTY_INITIALIZER;
> > -
> > -    ds_put_format(&s, "cookie=%"PRIx64", ", f->cookie);
> > -    ds_put_format(&s, "table_id=%"PRIu8", ", f->table_id);
> > -    ds_put_format(&s, "priority=%"PRIu16", ", f->priority);
> > -    minimatch_format(&f->match, NULL, NULL, &s, OFP_DEFAULT_PRIORITY);
> > -    ds_put_cstr(&s, ", actions=");
> > -    struct ofpact_format_params fp = { .s = &s };
> > -    ofpacts_format(f->ofpacts, f->ofpacts_len, &fp);
> > -    return ds_steal_cstr(&s);
> > -}
> > -
> > -static void
> > -ovn_flow_log(const struct ovn_flow *f, const char *action)
> > -{
> > -    if (VLOG_IS_DBG_ENABLED()) {
> > -        char *s = ovn_flow_to_string(f);
> > -        VLOG_DBG("%s flow: %s", action, s);
> > -        free(s);
> > -    }
> > -}
> > -
> > -static void
> > -ovn_flow_uninit(struct ovn_flow *f)
> > -{
> > -    minimatch_destroy(&f->match);
> > -    free(f->ofpacts);
> > -}
> > -
> > -static void
> > -desired_flow_destroy(struct desired_flow *f)
> > -{
> > -    if (f) {
> > -        ovs_assert(ovs_list_is_empty(&f->references));
> > -        ovs_assert(!f->installed_flow);
> > -        ovn_flow_uninit(&f->flow);
> > -        free(f);
> > -    }
> > -}
> > -
> > -static void
> > -installed_flow_destroy(struct installed_flow *f)
> > -{
> > -    if (f) {
> > -        ovs_assert(!installed_flow_get_active(f));
> > -        ovn_flow_uninit(&f->flow);
> > -        free(f);
> > -    }
> > -}
> >
> > -/* Desired flow table operations. */
> > -void
> > -ovn_desired_flow_table_init(struct ovn_desired_flow_table *flow_table)
> > -{
> > -    hmap_init(&flow_table->match_flow_table);
> > -    hmap_init(&flow_table->uuid_flow_table);
> > -    ovs_list_init(&flow_table->tracked_flows);
> > -    flow_table->change_tracked = false;
> > -}
> > -
> > -void
> > -ovn_desired_flow_table_clear(struct ovn_desired_flow_table *flow_table)
> > -{
> > -    flow_table->change_tracked = false;
> > -
> > -    struct desired_flow *f, *f_next;
> > -    LIST_FOR_EACH_SAFE (f, f_next, track_list_node,
> > -                        &flow_table->tracked_flows) {
> > -        ovs_list_remove(&f->track_list_node);
> > -        if (f->is_deleted) {
> > -            if (f->installed_flow) {
> > -                unlink_installed_to_desired(f->installed_flow, f);
> > -            }
> > -            desired_flow_destroy(f);
> > -        }
> > -    }
> > -
> > -    struct sb_to_flow *stf, *next;
> > -    HMAP_FOR_EACH_SAFE (stf, next, hmap_node,
> > -                        &flow_table->uuid_flow_table) {
> > -        remove_flows_from_sb_to_flow(flow_table, stf, NULL);
> > -    }
> > -}
> > -
> > -void
> > -ovn_desired_flow_table_destroy(struct ovn_desired_flow_table *flow_table)
> > -{
> > -    ovn_desired_flow_table_clear(flow_table);
> > -    hmap_destroy(&flow_table->match_flow_table);
> > -    hmap_destroy(&flow_table->uuid_flow_table);
> > -}
> > -
> > -
> > -/* Installed flow table operations. */
> > -static void
> > -ovn_installed_flow_table_clear(void)
> > -{
> > -    struct installed_flow *f, *next;
> > -    HMAP_FOR_EACH_SAFE (f, next, match_hmap_node, &installed_flows) {
> > -        hmap_remove(&installed_flows, &f->match_hmap_node);
> > -        unlink_all_refs_for_installed_flow(f);
> > -        installed_flow_destroy(f);
> > -    }
> > -}
> > -
> > -static void
> > -ovn_installed_flow_table_destroy(void)
> > -{
> > -    ovn_installed_flow_table_clear();
> > -    hmap_destroy(&installed_flows);
> > -}
> > -
> > -/* Flow table update. */
> > -
> >   static struct ofpbuf *
> >   encode_flow_mod(struct ofputil_flow_mod *fm)
> >   {
> > @@ -1566,13 +613,6 @@ encode_flow_mod(struct ofputil_flow_mod *fm)
> >       return ofputil_encode_flow_mod(fm, OFPUTIL_P_OF15_OXM);
> >   }
> >
> > -static void
> > -add_flow_mod(struct ofputil_flow_mod *fm, struct ovs_list *msgs)
> > -{
> > -    struct ofpbuf *msg = encode_flow_mod(fm);
> > -    ovs_list_push_back(msgs, &msg->list_node);
> > -}
> > -
> >   /* group_table. */
> >
> >   static struct ofpbuf *
> > @@ -1751,263 +791,6 @@ add_meter(struct ovn_extend_table_info *m_desired,
> >       free(mm.meter.bands);
> >   }
> >
> > -static void
> > -installed_flow_add(struct ovn_flow *d, struct ovs_list *msgs)
> > -{
> > -    /* Send flow_mod to add flow. */
> > -    struct ofputil_flow_mod fm = {
> > -        .match = d->match,
> > -        .priority = d->priority,
> > -        .table_id = d->table_id,
> > -        .ofpacts = d->ofpacts,
> > -        .ofpacts_len = d->ofpacts_len,
> > -        .new_cookie = htonll(d->cookie),
> > -        .command = OFPFC_ADD,
> > -    };
> > -    add_flow_mod(&fm, msgs);
> > -}
> > -
> > -static void
> > -installed_flow_mod(struct ovn_flow *i, struct ovn_flow *d,
> > -                   struct ovs_list *msgs)
> > -{
> > -    /* Update actions in installed flow. */
> > -    struct ofputil_flow_mod fm = {
> > -        .match = i->match,
> > -        .priority = i->priority,
> > -        .table_id = i->table_id,
> > -        .ofpacts = d->ofpacts,
> > -        .ofpacts_len = d->ofpacts_len,
> > -        .command = OFPFC_MODIFY_STRICT,
> > -    };
> > -    /* Update cookie if it is changed. */
> > -    if (i->cookie != d->cookie) {
> > -        fm.modify_cookie = true;
> > -        fm.new_cookie = htonll(d->cookie);
> > -        /* Use OFPFC_ADD so that cookie can be updated. */
> > -        fm.command = OFPFC_ADD;
> > -    }
> > -    add_flow_mod(&fm, msgs);
> > -
> > -    /* Replace 'i''s actions and cookie by 'd''s. */
> > -    free(i->ofpacts);
> > -    i->ofpacts = xmemdup(d->ofpacts, d->ofpacts_len);
> > -    i->ofpacts_len = d->ofpacts_len;
> > -    i->cookie = d->cookie;
> > -}
> > -
> > -static void
> > -installed_flow_del(struct ovn_flow *i, struct ovs_list *msgs)
> > -{
> > -    struct ofputil_flow_mod fm = {
> > -        .match = i->match,
> > -        .priority = i->priority,
> > -        .table_id = i->table_id,
> > -        .command = OFPFC_DELETE_STRICT,
> > -    };
> > -    add_flow_mod(&fm, msgs);
> > -}
> > -
> > -static void
> > -update_installed_flows_by_compare(struct ovn_desired_flow_table *flow_table,
> > -                                  struct ovs_list *msgs)
> > -{
> > -    ovs_assert(ovs_list_is_empty(&flow_table->tracked_flows));
> > -    /* Iterate through all of the installed flows.  If any of them are no
> > -     * longer desired, delete them; if any of them should have different
> > -     * actions, update them. */
> > -    struct installed_flow *i, *next;
> > -    HMAP_FOR_EACH_SAFE (i, next, match_hmap_node, &installed_flows) {
> > -        unlink_all_refs_for_installed_flow(i);
> > -        struct desired_flow *d = desired_flow_lookup(flow_table, &i->flow);
> > -        if (!d) {
> > -            /* Installed flow is no longer desirable.  Delete it from the
> > -             * switch and from installed_flows. */
> > -            installed_flow_del(&i->flow, msgs);
> > -            ovn_flow_log(&i->flow, "removing installed");
> > -
> > -            hmap_remove(&installed_flows, &i->match_hmap_node);
> > -            installed_flow_destroy(i);
> > -        } else {
> > -            if (!ofpacts_equal(i->flow.ofpacts, i->flow.ofpacts_len,
> > -                               d->flow.ofpacts, d->flow.ofpacts_len) ||
> > -                i->flow.cookie != d->flow.cookie) {
> > -                installed_flow_mod(&i->flow, &d->flow, msgs);
> > -                ovn_flow_log(&i->flow, "updating installed");
> > -            }
> > -            link_installed_to_desired(i, d);
> > -
> > -        }
> > -    }
> > -
> > -    /* Iterate through the desired flows and add those that aren't found
> > -     * in the installed flow table. */
> > -    struct desired_flow *d;
> > -    HMAP_FOR_EACH (d, match_hmap_node, &flow_table->match_flow_table) {
> > -        i = installed_flow_lookup(&d->flow);
> > -        if (!i) {
> > -            ovn_flow_log(&d->flow, "adding installed");
> > -            installed_flow_add(&d->flow, msgs);
> > -
> > -            /* Copy 'd' from 'flow_table' to installed_flows. */
> > -            i = installed_flow_dup(d);
> > -            hmap_insert(&installed_flows, &i->match_hmap_node, i->flow.hash);
> > -            link_installed_to_desired(i, d);
> > -        } else if (!d->installed_flow) {
> > -            /* This is a desired_flow that conflicts with one installed
> > -             * previously but not linked yet.  However, if this flow becomes
> > -             * active, e.g., it is less restrictive than the previous active
> > -             * flow then modify the installed flow.
> > -             */
> > -            if (link_installed_to_desired(i, d)) {
> > -                installed_flow_mod(&i->flow, &d->flow, msgs);
> > -                ovn_flow_log(&i->flow, "updating installed (conflict)");
> > -            }
> > -        }
> > -    }
> > -}
> > -
> > -/* Finds and returns a desired_flow in 'deleted_flows' that is exactly the
> > - * same as 'target', including cookie and actions.
> > - */
> > -static struct desired_flow *
> > -deleted_flow_lookup(struct hmap *deleted_flows, struct ovn_flow *target)
> > -{
> > -    struct desired_flow *d;
> > -    HMAP_FOR_EACH_WITH_HASH (d, match_hmap_node, target->hash,
> > -                             deleted_flows) {
> > -        struct ovn_flow *f = &d->flow;
> > -        if (f->table_id == target->table_id
> > -            && f->priority == target->priority
> > -            && minimatch_equal(&f->match, &target->match)
> > -            && f->cookie == target->cookie
> > -            && ofpacts_equal(f->ofpacts, f->ofpacts_len, target->ofpacts,
> > -                             target->ofpacts_len)) {
> > -            return d;
> > -        }
> > -    }
> > -    return NULL;
> > -}
> > -
> > -/* This function scans the tracked flow changes in the order and merges "add"
> > - * or "update" after "deleted" operations for exactly same flow (priority,
> > - * table, match, action and cookie), to avoid unnecessary OF messages being
> > - * sent to OVS. */
> > -static void
> > -merge_tracked_flows(struct ovn_desired_flow_table *flow_table)
> > -{
> > -    struct hmap deleted_flows = HMAP_INITIALIZER(&deleted_flows);
> > -    struct desired_flow *f, *next;
> > -    LIST_FOR_EACH_SAFE (f, next, track_list_node,
> > -                        &flow_table->tracked_flows) {
> > -        if (f->is_deleted) {
> > -            /* reuse f->match_hmap_node field since it is already removed from
> > -             * the desired flow table's match index. */
> > -            hmap_insert(&deleted_flows, &f->match_hmap_node,
> > -                        f->flow.hash);
> > -        } else {
> > -            struct desired_flow *del_f = deleted_flow_lookup(&deleted_flows,
> > -                                                             &f->flow);
> > -            if (!del_f) {
> > -                continue;
> > -            }
> > -
> > -            /* del_f must have been installed, otherwise it should have been
> > -             * removed during track_flow_add_or_modify. */
> > -            ovs_assert(del_f->installed_flow);
> > -
> > -            if (!f->installed_flow) {
> > -                /* f is not installed yet. */
> > -                replace_installed_to_desired(del_f->installed_flow, del_f, f);
> > -            } else {
> > -                /* f has been installed before, and now was updated to exact
> > -                 * the same flow as del_f. */
> > -                ovs_assert(f->installed_flow == del_f->installed_flow);
> > -                unlink_installed_to_desired(del_f->installed_flow, del_f);
> > -            }
> > -            hmap_remove(&deleted_flows, &del_f->match_hmap_node);
> > -            ovs_list_remove(&del_f->track_list_node);
> > -            desired_flow_destroy(del_f);
> > -
> > -            ovs_list_remove(&f->track_list_node);
> > -            ovs_list_init(&f->track_list_node);
> > -        }
> > -    }
> > -    HMAP_FOR_EACH_SAFE (f, next, match_hmap_node, &deleted_flows) {
> > -        hmap_remove(&deleted_flows, &f->match_hmap_node);
> > -    }
> > -    hmap_destroy(&deleted_flows);
> > -}
> > -
> > -static void
> > -update_installed_flows_by_track(struct ovn_desired_flow_table *flow_table,
> > -                                struct ovs_list *msgs)
> > -{
> > -    merge_tracked_flows(flow_table);
> > -    struct desired_flow *f, *f_next;
> > -    LIST_FOR_EACH_SAFE (f, f_next, track_list_node,
> > -                        &flow_table->tracked_flows) {
> > -        ovs_list_remove(&f->track_list_node);
> > -        if (f->is_deleted) {
> > -            /* The desired flow was deleted */
> > -            if (f->installed_flow) {
> > -                struct installed_flow *i = f->installed_flow;
> > -                bool was_active = unlink_installed_to_desired(i, f);
> > -                struct desired_flow *d = installed_flow_get_active(i);
> > -
> > -                if (!d) {
> > -                    installed_flow_del(&i->flow, msgs);
> > -                    ovn_flow_log(&i->flow, "removing installed (tracked)");
> > -
> > -                    hmap_remove(&installed_flows, &i->match_hmap_node);
> > -                    installed_flow_destroy(i);
> > -                } else if (was_active) {
> > -                    /* There are other desired flow(s) referencing this
> > -                     * installed flow, so update the OVS flow for the new
> > -                     * active flow (at least the cookie will be different,
> > -                     * even if the actions are the same). */
> > -                    installed_flow_mod(&i->flow, &d->flow, msgs);
> > -                    ovn_flow_log(&i->flow, "updating installed (tracked)");
> > -                }
> > -            }
> > -            desired_flow_destroy(f);
> > -        } else {
> > -            /* The desired flow was added or modified. */
> > -            struct installed_flow *i = installed_flow_lookup(&f->flow);
> > -            if (!i) {
> > -                /* Adding a new flow. */
> > -                installed_flow_add(&f->flow, msgs);
> > -                ovn_flow_log(&f->flow, "adding installed (tracked)");
> > -
> > -                /* Copy 'f' from 'flow_table' to installed_flows. */
> > -                struct installed_flow *new_node = installed_flow_dup(f);
> > -                hmap_insert(&installed_flows, &new_node->match_hmap_node,
> > -                            new_node->flow.hash);
> > -                link_installed_to_desired(new_node, f);
> > -            } else if (installed_flow_get_active(i) == f) {
> > -                /* The installed flow is installed for f, but f has change
> > -                 * tracked, so it must have been modified. */
> > -                installed_flow_mod(&i->flow, &f->flow, msgs);
> > -                ovn_flow_log(&i->flow, "updating installed (tracked)");
> > -            } else if (!f->installed_flow) {
> > -                /* Adding a new flow that conflicts with an existing installed
> > -                 * flow, so add it to the link.  If this flow becomes active,
> > -                 * e.g., it is less restrictive than the previous active flow
> > -                 * then modify the installed flow.
> > -                 */
> > -                if (link_installed_to_desired(i, f)) {
> > -                    installed_flow_mod(&i->flow, &f->flow, msgs);
> > -                    ovn_flow_log(&i->flow,
> > -                                 "updating installed (tracked conflict)");
> > -                }
> > -            }
> > -            /* The track_list_node emptyness is used to check if the node is
> > -             * already added to track list, so initialize it again here. */
> > -            ovs_list_init(&f->track_list_node);
> > -        }
> > -    }
> > -}
> > -
> >   /* The flow table can be updated if the connection to the switch is up and
> >    * in the correct state and not backlogged with existing flow_mods.  (Our
> >    * criteria for being backlogged appear very conservative, but the socket
> > @@ -2035,8 +818,7 @@ ofctrl_can_put(void)
> >    *
> >    * This should be called after ofctrl_run() within the main loop. */
> >   void
> > -ofctrl_put(struct ovn_desired_flow_table *flow_table,
> > -           struct shash *pending_ct_zones,
> > +ofctrl_put(struct shash *pending_ct_zones,
> >              const struct sbrec_meter_table *meter_table,
> >              uint64_t req_cfg,
> >              bool flow_changed)
> > @@ -2126,11 +908,7 @@ ofctrl_put(struct ovn_desired_flow_table *flow_table,
> >           }
> >       }
> >
> > -    if (flow_table->change_tracked) {
> > -        update_installed_flows_by_track(flow_table, &msgs);
> > -    } else {
> > -        update_installed_flows_by_compare(flow_table, &msgs);
> > -    }
> > +    dp_flow_populate_oflow_msgs(&msgs);
> >
> >       /* Iterate through the installed groups from previous runs. If they
> >        * are not needed delete them. */
> > @@ -2242,9 +1020,6 @@ ofctrl_put(struct ovn_desired_flow_table *flow_table,
> >           /* We were completely up-to-date before and still are. */
> >           cur_cfg = req_cfg;
> >       }
> > -
> > -    flow_table->change_tracked = true;
> > -    ovs_assert(ovs_list_is_empty(&flow_table->tracked_flows));
> >   }
> >
> >   /* Looks up the logical port with the name 'port_name' in 'br_int_'.  If
> > diff --git a/controller/ofctrl.h b/controller/ofctrl.h
> > index 88769566ac..0073227a3a 100644
> > --- a/controller/ofctrl.h
> > +++ b/controller/ofctrl.h
> > @@ -31,20 +31,6 @@ struct ovsrec_bridge;
> >   struct sbrec_meter_table;
> >   struct shash;
> >
> > -struct ovn_desired_flow_table {
> > -    /* Hash map flow table using flow match conditions as hash key.*/
> > -    struct hmap match_flow_table;
> > -
> > -    /* SB uuid index for the cross reference nodes that link to the nodes in
> > -     * match_flow_table.*/
> > -    struct hmap uuid_flow_table;
> > -
> > -    /* Is flow changes tracked. */
> > -    bool change_tracked;
> > -    /* Tracked flow changes. */
> > -    struct ovs_list tracked_flows;
> > -};
> > -
> >   /* Interface for OVN main loop. */
> >   void ofctrl_init(struct ovn_extend_table *group_table,
> >                    struct ovn_extend_table *meter_table,
> > @@ -52,8 +38,7 @@ void ofctrl_init(struct ovn_extend_table *group_table,
> >   void ofctrl_run(const struct ovsrec_bridge *br_int,
> >                   struct shash *pending_ct_zones);
> >   enum mf_field_id ofctrl_get_mf_field_id(void);
> > -void ofctrl_put(struct ovn_desired_flow_table *,
> > -                struct shash *pending_ct_zones,
> > +void ofctrl_put(struct shash *pending_ct_zones,
> >                   const struct sbrec_meter_table *,
> >                   uint64_t nb_cfg,
> >                   bool flow_changed);
> > @@ -68,53 +53,6 @@ char *ofctrl_inject_pkt(const struct ovsrec_bridge *br_int,
> >                           const char *flow_s, const struct shash *addr_sets,
> >                           const struct shash *port_groups);
> >
> > -/* Flow table interfaces to the rest of ovn-controller. */
> > -void ofctrl_add_flow(struct ovn_desired_flow_table *, uint8_t table_id,
> > -                     uint16_t priority, uint64_t cookie,
> > -                     const struct match *, const struct ofpbuf *ofpacts,
> > -                     const struct uuid *);
> > -
> > -void ofctrl_add_or_append_flow(struct ovn_desired_flow_table *desired_flows,
> > -                               uint8_t table_id, uint16_t priority,
> > -                               uint64_t cookie, const struct match *match,
> > -                               const struct ofpbuf *actions,
> > -                               const struct uuid *sb_uuid);
> > -
> > -/* Removes a bundles of flows from the flow table for a specific sb_uuid. The
> > - * flows are removed only if they are not referenced by any other sb_uuid(s).
> > - * For flood-removing all related flows referenced by other sb_uuid(s), use
> > - * ofctrl_flood_remove_flows(). */
> > -void ofctrl_remove_flows(struct ovn_desired_flow_table *,
> > -                         const struct uuid *sb_uuid);
> > -
> > -/* The function ofctrl_flood_remove_flows flood-removes flows from the desired
> > - * flow table for the sb_uuids provided in the flood_remove_nodes argument.
> > - * For each given sb_uuid in flood_remove_nodes, it removes all the flows
> > - * generated by the sb_uuid, and if any of the flows are referenced by another
> > - * sb_uuid, it continues removing all the flows used by that sb_uuid as well,
> > - * and so on, recursively.
> > - *
> > - * It adds all the sb_uuids that are actually removed in the
> > - * flood_remove_nodes. */
> > -struct ofctrl_flood_remove_node {
> > -    struct hmap_node hmap_node;
> > -    struct uuid sb_uuid;
> > -};
> > -void ofctrl_flood_remove_flows(struct ovn_desired_flow_table *,
> > -                               struct hmap *flood_remove_nodes);
> > -void ofctrl_flood_remove_add_node(struct hmap *flood_remove_nodes,
> > -                                  const struct uuid *sb_uuid);
> > -void ovn_desired_flow_table_init(struct ovn_desired_flow_table *);
> > -void ovn_desired_flow_table_clear(struct ovn_desired_flow_table *);
> > -void ovn_desired_flow_table_destroy(struct ovn_desired_flow_table *);
> > -
> > -void ofctrl_check_and_add_flow(struct ovn_desired_flow_table *,
> > -                               uint8_t table_id, uint16_t priority,
> > -                               uint64_t cookie, const struct match *,
> > -                               const struct ofpbuf *ofpacts,
> > -                               const struct uuid *, bool log_duplicate_flow);
> > -
> > -
> >   bool ofctrl_is_connected(void);
> >   void ofctrl_set_probe_interval(int probe_interval);
> >
> > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> > index 5dd643f52b..1dd3730353 100644
> > --- a/controller/ovn-controller.c
> > +++ b/controller/ovn-controller.c
> > @@ -30,6 +30,7 @@
> >   #include "compiler.h"
> >   #include "daemon.h"
> >   #include "dirs.h"
> > +#include "dp-flow-mgr.h"
> >   #include "openvswitch/dynamic-string.h"
> >   #include "encaps.h"
> >   #include "fatal-signal.h"
> > @@ -1745,8 +1746,6 @@ struct flow_output_persistent_data {
> >   };
> >
> >   struct ed_type_flow_output {
> > -    /* desired flows */
> > -    struct ovn_desired_flow_table flow_table;
> >       /* group ids for load balancing */
> >       struct ovn_extend_table group_table;
> >       /* meter ids for QoS */
> > @@ -1930,7 +1929,6 @@ static void init_lflow_ctx(struct engine_node *node,
> >       l_ctx_in->active_tunnels = &rt_data->active_tunnels;
> >       l_ctx_in->local_lport_ids = &rt_data->local_lport_ids;
> >
> > -    l_ctx_out->flow_table = &fo->flow_table;
> >       l_ctx_out->group_table = &fo->group_table;
> >       l_ctx_out->meter_table = &fo->meter_table;
> >       l_ctx_out->lfrr = &fo->lflow_resource_ref;
> > @@ -1945,7 +1943,6 @@ en_flow_output_init(struct engine_node *node OVS_UNUSED,
> >   {
> >       struct ed_type_flow_output *data = xzalloc(sizeof *data);
> >
> > -    ovn_desired_flow_table_init(&data->flow_table);
> >       ovn_extend_table_init(&data->group_table);
> >       ovn_extend_table_init(&data->meter_table);
> >       data->pd.conj_id_ofs = 1;
> > @@ -1957,7 +1954,6 @@ static void
> >   en_flow_output_cleanup(void *data)
> >   {
> >       struct ed_type_flow_output *flow_output_data = data;
> > -    ovn_desired_flow_table_destroy(&flow_output_data->flow_table);
> >       ovn_extend_table_destroy(&flow_output_data->group_table);
> >       ovn_extend_table_destroy(&flow_output_data->meter_table);
> >       lflow_resource_destroy(&flow_output_data->lflow_resource_ref);
> > @@ -1992,7 +1988,6 @@ en_flow_output_run(struct engine_node *node, void *data)
> >       ovs_assert(br_int && chassis);
> >
> >       struct ed_type_flow_output *fo = data;
> > -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> >       struct ovn_extend_table *group_table = &fo->group_table;
> >       struct ovn_extend_table *meter_table = &fo->meter_table;
> >       struct lflow_resource_ref *lfrr = &fo->lflow_resource_ref;
> > @@ -2001,7 +1996,6 @@ en_flow_output_run(struct engine_node *node, void *data)
> >       if (first_run) {
> >           first_run = false;
> >       } else {
> > -        ovn_desired_flow_table_clear(flow_table);
> >           ovn_extend_table_clear(group_table, false /* desired */);
> >           ovn_extend_table_clear(meter_table, false /* desired */);
> >           lflow_resource_clear(lfrr);
> > @@ -2018,18 +2012,19 @@ en_flow_output_run(struct engine_node *node, void *data)
> >       struct lflow_ctx_in l_ctx_in;
> >       struct lflow_ctx_out l_ctx_out;
> >       init_lflow_ctx(node, rt_data, fo, &l_ctx_in, &l_ctx_out);
> > +    dp_flow_switch_logical_oflow_tables();
> >       lflow_run(&l_ctx_in, &l_ctx_out);
> >
> >       if (l_ctx_out.conj_id_overflow) {
> >           /* Conjunction ids overflow. There can be many holes in between.
> >            * Destroy lflow cache and call lflow_run() again. */
> > -        ovn_desired_flow_table_clear(flow_table);
> >           ovn_extend_table_clear(group_table, false /* desired */);
> >           ovn_extend_table_clear(meter_table, false /* desired */);
> >           lflow_resource_clear(lfrr);
> >           fo->pd.conj_id_ofs = 1;
> >           lflow_cache_flush(fo->pd.lflow_cache);
> >           l_ctx_out.conj_id_overflow = false;
> > +        dp_flow_switch_logical_oflow_tables();
> >           lflow_run(&l_ctx_in, &l_ctx_out);
> >           if (l_ctx_out.conj_id_overflow) {
> >               VLOG_WARN("Conjunction id overflow.");
> > @@ -2039,7 +2034,8 @@ en_flow_output_run(struct engine_node *node, void *data)
> >       struct physical_ctx p_ctx;
> >       init_physical_ctx(node, rt_data, &p_ctx);
> >
> > -    physical_run(&p_ctx, &fo->flow_table);
> > +    dp_flow_switch_physical_oflow_tables();
> > +    physical_run(&p_ctx);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >   }
> > @@ -2070,7 +2066,8 @@ flow_output_sb_logical_flow_handler(struct engine_node *node, void *data)
> >   }
> >
> >   static bool
> > -flow_output_sb_mac_binding_handler(struct engine_node *node, void *data)
> > +flow_output_sb_mac_binding_handler(struct engine_node *node,
> > +                                   void *data OVS_UNUSED)
> >   {
> >       struct ovsdb_idl_index *sbrec_port_binding_by_name =
> >           engine_ovsdb_node_get_index(
> > @@ -2085,11 +2082,8 @@ flow_output_sb_mac_binding_handler(struct engine_node *node, void *data)
> >           engine_get_input_data("runtime_data", node);
> >       const struct hmap *local_datapaths = &rt_data->local_datapaths;
> >
> > -    struct ed_type_flow_output *fo = data;
> > -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> > -
> >       lflow_handle_changed_neighbors(sbrec_port_binding_by_name,
> > -            mac_binding_table, local_datapaths, flow_table);
> > +                                   mac_binding_table, local_datapaths);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >       return true;
> > @@ -2097,14 +2091,11 @@ flow_output_sb_mac_binding_handler(struct engine_node *node, void *data)
> >
> >   static bool
> >   flow_output_sb_port_binding_handler(struct engine_node *node,
> > -                                    void *data)
> > +                                    void *data OVS_UNUSED)
> >   {
> >       struct ed_type_runtime_data *rt_data =
> >           engine_get_input_data("runtime_data", node);
> >
> > -    struct ed_type_flow_output *fo = data;
> > -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> > -
> >       struct physical_ctx p_ctx;
> >       init_physical_ctx(node, rt_data, &p_ctx);
> >
> > @@ -2112,25 +2103,23 @@ flow_output_sb_port_binding_handler(struct engine_node *node,
> >        * only. flow_output runtime data handler takes care of processing
> >        * logical flows for any port binding changes.
> >        */
> > -    physical_handle_port_binding_changes(&p_ctx, flow_table);
> > +    physical_handle_port_binding_changes(&p_ctx);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >       return true;
> >   }
> >
> >   static bool
> > -flow_output_sb_multicast_group_handler(struct engine_node *node, void *data)
> > +flow_output_sb_multicast_group_handler(struct engine_node *node,
> > +                                       void *data OVS_UNUSED)
> >   {
> >       struct ed_type_runtime_data *rt_data =
> >           engine_get_input_data("runtime_data", node);
> >
> > -    struct ed_type_flow_output *fo = data;
> > -    struct ovn_desired_flow_table *flow_table = &fo->flow_table;
> > -
> >       struct physical_ctx p_ctx;
> >       init_physical_ctx(node, rt_data, &p_ctx);
> >
> > -    physical_handle_mc_group_changes(&p_ctx, flow_table);
> > +    physical_handle_mc_group_changes(&p_ctx);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >       return true;
> > @@ -2251,12 +2240,12 @@ flow_output_port_groups_handler(struct engine_node *node, void *data)
> >   }
> >
> >   static bool
> > -flow_output_physical_flow_changes_handler(struct engine_node *node, void *data)
> > +flow_output_physical_flow_changes_handler(struct engine_node *node,
> > +                                          void *data OVS_UNUSED)
> >   {
> >       struct ed_type_runtime_data *rt_data =
> >           engine_get_input_data("runtime_data", node);
> >
> > -    struct ed_type_flow_output *fo = data;
> >       struct physical_ctx p_ctx;
> >       init_physical_ctx(node, rt_data, &p_ctx);
> >
> > @@ -2264,22 +2253,19 @@ flow_output_physical_flow_changes_handler(struct engine_node *node, void *data)
> >       struct ed_type_pfc_data *pfc_data =
> >           engine_get_input_data("physical_flow_changes", node);
> >
> > -    /* If there are OVS interface changes. Try to handle them incrementally. */
> > -    if (pfc_data->ovs_ifaces_changed) {
> > -        if (!physical_handle_ovs_iface_changes(&p_ctx, &fo->flow_table)) {
> > -            return false;
> > -        }
> > -    }
> > -
> >       if (pfc_data->recompute_physical_flows) {
> >           /* This indicates that we need to recompute the physical flows. */
> > -        physical_clear_unassoc_flows_with_db(&fo->flow_table);
> > -        physical_clear_dp_flows(&p_ctx, &rt_data->ct_updated_datapaths,
> > -                                &fo->flow_table);
> > -        physical_run(&p_ctx, &fo->flow_table);
> > +        dp_flow_switch_physical_oflow_tables();
> > +        physical_run(&p_ctx);
> >           return true;
> >       }
> >
> > +    if (pfc_data->ovs_ifaces_changed) {
> > +        /* There are OVS interface changes. Try to handle them
> > +         * incrementally. */
> > +        return physical_handle_ovs_iface_changes(&p_ctx);
> > +    }
> > +
> >       return true;
> >   }
> >
> > @@ -2346,7 +2332,7 @@ flow_output_sb_load_balancer_handler(struct engine_node *node, void *data)
> >       struct lflow_ctx_out l_ctx_out;
> >       init_lflow_ctx(node, rt_data, fo, &l_ctx_in, &l_ctx_out);
> >
> > -    bool handled = lflow_handle_changed_lbs(&l_ctx_in, &l_ctx_out);
> > +    bool handled = lflow_handle_changed_lbs(&l_ctx_in);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >       return handled;
> > @@ -2363,7 +2349,7 @@ flow_output_sb_fdb_handler(struct engine_node *node, void *data)
> >       struct lflow_ctx_out l_ctx_out;
> >       init_lflow_ctx(node, rt_data, fo, &l_ctx_in, &l_ctx_out);
> >
> > -    bool handled = lflow_handle_changed_fdbs(&l_ctx_in, &l_ctx_out);
> > +    bool handled = lflow_handle_changed_fdbs(&l_ctx_in);
> >
> >       engine_set_node_state(node, EN_UPDATED);
> >       return handled;
> > @@ -2452,6 +2438,7 @@ main(int argc, char *argv[])
> >       patch_init();
> >       pinctrl_init();
> >       lflow_init();
> > +    dp_flow_tables_init();
> >
> >       /* Connect to OVS OVSDB instance. */
> >       struct ovsdb_idl_loop ovs_idl_loop = OVSDB_IDL_LOOP_INITIALIZER(
> > @@ -2960,8 +2947,7 @@ main(int argc, char *argv[])
> >
> >                       flow_output_data = engine_get_data(&en_flow_output);
> >                       if (flow_output_data && ct_zones_data) {
> > -                        ofctrl_put(&flow_output_data->flow_table,
> > -                                   &ct_zones_data->pending,
> > +                        ofctrl_put(&ct_zones_data->pending,
> >                                      sbrec_meter_table_get(ovnsb_idl_loop.idl),
> >                                      ofctrl_seqno_get_req_cfg(),
> >                                      engine_node_changed(&en_flow_output));
> > @@ -3132,6 +3118,7 @@ loop_done:
> >       free(ovn_version);
> >       unixctl_server_destroy(unixctl);
> >       lflow_destroy();
> > +    dp_flow_tables_destroy();
> >       ofctrl_destroy();
> >       pinctrl_destroy();
> >       patch_destroy();
> > diff --git a/controller/physical.c b/controller/physical.c
> > index fa5d0d692d..6d434b3f90 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -16,6 +16,7 @@
> >   #include <config.h>
> >   #include "binding.h"
> >   #include "byte-order.h"
> > +#include "dp-flow-mgr.h"
> >   #include "encaps.h"
> >   #include "flow.h"
> >   #include "ha-chassis.h"
> > @@ -248,8 +249,7 @@ put_remote_port_redirect_bridged(const struct
> >                                    const struct hmap *local_datapaths,
> >                                    struct local_datapath *ld,
> >                                    struct match *match,
> > -                                 struct ofpbuf *ofpacts_p,
> > -                                 struct ovn_desired_flow_table *flow_table)
> > +                                 struct ofpbuf *ofpacts_p)
> >   {
> >           if (strcmp(binding->type, "chassisredirect")) {
> >               /* bridged based redirect is only supported for chassisredirect
> > @@ -305,9 +305,10 @@ put_remote_port_redirect_bridged(const struct
> >                                &value, NULL);
> >
> >           put_resubmit(OFTABLE_LOG_TO_PHY, ofpacts_p);
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100,
> > -                        binding->header_.uuid.parts[0],
> > -                        match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(binding->datapath->tunnel_key,
> > +                                   OFTABLE_LOCAL_OUTPUT, 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   match, ofpacts_p, &binding->header_.uuid);
> >
> >   }
> >
> > @@ -320,8 +321,7 @@ put_remote_port_redirect_overlay(const struct
> >                                    const struct chassis_tunnel *tun,
> >                                    uint32_t port_key,
> >                                    struct match *match,
> > -                                 struct ofpbuf *ofpacts_p,
> > -                                 struct ovn_desired_flow_table *flow_table)
> > +                                 struct ofpbuf *ofpacts_p)
> >   {
> >       if (!is_ha_remote) {
> >           /* Setup encapsulation */
> > @@ -403,9 +403,10 @@ put_remote_port_redirect_overlay(const struct
> >           bundle->fields = NX_HASH_FIELDS_ETH_SRC;
> >           ofpact_finish_BUNDLE(ofpacts_p, &bundle);
> >       }
> > -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100,
> > -                    binding->header_.uuid.parts[0],
> > -                    match, ofpacts_p, &binding->header_.uuid);
> > +    dp_flow_add_physical_oflow(binding->datapath->tunnel_key,
> > +                               OFTABLE_REMOTE_OUTPUT, 100,
> > +                               binding->header_.uuid.parts[0],
> > +                               match, ofpacts_p, &binding->header_.uuid);
> >   }
> >
> >
> > @@ -481,8 +482,7 @@ free_remote_chassis_macs(void)
> >   static void
> >   put_chassis_mac_conj_id_flow(const struct sbrec_chassis_table *chassis_table,
> >                                const struct sbrec_chassis *chassis,
> > -                             struct ofpbuf *ofpacts_p,
> > -                             struct ovn_desired_flow_table *flow_table)
> > +                             struct ofpbuf *ofpacts_p)
> >   {
> >       struct match match;
> >       struct remote_chassis_mac *mac;
> > @@ -510,9 +510,10 @@ put_chassis_mac_conj_id_flow(const struct sbrec_chassis_table *chassis_table,
> >           conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID;
> >           conj->n_clauses = 2;
> >           conj->clause = 0;
> > -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
> > -                        mac->chassis_sb_cookie,
> > -                        &match, ofpacts_p, hc_uuid);
> > +        dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                   OFTABLE_PHY_TO_LOG, 180,
> > +                                   mac->chassis_sb_cookie,
> > +                                   &match, ofpacts_p, hc_uuid);
> >       }
> >
> >       free_remote_chassis_macs();
> > @@ -524,8 +525,7 @@ put_replace_chassis_mac_flows(const struct simap *ct_zones,
> >                                 sbrec_port_binding *localnet_port,
> >                                 const struct hmap *local_datapaths,
> >                                 struct ofpbuf *ofpacts_p,
> > -                              ofp_port_t ofport,
> > -                              struct ovn_desired_flow_table *flow_table)
> > +                              ofp_port_t ofport)
> >   {
> >       /* Packets arriving on localnet port, could have been routed on
> >        * source chassis and hence will have a chassis mac.
> > @@ -583,9 +583,10 @@ put_replace_chassis_mac_flows(const struct simap *ct_zones,
> >
> >           /* Resubmit to first logical ingress pipeline table. */
> >           put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
> > -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
> > -                        rport_binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, hc_uuid);
> > +        dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                   OFTABLE_PHY_TO_LOG, 180,
> > +                                   rport_binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, hc_uuid);
> >
> >           /* Provide second search criteria, i.e localnet port's
> >            * vlan ID for conjunction flow */
> > @@ -603,9 +604,10 @@ put_replace_chassis_mac_flows(const struct simap *ct_zones,
> >           conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID;
> >           conj->n_clauses = 2;
> >           conj->clause = 1;
> > -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
> > -                        rport_binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, hc_uuid);
> > +        dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                   OFTABLE_PHY_TO_LOG, 180,
> > +                                   rport_binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, hc_uuid);
> >       }
> >   }
> >
> > @@ -618,8 +620,7 @@ put_replace_router_port_mac_flows(struct ovsdb_idl_index
> >                                     const struct sset *active_tunnels,
> >                                     const struct hmap *local_datapaths,
> >                                     struct ofpbuf *ofpacts_p,
> > -                                  ofp_port_t ofport,
> > -                                  struct ovn_desired_flow_table *flow_table)
> > +                                  ofp_port_t ofport)
> >   {
> >       struct local_datapath *ld = get_local_datapath(local_datapaths,
> >                                                      localnet_port->datapath->
> > @@ -704,9 +705,10 @@ put_replace_router_port_mac_flows(struct ovsdb_idl_index
> >
> >           ofpact_put_OUTPUT(ofpacts_p)->port = ofport;
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 150,
> > -                        localnet_port->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &localnet_port->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOG_TO_PHY, 150,
> > +                                   localnet_port->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p,
> > +                                   &localnet_port->header_.uuid);
> >       }
> >   }
> >
> > @@ -714,8 +716,7 @@ static void
> >   put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> >                          uint32_t parent_port_key,
> >                          const struct zone_ids *zone_ids,
> > -                       struct ofpbuf *ofpacts_p,
> > -                       struct ovn_desired_flow_table *flow_table)
> > +                       struct ofpbuf *ofpacts_p)
> >   {
> >       struct match match;
> >
> > @@ -748,8 +749,9 @@ put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> >
> >       /* Resubmit to table 34. */
> >       put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> > -    ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100, 0,
> > -                    &match, ofpacts_p, hc_uuid);
> > +    dp_flow_add_physical_oflow(dp_key,
> > +                               OFTABLE_LOCAL_OUTPUT, 100, hc_uuid->parts[0],
> > +                               &match, ofpacts_p, hc_uuid);
> >
> >       /* Table 34, Priority 100.
> >        * =======================
> > @@ -763,8 +765,9 @@ put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> >                            0, MLF_ALLOW_LOOPBACK);
> >       match_set_reg(&match, MFF_LOG_INPORT - MFF_REG0, port_key);
> >       match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
> > -    ofctrl_add_flow(flow_table, OFTABLE_CHECK_LOOPBACK, 100, 0,
> > -                    &match, ofpacts_p, hc_uuid);
> > +    dp_flow_add_physical_oflow(dp_key,
> > +                               OFTABLE_CHECK_LOOPBACK, 100, hc_uuid->parts[0],
> > +                               &match, ofpacts_p, hc_uuid);
> >
> >       /* Table 64, Priority 100.
> >        * =======================
> > @@ -801,25 +804,15 @@ put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> >       put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> >       put_resubmit(OFTABLE_LOG_TO_PHY, ofpacts_p);
> >       put_stack(MFF_IN_PORT, ofpact_put_STACK_POP(ofpacts_p));
> > -    ofctrl_add_flow(flow_table, OFTABLE_SAVE_INPORT, 100, 0,
> > -                    &match, ofpacts_p, hc_uuid);
> > +    dp_flow_add_physical_oflow(dp_key, OFTABLE_SAVE_INPORT,
> > +                               100, hc_uuid->parts[0], &match, ofpacts_p,
> > +                               hc_uuid);
> >
> >       if (nested_container) {
> >           /* It's a nested container and when the packet from the nested
> >            * container is to be sent to the parent port, "nested_container"
> >            * flag will be set. We need to temporarily set the in_port to
> >            * OFPP_NONE as mentioned in the comment above.
> > -         *
> > -         * If a parent port has multiple child ports, then this if condition
> > -         * will be hit multiple times, but we want to add only one flow.
> > -         * ofctrl_add_flow() logs a warning message for duplicate flows.
> > -         * So use the function 'ofctrl_check_and_add_flow' which doesn't
> > -         * log a warning.
> > -         *
> > -         * Other option is to add this flow for all the ports which are not
> > -         * nested containers. In which case we will add this flow for all the
> > -         * ports even if they don't have any child ports which is
> > -         * unnecessary.
> >            */
> >           match_init_catchall(&match);
> >           ofpbuf_clear(ofpacts_p);
> > @@ -832,8 +825,9 @@ put_local_common_flows(uint32_t dp_key, uint32_t port_key,
> >           put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> >           put_resubmit(OFTABLE_LOG_TO_PHY, ofpacts_p);
> >           put_stack(MFF_IN_PORT, ofpact_put_STACK_POP(ofpacts_p));
> > -        ofctrl_check_and_add_flow(flow_table, OFTABLE_SAVE_INPORT, 100, 0,
> > -                                  &match, ofpacts_p, hc_uuid, false);
> > +        dp_flow_add_physical_oflow(dp_key,
> > +                                   OFTABLE_SAVE_INPORT, 100, hc_uuid->parts[0],
> > +                                   &match, ofpacts_p, hc_uuid);
> >       }
> >   }
> >
> > @@ -891,7 +885,6 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                         const struct hmap *local_datapaths,
> >                         const struct sbrec_port_binding *binding,
> >                         const struct sbrec_chassis *chassis,
> > -                      struct ovn_desired_flow_table *flow_table,
> >                         struct ofpbuf *ofpacts_p)
> >   {
> >       uint32_t dp_key = binding->datapath->tunnel_key;
> > @@ -914,7 +907,7 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >
> >           struct zone_ids binding_zones = get_zone_ids(binding, ct_zones);
> >           put_local_common_flows(dp_key, port_key, 0, &binding_zones,
> > -                               ofpacts_p, flow_table);
> > +                               ofpacts_p);
> >
> >           match_init_catchall(&match);
> >           ofpbuf_clear(ofpacts_p);
> > @@ -939,9 +932,9 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >           ofpacts_p->header = clone;
> >           ofpact_finish_CLONE(ofpacts_p, &clone);
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 100,
> > -                        binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOG_TO_PHY, 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, &binding->header_.uuid);
> >           return;
> >       }
> >
> > @@ -1009,9 +1002,9 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >           }
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100,
> > -                        binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOCAL_OUTPUT, 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, &binding->header_.uuid);
> >
> >           goto out;
> >       }
> > @@ -1117,7 +1110,7 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >           /* Pass the parent port tunnel key if the port is a nested
> >            * container. */
> >           put_local_common_flows(dp_key, port_key, parent_port_key, &zone_ids,
> > -                               ofpacts_p, flow_table);
> > +                               ofpacts_p);
> >
> >           /* Table 0, Priority 150 and 100.
> >            * ==============================
> > @@ -1162,9 +1155,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >
> >           /* Resubmit to first logical ingress pipeline table. */
> >           put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
> > -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG,
> > -                        tag ? 150 : 100, binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_PHY_TO_LOG,
> > +                                   tag ? 150 : 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, &binding->header_.uuid);
> >
> >           if (!tag && (!strcmp(binding->type, "localnet")
> >                        || !strcmp(binding->type, "l2gateway"))) {
> > @@ -1174,14 +1168,14 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                * action. */
> >               ofpbuf_pull(ofpacts_p, ofpacts_orig_size);
> >               match_set_dl_tci_masked(&match, 0, htons(VLAN_CFI));
> > -            ofctrl_add_flow(flow_table, 0, 100,
> > -                            binding->header_.uuid.parts[0], &match, ofpacts_p,
> > -                            &binding->header_.uuid);
> > +            dp_flow_add_physical_oflow(dp_key, 0, 100,
> > +                                       binding->header_.uuid.parts[0], &match,
> > +                                       ofpacts_p, &binding->header_.uuid);
> >           }
> >
> >           if (!strcmp(binding->type, "localnet")) {
> >               put_replace_chassis_mac_flows(ct_zones, binding, local_datapaths,
> > -                                          ofpacts_p, ofport, flow_table);
> > +                                          ofpacts_p, ofport);
> >           }
> >
> >           /* Table 65, Priority 100.
> > @@ -1208,15 +1202,15 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                * switch will also contain the tag. */
> >               ofpact_put_STRIP_VLAN(ofpacts_p);
> >           }
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 100,
> > -                        binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOG_TO_PHY, 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, &binding->header_.uuid);
> >
> >           if (!strcmp(binding->type, "localnet")) {
> >               put_replace_router_port_mac_flows(sbrec_port_binding_by_name,
> >                                                 binding, chassis, active_tunnels,
> >                                                 local_datapaths, ofpacts_p,
> > -                                              ofport, flow_table);
> > +                                              ofport);
> >           }
> >
> >       } else if (!tun && !is_ha_remote) {
> > @@ -1240,9 +1234,9 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >
> >           /* Resubmit to table 33. */
> >           put_resubmit(OFTABLE_LOCAL_OUTPUT, ofpacts_p);
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100,
> > -                        binding->header_.uuid.parts[0],
> > -                        &match, ofpacts_p, &binding->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOCAL_OUTPUT, 100,
> > +                                   binding->header_.uuid.parts[0],
> > +                                   &match, ofpacts_p, &binding->header_.uuid);
> >       } else {
> >
> >           const char *redirect_type = smap_get(&binding->options,
> > @@ -1266,13 +1260,11 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >
> >           if (redirect_type && !strcasecmp(redirect_type, "bridged")) {
> >               put_remote_port_redirect_bridged(binding, local_datapaths,
> > -                                             ld, &match, ofpacts_p,
> > -                                             flow_table);
> > +                                             ld, &match, ofpacts_p);
> >           } else {
> >               put_remote_port_redirect_overlay(binding, is_ha_remote,
> >                                                ha_ch_ordered, mff_ovn_geneve,
> > -                                             tun, port_key, &match, ofpacts_p,
> > -                                             flow_table);
> > +                                             tun, port_key, &match, ofpacts_p);
> >           }
> >       }
> >   out:
> > @@ -1286,8 +1278,7 @@ consider_mc_group(enum mf_field_id mff_ovn_geneve,
> >                     const struct simap *ct_zones,
> >                     const struct hmap *local_datapaths,
> >                     const struct sbrec_chassis *chassis,
> > -                  const struct sbrec_multicast_group *mc,
> > -                  struct ovn_desired_flow_table *flow_table)
> > +                  const struct sbrec_multicast_group *mc)
> >   {
> >       uint32_t dp_key = mc->datapath->tunnel_key;
> >       if (!get_local_datapath(local_datapaths, dp_key)) {
> > @@ -1364,9 +1355,9 @@ consider_mc_group(enum mf_field_id mff_ovn_geneve,
> >            * group as the logical output port. */
> >           put_load(mc->tunnel_key, MFF_LOG_OUTPORT, 0, 32, &ofpacts);
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_LOCAL_OUTPUT, 100,
> > -                        mc->header_.uuid.parts[0],
> > -                        &match, &ofpacts, &mc->header_.uuid);
> > +        dp_flow_add_physical_oflow(dp_key, OFTABLE_LOCAL_OUTPUT, 100,
> > +                                   mc->header_.uuid.parts[0],
> > +                                   &match, &ofpacts, &mc->header_.uuid);
> >       }
> >
> >       /* Table 32, priority 100.
> > @@ -1403,9 +1394,10 @@ consider_mc_group(enum mf_field_id mff_ovn_geneve,
> >               if (local_ports) {
> >                   put_resubmit(OFTABLE_LOCAL_OUTPUT, &remote_ofpacts);
> >               }
> > -            ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100,
> > -                            mc->header_.uuid.parts[0],
> > -                            &match, &remote_ofpacts, &mc->header_.uuid);
> > +            dp_flow_add_physical_oflow(dp_key, OFTABLE_REMOTE_OUTPUT, 100,
> > +                                       mc->header_.uuid.parts[0],
> > +                                       &match, &remote_ofpacts,
> > +                                       &mc->header_.uuid);
> >           }
> >       }
> >       ofpbuf_uninit(&ofpacts);
> > @@ -1425,8 +1417,7 @@ update_ofports(struct simap *old, struct simap *new)
> >   }
> >
> >   void
> > -physical_handle_port_binding_changes(struct physical_ctx *p_ctx,
> > -                                     struct ovn_desired_flow_table *flow_table)
> > +physical_handle_port_binding_changes(struct physical_ctx *p_ctx)
> >   {
> >       const struct sbrec_port_binding *binding;
> >       struct ofpbuf ofpacts;
> > @@ -1434,44 +1425,46 @@ physical_handle_port_binding_changes(struct physical_ctx *p_ctx,
> >       SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (binding,
> >                                                  p_ctx->port_binding_table) {
> >           if (sbrec_port_binding_is_deleted(binding)) {
> > -            ofctrl_remove_flows(flow_table, &binding->header_.uuid);
> > +            dp_flow_remove_physical_oflows(binding->datapath->tunnel_key,
> > +                                           &binding->header_.uuid);
> >           } else {
> >               if (!sbrec_port_binding_is_new(binding)) {
> > -                ofctrl_remove_flows(flow_table, &binding->header_.uuid);
> > +                dp_flow_remove_physical_oflows(binding->datapath->tunnel_key,
> > +                                               &binding->header_.uuid);
> >               }
> >               consider_port_binding(p_ctx->sbrec_port_binding_by_name,
> >                                     p_ctx->mff_ovn_geneve, p_ctx->ct_zones,
> >                                     p_ctx->active_tunnels,
> >                                     p_ctx->local_datapaths,
> >                                     binding, p_ctx->chassis,
> > -                                  flow_table, &ofpacts);
> > +                                  &ofpacts);
> >           }
> >       }
> >       ofpbuf_uninit(&ofpacts);
> >   }
> >
> >   void
> > -physical_handle_mc_group_changes(struct physical_ctx *p_ctx,
> > -                                 struct ovn_desired_flow_table *flow_table)
> > +physical_handle_mc_group_changes(struct physical_ctx *p_ctx)
> >   {
> >       const struct sbrec_multicast_group *mc;
> >       SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_TRACKED (mc, p_ctx->mc_group_table) {
> >           if (sbrec_multicast_group_is_deleted(mc)) {
> > -            ofctrl_remove_flows(flow_table, &mc->header_.uuid);
> > +            dp_flow_remove_physical_oflows(mc->datapath->tunnel_key,
> > +                                           &mc->header_.uuid);
> >           } else {
> >               if (!sbrec_multicast_group_is_new(mc)) {
> > -                ofctrl_remove_flows(flow_table, &mc->header_.uuid);
> > +                dp_flow_remove_physical_oflows(mc->datapath->tunnel_key,
> > +                                               &mc->header_.uuid);
> >               }
> >               consider_mc_group(p_ctx->mff_ovn_geneve, p_ctx->ct_zones,
> >                                 p_ctx->local_datapaths,
> > -                              p_ctx->chassis, mc, flow_table);
> > +                              p_ctx->chassis, mc);
> >           }
> >       }
> >   }
> >
> >   void
> > -physical_run(struct physical_ctx *p_ctx,
> > -             struct ovn_desired_flow_table *flow_table)
> > +physical_run(struct physical_ctx *p_ctx)
> >   {
> >       if (!hc_uuid) {
> >           hc_uuid = xmalloc(sizeof(struct uuid));
> > @@ -1611,7 +1604,7 @@ physical_run(struct physical_ctx *p_ctx,
> >       ofpbuf_init(&ofpacts, 0);
> >
> >       put_chassis_mac_conj_id_flow(p_ctx->chassis_table, p_ctx->chassis,
> > -                                 &ofpacts, flow_table);
> > +                                 &ofpacts);
> >
> >       /* Set up flows in table 0 for physical-to-logical translation and in table
> >        * 64 for logical-to-physical translation. */
> > @@ -1620,16 +1613,14 @@ physical_run(struct physical_ctx *p_ctx,
> >           consider_port_binding(p_ctx->sbrec_port_binding_by_name,
> >                                 p_ctx->mff_ovn_geneve, p_ctx->ct_zones,
> >                                 p_ctx->active_tunnels, p_ctx->local_datapaths,
> > -                              binding, p_ctx->chassis,
> > -                              flow_table, &ofpacts);
> > +                              binding, p_ctx->chassis, &ofpacts);
> >       }
> >
> >       /* Handle output to multicast groups, in tables 32 and 33. */
> >       const struct sbrec_multicast_group *mc;
> >       SBREC_MULTICAST_GROUP_TABLE_FOR_EACH (mc, p_ctx->mc_group_table) {
> >           consider_mc_group(p_ctx->mff_ovn_geneve, p_ctx->ct_zones,
> > -                          p_ctx->local_datapaths, p_ctx->chassis,
> > -                          mc, flow_table);
> > +                          p_ctx->local_datapaths, p_ctx->chassis, mc);
> >       }
> >
> >       /* Table 0, priority 100.
> > @@ -1670,8 +1661,9 @@ physical_run(struct physical_ctx *p_ctx,
> >
> >           put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> >
> > -        ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 100, 0, &match,
> > -                        &ofpacts, hc_uuid);
> > +        dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                   OFTABLE_PHY_TO_LOG, 100, hc_uuid->parts[0],
> > +                                   &match, &ofpacts, hc_uuid);
> >       }
> >
> >       /* Handle ramp switch encapsulations. */
> > @@ -1709,9 +1701,10 @@ physical_run(struct physical_ctx *p_ctx,
> >               put_load(1, MFF_LOG_FLAGS, MLF_RCV_FROM_RAMP_BIT, 1, &ofpacts);
> >               put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, &ofpacts);
> >
> > -            ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 110,
> > -                            binding->header_.uuid.parts[0],
> > -                            &match, &ofpacts, hc_uuid);
> > +            dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                       OFTABLE_PHY_TO_LOG, 110,
> > +                                       binding->header_.uuid.parts[0],
> > +                                       &match, &ofpacts, hc_uuid);
> >           }
> >       }
> >
> > @@ -1731,8 +1724,9 @@ physical_run(struct physical_ctx *p_ctx,
> >       /* Resubmit to table 33. */
> >       ofpbuf_clear(&ofpacts);
> >       put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150, 0,
> > -                    &match, &ofpacts, hc_uuid);
> > +    dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                               OFTABLE_REMOTE_OUTPUT, 150, hc_uuid->parts[0],
> > +                               &match, &ofpacts, hc_uuid);
> >
> >       /* Table 32, priority 150.
> >        * =======================
> > @@ -1745,8 +1739,9 @@ physical_run(struct physical_ctx *p_ctx,
> >       /* Resubmit to table 33. */
> >       ofpbuf_clear(&ofpacts);
> >       put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150, 0,
> > -                    &match, &ofpacts, hc_uuid);
> > +    dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                               OFTABLE_REMOTE_OUTPUT, 150, hc_uuid->parts[0],
> > +                               &match, &ofpacts, hc_uuid);
> >
> >       /* Table 32, priority 150.
> >        * =======================
> > @@ -1768,9 +1763,10 @@ physical_run(struct physical_ctx *p_ctx,
> >           if (pb && !strcmp(pb->type, "localport")) {
> >               match_set_reg(&match, MFF_LOG_INPORT - MFF_REG0, pb->tunnel_key);
> >               match_set_metadata(&match, htonll(pb->datapath->tunnel_key));
> > -            ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 150,
> > -                            pb->header_.uuid.parts[0],
> > -                            &match, &ofpacts, hc_uuid);
> > +            dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                       OFTABLE_REMOTE_OUTPUT, 150,
> > +                                       pb->header_.uuid.parts[0],
> > +                                       &match, &ofpacts, hc_uuid);
> >           }
> >       }
> >
> > @@ -1782,8 +1778,9 @@ physical_run(struct physical_ctx *p_ctx,
> >       match_init_catchall(&match);
> >       ofpbuf_clear(&ofpacts);
> >       put_resubmit(OFTABLE_LOCAL_OUTPUT, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 0, 0, &match,
> > -                    &ofpacts, hc_uuid);
> > +    dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                               OFTABLE_REMOTE_OUTPUT, 0, hc_uuid->parts[0],
> > +                               &match, &ofpacts, hc_uuid);
> >
> >       /* Table 34, Priority 0.
> >        * =======================
> > @@ -1797,8 +1794,9 @@ physical_run(struct physical_ctx *p_ctx,
> >           put_load(0, MFF_REG0 + i, 0, 32, &ofpacts);
> >       }
> >       put_resubmit(OFTABLE_LOG_EGRESS_PIPELINE, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_CHECK_LOOPBACK, 0, 0, &match,
> > -                    &ofpacts, hc_uuid);
> > +    dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                               OFTABLE_CHECK_LOOPBACK, 0, hc_uuid->parts[0],
> > +                               &match, &ofpacts, hc_uuid);
> >
> >       /* Table 64, Priority 0.
> >        * =======================
> > @@ -1808,8 +1806,9 @@ physical_run(struct physical_ctx *p_ctx,
> >       match_init_catchall(&match);
> >       ofpbuf_clear(&ofpacts);
> >       put_resubmit(OFTABLE_LOG_TO_PHY, &ofpacts);
> > -    ofctrl_add_flow(flow_table, OFTABLE_SAVE_INPORT, 0, 0, &match,
> > -                    &ofpacts, hc_uuid);
> > +    dp_flow_add_physical_oflow(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                               OFTABLE_SAVE_INPORT, 0, hc_uuid->parts[0],
> > +                               &match, &ofpacts, hc_uuid);
> >
> >       ofpbuf_uninit(&ofpacts);
> >
> > @@ -1817,8 +1816,7 @@ physical_run(struct physical_ctx *p_ctx,
> >   }
> >
> >   bool
> > -physical_handle_ovs_iface_changes(struct physical_ctx *p_ctx,
> > -                                  struct ovn_desired_flow_table *flow_table)
> > +physical_handle_ovs_iface_changes(struct physical_ctx *p_ctx)
> >   {
> >       const struct ovsrec_interface *iface_rec;
> >       OVSREC_INTERFACE_TABLE_FOR_EACH_TRACKED (iface_rec, p_ctx->iface_table) {
> > @@ -1848,11 +1846,13 @@ physical_handle_ovs_iface_changes(struct physical_ctx *p_ctx,
> >
> >           int64_t ofport = iface_rec->n_ofport ? *iface_rec->ofport : 0;
> >           if (ovsrec_interface_is_deleted(iface_rec)) {
> > -            ofctrl_remove_flows(flow_table, &lb->pb->header_.uuid);
> > +            dp_flow_remove_physical_oflows(lb->pb->datapath->tunnel_key,
> > +                                           &lb->pb->header_.uuid);
> >               simap_find_and_delete(&localvif_to_ofport, iface_id);
> >           } else {
> >               if (!ovsrec_interface_is_new(iface_rec)) {
> > -                ofctrl_remove_flows(flow_table, &lb->pb->header_.uuid);
> > +                dp_flow_remove_physical_oflows(lb->pb->datapath->tunnel_key,
> > +                                               &lb->pb->header_.uuid);
> >               }
> >
> >               simap_put(&localvif_to_ofport, iface_id, ofport);
> > @@ -1861,7 +1861,7 @@ physical_handle_ovs_iface_changes(struct physical_ctx *p_ctx,
> >                                     p_ctx->active_tunnels,
> >                                     p_ctx->local_datapaths,
> >                                     lb->pb, p_ctx->chassis,
> > -                                  flow_table, &ofpacts);
> > +                                  &ofpacts);
> >           }
> >       }
> >
> > @@ -1883,17 +1883,17 @@ get_tunnel_ofport(const char *chassis_name, char *encap_ip, ofp_port_t *ofport)
> >   }
> >
> >   void
> > -physical_clear_unassoc_flows_with_db(struct ovn_desired_flow_table *flow_table)
> > +physical_clear_unassoc_flows_with_db(void)
> >   {
> >       if (hc_uuid) {
> > -        ofctrl_remove_flows(flow_table, hc_uuid);
> > +        dp_flow_remove_physical_oflows(DP_FLOW_TABLE_GLOBAL_KEY,
> > +                                       hc_uuid);
> >       }
> >   }
> >
> >   void
> >   physical_clear_dp_flows(struct physical_ctx *p_ctx,
> > -                        struct hmapx *ct_updated_datapaths,
> > -                        struct ovn_desired_flow_table *flow_table)
> > +                        struct hmapx *ct_updated_datapaths)
> >   {
> >       const struct sbrec_port_binding *binding;
> >       SBREC_PORT_BINDING_TABLE_FOR_EACH (binding, p_ctx->port_binding_table) {
> > @@ -1902,9 +1902,11 @@ physical_clear_dp_flows(struct physical_ctx *p_ctx,
> >           }
> >           const struct sbrec_port_binding *peer =
> >               get_binding_peer(p_ctx->sbrec_port_binding_by_name, binding);
> > -        ofctrl_remove_flows(flow_table, &binding->header_.uuid);
> > +        dp_flow_remove_physical_oflows(binding->datapath->tunnel_key,
> > +                                       &binding->header_.uuid);
> >           if (peer) {
> > -            ofctrl_remove_flows(flow_table, &peer->header_.uuid);
> > +            dp_flow_remove_physical_oflows(peer->datapath->tunnel_key,
> > +                                           &peer->header_.uuid);
> >           }
> >       }
> >   }
> > diff --git a/controller/physical.h b/controller/physical.h
> > index 0bf13f2683..70849c091d 100644
> > --- a/controller/physical.h
> > +++ b/controller/physical.h
> > @@ -60,18 +60,13 @@ struct physical_ctx {
> >   };
> >
> >   void physical_register_ovs_idl(struct ovsdb_idl *);
> > -void physical_run(struct physical_ctx *,
> > -                  struct ovn_desired_flow_table *);
> > -void physical_clear_unassoc_flows_with_db(struct ovn_desired_flow_table *);
> > +void physical_run(struct physical_ctx *);
> > +void physical_clear_unassoc_flows_with_db(void);
> >   void physical_clear_dp_flows(struct physical_ctx *p_ctx,
> > -                             struct hmapx *ct_updated_datapaths,
> > -                             struct ovn_desired_flow_table *flow_table);
> > -void physical_handle_port_binding_changes(struct physical_ctx *,
> > -                                          struct ovn_desired_flow_table *);
> > -void physical_handle_mc_group_changes(struct physical_ctx *,
> > -                                      struct ovn_desired_flow_table *);
> > -bool physical_handle_ovs_iface_changes(struct physical_ctx *,
> > -                                       struct ovn_desired_flow_table *);
> > +                             struct hmapx *ct_updated_datapaths);
> > +void physical_handle_port_binding_changes(struct physical_ctx *);
> > +void physical_handle_mc_group_changes(struct physical_ctx *);
> > +bool physical_handle_ovs_iface_changes(struct physical_ctx *);
> >   bool get_tunnel_ofport(const char *chassis_name, char *encap_ip,
> >                          ofp_port_t *ofport);
> >   #endif /* controller/physical.h */
> >
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>



More information about the dev mailing list