[ovs-git] [ovn-org/ovn] 6aab70: controller: Avoid unnecessary load balancer flow p...

[Dumitru Ceara]

  Branch: refs/heads/master
  Home:   https://github.com/ovn-org/ovn
  Commit: 6aab704fb67d563bf66b199429f3721b771b5f6e
      https://github.com/ovn-org/ovn/commit/6aab704fb67d563bf66b199429f3721b771b5f6e
  Author: Dumitru Ceara <dceara at redhat.com>
  Date:   2021-07-12 (Mon, 12 Jul 2021)

  Changed paths:
    M controller/lflow.c
    M northd/ovn-northd.c
    M ovn-sb.ovsschema

  Log Message:
  -----------
  controller: Avoid unnecessary load balancer flow processing.

Whenever a Load_Balancer is updated, e.g., a VIP is added, the following
sequence of events happens:

1. The Southbound Load_Balancer record is updated.
2. The Southbound Datapath_Binding records on which the Load_Balancer is
   applied are updated.
3. Southbound ovsdb-server sends updates about the Load_Balancer and
   Datapath_Binding records to ovn-controller.
4. The IDL layer in ovn-controller processes the updates at #3, but
   because of the SB schema references between tables [0] all logical
   flows referencing the updated Datapath_Binding are marked as
   "updated".  The same is true for Logical_DP_Group records
   referencing the Datapath_Binding, and also for all logical flows
   pointing to the new "updated" datapath groups.
5. ovn-controller ends up recomputing (removing/readding) all flows for
   all these tracked updates.

>From the SB Schema:
        "Datapath_Binding": {
            "columns": {
                [...]
                "load_balancers": {"type": {"key": {"type": "uuid",
                                                   "refTable": "Load_Balancer",
                                                   "refType": "weak"},
                                            "min": 0,
                                            "max": "unlimited"}},
        [...]
        "Load_Balancer": {
            "columns": {
                "datapaths": {
                [...]
                    "type": {"key": {"type": "uuid",
                                     "refTable": "Datapath_Binding"},
                             "min": 0, "max": "unlimited"}},
        [...]
        "Logical_DP_Group": {
            "columns": {
                "datapaths":
                    {"type": {"key": {"type": "uuid",
                                      "refTable": "Datapath_Binding",
                                      "refType": "weak"},
                              "min": 0, "max": "unlimited"}}},
        [...]
        "Logical_Flow": {
            "columns": {
                "logical_datapath":
                    {"type": {"key": {"type": "uuid",
                                      "refTable": "Datapath_Binding"},
                              "min": 0, "max": 1}},
                "logical_dp_group":
                    {"type": {"key": {"type": "uuid",
                                      "refTable": "Logical_DP_Group"},

In order to avoid this unnecessary Logical_Flow notification storm we
now remove the explicit reference from Datapath_Binding to
Load_Balancer and instead store raw UUIDs.

This means that on the ovn-controller side we need to perform a
Load_Balancer table lookup by UUID whenever a new datapath is added,
but that doesn't happen too often and the cost of the lookup is
negligible compared to the huge cost of processing the unnecessary
logical flow updates.

This change is backwards compatible because the contents stored in the
database are not changed, just that the schema constraints are relaxed a
bit.

Some performance measurements, on a scale test deployment simulating an
ovn-kubernetes deployment with 120 nodes and a large load balancer
with 16K VIPs associated to each node's logical switch, the event
processing loop time in ovn-controller, when adding a new VIP, is
reduced from ~39 seconds to ~8 seconds.

There's no need to change the northd DDlog implementation.

Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1978605
Acked-by: Mark Michelson <mmichels at redhat.com>
Signed-off-by: Dumitru Ceara <dceara at redhat.com>