[ovs-dev] [PATCH v3 00/16] ovn-controller incremental processing

Han Zhou zhouhan at gmail.com
Fri Jun 1 17:41:50 UTC 2018

ovn-controller currently recomputes everything when there are any changes
of input, which leads to high CPU usages and slow in end-to-end flow
enforcement in response to changes. It even wastes CPU to recompute flows
for unrelated inputs such as pinctrl events.

This patch series implements incremental processing in ovn-controller to
solve above problems. There has been a similar attempt of solve the problem
earlier but was reverted (see commit: 926c34fd). This patch series takes
a different approach with an incremental processing engine, to make the
dependencies clear and easier to maintain. The engine is a DAG representing
dependencies between different nodes. Each node maintains its own data, which
depends on its inputs and the data can also be inputs of other nodes. Each
node implements a method to recompute its data based on all the inputs, but
also implements methods to handle changes of different inputs incrementally.
The engine will be responsible to try incremental processing for each node
based on the dependencies or fallback to recompute when changes cannot be
handled incrementally.

This patch series can incrementally process the most common changes:
logical flows and port bindings from OVNSB. It can be expanded further for
more fine grained incremental processing gradually.

In ovn-scale-test env [1], the total execution time of creating and
binding 10k ports on 1k HVs with 40 lswitches and 8 lrouters
(5 lswitches/lrouter), decreased from 3h40m to 1h50m because of the
less CPU on HVs. The CPU time of ovn-controller for additional 500
lports creating and binding (on top of already existed 10k lports)
decreased 90% comparing with master. Latency for end-to-end operations
of one extra port on top of the 10k lports, start from port-creation
until all flows installation on all related HVs is also improved
significantly from 20.6s to 7.3s.

[1] https://github.com/openvswitch/ovn-scale-test

    - Minor refactorings such as engine macros.
    - Coding style corrections.

    - Add incremental processing for address set and port group changes.
    - Rebased on master.

Han Zhou (16):
  ovn-controller: Incremental processing engine
  ovn-controller: Track OVSDB changes
  ovn-controller: Initial use of incremental engine in main
  ovn-controller: Split SB inputs as separate incremental engine nodes
  ovn-controller: split ovs_idl inputs in incremental engine
  ovn-controller: Incremental logical flow processing
  ovn-controller: runtime_data change handler for SB port-binding
  ovn-controller: port-binding incremental processing for physical flows
  ovn-controller: Avoid forced recompute when not needed
  ovn-controller: incremental processing for multicast group changes
  ovsdb-idl: Tracking - preserve data for deleted rows.
  ovn-controller: Split addr_sets from runtime_data.
  ovn-controller: Maintain resource references for logical flows.
  ovn-controller: Incremental processing for address-set changes.
  ovn-controller: Split port_groups from runtime_data.
  ovn-controller: Incremental processing for port-group changes.

 include/ovn/actions.h           |    3 +
 include/ovn/expr.h              |    5 +-
 lib/ovsdb-idl-provider.h        |    2 +
 lib/ovsdb-idl.c                 |   37 +-
 ovn/controller/bfd.c            |    4 +-
 ovn/controller/binding.c        |  108 ++++-
 ovn/controller/binding.h        |    4 +
 ovn/controller/encaps.c         |   12 +-
 ovn/controller/lflow.c          |  419 ++++++++++++++--
 ovn/controller/lflow.h          |   89 +++-
 ovn/controller/ofctrl.c         |  262 +++++++---
 ovn/controller/ofctrl.h         |   32 +-
 ovn/controller/ovn-controller.c | 1020 ++++++++++++++++++++++++++++++++-------
 ovn/controller/physical.c       |  229 ++++++---
 ovn/controller/physical.h       |   20 +-
 ovn/lib/actions.c               |    8 +-
 ovn/lib/automake.mk             |    4 +-
 ovn/lib/expr.c                  |   21 +-
 ovn/lib/extend-table.c          |   60 ++-
 ovn/lib/extend-table.h          |   16 +-
 ovn/lib/inc-proc-eng.c          |  161 ++++++
 ovn/lib/inc-proc-eng.h          |  206 ++++++++
 ovn/utilities/ovn-trace.c       |    2 +-
 tests/ovn.at                    |   71 +++
 tests/test-ovn.c                |    7 +-
 25 files changed, 2371 insertions(+), 431 deletions(-)
 create mode 100644 ovn/lib/inc-proc-eng.c
 create mode 100644 ovn/lib/inc-proc-eng.h


More information about the dev mailing list