[ovs-dev] [PATCH v3 0/5] Fast OVSDB resync after restart or fail-over.

[Ben Pfaff]

It didn't get caught in any filters for a different -dev list that I
happen to also moderate.  Doh!  Anyway, I released it now.

On Mon, Feb 25, 2019 at 11:24:21AM -0800, Ben Pfaff wrote:
> It didn't get caught in any filters this time.
> 
> I see all the patches, although maybe I was CCed.
> 
> On Mon, Feb 25, 2019 at 11:08:40AM -0800, Han Zhou wrote:
> > This seems not showing up in the mailing list. Retry sending using
> > different words in title.
> > On Mon, Feb 25, 2019 at 9:25 AM Han Zhou <zhouhan at gmail.com> wrote:
> > >
> > > In scalability test with ovn-scale-test, ovsdb-server SB load is not a
> > > problem at least with 1k HVs. However, if we restart the ovsdb-server,
> > > depending on the number of HVs and scale of logical objects, e.g. the
> > > number of logical ports, ovsdb-server of SB become an obvious bottleneck.
> > >
> > > In our test with 1k HVs and 20k logical ports (200 lport * 100 lswitches
> > > connected by one single logical router). Restarting ovsdb-server of SB
> > > resulted in 100% CPU of ovsdb-server for more than 1 hour. All HVs (and
> > > northd) are reconnecting and resyncing the big amount of data at the same
> > > time.
> > >
> > > Similar problem would happen in failover scenario. With active-active
> > > cluster, the problem can be aleviated slightly, because only 1/3 (assuming
> > > it is 3-node cluster) of the HVs will need to resync data from new servers,
> > > but it is still a serious problem.
> > >
> > > For detailed discussions for the problem and solutions, see:
> > > https://mail.openvswitch.org/pipermail/ovs-discuss/2018-October/047591.html
> > >
> > > The patches implements the proposal in that discussion. It introduces
> > > a new method monitor_cond_since to enable client to request changes that
> > > happened after a specific point so that the data has been cached already
> > > in client are not re-transfered. Scalability test shows dramatic improvement.
> > > All HVs finishes sync as soon as they reconnect since there is no new data
> > > to be transfered.
> > >
> > > The current patches supports all 3 modes of ovsdb-server, but only clustered
> > > mode can benefit from it, since it is the only one that supports transaction
> > > id out of the box.
> > >
> > > ---
> > > v1 -> v2: fix the unused variable in patch 6/7 reported by 0-day.
> > > v2 -> v3: first 2 in the 7 were merged. Revised 1/5 addressing Ben's
> > > comment on the json cache hmap.
> > >
> > > Han Zhou (5):
> > >   ovsdb-monitor: Refactor ovsdb monitor implementation.
> > >   ovsdb-server: Transaction history tracking.
> > >   ovsdb-monitor: Support monitor_cond_since.
> > >   ovsdb-idl.c: Support monitor_cond_since method in C IDL.
> > >   ovsdb-idl.c: Fast resync from server when connection reset.
> > >
> > >  Documentation/ref/ovsdb-server.7.rst |  78 +++++-
> > >  lib/ovsdb-idl.c                      | 229 ++++++++++++----
> > >  ovsdb/jsonrpc-server.c               | 101 +++++--
> > >  ovsdb/monitor.c                      | 516 ++++++++++++++++++++++-------------
> > >  ovsdb/monitor.h                      |  16 +-
> > >  ovsdb/ovsdb-client.1.in              |  28 +-
> > >  ovsdb/ovsdb-client.c                 | 104 ++++++-
> > >  ovsdb/ovsdb-server.c                 |  11 +
> > >  ovsdb/ovsdb.c                        |   3 +
> > >  ovsdb/ovsdb.h                        |  10 +
> > >  ovsdb/transaction.c                  | 117 +++++++-
> > >  ovsdb/transaction.h                  |   4 +
> > >  tests/ovsdb-monitor.at               | 301 +++++++++++++++++++-
> > >  13 files changed, 1232 insertions(+), 286 deletions(-)
> > >
> > > --
> > > 2.1.0
> > >
> > _______________________________________________
> > dev mailing list
> > dev at openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev