[ovs-git] [openvswitch/ovs] f70b61: log: Add async commit support.

GitHub noreply at github.com
Sat Mar 24 19:05:18 UTC 2018


  Branch: refs/heads/master
  Home:   https://github.com/openvswitch/ovs
  Commit: f70b61d33dd3a099cd51e95b1047c50c6bf331ca
      https://github.com/openvswitch/ovs/commit/f70b61d33dd3a099cd51e95b1047c50c6bf331ca
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M ovsdb/file.c
    M ovsdb/log.c
    M ovsdb/log.h
    M ovsdb/ovsdb-tool.c
    M tests/test-ovsdb.c

  Log Message:
  -----------
  log: Add async commit support.

The OVSDB log code has always had the ability to commit the log to disk and
wait for the commit to finish.  This patch introduces a new feature that
allows the client to start a commit in the background and then to determine
asynchronously that the commit has completed.  This will be especially
useful later for the distributed database feature.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Reviewed-by: Yifeng Sun <pkusunyifeng at gmail.com>


  Commit: 5ee527e223b266d06c37d93d54695dc48e890cbf
      https://github.com/openvswitch/ovs/commit/5ee527e223b266d06c37d93d54695dc48e890cbf
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M lib/reconnect.c
    M lib/reconnect.h
    M python/ovs/reconnect.py
    M tests/reconnect.at
    M tests/test-reconnect.c
    M tests/test-reconnect.py

  Log Message:
  -----------
  reconnect: Add ability to do a number of retries without backoff.

This is aimed at an upcoming database clustering implementation, where it's
desirable to try all of the cluster members quickly before backing off to
retry them again in sequence.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Russell Bryant <russell at ovn.org>
Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 8cf6bbb184c319b05f2a78a6fc7fc5b93297eafd
      https://github.com/openvswitch/ovs/commit/8cf6bbb184c319b05f2a78a6fc7fc5b93297eafd
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M lib/jsonrpc.c
    M lib/jsonrpc.h
    M lib/svec.c
    M lib/svec.h

  Log Message:
  -----------
  jsonrpc: Allow jsonrpc_session to have more than one remote.

The implementation cycles through the remotes in random order.  This allows
clients to perform some load balancing across alternative implementations
of a service.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Russell Bryant <russell at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 120fb2ca52cd8c2abedf6ee44032d17eb854fd77
      https://github.com/openvswitch/ovs/commit/120fb2ca52cd8c2abedf6ee44032d17eb854fd77
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M ovsdb/file.c
    M ovsdb/file.h
    M ovsdb/monitor.c
    M ovsdb/ovsdb.c
    M ovsdb/ovsdb.h
    M ovsdb/transaction.c

  Log Message:
  -----------
  ovsdb-server: Distinguish logs from other replicas.

Until now, ovsdb-server has internally chained a list of replicas from each
database.  Whenever ovsdb_txn_commit() commits a transaction, it passes the
transaction to each replica.  The first replica, which is always the disk
file that stores the database, is special because it is the only replica
that can report an error and thereby abort the transaction.  This is a very
special property that genuinely distinguishes this first replica from the
others on the chain.  This commit breaks that first replica out as a
separate kind of entity that is not on the list of replicas.  When later
commits add support for clustering, there will only be more and more
special cases for the "first replica", so it makes sense to distinguish it
this way.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 009bf21fdd3554065f4f546d48bf32e6aa7bf309
      https://github.com/openvswitch/ovs/commit/009bf21fdd3554065f4f546d48bf32e6aa7bf309
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M ovsdb/monitor.c
    M ovsdb/monitor.h
    M ovsdb/ovsdb.c
    M ovsdb/ovsdb.h
    M ovsdb/transaction.c

  Log Message:
  -----------
  ovsdb: Drop distinction between monitors and replicas.

Until now, OVSDB distinguished "monitors", which are associated with OVSDB
JSON-RPC client sessions and allow clients to find out about database
changes, from "replicas", which are associated with databases and also find
out about database changes and act on them in some way.  Now that
committing to disk has been broken into a separate concept, there is a
one-to-one and "onto" relationship between monitors and replicas: every
monitor M has a replica R and R is associated with M as well.  It's easier
if we just treat them as a single entity, and that's what this commit
implements.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 077f03028bbea3073c362b146a0891bf5040ca15
      https://github.com/openvswitch/ovs/commit/077f03028bbea3073c362b146a0891bf5040ca15
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M ovsdb/jsonrpc-server.c
    M ovsdb/jsonrpc-server.h
    M ovsdb/ovsdb-server.c

  Log Message:
  -----------
  jsonrpc-server: Separate changing read_only status from reconnecting.

The code in jsonrpc-server conflated two different kinds of functionality.
It makes sense for the client to be able to change whether a particular
server is read-only.  It also makes sense for the client to tell a server
to reconnect.  The code in jsonrpc-server only provided a single function
that does both, which is weird.  This commit breaks these apart.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 1456335dc59917535a5bc6c3fc74231395100263
      https://github.com/openvswitch/ovs/commit/1456335dc59917535a5bc6c3fc74231395100263
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M lib/ovsdb-idl-provider.h
    M lib/ovsdb-idl.c
    M lib/ovsdb-idl.h

  Log Message:
  -----------
  ovsdb-idl: Break out database-specific stuff into new data structure.

Until now, a given ovsdb-idl instances has only monitored a single
database.  In an upcoming commit, it will grow to also monitor a second
database that represents the state of the database server itself.  Much of
the work is the same for both databases, so this commit breaks the common
code and data out into new data structures and functions.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 00d5d6310da15bf0530cb9a7a73acce325636b44
      https://github.com/openvswitch/ovs/commit/00d5d6310da15bf0530cb9a7a73acce325636b44
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M ovn/utilities/ovn-sbctl.c

  Log Message:
  -----------
  ovn-sbctl: Allow retries by default.

Most of the OVS database-manipulation utilities (ovn-sbctl, ovn-nbctl,
ovs-vsctl, vtep-ctl) don't retry their connections by default because
they assume that the database is either up or down and likely to stay
that way.  The OVN southbound database, however, is a likely candidate
for high availability clustering, so that even if it appears to be
down for a moment it will be available again soon.  So, prepare for
the clustering implementation by enabling retry by default in
ovn-sbctl.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 6bb9b060d5b0f972461f0c061a865393611fd86a
      https://github.com/openvswitch/ovs/commit/6bb9b060d5b0f972461f0c061a865393611fd86a
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M Documentation/ref/index.rst
    M Makefile.am
    M NEWS
    M build-aux/automake.mk
    A build-aux/text2c
    M debian/openvswitch-switch.manpages
    M ovsdb/.gitignore
    A ovsdb/_server.ovsschema
    A ovsdb/_server.xml
    M ovsdb/automake.mk
    M ovsdb/ovsdb-server.1.in
    M ovsdb/ovsdb-server.c
    M ovsdb/ovsdb-util.c
    M ovsdb/ovsdb-util.h
    M rhel/openvswitch-fedora.spec.in
    M rhel/openvswitch.spec.in
    M tests/ovsdb-server.at
    M xenserver/openvswitch-xen.spec.in

  Log Message:
  -----------
  ovsdb-server: Add support for a built-in _Server database.

The _Server database is valuable primarily because it provides database
clients a way to find out the details of changes to databases, schemas,
etc. in a granular, natural way.  Until now, the only way that the server
could notify clients about these kinds of changes was to close the session;
when the client reconnects, it is expected to reassess the server's state.
One way to provide this kind of granular information would be to add
specific JSON-RPC requests to obtain notifications for different kinds of
changes, but since ovsdb-server already provides granular and flexible
notification support for databases, using a database for the purpose is
convenient and avoids duplicating functionality.

Initially this database only reports databases' names and schemas, but
when clustering support is added in a later commit it will also report
important aspects of clustering and cluster status.  Thus, this database
also reduces the need to add JSON-RPC calls to retrieve information about
new features.

Signed-off-by: Ben Pfaff <blp at ovn.org>


  Commit: 10621d795331a3ddddb3e96086a9fda7ad156efc
      https://github.com/openvswitch/ovs/commit/10621d795331a3ddddb3e96086a9fda7ad156efc
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M Documentation/ref/ovsdb-server.7.rst
    M ovsdb/_server.xml
    M ovsdb/jsonrpc-server.c
    M ovsdb/jsonrpc-server.h
    M ovsdb/ovsdb-client.c
    M ovsdb/ovsdb-server.c
    M tests/ovsdb-server.at

  Log Message:
  -----------
  ovsdb-server: Add new RPC "set_db_change_aware".

The _Server database recently added to ovsdb-server can be used to dump out
information about databases, but monitoring updates to _Server is not yet
very useful because for historical reasons ovsdb-server drops all of its
OVSDB connections whenever databases are added or removed or otherwise
change in some major way.  It is not a good idea to change this behavior
for all clients, because some of them rely on it, but this commit
introduces a new RPC that allows clients that understand _Server to
suppress the connection-closing behavior.

Signed-off-by: Ben Pfaff <blp at ovn.org>


  Commit: 53178986d7fc86bcfc2f297b547a97ee71a21bb7
      https://github.com/openvswitch/ovs/commit/53178986d7fc86bcfc2f297b547a97ee71a21bb7
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M Documentation/ref/ovsdb-server.7.rst
    M Documentation/ref/ovsdb.7.rst
    M NEWS
    M lib/ovsdb-data.c
    M lib/ovsdb-data.h
    M ovsdb/file.c
    M ovsdb/file.h
    M ovsdb/jsonrpc-server.c
    M ovsdb/jsonrpc-server.h
    M ovsdb/monitor.c
    M ovsdb/monitor.h
    M ovsdb/ovsdb-client.1.in
    M ovsdb/ovsdb-client.c
    M ovsdb/ovsdb-server.c
    M ovsdb/ovsdb.c
    M ovsdb/ovsdb.h
    M ovsdb/transaction.c
    M ovsdb/transaction.h
    M ovsdb/trigger.c
    M ovsdb/trigger.h
    M tests/ovsdb-monitor.at
    M tests/ovsdb-server.at
    M tests/test-ovsdb.c

  Log Message:
  -----------
  ovsdb: Add support for online schema conversion.

With this change, "ovsdb-client convert" can be used to convert a database
from one schema to another without taking the database offline.

This can be useful to minimize downtime for a database during a software
upgrade.

Signed-off-by: Ben Pfaff <blp at ovn.org>
Acked-by: Justin Pettit <jpettit at ovn.org>


  Commit: 1b1d2e6daa563cc91f974ffdc082fb3a8b424801
      https://github.com/openvswitch/ovs/commit/1b1d2e6daa563cc91f974ffdc082fb3a8b424801
  Author: Ben Pfaff <blp at ovn.org>
  Date:   2018-03-24 (Sat, 24 Mar 2018)

  Changed paths:
    M Documentation/ref/ovsdb.5.rst
    M Documentation/ref/ovsdb.7.rst
    M NEWS
    M lib/.gitignore
    M lib/automake.mk
    M lib/jsonrpc.c
    M lib/jsonrpc.h
    M lib/ovsdb-idl.c
    M lib/ovsdb-idl.h
    A lib/ovsdb-server-idl.ann
    A lib/ovsdb-session.c
    A lib/ovsdb-session.h
    M lib/smap.c
    M lib/smap.h
    M lib/uuid.h
    M ovn/controller/ovn-controller.c
    M ovn/utilities/ovn-nbctl.8.xml
    M ovn/utilities/ovn-nbctl.c
    M ovn/utilities/ovn-sbctl.8.in
    M ovn/utilities/ovn-sbctl.c
    A ovsdb/TODO.rst
    M ovsdb/_server.ovsschema
    M ovsdb/_server.xml
    M ovsdb/automake.mk
    M ovsdb/execution.c
    M ovsdb/file.c
    M ovsdb/file.h
    M ovsdb/jsonrpc-server.c
    M ovsdb/jsonrpc-server.h
    M ovsdb/log.c
    M ovsdb/log.h
    M ovsdb/ovsdb-client.1.in
    M ovsdb/ovsdb-client.c
    M ovsdb/ovsdb-server.1.in
    M ovsdb/ovsdb-server.c
    M ovsdb/ovsdb-tool.1.in
    M ovsdb/ovsdb-tool.c
    M ovsdb/ovsdb-util.c
    M ovsdb/ovsdb-util.h
    M ovsdb/ovsdb.c
    M ovsdb/ovsdb.h
    A ovsdb/raft-private.c
    A ovsdb/raft-private.h
    A ovsdb/raft-rpc.c
    A ovsdb/raft-rpc.h
    A ovsdb/raft.c
    A ovsdb/raft.h
    M ovsdb/replication.c
    M ovsdb/row.c
    M ovsdb/server.c
    M ovsdb/server.h
    A ovsdb/storage.c
    A ovsdb/storage.h
    M ovsdb/transaction.c
    M ovsdb/transaction.h
    M ovsdb/trigger.c
    M ovsdb/trigger.h
    M tests/.gitignore
    M tests/automake.mk
    M tests/ovs-macros.at
    A tests/ovsdb-cluster.at
    M tests/ovsdb-idl.at
    M tests/ovsdb-monitor.at
    M tests/ovsdb-server.at
    M tests/ovsdb-tool.at
    M tests/ovsdb.at
    M tests/test-ovsdb.c
    M tutorial/ovs-sandbox

  Log Message:
  -----------
  ovsdb: Introduce experimental support for clustered databases.

This commit adds support for OVSDB clustering via Raft.  Please read
ovsdb(7) for information on how to set up a clustered database.  It is
simple and boils down to running "ovsdb-tool create-cluster" on one server
and "ovsdb-tool join-cluster" on each of the others and then starting
ovsdb-server in the usual way on all of them.

One you have a clustered database, you configure ovn-controller and
ovn-northd to use it by pointing them to all of the servers, e.g. where
previously you might have said "tcp:1.2.3.4" was the database server,
now you say that it is "tcp:1.2.3.4,tcp:5.6.7.8,tcp:9.10.11.12".

This also adds support for database clustering to ovs-sandbox.

Acked-by: Justin Pettit <jpettit at ovn.org>
Tested-by: aginwala <aginwala at asu.edu>
Signed-off-by: Ben Pfaff <blp at ovn.org>


Compare: https://github.com/openvswitch/ovs/compare/83e866067ea6...1b1d2e6daa56


More information about the git mailing list