[ovs-dev] [PATCH v2 0/9] OVSDB Relay Service Model. (Was: OVSDB 2-Tier deployment)

Ilya Maximets i.maximets at ovn.org
Sat Jun 12 01:59:59 UTC 2021

Replication can be used to scale out read-only access to the database.
But there are clients that are not read-only, but read-mostly.
One of the main examples is ovn-controller that mostly monitors
updates from the Southbound DB, but needs to claim ports by sending
transactions that changes some database tables.

Southbound database serves lots of connections: all connections
from ovn-controllers and some service connections from cloud
infrastructure, e.g. some OpenStack agents are monitoring updates.
At a high scale and with a big size of the database ovsdb-server
spends too much time processing monitor updates and it's required
to move this load somewhere else.  This patch-set aims to introduce
required functionality to scale out read-mostly connections by
introducing a new OVSDB 'relay' service model .

In this new service model ovsdb-server connects to existing OVSDB
server and maintains in-memory copy of the database.  It serves
read-only transactions and monitor requests by its own, but forwards
write transactions to the relay source.

Key differences from the active-backup replication:
- support for "write" transactions.
- no on-disk storage. (probably, faster operation)
- support for multiple remotes (connect to the clustered db).
- doesn't try to keep connection as long as possible, but
  faster reconnects to other remotes to avoid missing updates.
- No need to know the complete database schema beforehand,
  only the schema name.
- can be used along with other standalone and clustered databases
  by the same ovsdb-server process. (doesn't turn the whole
  jsonrpc server to read-only mode)
- supports modern version of monitors (monitor_cond_since),
  because based on ovsdb-cs.
- could be chained, i.e. multiple relays could be connected
  one to another in a row or in a tree-like form.

Bringing all above functionality to the existing active-backup
replication doesn't look right as it will make it less reliable
for the actual backup use case, and this also would be much
harder from the implementation point of view, because current
replication code is not based on ovsdb-cs or idl and all the required
features would be likely duplicated or replication would be fully
re-written on top of ovsdb-cs with severe modifications of the former.

Relay is somewhere in the middle between active-backup replication and
the clustered model taking a lot from both, therefore is hard to
implement on top of any of them.

To run ovsdb-server in relay mode, user need to simply run:

  ovsdb-server --remote=punix:db.sock relay:<schema-name>:<remotes>


  ovsdb-server --remote=punix:db.sock relay:OVN_Southbound:tcp:

More details and examples in the documentation in the last patch
of the series.

I actually tried to implement transaction forwarding on top of
active-backup replication in v1 of this seies, but it required
a lot of tricky changes, including schema format changes in order
to bring required information to the end clients, so I decided
to fully rewrite the functionality in v2 with a different approach.

Future work:
- Add support for transaction history (it could be just inherited
  from the transaction ids received from the relay source).  This
  will allow clients to utilize monitor_cond_since while working
  with relay.
- Possibly try to inherit min_index from the relay source to give
  clients ability to detect relays with stale data.
- Probably, add support for both above things to standalone databases,
  so relays will be able to inherit not only from clustered ones.

Version 2:
  * Dropped implementation on top of active-backup replication.
  * Implemented new 'relay' service model.
  * Updated documentation and wrote a separate topic with examples
    and ascii-graphics.  That's why v2 seem larger.

Ilya Maximets (9):
  jsonrpc-server: Wake up jsonrpc session if there are completed
  ovsdb: storage: Allow setting the name for the unbacked storage.
  ovsdb: table: Expose functions to execute operations on ovsdb tables.
  ovsdb: row: Add support for xor-based row updates.
  ovsdb: New ovsdb 'relay' service model.
  ovsdb: relay: Add support for transaction forwarding.
  ovsdb: relay: Reflect connection status in _Server database.
  ovsdb: Make clients aware of relay service model.
  docs: Add documentation for ovsdb relay mode.

 Documentation/automake.mk            |   1 +
 Documentation/ref/ovsdb.7.rst        |  62 ++++-
 Documentation/topics/index.rst       |   1 +
 Documentation/topics/ovsdb-relay.rst | 124 +++++++++
 NEWS                                 |   3 +
 lib/ovsdb-cs.c                       |  15 +-
 ovsdb/_server.ovsschema              |   7 +-
 ovsdb/_server.xml                    |  33 +--
 ovsdb/automake.mk                    |   4 +
 ovsdb/execution.c                    |  18 +-
 ovsdb/file.c                         |   2 +-
 ovsdb/jsonrpc-server.c               |   3 +-
 ovsdb/ovsdb-client.c                 |   2 +-
 ovsdb/ovsdb-server.1.in              |  27 +-
 ovsdb/ovsdb-server.c                 | 102 ++++---
 ovsdb/ovsdb.c                        |  11 +
 ovsdb/ovsdb.h                        |   9 +-
 ovsdb/relay.c                        | 381 +++++++++++++++++++++++++++
 ovsdb/relay.h                        |  38 +++
 ovsdb/replication.c                  |  83 +-----
 ovsdb/row.c                          |  30 ++-
 ovsdb/row.h                          |   6 +-
 ovsdb/storage.c                      |  13 +-
 ovsdb/storage.h                      |   2 +-
 ovsdb/table.c                        |  70 +++++
 ovsdb/table.h                        |  14 +
 ovsdb/transaction-forward.c          | 182 +++++++++++++
 ovsdb/transaction-forward.h          |  44 ++++
 ovsdb/trigger.c                      |  48 +++-
 ovsdb/trigger.h                      |  41 +--
 python/ovs/db/idl.py                 |  16 ++
 tests/ovsdb-server.at                |  85 +++++-
 tests/test-ovsdb.c                   |   6 +-
 33 files changed, 1287 insertions(+), 196 deletions(-)
 create mode 100644 Documentation/topics/ovsdb-relay.rst
 create mode 100644 ovsdb/relay.c
 create mode 100644 ovsdb/relay.h
 create mode 100644 ovsdb/transaction-forward.c
 create mode 100644 ovsdb/transaction-forward.h


More information about the dev mailing list