[ovs-git] [openvswitch/ovs] b0bca6: json: Add support for partially serialized json ob...

Tue Aug 31 22:16:43 UTC 2021

  Branch: refs/heads/master
  Home:   https://github.com/openvswitch/ovs
  Commit: b0bca6f27aae845c3ca8b48d66a7dbd3d978162a
      https://github.com/openvswitch/ovs/commit/b0bca6f27aae845c3ca8b48d66a7dbd3d978162a
  Author: Ilya Maximets <i.maximets at ovn.org>
  Date:   2021-08-31 (Tue, 31 Aug 2021)

  Changed paths:
    M include/openvswitch/json.h
    M lib/json.c

  Log Message:
  -----------
  json: Add support for partially serialized json objects.

Introducing a new json type JSON_SERIALIZED_OBJECT.  It's not an
actual type that can be seen in a json message on a wire, but
internal type that is intended to hold a serialized version of
some other json object.  For this reason it's defined after the
JSON_N_TYPES to not confuse parsers and other parts of the code
that relies on compliance with RFC 4627.

With this JSON type internal users may construct large JSON objects,
parts of which are already serialized.  This way, while serializing
the larger object, data from JSON_SERIALIZED_OBJECT can be added
directly to the result, without additional processing.

This will be used by next commits to add pre-serialized JSON data
to the raft_header structure, that can be converted to a JSON
before writing the file transaction on disk or sending to other
servers.  Same technique can also be used to pre-serialize json_cache
for ovsdb monitors, this should allow to not perform serialization
for every client and will save some more memory.

Since serialized JSON is just a string, reusing the 'json->string'
pointer for it.

Acked-by: Dumitru Ceara <dceara at redhat.com>
Acked-by: Han Zhou <hzhou at ovn.org>
Signed-off-by: Ilya Maximets <i.maximets at ovn.org>

  Commit: 0de882954032aa37dc943bafd72c33324aa0c95a
      https://github.com/openvswitch/ovs/commit/0de882954032aa37dc943bafd72c33324aa0c95a
  Author: Ilya Maximets <i.maximets at ovn.org>
  Date:   2021-09-01 (Wed, 01 Sep 2021)

  Changed paths:
    M ovsdb/ovsdb-tool.c
    M ovsdb/raft-private.c
    M ovsdb/raft-private.h
    M ovsdb/raft.c
    M ovsdb/raft.h
    M ovsdb/storage.c

  Log Message:
  -----------
  raft: Don't keep full json objects in memory if no longer needed.

Raft log entries (and raft database snapshot) contains json objects
of the data.  Follower receives append requests with data that gets
parsed and added to the raft log.  Leader receives execution requests,
parses data out of them and adds to the log.  In both cases, later
ovsdb-server reads the log with ovsdb_storage_read(), constructs
transaction and updates the database.  On followers these json objects
in common case are never used again.  Leader may use them to send
append requests or snapshot installation requests to followers.
However, all these operations (except for ovsdb_storage_read()) are
just serializing the json in order to send it over the network.

Json objects are significantly larger than their serialized string
representation.  For example, the snapshot of the database from one of
the ovn-heater scale tests takes 270 MB as a string, but 1.6 GB as
a json object from the total 3.8 GB consumed by ovsdb-server process.

ovsdb_storage_read() for a given raft entry happens only once in a
lifetime, so after this call, we can serialize the json object, store
the string representation and free the actual json object that ovsdb
will never need again.  This can save a lot of memory and can also
save serialization time, because each raft entry for append requests
and snapshot installation requests serialized only once instead of
doing that every time such request needs to be sent.

JSON_SERIALIZED_OBJECT can be used in order to seamlessly integrate
pre-serialized data into raft_header and similar json objects.

One major special case is creation of a database snapshot.
Snapshot installation request received over the network will be parsed
and read by ovsdb-server just like any other raft log entry.  However,
snapshots created locally with raft_store_snapshot() will never be
read back, because they reflect the current state of the database,
hence already applied.  For this case we can free the json object
right after writing snapshot on disk.

Tests performed with ovn-heater on 60 node density-light scenario,
where on-disk database goes up to 97 MB, shows average memory
consumption of ovsdb-server Southbound DB processes decreased by 58%
(from 602 MB to 256 MB per process) and peak memory consumption
decreased by 40% (from 1288 MB to 771 MB).

Test with 120 nodes on density-heavy scenario with 270 MB on-disk
database shows 1.5 GB memory consumption decrease as expected.
Also, total CPU time consumed by the Southbound DB process reduced
from 296 to 256 minutes.  Number of unreasonably long poll intervals
reduced from 2896 down to 1934.

Acked-by: Dumitru Ceara <dceara at redhat.com>
Acked-by: Han Zhou <hzhou at ovn.org>
Signed-off-by: Ilya Maximets <i.maximets at ovn.org>

  Commit: 43e66fc27659af2a5c976bdd27fe747b442b5554
      https://github.com/openvswitch/ovs/commit/43e66fc27659af2a5c976bdd27fe747b442b5554
  Author: Ilya Maximets <i.maximets at ovn.org>
  Date:   2021-09-01 (Wed, 01 Sep 2021)

  Changed paths:
    M ovsdb/monitor.c

  Log Message:
  -----------
  ovsdb: monitor: Store serialized json in a json cache.

Same json from a json cache is typically sent to all the clients,
e.g., in case of OVN deployment with ovn-monitor-all=true.

There could be hundreds or thousands connected clients and ovsdb
will serialize the same json object for each of them before sending.

Serializing it once before storing into json cache to speed up
processing.

This change allows to save a lot of CPU cycles and a bit of memory
since we need to store in memory only a string and not the full json
object.

Testing with ovn-heater on 120 nodes using density-heavy scenario
shows reduction of the total CPU time used by Southbound DB processes
from 256 minutes to 147.  Duration of unreasonably long poll intervals
also reduced dramatically from 7 to 2 seconds:

           Count   Min    Max   Median    Mean   95 percentile
 -------------------------------------------------------------
  Before   1934   1012   7480   4302.5   4875.3     7034.3
  After    1909   1004   2730   1453.0   1532.5     2053.6

Acked-by: Dumitru Ceara <dceara at redhat.com>
Acked-by: Han Zhou <hzhou at ovn.org>
Signed-off-by: Ilya Maximets <i.maximets at ovn.org>

Compare: https://github.com/openvswitch/ovs/compare/c2fb5bdae6e3...43e66fc27659