[ovs-dev] [PATCH 0/2] raft: Reduce memory consumption by freeing unnecessary json objects.

[Ilya Maximets]

On 8/20/21 6:51 PM, Ilya Maximets wrote:
> In short, database snapshot stored inside the raft module is
> a huge json object.  E.g. in ovn-heater scale tests with 270 MB
> on-disk Southbound DB, json object of a snapshot takes 1.6 GB of
> RAM out of total 3.8 GB of the RSS of an ovsdb-server process.
> 
> This change is targeted to free that json object as soon as it
> no longer needed and keep its serialized version instead.
> 
> Testing on a bit smaller cluster with 97 MB of on-disk database
> showed *58%* of the memory consumption decrease with this change.
> 
> More details on testing and implementation in commit messages.
> 
> Ilya Maximets (2):
>   json: Add support for partially serialized json objects.
>   raft: Don't keep full json objects in memory if no longer needed.
> 
>  include/openvswitch/json.h |   9 ++-
>  lib/json.c                 |  34 ++++++++++++
>  ovsdb/ovsdb-tool.c         |  10 ++--
>  ovsdb/raft-private.c       | 111 +++++++++++++++++++++++++++++++++----
>  ovsdb/raft-private.h       |  12 +++-
>  ovsdb/raft.c               |  94 +++++++++++++++++--------------
>  ovsdb/raft.h               |   3 +-
>  ovsdb/storage.c            |   4 +-
>  8 files changed, 211 insertions(+), 66 deletions(-)
> 

I sent a v2 with one more patch that allows to reduce CPU
usage of ovsdb-server by more than 50% using same technique:
  https://patchwork.ozlabs.org/project/openvswitch/list/?series=259477&state=*

Best regards, Ilya Maximets.