[ovs-dev] [PATCH 0/5] Stream Record/Replay.

[Dumitru Ceara]

On 6/30/20 3:24 AM, Ilya Maximets wrote:
> This patch set adds new stream provider and other functionality in
> order to record all the incoming data on all the steams (ssl, tcp,
> unixctl) of openvswitch library based applications and replay these
> streams later for debugging purposes or performance tests.
> 
> For example, these changes allowed me to record the lifecycle of
> a standalone ovsdb-server in a fake-multinode cluster from ovn
> scale testing setup with 100 nodes.  While having only replay files,
> user could reproduce transactions and connections in a big cluster on
> a local PC wile being able to tweak various log levels, run everything
> under debugger or tracer, measure performance difference with perf.
> 
> Current implementation doesn't work with clustered databases since
> raft heavily depends on time events.  This is a point of further
> improvement.
> 
> Some more details in individual commit messages.  Some documentation
> updates are missing, will be in next version of the patch-set.  Code
> maybe not that elegant in a couple of places, I'm going to work on
> this too.
> 
> ovn-northd might be a good candidate to integrate this functionality
> from the OVN side.
> 
> Ilya Maximets (5):
>   ovsdb-server: Allow using predictable UUIDs.
>   stream: Add record/replay functionality.
>   ovsdb-server: Integrate stream replay engine.
>   ovsdb-server: Don't update manager status if replay engine is active.
>   jsonrpc: Disable inactivity probes if replay engine is active.
> 
>  lib/automake.mk         |   1 +
>  lib/jsonrpc.c           |  10 +-
>  lib/stream-provider.h   |   4 +
>  lib/stream-replay.c     | 561 ++++++++++++++++++++++++++++++++++++++++
>  lib/stream.c            |  38 ++-
>  lib/stream.h            |  46 +++-
>  lib/uuid.c              |  34 +++
>  lib/uuid.h              |   1 +
>  ovsdb/ovsdb-client.c    |   2 +-
>  ovsdb/ovsdb-server.c    |  30 ++-
>  tests/test-jsonrpc.c    |   2 +-
>  utilities/ovs-vsctl.c   |   2 +-
>  vswitchd/ovs-vswitchd.c |   2 +-
>  vtep/vtep-ctl.c         |   2 +-
>  14 files changed, 717 insertions(+), 18 deletions(-)
>  create mode 100644 lib/stream-replay.c
> 

Hi Ilya,

This is really great and it will make performance testing easier indeed!

This is not a complete review as I just played a bit with the
functionality and recorded and replayed some OVN DB executions but
something that I missed was a bit of documentation or maybe a short
example (apart from the samples in the commit logs).

Also, I'm wondering if there's any way possibility to expand this
feature and allow users to start/stop recording while the application is
already running. If that could be done in a reliable way it would make
it usable "online", potentially even in production.

I have a few minor comments on the individual patches.

Again, great work!

Thanks,
Dumitru