[ovs-dev] [ovsdb-server multithreading RFC 6/9] ovsdb: Add IPC messages for thread communication

[Andy Zhou]

On Fri, Mar 18, 2016 at 6:24 PM, Ben Pfaff <blp at ovn.org> wrote:

> On Thu, Mar 03, 2016 at 12:13:25AM -0800, Andy Zhou wrote:
> > Add a command queue for the main thread to pass commands to the
> > sessions thread.
> >
> > Defined an implemented a set of IPC messages for creating,
> > closing and managing jsonrpc sessions from the main process to jsonrpc
> > sessions threads.
> >
> > Add a doorbell mechanism for a jsonrpc session to inform other sessions
> > of possible OVSDB changes.
> >
> > Although all supporting functions for running a multithreading OVSDB
> > server have been implemented, multithread is still not turned on,
> > because 'n_max_threads' is still hard coded to zero.  Future patches
> > will remove this restriction.
> >
> > Signed-off-by: Andy Zhou <azhou at ovn.org>
>
> This is one way to synchronize threads.  Maybe it is one that we should
> use.  It is nice from the point of view that the sending thread can just
> post a message and does not have to wait for it to be received, but it
> has the disadvantage that it is necessary to define a data structure and
> write marshaling and de-marshaling code for each possible form of
> sychronization.
>


> Another form of synchronization with basically opposite properties is to
> give each thread a mutex.  Whenever the thread wakes up, take the mutex;
> just before it goes to sleep, release the mutex.  To synchronize with
> the thread, take its mutex.  To synchronize with every other thread,
> take all of their mutexes.
>
> I'd like to hear your thoughts.
>

Using messaging passing has a few advantages:

1. It can be faster as you have pointed out, a message can be sent without
waiting.

2, It is simpler to reason about locking in some cases. For example, when
two threads
want to synchronize with every other thread at the same time, the mutex
scheme
proposed above will require additional mechanism to avoid dead lock, while
it is more
straight forward with message passing.

I don't feel the marshaling and de-marshing code is much of a burden in
this case. Since
all threads are running within the same address space, we can just pass
pointer of structs
back and forth, so it is not too back.

What do you think?