[ovs-discuss] etcd for OVN status update (was: Re: more about etcd (can it support big transactions and many monitors?))

[Han Zhou]

Hi Andy,

Sorry #1 seems not clear to me. It sounds like a etcd cluster running
behind a ovsdb-server cluster? Then what would be the HA mechanism for the
ovsdb-server layer?

Han

On Thu, Jul 7, 2016 at 11:07 AM, Andy Zhou <azhou at ovn.org> wrote:
>
> Ben and I discussed briefly,  and came up with two possible alternatives:
>
> 1. Deploy ovsdb and etcd in tendon.
> The Ovsdb clients still connect to ovsdb server. OVSDB server(s) stores
the db in etcd, rather than
> a file.  Referential integrity will be enforced by ovsdb server.
>
> 2. Absorb some ovsdb server functions into the idl layer.  The ovsdb
client connects directly to etcd,
> but need to guarantee referential integrity for each transaction.
>
> #1 is more straight forward to implement.  ovsdb client does not need to
change; they are still talking
> to ovsdb servers. However, the number of transactions will be 2x, so
transaction latency will increase.
> In addition, the overall system can be more complex because two database
are in use.
>
> #2 can be more efficient than #1. But it seems odd to put the burden of
referential integrity into client. (although
> this must be the case if we were to write a native etcd client from
ground up).  Technically, ovsdb clients
> may not be possible when ovsdb clients only maintains a subset of the DB
replica. Of course this can be solved
> by replicating more data to the client than otherwise necessary.
>
>
> Ben, please feel free to chime in in case I missed anything.
>
> I plan to prototype #1, and keep the code modular so that they can be
applied to #2, just in case we like #2 more.
>
>
>
>
>
>
>
> On Fri, Jul 1, 2016 at 1:23 PM, Ben Pfaff <blp at ovn.org> wrote:
>>
>> On Wed, Jun 29, 2016 at 03:38:26PM -0700, Andy Zhou wrote:
>> > This is one possible way to implement reference integrity with etcd:
>> >
>> > * DB wide versioning.
>> >
>> > Assign a key db/version that stores db wide transaction id. Assume the
id
>> > starts with 0. Any client issued transaction on the DB should also
include
>> > this key; A transaction will increase its value by 1; Any etcd client
>> > transaction
>> > will always bring this version number from even to an odd number.
>> >
>> > No further transaction can be issued until "db/version"'s value become
>> > even.
>> >
>> > * A dedicated client enforces referential integrity
>> >
>> > There is a dedicated etcd client whose job is to enforce referential
>> > integrity.
>> > It starts to run when the version number is odd, commit the next
transaction
>> > that "fixes"  the etcd.  The version number is increased even if there
is
>> > nothing
>> > to fix.
>> >
>> > In the HA setup, referential integrity checking clients should run on
the
>> > same machines
>> > that run etcd. Only the etcd client that runs on the same machine as
the
>> > etcd leader
>> > will actively enforce referential integrity.  Other clients will be
running
>> > in standby mode,
>> > and only become active when its local etcd server become the leader.
>> >
>> > Will this work?
>>
>> I agree with Russell that this sounds really expensive.
>
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160707/a834feb5/attachment-0002.html>