[ovs-discuss] etcd for OVN status update (was: Re: more about etcd (can it support big transactions and many monitors?))
Ben Pfaff
blp at ovn.org
Wed Jun 15 17:02:02 UTC 2016
On Fri, Jun 10, 2016 at 03:59:34PM -0700, Ben Pfaff wrote:
> After talking to Justin, I think I'm going to take a few days (maybe
> Wednesday through Friday next week) to hack on etcd related stuff, with
> the goal being to come up with a detailed to-do list and try to verify
> that the stuff that I think should work really will. (For example, Andy
> thinks that the gRPC implementation for C is incomplete, but neither of
> us has really tried anything.)
>
> I guess the first step for me personally is an x86-64 container or
> chroot or something.
Here are the questions I started out with:
1. Can I get etcd v3 working on an i386 laptop somehow, given etcd's
hostility to non-x86-64 architectures?
2. Can I get a C program to talk to etcd v3, given:
- etcd is layered on grpc, which does not have a full set of bindings
for C but instead infrastructure bindings on which the other
language bindings are layered.
- etcd uses protobufs (as is typical) inside grpc, but protobufs do
not have official C bindings either.
3. How should the OVN databases be arranged within etcd? There are
multiple possibilities:
- Define OVSDB bindings to etcd and implement those bindings in the
OVSDB client libraries (C and Python).
- Define OVSDB bindings to etcd and implement those bindings in the
OVSDB server (so that ovsdb-server uses etcd as a storage layer).
- Define a native etcd schema for OVN SB (and probably NB) database
and make ovn-controller and ovn-northd use it natively.
Here's what I've accomplished so far:
* Managed to get etcd v3 beta running in a docker container on my
laptop. This solves the problem I personally have due to running i386
userspace on my laptop, because Docker containers run x86-64 userspace
internally regardless of whether the host uses i386 or x86-64
userspace.
(This does not answer the larger question of how archs other than
x86-64 will use OVN.)
I suspect this is not the same reason that the rest of the world is so
excited about containers, but it works for me.
* Managed to interact with etcd in the x86-64 container from the etcdctl
v3 command-line client running on my i386 host. This does not seem to
have the same arch problem as running the etcd server, or at least the
client does useful work instead of immediately aborting with an error
due to an unsupported arch.
* Managed to build the grpc C libraries.
* Managed to use "nanopb" (https://github.com/nanopb/nanopb) to generate
C bindings for the Protocol Buffer definitions for the etcd v3
protocol. This took a little bit of coaxing. I have not yet tried to
compile or use what it generated. (grpc itself uses nanopb
internally, which gives me hope that nanopb works OK.)
Here's what is left:
* Write a C test program that tries to tie together the grpc C library
and the nanopb-generated etcd bindings into something that can
actually talk to etcd, probably something comparable to a simplified
etcdctl in C.
This has multiple steps, at least the following:
- Figure out what's needed to tie together all the grpc C
infrastructure code into a C library that can actually implement a
grpc service. I don't know whether this is on the spectrum from
trivial to troublesome.
- Layer a library on top of the previous one from the previous bullet
that can send and receive etcd messages.
- Layer a test program on top of the previous library.
- Get grpc and nanopb-generated code and everything else to actually
link together. This is always more of a challenge than it ought to
be.
* Figure out OVN database handling (question 3 above).
So far, I'm feeling fairly optimistic.
Thanks,
Ben.
More information about the discuss
mailing list