[ovs-dev] [PATCH v1 ovn] ovn-nb/sbctl.c: Set no-leader-only as default for clustered dbs

Han Zhou zhouhan at gmail.com
Thu Oct 3 17:19:35 UTC 2019

On Thu, Oct 3, 2019 at 9:09 AM Ben Pfaff <blp at ovn.org> wrote:
> On Tue, Oct 01, 2019 at 05:05:28PM -0700, amginwal at gmail.com wrote:
> > From: Aliasgar Ginwala <aginwala at ebay.com>
> >
> > When using ovn-nb/sbctl running in cluster, one can use local
> > socket to run different commands. It is very inconvenient to pass
> > no-leader-only in different tools using ovn-nb/sbctl instead of
> > allowing one to to connect to any nodes in the cluster including
> > itself.
> > e.g common usage ovn-nb/sbctl show.
> > Hence, this commit handles the same.
> >
> > Signed-off-by: Aliasgar Ginwala <aginwala at ebay.com>
> This change makes more of a difference than its size implies, because it
> means that scripts that previously were guaranteed to get up-to-date
> data can now get inconsistent results.  It loses read-after-write
> consistency, for example.  I'd really prefer to avoid surprising users
> with that kind of thing (especially as a change).
> If it's common to want that kind of behavior, though, perhaps there
> could be a nice way to set it as the default for a session.  In daemon
> mode, of course, it's already possible to control it for the daemon's
> users.  One option for outside daemon mode might be to introduce an
> environment variable.  The environment variable could be specific to
> this feature, e.g. OVN_LEADER_ONLY=0 or OVN_LEADER_ONLY=1, or it could
> be a general-purpose options variable,
> e.g. OVN_OPTIONS=--no-leader-only.  I don't know whether it should be
> specific to one of ovn-sbctl and ovn-nbctl or apply to both.
> Have you thought about these possibilities?
Thanks Ben for the comments. Let's compare the pros and cons of each

1) keep current implementation - default to leader-only

pros: Consistency is preserved with default parameter

- Existing scripts with ovn-nb/sbctl using default unix socket will fail.
E.g. when restarting a central node service using: ovn-ctl start_northd.
The command will fail when executing the step ovn-nbctl init, because the
current node is not the leader. There can be other scripts built by user
that is supposed to work with the default option now might fail as well.
All these can be fixed by updating all the scripts that doesn't require
strong consistency with --no-leader-only.
- Inconvenience of manual command execution for regular operations (mostly
not care about occasionally reading staled data).

2) apply this patch - default to no-leader-only, except daemon mode

pros: Existing scripts will not *fail*.

cons: There are *small* chances that inconsistent (stale) data can be read
by the existing tools (except daemon mode, because this patch enforces
leader-only as default for daemon mode). To avoid that, the scripts that
has strong consistency requirement should be updated to add --leader-only.

3) default to leader-only, but let user to change the default behavior by
setting environment variable

- Consistency is preserved with default parameter, if user doesn't set
environment variable
- User has the freedom to change default behavior by setting environment

- Existing scripts with ovn-nb/sbctl using default unix socket will still
fail, unless they are updated to set the environment variable, which is
even less straightforward change compared with adding --no-leader-only
- If user set the environment variable to use --no-leader-only by default
(e.g. in bashrc of central node), then I think it is the same problem that
by default user could get inconsistent data.
- One more place to control the default behavior, which sometimes leads to
more confusion.

I just listed the pros and cons for discussion. I don't have a strong
opinion which way is the best. (The number of pros/cons doesn't necessarily
mean it is better/worse)

More information about the dev mailing list