[ovs-dev] [RFC] ovn: minimize the impact of a compromisedchassis

[Ryan Moats]

"dev" <dev-bounces at openvswitch.org> wrote on 08/21/2016 03:02:12 PM:

> From: Justin Pettit <jpettit at ovn.org>
> To: Russell Bryant <rbryant at redhat.com>
> Cc: ovs dev <dev at openvswitch.org>
> Date: 08/21/2016 07:30 PM
> Subject: Re: [ovs-dev] [RFC] ovn: minimize the impact of a compromised
chassis
> Sent by: "dev" <dev-bounces at openvswitch.org>
>
>
> > On Aug 16, 2016, at 6:52 AM, Russell Bryant <rbryant at redhat.com> wrote:
> >
> > ​Thanks for starting this discussion.​
> >
> > ​I do think making ovn-controller a read-only client of the database
seems
> > like the simplest path forward.  We should pursue that until we hit a
> > strong reason not to.
> >
> > One of the biggest questions remaining for me is how we deal with
backwards
> > compatibility.  Whatever we do here will have to account for what
happens
> > for environments running OVN from OVS 2.6 when they upgrade.
> >
> > Perhaps the most straight forward way to do that is to make this new
more
> > secure mode optional and off by default.  The downside is that it makes
the
> > system more complex, since it will have different modes it can work.
>
> I think we should avoid providing two modes if at all possible.  As
> you mentioned, it increases the complexity, which will likely make
> it more difficult to test thoroughly and deploy consistently.
>
> > An alternative would be to provide some tooling to assist with the
upgrade:
> >
> > - Document the new requirements for creating Chassis and Encap rows
> > manually
> >
> > - Provide an upgrade tool (via ovn-nbctl/ovn-sbctl/something-else) that
> > will add the "chassis" option to logical ports set based on where ports
are
> > currently bound in the southbound database.
> >
> > ​ - Provide an upgrade tool that converts the MAC_Binding table to
whatever
> > the new chassis local storage for that information would be.
>
> I think the tool approach is better.  I imagine people deploying 2.6
> will be pretty closely tied to the community still, so walking them
> through an upgrade shouldn't be too bad--especially if we make it
> easy.  Obviously, we'll try to avoid the need for that again in the
future.
>
> --Justin

I agree that the ovn-controller process should be limited to read-only
for the following tables
     SB_Global
     Logical_Flow
     Multicast_Group
     Datapath_Binding
     Address_Set
     DHCP_Options
     DHCPv6_Options

However, I'm going to argue that the suggestions for making ovn-controller
read-only for

 Chassis
 Encap
 Port_Binding
 MAC_Binding

need some more discussion as I don't think all of the suggestions to date
are entirely feasible...

First, putting the responsibility for updating the Chassis and Encap tables
on the CMS require (a) that the CMS have the ability to provide that
information
and (b) that the CMS have an OVN plugin.  I see this as moving OVN out of
the
pure networking space (at least with respect to OpenStack) as I don't see
Neutron getting this functionality any time soon.  I agree with Justin that
the right approach is to provide documentation for how to add/remove/update
entries via ovn-sbctl and then leave it to operators to incorporate that
tooling into their deployment scenarios.

For Port_Binding, while it is possible to get that information from the
CMS,
I worry (at least in the OpenStack case) about a potential race condition
during instance boot - we've already seen cases during scale testing where
the
current method (having the chassis update the port binding) doesn't
percolate
through Neutron back to Nova fast enough, and using the CMS to set the
port binding will add another half round trip of delay.  Couple that with
the
needed modifications of ovn-controller, and I think that needs some more
thought before we decide on how to change this.

MAC_Binding is a bit tricky - the problem here is how to deal where dynamic
MAC bindings need to be transferred from one chassis to another for either
HA or live migration scenarios. My preference here is to leave this alone
(i.e. allow ovn-controller to continue to write this table) and see what we
can apply from various anti-arp cache poisoning technologies to either the
IDL
or ovsdb-server itself.