[ovs-dev] ovn: Improving southbound database security

Darrell Ball dlu998 at gmail.com
Wed Oct 19 03:34:33 UTC 2016

On Wed, Oct 12, 2016 at 10:51 AM, Russell Bryant <russell at ovn.org> wrote:

> Hello, I'm back to looking at southbound database security concerns in
> OVN.  A previous thread discussing approaches was here:
>     http://openvswitch.org/pipermail/dev/2016-August/078106.html
> I'm now working with a few others on implementing a proposed solution.  The
> overview is that we'd like to make ovn-controller a read-only client of the
> southbound database.
> The task breakdown is:
> 1) Add support to ovsdb-server for read-only remotes.  The port reachable
> by ovn-controller would only accept read-only connections.

Instead of read-only, why can't rbac be used. rbac seems the more optimal
and flexible option.
rbac could also limit the amount of information written.

Preventing ovn-controller from writing to the common SB DB essentially
means ovn-controller cannot communicate information it learns like HA
failures of remote HVs, mac bindings etc. It seems like a pretty big
constraint to
not be able to trust any information sourced from an ovn-controller.

Presumably ovn-controller still needs to tell northd (directly or
indirectly) some
information which northd would write to the SB DB after vetting it, which
is an
absolute minimum of physical and operational state.

rbac seems a cleaner and more structured approach for this.
etcd has support for rbac; if etcd is not migrated to, then
it seems valuable to add support to ovsdb.

Even restricting ovn-controller to read only for the SB DB leaves big holes
such as arp spoofing, DOS attacks etc. This is made worse because there is
no security
"at the packet level" between HVs. Also, if an ovn-controller is hacked, it
can still read
the whole SB DB with this proposal; this makes hacking the network easier.
Also, what happens when a gateway ovn-controller is hacked. Are all
gateway HVs off limits for hosting containers now.

In addition to rbac support, it seems making efforts to make it more
difficult to hack
ovn-controller and/or actively detect this and take action seems like a
effective solution overall and sacrifices less advantages of ovn-controller
SB write

> 2) Remove support from ovn-controller for automatically creating chassis
> and encap records.  Document this as an administrative step for adding a
> new chassis to the system, instead.

ovn-controller is the natural owner of these records linked to it's
operational state/health.

> 3) Remove support from ovn-controller for setting the chassis column of
> Port_Binding records.  Instead, have this handled by ovn-northd based on
> binding instructions given in the northbound database.

ovn-controller is the natural owner of these records since it knows when
a logical port is active/up and vice versa.

> As a nice side effect, this helps solve one of the difficulties with live
> migration (two chassis fighting to own a port while the migration is in
> progress).  Instead, we would update OVN when we know the migration is
> complete.

hmm, can this also be done with a hold down period that could be added to
the southbound DB
schema and read by the ovn-controllers ?

> I was originally thinking we may need an upgrade utility to help existing
> environments, but I think ovn-northd can handle this automatically.

Can you elaborate on the proposed upgrade marshalling and why ovn-controller
having partial write access during <most> timeframes, but not necessarily
some upgrade window, creates a problem ?

> For the northbound database, I was thinking of adding a new option for
> logical ports called "chassis" with a value of the hostname of the chassis
> the port should be bound to.

That is logical configuration not operational state or physical state.
ovn-controller is the master of it's local physical and operational state
which always gets fed upwards.

> 4) Remove use of MAC_bindings table from ovn-controller.  Replace it with a
> local cache, instead.  I'm proposing just keeping it in memory in
> ovn-controller, but we could also make use of the openvswitch db.

This is a pretty big optimization loss, since sharing arp bindings by HVs
is very useful for lower latency.
arp aging support will aggravate this situation more.
The scale here is future container/vm scale which presumably should be
100s of thousands.
rbac could also be used to limit the amount of writing.

> One detail I haven't fully thought through: what should happen to the
> MAC_Bindings table?  Dropping the table seems not backwards compatible and
> would break a rolling upgrade scenario.  Should we leave it as unused for
> one release, and then remove it in the next release?  More generally, I
> think we need to document our upgrade strategy and related rules for
> database schema changes.

> --
> Russell Bryant
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev

More information about the dev mailing list