[ovs-discuss] [ovs-dev] Somebody making --user and dpdk compatible again?

[Ansis Atteka]

On 29 January 2016 at 12:10, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:

> Quoting Christian Ehrhardt (christian.ehrhardt at canonical.com):
> > On Wed, Jan 27, 2016 at 8:26 PM, Ansis Atteka <ansisatteka at gmail.com>
> wrote:
> >
> > >
> > >
> > > On 27 January 2016 at 02:30, Christian Ehrhardt <
> > > christian.ehrhardt at canonical.com> wrote:
> > >
> > >>
> > >> On Wed, Jan 27, 2016 at 9:29 AM, Ansis Atteka <ansisatteka at gmail.com>
> > >> wrote:
> > >>
> > >>>
> > >>>
> > >>> On 26 January 2016 at 11:07, Christian Ehrhardt <
> > >>> christian.ehrhardt at canonical.com> wrote:
> > >>>
> > >>>> Hi Ansis,
> > >>>> the links I referred were older and yes some had apparmor/selinux
> > >>>> issues.
> > >>>> I didn't refer so much to the issues they had, but more to the fact
> that
> > >>>> they usually ended up running vswitchd as different user.
> > >>>> But most of that is fixed in the respective distributions already.
> > >>>> I did a quick test a few days ago disabling all of the which didn't
> > >>>> change
> > >>>> the situation, so I reenabled them again.
> > >>>>
> > >>>> I already checked the processes and sockets with ps/ls and such.
> > >>>> Vswitchd ran as root - so was the ownership of the sockets
> (root:root
> > >>>> with
> > >>>> srwxr-xr-x).
> > >>>> kvm/libvirt/qemu in my case is running as user libvirt-qemu and that
> > >>>> mismatch to the socket is what caused the permission denied, just
> basic
> > >>>> file permissions/ownership.
> > >>>>
> > >>>> This is what got me to --user, which was working nice in general
> but as
> > >>>> mentioned is mutually exclusive to dpdk as of now.
> > >>>>
> > >>>> I found the following to be a simplification of the Test with kvm:
> > >>>>   socat - UNIX-CONNECT:/var/run/openvswitch/vhost-user-1
> > >>>> You can run that from various users as you need.
> > >>>>
> > >>>> I'm happy if we at some point want to do more fine grained
> > >>>> apparmor/selinux
> > >>>> profiles to confine things more.
> > >>>> But IMHO they won't overcome the basic access issue as long as qemu
> just
> > >>>> isn't allowed to access the unix socket.
> > >>>>
> > >>>
> > >>> Hi Christian.
> > >>>
> > >>> Yes, MAC (SELinux and Apparmor) can't magically allow actions that
> are
> > >>> blocked by DAC (Linux user:group) or vice versa. Both of them are two
> > >>> independent access control solutions.
> > >>>
> > >>> Here are the 4 possible solutions that I see:
> > >>>
> > >>> 1. OVS downgrades from "root" to "openvswitch"; The user under which
> KVM
> > >>> runs gets added to "openvswitch" group.
> > >>> 2. OVS runs as root, but allows to set through OVSDB "kvm" or any
> other
> > >>> user as owner for UNIX domain socket; After that KVM process simply
> > >>> connects to that socket.
> > >>> 3. OVS runs as root; KVM process starts as root, opens the socket to
> > >>> OVS and then downgrades to "kvm" user.
> > >>> 4. OVS runs as root; KVM runs as root; Both processes are confined
> > >>> through Mandatory Access Control instead.
> > >>>
> > >>> Basically solution #1 is what --user flag was intended to do. Though,
> > >>> you would have to run Open vSwitch under "openvswitch" user and not
> "kvm".
> > >>> Also, Open vSwitch may need to be restarted on certain configuration
> > >>> changes that need OVS to have access again to only-root resources.
> > >>>
> > >>> However, based on the chown/chmod commands that you invoked you are
> > >>> expecting something like #2. You still need "root" access at least
> for
> > >>> short time because the right socket owner needs to be specified for
> > >>> that UNIX domain socket (either through OVSDB socket or other means).
> > >>>
> > >>> I believe solution #3 would work if KVM's --runas flag would
> > >>> make KVM process to downgrade user only after it opened that socket.
> I
> > >>> suppose this did not work for you? This would also not survive "only
> OVS
> > >>> reboot". Note that solution #1 and #3 are like-symmetric opposites.
> > >>>
> > >>> Solution #4 is what I think is the right solution. However, even this
> > >>> may not be silver bullet for all the situations (see the other
> email). What
> > >>> I like about it is that discretionary access control policy does not
> > >>> pollute domain logic.
> > >>>
> > >>> Let me know which you think would be the right solution?
> > >>>
> > >> [...]
> > >>
> > >> Hi Ansis,
> > >> thanks for sharing your further evolved thoughts on DAC/MAC for
> > >> openvswitch.
> > >> I must admit that I'm no expert in MAC/DAC topics. But reading through
> > >> the related link and posts on the mailing list it seems so far no one
> stood
> > >> up and said - "here I am, expert in that stuff and solving it for
> you" :-)
> > >>
> > >
> > > Sorry, if I sounded as if MAC is the only option - that wasn't my
> goal. I
> > > am open to DAC as well and that is the reason why I listed all those 4
> > > options to see which one would fit you the best to overcome the
> problem you
> > > are facing. :)
> > >
> > > Also, you were oversimplifying some things for solution #1 - see my
> > > response below.
> > >
> > >
> > >> I guess we all have certain use cases and visions in mind how we can
> help
> > >> to improve openvswitch - in the current case regarding access control.
> > >> Mine for example is to care about the users consuming openvswitch on
> > >> ubuntu and at the moment those that will use it with openvswitch+dpdk
> in
> > >> particular.
> > >> Just as Aaron likely cares for the same in an RHEL/Fedora/CentOS
> context.
> > >> So if I today would write a "how to use" guide it would have to state
> > >> "after every start/restart of openvswitch you have to chown some
> files"
> > >> which clearly isn't what I would expect.
> > >>
> > > I want to assure you that I care about this just as much as you and I
> > > agree that we need to come up with user-friendly solution. :)
> > >
> > > Yet I don't expect it to break other confinement either to get it done
> > >> (e.g. run kvm as root just to be able to connect).
> > >>
> > >> That said here my opinion to your list of DAC/MAC options:
> > >> #1&#2 - I don't see those to be mutually exclusive. By extending upon
> > >> existing functionality it can be achieved in the short term.
> > >>
> > > #1 and #2 are mutually exclusive because Open vSwitch can't chown
> socket
> > > from "openvswitch" to "kvm" if Open vSwitch would run purely under
> > > "openvswitch" user. Here is example:
> > >
> > > openvswitch_test at aatteka-MacBookPro:/tmp$ echo "creating file as
> > > openvswitch" > ovs_file
> > > openvswitch_test at aatteka-MacBookPro:/tmp$ chown kvm_test:kvm_test
> ovs_file
> > > chown: changing ownership of ‘ovs_file’: Operation not permitted
> > >
> > > So, for #2 to work while you also have #1 then
> > > 1. OVS needs to (at least) partially remain as root that calls chown
> and
> > > have another piece of OVS to run under restricted user (invasive for
> OVS);
> > > OR
> > > 2. openvswitch user needs to be put under kvm group to allow Open
> vSwitch
> > > to chown sockets to kvm user (ie white list each group)
> > >
> >
> > You are totally right for chown or arbitrary groups you need to be root.
> > The second option is what I had in mind.
> > And that is what I'd consider the most likely use case in the near term
> > combining #1&#2.
>
> Sorry I've not really had anything to add here, I'm just not familiar
> enough with the ovs codebase.  Absolutely running as non-root as much as
> possible would be far preferred, so that the MAC policy can be just another
> layer in our defense in depth rather than our only card.  It sounds from
> the above as though you don't just create one socket but create them on
> the fly, one per started VM?  I realize this becomes more work for ovs,
> but perhaps ovs could fork off a privileged child which just creates
> and chowns sockets on demand and hands them back over a socketpair to

the parent, which drops privs immediately after the fork?  (I would have
> thought ovs needed privs for a lot more of its work than that, but if
> not...)


Hi Serge,

I think that there is catch-22 paradox with Discretionary Access Control.
Here is why:
1. Processes need access control because domain logic does not implement
proper input and error code validation (e.g. allows shell escape, path
escape ...).
2. Discretionary Access Control is intertwined with Domain Logic (ie access
control at least partially resides in the same process space as domain
logic, at least the "chown" invocations); This means that developers are
tempted to reuse the same input validation for access control and hence
have the same set of bugs.

I think [http://openvswitch.org/pipermail/dev/2016-January/065351.html]
patch demonstrates this Catch-22 paradox very well where with `ovs-vsctl
add-port "../../../ ...` command it became possible to chown arbitrary
files owned by root because domain logic had "path escape" flaw that was
inherited in access control too.

Of course this could be solved by having a separate "Access Control Policy
Engine" in the OVS user-space that runs as root and has some restricting
white list rules. However, if we would be going this far, then wouldn't we
be reinventing SELinux and Apparmor? It would be just like SELinux/Apparmom
Linux Security Module that sits between the confined user space part and
the actual system call.

Note that dnsmasq has similar issue where you can create Trojan files under
file system even when using "--user nobody" flag. Try, `dnsmasq
--log-file=/etc/rsyslog.d/trojan.conf --user=nobody`. Yes, if input is
hardcoded this does not sound like a big deal, but OVSDB receives most of
configuration at run time from OVSDB and I believe most of the bugs would
be of nature where input is not properly validated.

Also, I agree that there are cases when Discretionary Access Control makes
total sense, for example, when one is running webserver on port >1024 and
can start process under the restricted user from the very beginning.

The arguments I have heard so far in favor of DAC are:
1. Hard to disable (but only because DAC is hard-coded in the Application
itself and admin would have to edit systemd or binaries themselves to
switch back to root).
2. More consistent across different Linux distributions.
3. A lot of processes have been already ported to use some sort of DAC
hence this is least resistance path when we look at the distribution as
whole where process need to talk with each other.


Ansis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160201/eb07e0ce/attachment-0002.html>