[ovs-dev] [ovs-discuss] Somebody making --user and dpdk compatible again?

[Christian Ehrhardt]

On Wed, Jan 27, 2016 at 8:26 PM, Ansis Atteka <ansisatteka at gmail.com> wrote:

>
>
> On 27 January 2016 at 02:30, Christian Ehrhardt <
> christian.ehrhardt at canonical.com> wrote:
>
>>
>> On Wed, Jan 27, 2016 at 9:29 AM, Ansis Atteka <ansisatteka at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On 26 January 2016 at 11:07, Christian Ehrhardt <
>>> christian.ehrhardt at canonical.com> wrote:
>>>
>>>> Hi Ansis,
>>>> the links I referred were older and yes some had apparmor/selinux
>>>> issues.
>>>> I didn't refer so much to the issues they had, but more to the fact that
>>>> they usually ended up running vswitchd as different user.
>>>> But most of that is fixed in the respective distributions already.
>>>> I did a quick test a few days ago disabling all of the which didn't
>>>> change
>>>> the situation, so I reenabled them again.
>>>>
>>>> I already checked the processes and sockets with ps/ls and such.
>>>> Vswitchd ran as root - so was the ownership of the sockets (root:root
>>>> with
>>>> srwxr-xr-x).
>>>> kvm/libvirt/qemu in my case is running as user libvirt-qemu and that
>>>> mismatch to the socket is what caused the permission denied, just basic
>>>> file permissions/ownership.
>>>>
>>>> This is what got me to --user, which was working nice in general but as
>>>> mentioned is mutually exclusive to dpdk as of now.
>>>>
>>>> I found the following to be a simplification of the Test with kvm:
>>>>   socat - UNIX-CONNECT:/var/run/openvswitch/vhost-user-1
>>>> You can run that from various users as you need.
>>>>
>>>> I'm happy if we at some point want to do more fine grained
>>>> apparmor/selinux
>>>> profiles to confine things more.
>>>> But IMHO they won't overcome the basic access issue as long as qemu just
>>>> isn't allowed to access the unix socket.
>>>>
>>>
>>> Hi Christian.
>>>
>>> Yes, MAC (SELinux and Apparmor) can't magically allow actions that are
>>> blocked by DAC (Linux user:group) or vice versa. Both of them are two
>>> independent access control solutions.
>>>
>>> Here are the 4 possible solutions that I see:
>>>
>>> 1. OVS downgrades from "root" to "openvswitch"; The user under which KVM
>>> runs gets added to "openvswitch" group.
>>> 2. OVS runs as root, but allows to set through OVSDB "kvm" or any other
>>> user as owner for UNIX domain socket; After that KVM process simply
>>> connects to that socket.
>>> 3. OVS runs as root; KVM process starts as root, opens the socket to
>>> OVS and then downgrades to "kvm" user.
>>> 4. OVS runs as root; KVM runs as root; Both processes are confined
>>> through Mandatory Access Control instead.
>>>
>>> Basically solution #1 is what --user flag was intended to do. Though,
>>> you would have to run Open vSwitch under "openvswitch" user and not "kvm".
>>> Also, Open vSwitch may need to be restarted on certain configuration
>>> changes that need OVS to have access again to only-root resources.
>>>
>>> However, based on the chown/chmod commands that you invoked you are
>>> expecting something like #2. You still need "root" access at least for
>>> short time because the right socket owner needs to be specified for
>>> that UNIX domain socket (either through OVSDB socket or other means).
>>>
>>> I believe solution #3 would work if KVM's --runas flag would
>>> make KVM process to downgrade user only after it opened that socket. I
>>> suppose this did not work for you? This would also not survive "only OVS
>>> reboot". Note that solution #1 and #3 are like-symmetric opposites.
>>>
>>> Solution #4 is what I think is the right solution. However, even this
>>> may not be silver bullet for all the situations (see the other email). What
>>> I like about it is that discretionary access control policy does not
>>> pollute domain logic.
>>>
>>> Let me know which you think would be the right solution?
>>>
>> [...]
>>
>> Hi Ansis,
>> thanks for sharing your further evolved thoughts on DAC/MAC for
>> openvswitch.
>> I must admit that I'm no expert in MAC/DAC topics. But reading through
>> the related link and posts on the mailing list it seems so far no one stood
>> up and said - "here I am, expert in that stuff and solving it for you" :-)
>>
>
> Sorry, if I sounded as if MAC is the only option - that wasn't my goal. I
> am open to DAC as well and that is the reason why I listed all those 4
> options to see which one would fit you the best to overcome the problem you
> are facing. :)
>
> Also, you were oversimplifying some things for solution #1 - see my
> response below.
>
>
>> I guess we all have certain use cases and visions in mind how we can help
>> to improve openvswitch - in the current case regarding access control.
>> Mine for example is to care about the users consuming openvswitch on
>> ubuntu and at the moment those that will use it with openvswitch+dpdk in
>> particular.
>> Just as Aaron likely cares for the same in an RHEL/Fedora/CentOS context.
>> So if I today would write a "how to use" guide it would have to state
>> "after every start/restart of openvswitch you have to chown some files"
>> which clearly isn't what I would expect.
>>
> I want to assure you that I care about this just as much as you and I
> agree that we need to come up with user-friendly solution. :)
>
> Yet I don't expect it to break other confinement either to get it done
>> (e.g. run kvm as root just to be able to connect).
>>
>> That said here my opinion to your list of DAC/MAC options:
>> #1&#2 - I don't see those to be mutually exclusive. By extending upon
>> existing functionality it can be achieved in the short term.
>>
> #1 and #2 are mutually exclusive because Open vSwitch can't chown socket
> from "openvswitch" to "kvm" if Open vSwitch would run purely under
> "openvswitch" user. Here is example:
>
> openvswitch_test at aatteka-MacBookPro:/tmp$ echo "creating file as
> openvswitch" > ovs_file
> openvswitch_test at aatteka-MacBookPro:/tmp$ chown kvm_test:kvm_test ovs_file
> chown: changing ownership of ‘ovs_file’: Operation not permitted
>
> So, for #2 to work while you also have #1 then
> 1. OVS needs to (at least) partially remain as root that calls chown and
> have another piece of OVS to run under restricted user (invasive for OVS);
> OR
> 2. openvswitch user needs to be put under kvm group to allow Open vSwitch
> to chown sockets to kvm user (ie white list each group)
>

You are totally right for chown or arbitrary groups you need to be root.
The second option is what I had in mind.
And that is what I'd consider the most likely use case in the near term
combining #1&#2.

Near Term use cases for admins using OVS:
#2 implemented - OVS stays root (maybe because some other OVS pieces still
need it) and can change the socket group to kvm (because root can)
#2 implemented and all req. components compatible with #1 - OVS drops
privileges and downgrades to user openvswitch for confinement, but it can
still change socket group to kvm (because admin made openvswitch member of
kvm group)

In the long term I agree and see the beauty of splitting into two privilege
levels to maintain a way to access privileged things at runtime while at
the same time have the majority of code running with reduced privileges.
But I see it - just as MAC - as a long term solution that needs to be
worked out more clearly.



> Also this is the least invasive option to kvm/libvirt.
>>
> What isn't invasive to kvm is invasive to openvswitch; and vice versa.
> This is what I meant with "#1 and #3 are symmetrically opposite solutions".
> So far no one has implemented the right DAC solution because neither of
> both processes *in its current shape* would work with their own hot-plug
> features after it self-downgraded its user (ie KVM's --run-as flag is just
> as flawed as OVS's --user flag).
>
> However, if KVM process insisted to be confined (ie run under kvm user)
> and OVS did not insist to be confined (ie remain running as root), then my
> point is that KVM and not OVS should split into two processes - one running
> as root and opening sockets belonging to root and the other running under
> confined kvm user (I this this is what libvirtd does, no?). If we don't
> want to or can't to do this in KVM, then we are moving the same invasive
> code to OVS. Maybe, yes, least resistance path to get this done is by
> changing OVS code,
>

You are right, sorry - I didn't want to question the symetric relationship
between #1 and #3 - neither that it will end up being "invasive" somewhere.
There could be a lot of arguing which of the projects is at an higher
"risk" (doability, security, ...) when implementing that. But I clearly
don't want to start that discussion as it would be endless and provide no
huge benefit.


> but I want everyone to understand implications of this before we move
> forward...
>

I totally agree and consider to that part to be very important.

Eventually the good thing is that none of your/my/aarons suggestion let OVS
degrade in functionality/features/security in any way - we are just
discussing which way to go.
And IMHO there can be different short/long term targets to achieve.
Personally I'm just a fan of staged approaches where the implications of
the smaller near term things can be more easily considered and thereby
accepted.



>
>> #3 - just as your examples with vswitch needing resources later this
>> would fail with all kind of hot-add features; and it is also invasive to
>> other peojects.
>>
>
> OVS also has hot-add features that are enabled/disabled through OVSDB
> ad-hoc. For example, to install IPsec policies for tunnels when it calls
> "ipsec restart". Note that strongSwan (and racoon) still run as root on
> Ubuntu 15.10 so it is not like we can "downgrade" Open vSwitch user from
> "root" to "openvswitch" and expect OVS to work just fine.
>

Aarons approach is trying to remove one of that incompatibilities (dpdk &
user dropping), while later on other work might address more.
In general it is fine that some options like --user are incompatible with
other options (just as they are today) as long as there is either a reason
for it or a thought/roadmap to get it interoperable someday.


> #4 - even with MAC in place "running as root" doesn't sound 100% right to
>> me, but I agree that MAC based confinement could be a great step forward.
>>
> I agree that running as root should not sound right if there is no proper
> MAC policy in place.
>
>
>> I'd add a missing one
>> #5 - by implementing #1&#2 neither OVS nor KVM will run as root, and we
>> add further fine grained confinement through Mandatory Access Control on
>> top to further bolster the trust for for security/isolation of our SW in
>> enterprise/large scale deployments.
>>
>> So I'd suggest we go #1&#2 soon, then evaluate #5 for long term
>> development.
>>
> If the requirement here is to confine Open vSwitch, then we need #1.
> If the requirement is to get up and running other confined daemons (e.g
> kvm), then #2 should be sufficient.
>

The (my) immediate requirement is to get things going - so #2 which Aarons
patches provide are great, but that is just me.

In general confining OVS is a target worth on its own, but as you pointed
out some components are not yet compatible with it.
So making the existing downgrading (#1) via --user compatible with more
components like racoon, strongswan, dpdk that is fine and might open up a
full #1 solution some day.
We can work on reducing these restrictions one by one if needed, while
later on improved solutions (like the split privilege and MAC ideas) might
appear.


>
>> My (IMHO) preferred ordered approach in detail would be:
>> 1. Help Aaron by reviewing/testing his patches so quickly let users
>> overcome that hurdle of the socket ownership issues (I'd be super happy if
>> these would still slip into 2.5)
>> 2. I'd also suppose to help Aaron with his goal to allow ovs+dpdk work by
>> downgrading 'after' the initializations are made
>>     Yes that might have some drawbacks to access additional privileged
>> resources later, but then it has also huge benefits in terms of confinement.
>> 3. MAC might be the best, but also more complex approach to all of that.
>> Also it clearly seems that there is an agreement for the need to "do it
>> right (or not at all).
>>     So I'd suggest to take some extra time for MAC discussions, code
>> suggestions, reviews, ...
>>     Implementing #1&#2 as short term solutions to the most urgent needs
>> frees up that time for all of us.
>>
>> P.S. Since most thoughts in the discussion linked so far seemed to be
>> about selinux (only) so far I'll be adding James Page and Serge Hallyn to
>> give them a chance to share a more educated thought on all that especially
>> regarding Apparmor'ing of the involved components.
>>
>
> Thanks, I would really appreciate their feedback!
>
> I will take closer look into Aaron's and your patch.
>

Down to just Aarons patches as he had a good head start, he just submitted
v7 of them.