[ovs-discuss] Handling conf.db ownership on OVS_USER_ID changes

Aaron Conole aconole at redhat.com
Mon Apr 29 16:02:36 UTC 2019 

Jaime Caamaño Ruiz <jcaamano at suse.de> writes:

>> As a "security concern" you mean something among the lines where one
>> of ovs-* processes running under openvswitch user would go ahead and
>> create a file with its owner that later one of ovn processes would
>> blindly reuse without checking that it actually belongs to root:root?
>
> One example is that the OVS user could create link as any OVN log file
> to other file owned by root and then OVN would write to that file.

There's a long-standing issue of OVN running as root.  I don't see any
reason it should.

>> Can you give a more concrete example? I believe logrotate is running
>> under root and should be able to rotate everything?
>
> The logrotate configuration for openvswitch logs has a 'su' directive
> to run under the openvswitch user, precisely to prevent something
> similar to the above. So it wont be able to rotate root owned logs in
> the openvswitch directory. How it fails precisely depends on global
> logrotate configuration. For example it will fail to create the new log
> file after rotation if the 'create' directive is used. Or it may fail
> to compress the rotated log file as it wont be able to read it.
>
> BR
> Jaime
>
> -----Original Message-----
> From: Ansis Atteka <ansisatteka at gmail.com>
> To: jcaamano at suse.de
> Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
> Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID
> changes
> Date: Wed, 24 Apr 2019 19:07:24 -0700
>
> On Tue, 23 Apr 2019 at 09:30, Jaime Caamaño Ruiz <jcaamano at suse.de>
> wrote:
>> 
>> Hello
>> 
>> So the non root owned log directory (and run directory) is shared
>> between non root OVS processes and root OVN processes. Doesn't this
>> raise some security concerns?
>
> As a "security concern" you mean something among the lines where one
> of ovs-* processes running under openvswitch user would go ahead and
> create a file with its owner that later one of ovn processes would
> blindly reuse without checking that it actually belongs to root:root?
>
>
>> 
>> Also, since logrotate rotates the logs with the OVS user, it may fail
>> to some extent to rotate the root owned OVN log files. Consequences
>> vary but in it's current state some logging might be lost. This could
>> be improved but I would guess that the approprioate thing to do is to
>> etiher use a different log directory for OVN or make its processes
>> run
>> with the OVS user also. Has any of this been considered?
>
> Can you give a more concrete example? I believe logrotate is running
> under root and should be able to rotate everything?
>
>
>> 
>> BR
>> Jaime.
>> 
>> 
>> -----Original Message-----
>> From: Jaime Caamaño Ruiz <jcaamano at suse.de>
>> Reply-to: jcaamano at suse.com
>> To: Ansis Atteka <ansisatteka at gmail.com>
>> Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
>> Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID
>> changes
>> Date: Wed, 17 Apr 2019 12:52:32 +0200
>> 
>> > You also need to chown /var/log/openvswitch.*.log files.
>> 
>> OVS seems to be already handling this. I dont know the details but I
>> guess that before dropping capabilities, OVS chowns these by itself.
>> 
>> > However,
>> > what about other daemons, like ovn? Do they share run time dir?
>> > Haven't looked into this in a while and don't have a setup to check
>> > myself.
>> 
>> The permisions of the openvswitch run directory are already being
>> handled by the service unit file. There is read access to files
>> created
>> there by OVS and AFAIK what it ultimately makes it a no problem for
>> OVN
>> is that it still runs as root with the provided unit files. If anyone
>> changes this to a user different than the openvswitch user, then that
>> is an already existing issue. Am I missing something here?
>> 
>> Agree with you other considerations.
>> 
>> BR
>> Jaime.
>> 
>> -----Original Message-----
>> From: Ansis Atteka <ansisatteka at gmail.com>
>> To: jcaamano at suse.de
>> Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
>> Subject: Re: [ovs-discuss] Handling conf.db ownership on OVS_USER_ID
>> changes
>> Date: Tue, 16 Apr 2019 18:21:48 -0700
>> 
>> On Tue, 16 Apr 2019 at 17:20, Jaime Caamaño Ruiz <jcaamano at suse.de>
>> wrote:
>> > 
>> > My intention was doing it at systemd unit (prestart) for file
>> > conf.db
>> > (and .conf.db.~lock~ that stays around) only.
>> 
>> You also need to chown /var/log/openvswitch.*.log files.
>> > 
>> > I was not thinking on absolutely fool proof mechanism, among other
>> > things because the admin might have customized the location for the
>> > database. Also, updating and restarting the service are things that
>> > would usually be monitored. So best effort.
>> 
>> make sure that best effort solution informs admin in case of error so
>> that he does not have to go file by file to find the offending one.
>> 
>> > 
>> > At systemd unit prestart the service is stopped and OVS nor nobody
>> > should really be messing with the database files owned by OVS. If
>> > the
>> > spec file is changing things back to root unappropriately then that
>> > should be changed too.
>> 
>> Yes.
>> 
>> > 
>> > The runtime /run/openvswitch directory is also controlled by
>> > systemd
>> > and at least on my system is removed if the service is not active,
>> > either by graceful or ungraceful exit.
>> 
>> I somewhat agree with this, because back in those days we were using
>> SystemV where the init.d managed lifecycle of run directory..
>> However,
>> what about other daemons, like ovn? Do they share run time dir?
>> Haven't looked into this in a while and don't have a setup to check
>> myself.
>> 
>> > 
>> > I guess we will need to look for specific issues. Any hint to find
>> > that
>> > patch?
>> 
>> 1. test upgrade to your patched OVS and also from your patched OVS to
>> another future version
>> 2. if you change existing spec files that are shared with on Fedora
>> or
>> RHEL/CentOS make sure you don't break anything for them.
>> 3. prefer consistency w.r.t. default behavior across deb and rpm
>> packages (if you change user by default consider to do that for all
>> platforms)
>> 4. besides database you will also need to set right access bits to
>> log
>> files. On my Ubuntu it is "-rw-r----- 1 root adm " which means:
>> 4.1. you have to chown it; OR
>> 4.2. add openvswitch user to adm group and chmod that file
>> 5. test logrotation. Maybe you can use a one forced logrotation
>> invocation to change the ownrer.
>> 6. check that other stuff like ovs-monitor-ipsec is not affected. It
>> stil need to talk with StrongSwan or libreswan that best to my
>> knowledge still runs as root.
>> 7. if you use USER= then:
>> 7.1. For Linux datapath make sure you retain CAP_NET_ADMIN Linux
>> Capabilities
>> 7.2. I don't know if there are special privileges you need to retain
>> for DPDK. And in future possibly AF_XDP.
>> 8. I haven't looked in a while but check if ovn (that has its own
>> Systemd service file) shares run dir with OVS
>> 
>> > 
>> > BR
>> > Jaime.
>> > 
>> > -----Original Message-----
>> > From: Ansis Atteka <ansisatteka at gmail.com>
>> > To: jcaamano at suse.de
>> > Cc: ovs-discuss at openvswitch.org, Ben Pfaff <blp at ovn.org>
>> > Subject: Re: [ovs-discuss] Handling conf.db ownership on
>> > OVS_USER_ID
>> > changes
>> > Date: Tue, 16 Apr 2019 16:11:44 -0700
>> > 
>> > On Tue, 16 Apr 2019 at 15:36, Jaime Caamaño Ruiz <jcaamano at suse.de>
>> > wrote:
>> > > 
>> > > > On any given system, I would expect ovsdb-server to run as the
>> > > > same
>> > > > user
>> > > > every time.  Is there a reason to sometimes use a different
>> > > > user?
>> > > 
>> > > Well, changing from root which was the only supported option in
>> > > the
>> > > past and current default to a different user (like the suggested
>> > > openvswitch), this one time, might be somewhat common. And the
>> > > scenario
>> > > that I am looking at is suporting a pianless upgrade through it.
>> > 
>> > Actually debian packages still by default use "root" user. Not
>> > "openvswitch".
>> > 
>> > Long time ago there was an attempt to change the default user
>> > across
>> > all different flavor packages to openvswitch (you can lookup the
>> > patch
>> > on mailing list). However, as you noticed upgrades are tricky.
>> > Hence
>> > packages built from our debian/rules and rhel/openvswitch.spec.in
>> > files still use "root" as default user. The packages built with
>> > rhel/openvswitch-fedora.spec.in are an exception where the user
>> > indeed
>> > is "openvswitch". Unless you passed --without libcapng flag to
>> > rpmbuild invocation. Then the user would still be "root".
>> > 
>> > Here are the difficulties with automating the change of file
>> > ownership:
>> > 1. from which context to change ownership? As chown must be invoked
>> > from something that runs under root then there is not much choice -
>> > either package installation time (%post scriptlet) or daemon
>> > startup
>> > time (assuming you are not using Systemd's USER= feature in spec
>> > file)
>> > or something that admin explicitly has to invoke from bash as
>> > "root".
>> > 1.1. if it is package installation time then you can't update user
>> > gracefully at daemon init time. It will be possible only at
>> > installation time.
>> > 1.2. if it is installation time then you can run into weird race
>> > conditions where some code that was executed under root (as package
>> > %post scriptlets) or previous instance of OVS that was still
>> > running
>> > as root could change ownership back to root for some files and
>> > cause
>> > unexpected "permission denied" issues (the patch I mentioned was
>> > affected with these rare race conditions)
>> > 2. for which files to change ownership? What happens if Unix domain
>> > socket file remained on fileystem with "root" permissions when OVS
>> > was
>> > abruptly killed? What if someone else put a file under ovs
>> > directories
>> > with non-root user - should you blindly chown that too? What if
>> > %post
>> > scriptlet that runs under root non-atomically created a file and in
>> > next line chown() it back - is there a window for race? Should you
>> > follow directories/symlinks recursively? Changing user for conf.db
>> > file is not enough...
>> > 
>> > I am not saying it is impossible to change user gracefully on
>> > upgrades. I am saying that code may have to be reorganized quite a
>> > bit
>> > to eliminate the issues I raised above.
>> > 
>> > Alternatively, for Centos, RHEL and Fedora one can use SElinux to
>> > confine processes and leave them running as root (instead of
>> > running
>> > them under limited privilege user). I believe on SUSE (and Debian)
>> > once could use AppArmor to achieve the same results. At least for
>> > SElinux it was a little bit easier to make upgrade seamless than
>> > with
>> > Linux user confinement approach.
>> > 
>> > > 
>> > > So if that makes sense to you, I will go ahead with the patch.
>> > > 
>> > > BR
>> > > Jaime.
>> > > 
>> > > -----Original Message-----
>> > > From: Ben Pfaff <blp at ovn.org>
>> > > To: jcaamano at suse.de
>> > > Cc: ovs-discuss at openvswitch.org
>> > > Subject: Re: [ovs-discuss] Handling conf.db ownership on
>> > > OVS_USER_ID
>> > > changes
>> > > Date: Tue, 16 Apr 2019 15:25:24 -0700
>> > > 
>> > > On Tue, Apr 16, 2019 at 11:32:21PM +0200, Jaime Caamaño Ruiz
>> > > wrote:
>> > > > When sysconfig OVS_USER_ID is changed to a different user, it
>> > > > requires
>> > > > a manual ownership change of the OVS conf.db database if
>> > > > existing
>> > > > or
>> > > > otherwise ovsdb-server will fail to (re)start. I was wondering
>> > > > if
>> > > > I
>> > > > am
>> > > > missing any particular reason why this is change of ownership
>> > > > is
>> > > > not
>> > > > automatically being handled through the service unit file as it
>> > > > is
>> > > > being done with other items.
>> > > 
>> > > On any given system, I would expect ovsdb-server to run as the
>> > > same
>> > > user
>> > > every time.  Is there a reason to sometimes use a different user?
>> > > 
>> > > Or, perhaps you are saying that, just before invoking ovsdb-
>> > > server,
>> > > the
>> > > service unit should chown (or whatever) the database file to the
>> > > user
>> > > that it is going to use to invoke ovsdb-server.  That might be
>> > > reasonable, but no one has thought to do it yet.  Do you want to
>> > > submit
>> > > a patch?
>> > > 
>> > > Thanks,
>> > > 
>> > > Ben.
>> > > 
>> > > _______________________________________________
>> > > discuss mailing list
>> > > discuss at openvswitch.org
>> > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>> > 
>> > 
>> 
>> 
>
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

More information about the discuss mailing list